Guides

Provider fallback

When a provider returns 429/5xx or its circuit breaker opens, the outbound transport can route the call to the next OpenAI-compatible provider in a chain — DeepSeek, Groq, Cerebras, self-hosted vLLM, and friends. Responses served by a fallback carry X-RateGuard-Fallback: true.

client := rg.WrapClient(nil, rateguard.OutboundOptions{
    Chain: rateguard.NewProviderChain(
        rateguard.Provider("openai", "gpt-4o", "https://api.openai.com/v1"),
        rateguard.ProviderEntry{
            Name:    "deepseek",
            Model:   "deepseek-chat",
            BaseURL: "https://api.deepseek.com/v1",
            Headers: map[string]string{
                "Authorization": "Bearer " + deepseekKey,
            },
        },
    ),
})

Honest scope

Fallback works across OpenAI-compatible endpoints only — same request schema. Cross-schema fallback (OpenAI → Anthropic's native API) is impossible at the transport layer and RateGuard does not claim it. Credentials never transfer between providers; each chain entry carries its own headers.

Built-in chains (Go)

chain := rateguard.DefaultProviderChain()
// OpenAI → Anthropic → Google (cost-optimized)

chain = rateguard.BudgetProviderChain()
// Gemini Flash → GPT-4o Mini → Claude Haiku (cheapest first)

chain = rateguard.QualityProviderChain()
// Claude Opus → GPT-4o → Gemini Pro (best quality first)

entry, provider, fallback := chain.Route("openai", CircuitBreakerOpen)
// entry = Anthropic provider, fallback = true

Per-provider circuit breakers

Each provider gets its own breaker (closed → open → half-open) — an OpenAI outage doesn't trip DeepSeek. Agents can check breaker state before calling via the get_circuit_breaker_state MCP tool.