Configuration

llm-rotate is configured with a registry — a plain Python dict of keys (and optional provider overrides) — plus the list of keys you want active. The package ships a built-in provider catalog, so most configs only need a keys block.

The two ways to configure#

from llm_rotate import configure
 
# 1. Registry + use_keys (most common)
configure(registry={"keys": [...]}, use_keys=["openai-1"])
 
# 2. A pre-built LMRotateConfig (full control)
from llm_rotate import LMRotateConfig
configure(config=LMRotateConfig(...))

You can also build a config object without touching the singleton:

from llm_rotate import configure_from_dict, LMRotate
 
config = configure_from_dict(registry={"keys": [...]}, use_keys=["openai-1"])
rot = LMRotate(config)

Key entries#

Each entry in registry["keys"] describes one credential. Four fields are required:

Optional key fields#

{
    "key_id": "vertex-prod-1",
    "provider": "google_vertex",
    "secret_ref": "env://VERTEX_SA_JSON",
    "auth_method": "service_account",
    "auth_config": {"project": "my-proj", "region": "us-central1"},
    "models": ["gemini-2.0-flash"],
    "priority": 10,
}

Overriding provider defaults#

Every built-in provider has sensible cooldown/quarantine defaults. To change them, add a providers block with only the entries you want to override — everything else keeps its built-in value.

configure(
    registry={
        "providers": {
            "openai": {
                "provider_type": "direct",
                "display_name": "OpenAI",
                "default_cooldown_seconds": 10,
            }
        },
        "keys": [...],
    },
    use_keys=[...],
)

Defaults#

These are the global defaults (DefaultsConfig), overridable per provider and, for max_retries, per call:

Merge rules#

1. The five built-in providers are always the base layer.
2. A provider entry in your registry's providers block replaces the matching built-in by name; providers you don't mention keep their defaults.
3. keys always come from your dict — the package ships no keys, credentials, or secret_ref values.

State store#

By default, key health, cooldowns, and leases live in an in-process store — zero setup, but not shared across workers. To coordinate rotation across multiple processes, switch to the Redis backend (pip install "llm-rotate[redis]"):

config = configure_from_dict(
    registry={
        "state_store": {
            "backend": "redis",
            "redis_url": "redis://localhost:6379/0",
            "redis_namespace": "llmrotate",
        },
        "keys": [...],
    },
    use_keys=[...],
)

See Advanced for the operational details.

Loading config from a file#

configure_from_file() reads the same shape from JSON or YAML (YAML needs pip install "llm-rotate[yaml]"). use_keys can live in the file or be passed in:

from llm_rotate import configure_from_file
 
config = configure_from_file("llm-rotate.yaml")

# llm-rotate.yaml
use_keys: [openai-1, anthropic-1]
strategy: health_aware
state_store:
  backend: redis
  redis_url: redis://localhost:6379/0
keys:
  - key_id: openai-1
    provider: openai
    secret_ref: env://OPENAI_API_KEY
    models: [gpt-4o-mini]
  - key_id: anthropic-1
    provider: anthropic
    secret_ref: env://ANTHROPIC_API_KEY
    models: [claude-haiku-4-5-20251001]

The built-in provider catalog is merged automatically, so only keys (and use_keys) are strictly required.

Top-level configure() options#

Enterprise pattern#

For multi-app deployments, keep a central registry (in a cloud secret manager) and let each app pick the keys it needs:

import llm_rotate
 
llm_rotate.configure(
    use_keys=["vertex-prod-1", "anthropic-prod-1"],
    priorities={"vertex-prod-1": 10, "anthropic-prod-1": 5},
    strategy="health_aware",
    fallback_chains={"google_vertex": [{"provider": "anthropic"}]},
)

The recommended secret flow is: cloud secret manager → a load_secrets.sh step that exports env vars → env:// resolution at call time. Secrets never live in the registry dict.