Research CommonsResearch Commons
llm-rotate/Roadmap

Roadmap

What's shipped, what's planned, and what the placeholders mean.

llm-rotate is alpha (0.3.x). The core — unified chat, key rotation, health-aware selection, fallback chains, streaming, and Google multimodal — is implemented and stable for v1, and the surface keeps growing while staying small: new capabilities land as opt-in extras rather than changes to the everyday configure() / chat() API.

Shipped today

  • Unified chat() / chat_stream() / chat_sync() across seven providers (OpenAI, Anthropic, Google AI Studio, Google Vertex, OpenRouter, Azure OpenAI, AWS Bedrock)
  • Google generate_content() for multimodal inputs
  • Automatic key rotation with a health state machine
  • Four selection strategies (health_aware, round_robin, priority, weighted)
  • Cross-provider fallback chains and broker-aware routing for OpenRouter
  • Retry with exponential backoff and jitter
  • Client-side rate limiting (rate_limit_rpm / rate_limit_tpm) and header-driven proactive cooldowns
  • Advisory key leases to spread concurrent load across the pool
  • Redis state store for shared key health across workers ([redis] extra)
  • OpenTelemetry spans + metrics ([otel] extra; no-op otherwise)
  • In-memory event buffer + read-only control server / dashboard ([server] extra)
  • configure_from_file() for JSON / YAML configs ([yaml] extra for YAML)
  • Structured per-call usage logging, health(), and usage_summary()
  • Pluggable secret resolution (env://, file://, literal://)
  • Bundled vendor SDKs; fully typed (py.typed, strict mypy)

Planned

AreaStatusNotes
Prometheus metricsplannedCounters/histograms for calls, rotations, latency (complements the OpenTelemetry hooks).
structlog integrationplannedFirst-class structured logging.
Central registry env loadingdocumentedA LLM_ROTATE_REGISTRY env pattern for enterprise deploys is described in the config architecture but not yet wired into configure_from_dict.
Additional providersexploringMore direct integrations as demand warrants.

Contributing

Contributions are welcome. Install the dev extras (pip install -e ".[dev]") and make sure CI passes locally:

ruff check .
ruff format --check .
mypy src/ --strict
pytest --ignore=tests/e2e --cov=llm_rotate --cov-fail-under=90

Releases are cut by bumping _version.py and tagging vX.Y.Z, which triggers a GitHub Actions publish to PyPI via Trusted Publishing (OIDC). See the repository for full contributor and release docs.