TL;DR — AI Gateway

Design an AI gateway that acts as a unified proxy between applications and multiple LLM providers (OpenAI, Anthropic, Google, self-hosted). The gateway handles model routing, automatic fallbacks, rate limiting, cost tracking, caching, and observability across all AI API calls. Key features: Unified API that proxies to multiple LLM providers. Model routing based on task, cost, latency, or custom rules.