AI Gateway 2026: LiteLLM vs Portkey vs Cloudflare

Managing multiple LLM providers without an AI gateway for LLMs in 2026 is like running a highway without traffic lights. Enterprise LLM adoption has crossed 80%, and teams juggling OpenAI, Anthropic, Google Gemini, and open-source models need a unified layer for routing, rate limiting, caching, and observability. That’s exactly what AI gateways deliver.

In this guide, we compare the three most popular options: LiteLLM, Portkey, and Cloudflare AI Gateway. We’ll break down their features, pricing, deployment models, and ideal use cases so you can pick the right tool for your stack.

What Is an AI Gateway for LLMs?

An AI gateway is a proxy layer that sits between your application and one or more LLM providers. Instead of hardcoding API keys and provider-specific SDKs throughout your codebase, your app sends requests to the gateway, which handles routing, retries, caching, cost tracking, and guardrails.

Key capabilities of a modern AI gateway include:

• Multi-provider routing — send requests to OpenAI, Anthropic, Cohere, Gemini, or local models from a single endpoint
• Fallback and load balancing — automatically retry on failure or distribute load across providers
• Semantic caching — cache similar prompts to reduce costs and latency
• Rate limiting and budgets — enforce per-user or per-team token budgets
• Observability — log requests, track costs, and monitor latency across all providers
• Guardrails — filter harmful inputs/outputs before they reach your app

The LLM middleware gateway market is projected to grow at a 49.6% CAGR through 2034, with roughly 42% of enterprises already using a middleware layer to manage AI infrastructure. Let’s look at the top three contenders.

LiteLLM: The Open-Source Standard

LiteLLM is the go-to open-source AI gateway for developers who want full control. It provides both a Python SDK and a self-hosted proxy server that exposes an OpenAI-compatible interface across 100+ LLM providers.

Key Features

• 100+ provider support — OpenAI, Anthropic, Azure, Bedrock, Gemini, Cohere, Mistral, Ollama, and more
• Virtual keys — issue per-team or per-project API keys mapped to real provider credentials
• Budget and rate limits — enforce RPM (requests per minute) and TPM (tokens per minute) limits per key
• Load balancing — round-robin or weighted routing across multiple deployments
• Guardrails — integrations with LlamaGuard, Azure Content Safety, and custom Python hooks
• Logging — integrates with Langfuse, Helicone, Datadog, S3, and more

Pricing

LiteLLM is free and open-source under the MIT license. The hosted LiteLLM Cloud starts at approximately $50/month for teams that don’t want to self-manage the server. Enterprise plans with SSO, dedicated support, and custom SLAs are available on request.

Best For

LiteLLM is ideal for engineering teams at startups and mid-size companies who want to self-host, have diverse provider needs, and are comfortable running a Docker container or Kubernetes deployment.

Portkey: Managed AI Gateway with Enterprise Guardrails

Portkey is a fully managed, cloud-hosted AI gateway built for production-grade deployments. It advertises the widest model catalog — 1,600+ models across 45+ providers — and ships enterprise-grade features out of the box.

Key Features

• 1,600+ models — the broadest provider coverage of any managed gateway
• Semantic caching — cache semantically similar prompts, not just exact matches, saving 20–40% on LLM costs
• 20+ native guardrails — deterministic checks plus LLM-based content filtering and PII detection
• MCP gateway — route Model Context Protocol traffic across your agent workflows
• Prompt management — versioned prompt templates with A/B testing support
• Observability dashboard — real-time cost, latency, and error-rate dashboards per provider

Pricing

Portkey has a generous free tier (10,000 requests/month). Paid plans start at $49/month for 500,000 requests. Enterprise pricing is available for high-volume teams with dedicated support and SLAs.

Best For

Portkey is best for teams that want enterprise-grade guardrails, semantic caching, and a polished UI without the operational overhead of managing infrastructure. It’s a strong fit for AI product teams building customer-facing LLM features.

Cloudflare AI Gateway: Zero-Ops Edge Caching

Cloudflare AI Gateway is a fully managed proxy that runs on Cloudflare’s global edge network. It’s the easiest to get started with — no infrastructure, no containers, just point your API calls at Cloudflare’s endpoint.

Key Features

• Zero infrastructure setup — configured entirely in the Cloudflare dashboard
• 20+ provider support — OpenAI, Anthropic, Google AI Studio, Bedrock, Workers AI, and more
• Exact-match caching — cache identical prompts at the edge for near-zero latency
• Spend limits — set hard caps on token spend per gateway
• Guardrails — evaluate prompts and responses for content categories with optional DLP
• Unified billing — new in 2026: pay for OpenAI, Anthropic, and Google AI through a single Cloudflare invoice
• Workers AI integration — seamlessly route to Cloudflare’s own serverless GPU inference

Pricing

Cloudflare AI Gateway is free for all Cloudflare users (including the Free plan). Usage-based charges only apply when using Workers AI for inference. The unified billing feature is available on Pro plans and above.

Best For

Cloudflare AI Gateway is perfect for developers already on Cloudflare’s platform, side projects, and teams that want analytics and caching with minimal setup. It’s the fastest path from zero to a production gateway.

LiteLLM vs Portkey vs Cloudflare: Side-by-Side Comparison

How to Choose the Right AI Gateway for LLMs

Here’s a simple decision framework:

• Choose LiteLLM if you need open-source, self-hosted control, have 5+ LLM providers to manage, and your team can handle a Docker or Kubernetes deployment.
• Choose Portkey if you’re building production AI products, need semantic caching and guardrails, and want a managed service with a great developer experience.
• Choose Cloudflare AI Gateway if you’re already on Cloudflare, want zero-infrastructure setup, or are in the early exploration phase and need basic analytics and caching fast.

Many teams use a combination: Cloudflare AI Gateway for edge caching and cost caps at the perimeter, with LiteLLM or Portkey handling routing and guardrails at the application layer.

Frequently Asked Questions

What is the best open-source AI gateway for LLMs?

LiteLLM is the leading open-source AI gateway for LLMs in 2026. It supports 100+ providers via an OpenAI-compatible interface and offers virtual keys, budget controls, load balancing, and logging integrations — all fully self-hostable under the MIT license.

Does Cloudflare AI Gateway support Anthropic Claude?

Yes. Cloudflare AI Gateway supports Anthropic Claude (all current models), OpenAI GPT-4o, Google Gemini, Bedrock, and 20+ other providers. In 2026, Cloudflare added unified billing so you can pay for Anthropic usage through your Cloudflare invoice.

Is Portkey better than LiteLLM?

Neither is strictly “better” — they target different needs. Portkey offers a polished managed experience with 1,600+ models, semantic caching, and 20+ native guardrails, while LiteLLM gives you full control via self-hosting and is completely free. Teams prioritizing operational simplicity and guardrails tend to prefer Portkey; teams prioritizing open-source control and provider breadth tend to prefer LiteLLM.

Can I use an AI gateway with local models like Ollama?

Yes — LiteLLM has first-class support for local models including Ollama, LM Studio, and vLLM. Portkey also supports self-hosted and local model endpoints. Cloudflare AI Gateway is designed primarily for cloud providers and does not natively proxy requests to local inference servers.

Conclusion

In 2026, an AI gateway for LLMs is no longer optional for teams running LLMs in production — it’s essential infrastructure. LiteLLM wins for open-source control and provider breadth, Portkey wins for managed production features and guardrails, and Cloudflare AI Gateway wins for zero-ops simplicity at the edge.

Start with Cloudflare AI Gateway if you want to be up and running in 10 minutes. Graduate to LiteLLM or Portkey when your team needs more sophisticated routing, semantic caching, or enterprise guardrails. Whichever you choose, having a gateway in place will cut your LLM costs, improve reliability, and give you the observability your team needs to ship with confidence.

Have questions about which AI gateway fits your stack? Drop a comment below or explore our related guides on LLM tools and comparisons.