The best litellm alternatives

headroom

headroomlabs-ai/headroom

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

Choose headroom if…

• you need a detected license — this one reports Apache License 2.0
• you want something your agent can install directly rather than a library to wire up yourself
• you specifically need agent or ai

Shares anthropic, langchain, llm, openai with litellm.

langchain

langchain-ai/langchain

The agent engineering platform.

Choose langchain if…

• you need a detected license — this one reports MIT License
• you want the more widely starred option (148.9k)
• you specifically need agents or ai

Shares anthropic, langchain, llm, openai with litellm.

harness-sdk

strands-agents/harness-sdk

Build an agent harness and control it end-to-end. Open-source SDK for production AI agents in Python & TypeScript - any model, any cloud.

Choose harness-sdk if…

• you need a detected license — this one reports Apache License 2.0
• you prefer a smaller, narrower project to a large one
• you specifically need agent-framework or agentic

Shares anthropic, bedrock, llm, openai with litellm.

langfuse

langfuse/langfuse

🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

Choose langfuse if…

• you would rather work in TypeScript than Python
• you specifically need analytics or autogen

Shares langchain, llm, llmops, openai with litellm.

gateway

Portkey-AI/gateway

A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

Choose gateway if…

• you would rather work in TypeScript than Python
• you need a detected license — this one reports MIT License
• you want something your agent can install directly rather than a library to wire up yourself
• you prefer a smaller, narrower project to a large one

Shares ai-gateway, gateway, langchain, llm with litellm.

langgraph

langchain-ai/langgraph

Build resilient agents.

Choose langgraph if…

• you need a detected license — this one reports MIT License
• you specifically need agents or ai

Shares langchain, llm, openai with litellm.

kong

Kong/kong

🦍 The API and AI Gateway

Choose kong if…

• you would rather work in Lua than Python
• you need a detected license — this one reports Apache License 2.0
• you specifically need ai or api-gateway

Shares ai-gateway, llm-gateway, mcp-gateway, openai-proxy with litellm.

plano

katanemo/plano

Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.

Choose plano if…

• you would rather work in Rust than Python
• you need a detected license — this one reports Apache License 2.0
• you prefer a smaller, narrower project to a large one
• you specifically need ai-gateway-support or envoy

Shares ai-gateway, gateway, llm-gateway, llmops with litellm.

SuperAGI

TransformerOptimus/SuperAGI

<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

Choose SuperAGI if…

• you need a detected license — this one reports MIT License
• you prefer a smaller, narrower project to a large one
• you specifically need agents or agi

Shares llm, llmops, openai with litellm.

deer-flow

bytedance/deer-flow

An open-source long-horizon SuperAgent harness that researches, codes, and creates. With the help of sandboxes, memories, tools, skill, subagents and message gateway, it handles different levels of tasks that could take minutes to hours.

Choose deer-flow if…

• you need a detected license — this one reports MIT License
• you specifically need agent or agentic

Shares langchain, llm with litellm.

OmniRoute

diegosouzapw/OmniRoute

Never stop coding. Free MIT AI gateway: one endpoint, 352 providers (150+ free), 1200+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A, Desktop/PWA. Built by 550+ contributors

Choose OmniRoute if…

• you would rather work in TypeScript than Python
• you need a detected license — this one reports MIT License
• you specifically need a2a or ai-agents

Shares ai-gateway, anthropic, llm-gateway, openai with litellm.

oh-my-pi

can1357/oh-my-pi

⌥ Coding agent with the IDE wired in

Choose oh-my-pi if…

• you would rather work in TypeScript than Python
• you need a detected license — this one reports MIT License
• you specifically need ai-agent or ai-coding-agent

Shares anthropic, llm, openai, rust with litellm.