The best vllm alternatives

LlamaFactory

hiyouga/LlamaFactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Choose LlamaFactory if…

• you specifically need agent or ai

Shares deepseek, gpt, llama, llm with vllm.

inference

xorbitsai/inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Choose inference if…

• you prefer a smaller, narrower project to a large one
• you specifically need artificial-intelligence or deployment

Shares inference, kimi, llm, pytorch with vllm.

transformers

huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Choose transformers if…

• you want the more widely starred option (168.5k)
• you specifically need audio or deep-learning

Shares deepseek, llm, pytorch, qwen with vllm.

AstrBot

AstrBotDevs/AstrBot

AI Agent Assistant & development framework that integrates lots of IM platforms, LLMs, plugins and AI feature, and can be your openclaw alternative. ✨

Choose AstrBot if…

• GNU Affero General Public License v3.0 suits your use better than Apache License 2.0
• you prefer a smaller, narrower project to a large one
• you specifically need agent or ai

Shares gpt, llama, llm, openai with vllm.

sglang

sgl-project/sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

Choose sglang if…

• you prefer a smaller, narrower project to a large one
• you specifically need attention or diffusion

Shares blackwell, cuda, deepseek, gpt-oss with vllm.

unsloth

unslothai/unsloth

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.

Choose unsloth if…

• you specifically need agent or ai

Shares deepseek, llama, llm, openai with vllm.

Langchain-Chatchat

chatchat-space/Langchain-Chatchat

Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM, Qwen 与 Llama 等语言模型的 RAG 与 Agent 应用 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM, Qwen and Llama) RAG and Agent app with langchain

Choose Langchain-Chatchat if…

• you prefer a smaller, narrower project to a large one
• you specifically need chatbot or chatchat

Shares gpt, llama, llm, qwen with vllm.

ollama

ollama/ollama

Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

Choose ollama if…

• you would rather work in Go than Python
• MIT License suits your use better than Apache License 2.0
• you want the more widely starred option (185.9k)
• you specifically need gemma or gemma3

Shares deepseek, gpt-oss, llama, llm with vllm.

repomix

yamadashy/repomix

📦 Repomix is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like Claude, ChatGPT, DeepSeek, Perplexity, Gemini, Gemma, Llama, Grok, and more.

Choose repomix if…

• you would rather work in TypeScript than Python
• MIT License suits your use better than Apache License 2.0
• you prefer a smaller, narrower project to a large one
• you specifically need ai or anthropic

Shares deepseek, gpt, llama, llm with vllm.

dyad

dyad-sh/dyad

Local, open-source AI app builder for power users ✨ v0 / Lovable / Replit / Bolt alternative 🌟 Star if you like it!

Choose dyad if…

• you would rather work in TypeScript than Python
• you prefer a smaller, narrower project to a large one
• you specifically need ai-app-builder or anthropic

Shares deepseek, llm, openai, qwen with vllm.

llm-scraper

mishushakov/llm-scraper

Turn any webpage into structured data using LLMs

Choose llm-scraper if…

• you would rather work in TypeScript than Python
• MIT License suits your use better than Apache License 2.0
• you prefer a smaller, narrower project to a large one
• you specifically need ai or artificial-intelligence

Shares gpt, llama, llm, openai with vllm.

codecompanion.nvim

olimorris/codecompanion.nvim

✨ AI Coding, Vim Style

Choose codecompanion.nvim if…

• you would rather work in Lua than Python
• you want something your agent can install directly rather than a library to wire up yourself
• you prefer a smaller, narrower project to a large one
• you specifically need acp or agent

Shares deepseek, kimi, llm, openai with vllm.