The best llamafile alternatives

Soup

MakazhanAlpamys/Soup

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

Choose Soup if…

• you would rather work in Python than C++
• you need a detected license — this one reports Apache License 2.0
• you prefer a smaller, narrower project to a large one
• you specifically need cli or consumer-gpu

Shares gguf, local-ai, local-llm with llamafile.

ODS

Osmantic/ODS

Turn your PC, Mac, or Linux box into an AI server. LLM inference, chat UI, voice, agents, workflows, RAG, and image generation.

Choose ODS if…

• you would rather work in Python than C++
• you need a detected license — this one reports Apache License 2.0
• you prefer a smaller, narrower project to a large one
• you specifically need ai-agents or amd

Shares llama-cpp, local-ai, speech-to-text with llamafile.

OGAM

off-grid-ai/OGAM

The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.

Choose OGAM if…

• you would rather work in TypeScript than C++
• you need a detected license — this one reports MIT License
• you prefer a smaller, narrower project to a large one
• you specifically need android or edge-ai

Shares gguf, llama-cpp, local-ai with llamafile.

meetily

Zackriya-Solutions/meetily

Privacy first, AI meeting assistant with 4x faster Parakeet/Whisper live transcription, speaker diarization, and Ollama summarization built on Rust. 100% local processing. no cloud required. Meetily (Meetly Ai - https://meetily.ai) is the #1 Self-hosted, Open-source Ai meeting note taker for macOS & Windows. Understand How to write meeting minutes

Choose meetily if…

• you would rather work in Rust than C++
• you need a detected license — this one reports MIT License
• you specifically need ai or ai-meeting-assistant

Shares local-ai, speech-to-text with llamafile.

foundry-local

microsoft/foundry-local

No description provided by the repository owner.

Choose foundry-local if…

• you prefer a smaller, narrower project to a large one
• you specifically need ai-sdk or chat-completions

Shares local-ai, speech-to-text with llamafile.

screenpipe

screenpipe/screenpipe

YC (S26) | Open Computer History | Record your screen continuously locally and provide context to your agents (Claude, Codex, Openclaw, Hermes, Runner...)

Choose screenpipe if…

• you would rather work in Rust than C++
• you specifically need agents or agi

Shares local-ai, speech-to-text with llamafile.

shimmy

Michael-A-Kuykendall/shimmy

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

Choose shimmy if…

• you would rather work in Rust than C++
• you need a detected license — this one reports Apache License 2.0
• you prefer a smaller, narrower project to a large one
• you specifically need api-server or command-line-tool

Shares gguf, local-ai with llamafile.

claude-code-local

nicedreamzapp/claude-code-local

Run Claude Code 100% on-device with local AI on Apple Silicon. MLX-native Anthropic-API server. 6 fighters incl. Muse-Glimmer 30B (now multimodal — reads images, abliterated), Gemma 4 31B, Qwen 3.5 122B (65 tok/s), DeepSeek V4 Flash (1M ctx). Private, offline, airgap-ready. Built for NDA / legal / healthcare workflows.

Choose claude-code-local if…

• you would rather work in Python than C++
• you need a detected license — this one reports MIT License
• you prefer a smaller, narrower project to a large one
• you specifically need abliterated or ai-privacy

Shares local-ai, local-llm with llamafile.

DeepCamera

SharpAI/DeepCamera

Open-Source AI Camera Skills Platform, AI NVR & CCTV Surveillance. Local VLM video analysis with Qwen, DeepSeek, SmolVLM, LLaVA, YOLO26. LLM-powered agentic security camera agent — watches, understands, remembers & guards your home via Telegram, Discord or Slack. Pluggable AI skills. OpenAI, Google, Anthropic or local AI. Runs on Mac Mini & AI PC.

Choose DeepCamera if…

• you would rather work in JavaScript than C++
• you need a detected license — this one reports MIT License
• you prefer a smaller, narrower project to a large one
• you specifically need ai or ai-camera

Shares llama-cpp, local-ai with llamafile.

lucebox

Luce-Org/lucebox

LLM speculative inference server for heterogeneous hardware & consumer GPUs

Choose lucebox if…

• you need a detected license — this one reports Apache License 2.0
• you prefer a smaller, narrower project to a large one
• you specifically need cuda or cuda-kernels

Shares llama-cpp, local-ai with llamafile.

nanobot

HKUDS/nanobot

Ultra-lightweight, open-source, self-hosted personal AI agent framework in Python with WebUI, tools, memory, MCP, multi-agent workflows, automation, and chat apps

Choose nanobot if…

• you would rather work in Python than C++
• you need a detected license — this one reports MIT License
• you want the more widely starred option (49.8k)
• you specifically need agent-framework or ai-agent

Shares local-llm with llamafile.

llmfit

AlexsJones/llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

Choose llmfit if…

• you would rather work in Rust than C++
• you need a detected license — this one reports MIT License
• you want something your agent can install directly rather than a library to wire up yourself
• you specifically need llm or localai

Shares gguf with llamafile.