The best shimmy alternatives

inference

xorbitsai/inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Choose inference if…

• you would rather work in Python than Rust
• you want the more widely starred option (9.6k)
• you specifically need artificial-intelligence or deployment

Shares llamacpp, machine-learning, transformers with shimmy.

Soup

MakazhanAlpamys/Soup

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

Choose Soup if…

• you would rather work in Python than Rust
• you want the more widely starred option (10.1k)
• you specifically need cli or consumer-gpu

Shares gguf, huggingface, local-ai, machine-learning with shimmy.

LlamaFactory

hiyouga/LlamaFactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Choose LlamaFactory if…

• you would rather work in Python than Rust
• you want the more widely starred option (75.9k)
• you specifically need agent or ai

Shares llama, transformers with shimmy.

rtk

rtk-ai/rtk

CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies

Choose rtk if…

• you want the more widely starred option (85.4k)
• you specifically need agentic-coding or ai-coding

Shares command-line-tool, developer-tools, rust with shimmy.

screenpipe

screenpipe/screenpipe

YC (S26) | Open Computer History | Record your screen continuously locally and provide context to your agents (Claude, Codex, Openclaw, Hermes, Runner...)

Choose screenpipe if…

• you want the more widely starred option (22.2k)
• you specifically need agents or agi

Shares local-ai, machine-learning with shimmy.

nautilus_trader

nautechsystems/nautilus_trader

Production-grade Rust-native trading engine with deterministic event-driven architecture

Choose nautilus_trader if…

• GNU Lesser General Public License v3.0 suits your use better than Apache License 2.0
• you want the more widely starred option (31.2k)
• you specifically need algorithmic-trading-engine or artificial-intelligence

Shares machine-learning, rust with shimmy.

turbo-fieldfare

drumih/turbo-fieldfare

Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

Choose turbo-fieldfare if…

• you would rather work in Swift than Rust
• you specifically need apple-silicon or gemma

Shares llm-inference, local-ai with shimmy.

claude-code-local

nicedreamzapp/claude-code-local

Run Claude Code 100% on-device with local AI on Apple Silicon. MLX-native Anthropic-API server. 6 fighters incl. Muse-Glimmer 30B (now multimodal — reads images, abliterated), Gemma 4 31B, Qwen 3.5 122B (65 tok/s), DeepSeek V4 Flash (1M ctx). Private, offline, airgap-ready. Built for NDA / legal / healthcare workflows.

Choose claude-code-local if…

• you would rather work in Python than Rust
• MIT License suits your use better than Apache License 2.0
• you specifically need abliterated or ai-privacy

Shares llama, local-ai with shimmy.

vllm

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Choose vllm if…

• you would rather work in Python than Rust
• you want the more widely starred option (96.2k)
• you specifically need amd or blackwell

Shares llama with shimmy.

meetily

Zackriya-Solutions/meetily

Privacy first, AI meeting assistant with 4x faster Parakeet/Whisper live transcription, speaker diarization, and Ollama summarization built on Rust. 100% local processing. no cloud required. Meetily (Meetly Ai - https://meetily.ai) is the #1 Self-hosted, Open-source Ai meeting note taker for macOS & Windows. Understand How to write meeting minutes

Choose meetily if…

• MIT License suits your use better than Apache License 2.0
• you want the more widely starred option (32.4k)
• you specifically need ai or ai-meeting-assistant

Shares local-ai, rust with shimmy.

repomix

yamadashy/repomix

📦 Repomix is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like Claude, ChatGPT, DeepSeek, Perplexity, Gemini, Gemma, Llama, Grok, and more.

Choose repomix if…

• you would rather work in TypeScript than Rust
• MIT License suits your use better than Apache License 2.0
• you want the more widely starred option (29.1k)
• you specifically need ai or anthropic

Shares developer-tools, llama with shimmy.

lemonade

lemonade-sdk/lemonade

Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

Choose lemonade if…

• you would rather work in C++ than Rust
• you want something your agent can install directly rather than a library to wire up yourself
• you specifically need ai or amd

Shares llama, llm-inference with shimmy.