The best trnrt alternatives

inference

xorbitsai/inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Choose inference if…

• you want the more widely starred option (9.7k)
• you specifically need artificial-intelligence or deployment

Shares glm, inference, llm, openai-api with trnrt.

vllm

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Choose vllm if…

• you want the more widely starred option (96.8k)
• you specifically need amd or blackwell

Shares inference, llm, llm-serving with trnrt.

sglang

sgl-project/sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

Choose sglang if…

• you want the more widely starred option (38.8k)
• you specifically need attention or blackwell

Shares glm, inference, llm with trnrt.

transformers

huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Choose transformers if…

• you want the more widely starred option (168.9k)
• you specifically need audio or deep-learning

Shares glm, llm with trnrt.

ray

ray-project/ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

Choose ray if…

• you want the more widely starred option (44.3k)
• you specifically need data-science or deep-learning

Shares llm, llm-serving with trnrt.

omlx

jundot/omlx

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

Choose omlx if…

• you want the more widely starred option (23.6k)
• you specifically need apple-silicon or inference-server

Shares llm, openai-api with trnrt.

BentoML

bentoml/BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Choose BentoML if…

• you want the more widely starred option (8.9k)
• you specifically need ai-inference or deep-learning

Shares llm, llm-serving with trnrt.

StartLux-Decision

StartLuxLabs/StartLux-Decision

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.

Choose StartLux-Decision if…

• you specifically need agents or calibration

Shares inference, llm with trnrt.

project-maya

mw00/project-maya

Run GLM-5.3 (744B), GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one dashboard, OpenAI- and Anthropic-compatible API.

Choose project-maya if…

• you would rather work in C++ than Python
• MIT License suits your use better than Apache License 2.0
• you want the more widely starred option (2.8k)
• you specifically need anthropic-api or cuda

Shares glm, llm, openai-api with trnrt.

ms-swift

modelscope/ms-swift

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).

Choose ms-swift if…

• you want the more widely starred option (16.1k)
• you specifically need deepseek or embedding

Shares glm, llm with trnrt.

metaflow

Netflix/metaflow

Build, Manage and Deploy AI/ML Systems

Choose metaflow if…

• you want the more widely starred option (10.3k)
• you specifically need agents or ai

Shares aws, llm with trnrt.

graphify

Graphify-Labs/graphify

Turn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph. A /graphify skill for Claude Code, Cursor, Codex, and Gemini CLI: local deterministic AST parsing, every edge explained, no vector store.

Choose graphify if…

• you want something your agent can install directly rather than a library to wire up yourself
• you want the more widely starred option (146.7k)
• you specifically need ai-agents or antigravity

Shares llm with trnrt.