Skip to content

The Great Hall

Every work in the library, top first

Free to explore · curated by OneWave AI

1,407 apps, agents and plugins from GitHub, ranked on how actively they are maintained rather than on stars. Security review is in progress and every project shows exactly what has and has not been checked.

tech·an·dri·a/tɛkˈændriə/n.
A library for the tools of a new age. After the Library of Alexandria, which copied every scroll that reached its harbour before it went on the shelf.
AI searches are sent to OpenAI.

Review is in progress. 1,407 projects are queued and none has been approved yet. Everything unreviewed is discovery metadata imported from GitHub — listed, not vetted.

Request a review · What we check

Ranked by total stars at the last daily refresh. This is not stars earned during the selected period. Star counts in the AI category are known to be inflated by fake-star campaigns, so activity and license are shown first.

38 projects matching “inference” · page 1 of 2
huggingface organisation mark

transformers

huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Pushed today.Apache License 2.0PythonDeveloper tools168.9k+351
vllm-project organisation mark

vllm

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Pushed today.Apache License 2.0PythonDeveloper tools96.8k+444
zylon-ai organisation mark

private-gpt

zylon-ai/private-gpt

Complete API layer for private AI applications on local models: RAG, skills, tools, MCP, text-to-sql, and more. Works with any OpenAI-compatible inference server.

Pushed yesterday.Apache License 2.0PythonDeveloper tools57.6k+7
ray-project organisation mark

ray

ray-project/ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

Pushed today.Apache License 2.0PythonDeveloper tools44.3k+64
lyogavin organisation mark

airllm

lyogavin/airllm

AirLLM 70B inference with single 4GB GPU

Pushed yesterday.Apache License 2.0Jupyter NotebookDeveloper tools42.3k+481
hpcaitech organisation mark

ColossalAI

hpcaitech/ColossalAI

Making large AI models cheaper, faster and more accessible

Pushed yesterday.Apache License 2.0PythonApps41.4k
2noise organisation mark

ChatTTS

2noise/ChatTTSSlowing

A generative speech model for daily dialogue.

Last push 7 months ago.GNU Affero General Public License v3.0PythonApps40.0k+16
sgl-project organisation mark

sglang

sgl-project/sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

Pushed today.Apache License 2.0PythonDeveloper tools38.8k+301
chatchat-space organisation mark

Langchain-Chatchat

chatchat-space/Langchain-ChatchatDormant

Langchain-Chatchat(原Langchain-ChatGLM)基于 Langchain 与 ChatGLM, Qwen 与 Llama 等语言模型的 RAG 与 Agent 应用 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM, Qwen and Llama) RAG and Agent app with langchain

No push in over a year ago.Apache License 2.0PythonApps38.7k
gitleaks organisation mark

gitleaks

gitleaks/gitleaks

Find secrets with Gitleaks 🔑

Pushed 16 days ago.MIT LicenseGoApps31.0k+270
mozilla-ai organisation mark

llamafile

mozilla-ai/llamafile

Distribute and run LLMs with a single file.

Pushed 4 days ago.OtherC++Apps26.6k+57
Tencent organisation mark

ncnn

Tencent/ncnn

ncnn is a high-performance neural network inference framework optimized for the mobile platform

Pushed 2 days ago.OtherC++Developer tools24.0k+22
jundot organisation mark

omlx

jundot/omlx

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

Pushed today.Apache License 2.0PythonDeveloper tools23.6k
NVIDIA organisation mark

NemoClaw

NVIDIA/NemoClaw

Run agents like Hermes, LangChain Deep Agents, and OpenClaw more securely inside NVIDIA OpenShell with managed inference

Pushed today.Apache License 2.0TypeScriptDeveloper tools23.0k
modelscope organisation mark

FunASR

modelscope/FunASR

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

Pushed today.MIT LicensePythonPlugins21.0k+60
mlc-ai organisation mark

web-llm

mlc-ai/web-llm

High-performance In-browser LLM Inference Engine

Pushed 3 days ago.Apache License 2.0TypeScriptDeveloper tools19.4k
alibaba organisation mark

MNN

alibaba/MNN

MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.

Pushed yesterday.Apache License 2.0C++Developer tools16.5k
Lightning-AI organisation mark

litgpt

Lightning-AI/litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Pushed 2 days ago.Apache License 2.0PythonApps13.7k
openvinotoolkit organisation mark

openvino

openvinotoolkit/openvino

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Pushed today.Apache License 2.0C++Developer tools11.2k+37
Lightricks organisation mark

LTX-2

Lightricks/LTX-2

Official Python inference and LoRA trainer package for the LTX-2 audio–video generative model.

Pushed yesterday.OtherPythonDeveloper tools10.2k+115
xorbitsai organisation mark

inference

xorbitsai/inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Pushed today.Apache License 2.0PythonDeveloper tools9.7k+10
bentoml organisation mark

BentoML

bentoml/BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Pushed 10 days ago.Apache License 2.0PythonDeveloper tools8.9k+5
flyteorg organisation mark

flyte

flyteorg/flyte

Type-safe, distributed orchestration of agents, ML pipelines, and real-time inference on your k8s — in pure Python with async/await, also other languages (rust, go and ts)

Pushed today.Apache License 2.0GoDeveloper tools7.8k
Osmantic organisation mark

ODS

Osmantic/ODS

Turn your PC, Mac, or Linux box into an AI server. LLM inference, chat UI, voice, agents, workflows, RAG, and image generation.

Pushed today.Apache License 2.0PythonDeveloper tools7.8k+51
The Great Hall: top open-source AI tools — Techandria