Skip to content

Free to explore · curated by OneWave AI

Developer tools

130 developer tools projects. Libraries, CLIs and infrastructure for building with AI.

AI searches are sent to OpenAI.

Review is in progress. 1,004 projects are queued and none has been approved yet. Everything unreviewed is discovery metadata imported from GitHub — listed, not vetted.

Request a review · What we check

Ranked by total stars at the last daily refresh. This is not stars earned during the selected period. Star counts in the AI category are known to be inflated by fake-star campaigns, so activity and license are shown first.

18 projects matching “inference”Page 1 of 1
huggingface organisation mark

transformers

huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Pushed yesterday.Apache License 2.0PythonDeveloper tools168.5k+255
vllm-project organisation mark

vllm

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Pushed yesterday.Apache License 2.0PythonDeveloper tools96.2k+542
zylon-ai organisation mark

private-gpt

zylon-ai/private-gpt

Complete API layer for private AI applications on local models: RAG, skills, tools, MCP, text-to-sql, and more. Works with any OpenAI-compatible inference server.

Pushed yesterday.Apache License 2.0PythonDeveloper tools57.6k+17
ray-project organisation mark

ray

ray-project/ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

Pushed yesterday.Apache License 2.0PythonDeveloper tools44.2k+62
lyogavin organisation mark

airllm

lyogavin/airllm

AirLLM 70B inference with single 4GB GPU

Pushed yesterday.Apache License 2.0Jupyter NotebookDeveloper tools41.7k+735
sgl-project organisation mark

sglang

sgl-project/sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

Pushed yesterday.Apache License 2.0PythonDeveloper tools38.4k+151
Tencent organisation mark

ncnn

Tencent/ncnn

ncnn is a high-performance neural network inference framework optimized for the mobile platform

Pushed yesterday.OtherC++Developer tools24.0k+18
openvinotoolkit organisation mark

openvino

openvinotoolkit/openvino

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Pushed yesterday.Apache License 2.0C++Developer tools11.1k+27
Lightricks organisation mark

LTX-2

Lightricks/LTX-2

Official Python inference and LoRA trainer package for the LTX-2 audio–video generative model.

Pushed 17 days ago.OtherPythonDeveloper tools10.1k+151
xorbitsai organisation mark

inference

xorbitsai/inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Pushed yesterday.Apache License 2.0PythonDeveloper tools9.6k+11
bentoml organisation mark

BentoML

bentoml/BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Pushed 2 days ago.Apache License 2.0PythonDeveloper tools8.9k+12
Osmantic organisation mark

ODS

Osmantic/ODS

Turn your PC, Mac, or Linux box into an AI server. LLM inference, chat UI, voice, agents, workflows, RAG, and image generation.

Pushed yesterday.Apache License 2.0PythonDeveloper tools7.8k+53
drumih organisation mark

turbo-fieldfare

drumih/turbo-fieldfare

Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

Pushed 9 days ago.Apache License 2.0SwiftDeveloper tools7.1k+58
katanemo organisation mark

plano

katanemo/plano

Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.

Pushed 11 days ago.Apache License 2.0RustDeveloper tools7.1k+8
Michael-A-Kuykendall organisation mark

shimmy

Michael-A-Kuykendall/shimmy

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

Pushed 3 days ago.Apache License 2.0RustDeveloper tools6.0k+12
Blaizzy organisation mark

mlx-vlm

Blaizzy/mlx-vlm

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

Pushed yesterday.MIT LicensePythonDeveloper tools5.8k+92
FedML-AI organisation mark

FedML

FedML-AI/FedMLDormant

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

No push in over a year ago.Apache License 2.0PythonDeveloper tools4.1k
Luce-Org organisation mark

lucebox

Luce-Org/lucebox

LLM speculative inference server for heterogeneous hardware & consumer GPUs

Pushed yesterday.Apache License 2.0C++Developer tools3.0k+13
Developer tools — open-source AI developer tools — Techandria