vllm
vllm-project/vllmA high-throughput and memory-efficient inference and serving engine for LLMs
Pushed today.Apache License 2.0PythonDeveloper tools96.8k+444
The Stacks
Wing of Developer tools
Libraries, CLIs and infrastructure for building with AI.
Free to explore · curated by OneWave AI
215 developer tools projects in this wing of the Stacks. Review status is shown for every entry.
Regular searchNames, topics, and keywords
AI searchDescribe a task, find the right tools
AI searches are sent to OpenAI.
Review is in progress. 1,407 projects are queued and none has been approved yet. Everything unreviewed is discovery metadata imported from GitHub — listed, not vetted.
Ranked by total stars at the last daily refresh. This is not stars earned during the selected period. Star counts in the AI category are known to be inflated by fake-star campaigns, so activity and license are shown first.
4 projects matching “model-serving”
BentoML
bentoml/BentoMLThe easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
Pushed 10 days ago.Apache License 2.0PythonDeveloper tools8.9k+5
flyte
flyteorg/flyteType-safe, distributed orchestration of agents, ML pipelines, and real-time inference on your k8s — in pure Python with async/await, also other languages (rust, go and ts)
Pushed today.Apache License 2.0GoDeveloper tools7.8k
FedML
FedML-AI/FedMLFEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.
Pushed today.Apache License 2.0PythonDeveloper tools4.1k