sglang
sgl-project/sglangSGLang is a high-performance serving framework for large language models and multimodal models.
Pushed yesterday.Apache License 2.0PythonDeveloper tools38.4k+151
vllm
vllm-project/vllm
Not yet reviewedDeveloper tools
A high-throughput and memory-efficient inference and serving engine for LLMs
Topics
Install
Not available through Techandria. A library has no Techandria install path. Running an arbitrary program is a different risk from adding a reviewed skill.
You can still get the code from the source and read it yourself:
git clone https://github.com/vllm-project/vllmWhat we checked
Not yet reviewedDiscovery metadata only. Nothing has been scanned or approved, so this is a listing — not a recommendation to run the code.
Nothing has been scanned for this repository yet, so there is nothing to report. Listing here is not permission or a recommendation to run the code.
Maintain this project? Add the review badge to your README
The badge shows this project's current review state and updates on its own — including if the state changes. It links back to this page.
[](https://techandria.com/project/vllm-project/vllm)README
No README could be fetched from GitHub for this repository.
More in Developer tools
LlamaFactory
hiyouga/LlamaFactoryUnified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
Pushed yesterday.Apache License 2.0PythonDeveloper tools75.9k+126
transformers
huggingface/transformers🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Pushed yesterday.Apache License 2.0PythonDeveloper tools168.5k+255
inference
xorbitsai/inferenceSwap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
Pushed yesterday.Apache License 2.0PythonDeveloper tools9.6k+11