discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens
sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
History and source changes · Reports and owner responses · Compare
Similar
- vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Why this match
topics blackwell, cuda, deepseek
- FlashML-org/FreeToken
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
Why this match
topics glm, inference, minimax
- ollama/ollama
Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
Why this match
topics deepseek, glm, gpt-oss; language differs: Python → Go; license differs: Apache-2.0 → MIT
- NVIDIA/TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
Why this match
topics blackwell, cuda, moe
- oumi-ai/oumi
Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
Why this match
topics gpt-oss, inference, llama
- huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Why this match
topics deepseek, glm, llm
- GeeeekExplorer/nano-vllm
Nano vLLM
Why this match
topics inference, llm, transformer; license differs: Apache-2.0 → MIT
- cocktailpeanut/dalai
The simplest way to run LLaMA on your local machine
Why this match
topics llama, llm; language differs: Python → CSS