discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens
NVIDIA/TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
History and source changes · Reports and owner responses · Compare
Similar
- vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Why this match
topics blackwell, cuda, llm-serving
- flashinfer-ai/flashinfer
FlashInfer: Kernel Library for LLM Serving
Why this match
topics cuda, moe, pytorch; language differs: Python → Cuda
- kuangliu/pytorch-cifar
95.47% on CIFAR10 with PyTorch
Why this match
topics pytorch
- sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
Why this match
topics blackwell, cuda, moe
- vosen/ZLUDA
CUDA on non-NVIDIA GPUs
Why this match
topics cuda; language differs: Python → Rust
- replicate/cog
Containers for machine learning
Why this match
topics cuda, pytorch; language differs: Python → Go
- srush/GPU-Puzzles
Solve puzzles. Learn CUDA.
Why this match
topics cuda; language differs: Python → Jupyter Notebook
- liguodongiot/llm-action
本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
Why this match
topics llm-serving; language differs: Python → HTML