discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens
flashinfer-ai/flashinfer
FlashInfer: Kernel Library for LLM Serving
History and source changes · Reports and owner responses · Compare
Similar
- NVIDIA/TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
Why this match
topics cuda, moe, pytorch; language differs: Cuda → Python
- NVIDIA/cutlass
CUDA Templates and Python DSLs for High-Performance Linear Algebra
Why this match
topics cuda, gpu, nvidia; language differs: Cuda → C++
- NVIDIA/warp
A Python framework for GPU-accelerated simulation, robotics, and machine learning.
Why this match
topics cuda, gpu, nvidia; language differs: Cuda → Python
- uccl-project/uccl
UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)
Why this match
topics cuda, gpu, moe; language differs: Cuda → C++
- NVIDIA/nvidia-docker
Build and run Docker containers leveraging NVIDIA GPUs
Why this match
topics cuda, gpu
- intel/ipex-llm
Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.
Why this match
topics gpu, pytorch; language differs: Cuda → Python
- kuangliu/pytorch-cifar
95.47% on CIFAR10 with PyTorch
Why this match
topics pytorch; language differs: Cuda → Python; license differs: Apache-2.0 → MIT
- jadore801120/attention-is-all-you-need-pytorch
A PyTorch implementation of the Transformer model in "Attention is All You Need".
Why this match
topics attention, pytorch; language differs: Cuda → Python; license differs: Apache-2.0 → MIT