discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens
xorbitsai/inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
History and source changes · Reports and owner responses · Compare
Similar
- huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Why this match
topics gemma, glm, llm
- intel/ipex-llm
Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.
Why this match
topics llm, pytorch, transformers
- kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Why this match
topics inference, llm, sglang; language differs: Python → C++
- jundot/omlx
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
Why this match
topics llm, mlx, openai-api
- ml-explore/mlx
MLX: An array framework for Apple silicon
Why this match
topics mlx; language differs: Python → C++; license differs: Apache-2.0 → MIT
- ml-explore/mlx-examples
Examples in the MLX framework
Why this match
topics mlx; license differs: Apache-2.0 → MIT
- ollama/ollama
Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
Why this match
topics gemma, glm, llm; language differs: Python → Go; license differs: Apache-2.0 → MIT
- facebookresearch/fairseq
Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
Why this match
topics artificial-intelligence, pytorch; license differs: Apache-2.0 → MIT