discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens
OpenBMB/MiniCPM-V · source
Snapshot 0 · 2026-11-04T12:47:27Z · Python · Apache-2.0
Content fingerprint
eb3ab23c1489710c3863009d8a946bc99bd3570a912c25173e54ace4375c85f2GitHub description at capture
See, hear, and interact on your local devices.
README at capture
<div align="center">
<a href="./docs/minicpm_v4dot7_en.md"><img src="./assets/minicpmv4.7/release-hero.svg" alt="MiniCPM-V 4.7 — A new chapter in on-device multimodal AI. 35B-A3B and 1B models for images, video and documents." width="100%"></a>
**MiniCPM-V & MiniCPM-o · See, hear, and interact on your local devices.**
<p>
<strong><a href="./docs/minicpm_v4dot7_en.md">MiniCPM-V 4.7</a></strong> <a href="https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B" title="MiniCPM-V 4.7 35B-A3B on Hugging Face">🤗</a> 35B-A3B <a href="https://huggingface.co/openbmb/MiniCPM-V-4.7-1B" title="MiniCPM-V 4.7 1B on Hugging Face">🤗</a> 1B <a href="https://huggingface.co/spaces/openbmb/MiniCPM-V-4.7-Demo" title="MiniCPM-V 4.7 Demo">🤖</a>
 
<strong><a href="./docs/minicpm_o4dot5_en.md">MiniCPM-o 4.5</a></strong> <a href="https://huggingface.co/openbmb/MiniCPM-o-4_5" title="MiniCPM-o 4.5 on Hugging Face">🤗</a> <a href="https://minicpmo45.modelbest.cn" title="MiniCPM-o 4.5 Demo">🤖</a>
</p>
<p>
<sub><a href="#model-overview">Overview</a> · <a href="#get-started">Get started</a> · <a href="#project-resources">Resources</a> | <a href="./README_zh.md">中文</a> · English</sub>
</p>
</div>
## News
- [2026.10.28] 🔥🔥🔥 We open-source the [MiniCPM-OCR suite](https://github.com/OpenBMB/MiniCPM-OCR), pairing two models for **fast, accurate parsing and result verification**: **MiniCPM-OCR 1.0** (based on **MiniCPM-V 4.6**, SOTA on Real5-OmniDocBench and OCRBench v2) and **MiniCPM-OCR Jev** for result correctness checks.
- **2026-10-15** · We release **DSpark speculative decoding heads** in **BF16 and GGUF** for both MiniCPM-V 4.7 variants: **1B** ([**DSpark-BF16**](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-DSpark), [**DSpark-GGUF**](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-DSpark-gguf)) and **35B-A3B** ([**DSpark-BF16**](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-DSpark), [**DSpark-GGUF**](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-DSpark-gguf)).
- **2026-10-12 · MiniCPM-V 4.7 released** — the first open-source MoE model to run smoothly on-device without compromising performance, setting a new multimodal performance–efficiency Pareto frontier. [See what's new](./docs/minicpm_v4dot7_en.md).
## MiniCPM-V 4.7
MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in **35B-A3B and 1B variants**. The 35B-A3B variant is the series' first MoE model. Its main features include:
- **🚀 A new performance–efficiency Pareto frontier.** The 35B-A3B and 1B variants each advance the performance–efficiency Pareto frontier for multimodal large language models. In Instruct mode, 35B-A3B **achieves a higher overall score in less inference time than Qwen3.6 35B-A3B**; the 1B variant further improves on MiniCPM-V 4.6's core capabilities and **substantially outperforms Qwen3.5 0.8B in both overall score and inference efficiency**.
- **📱 The first open-source MoE to run smoothly on-device without compromising performance.** Powered by the latest [Lookahead Expert Scheduling (LES)](https://github.com/OpenBMB/llama.cpp-omni/blob/main/docs/les/lookahead-expert-scheduling.md) inference acceleration technique, MiniCPM-V 4.7 35B-A3B becomes the first open-source MoE model to run smoothly on-device without compromising performance, achieving a **decoding speed of 13+ tokens/s with just 3 GB of memory** on iPhone 17 Pro Max. We also support **DSpark speculative decoding**, with dedicated heads available for **1B** ([**DSpark-BF16**](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-DSpark), [**DSpark-GGUF**](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-DSpark-gguf)) and **35B-A3B** ([**DSpark-BF16**](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-DSpark), [**DSpark-GGUF**](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-DSpark-gguf)).
- **🛠️ Broad platform support and easy customization.** MiniCPM-V 4.7 supports **iOS, Android, and HarmonyOS**, and **supports chip platforms from Intel, AMD, Arm, Rockchip, and Houmo**, covering a diverse range of mobile devices and local computing scenarios. We also open-source [Self-Replay Fine-Tuning (SRFT)](https://openbmb.cn/en/blog/self-replay-fine-tuning), helping developers **extend model capabilities at low cost while reducing forgetting of existing capabilities**.
<a name="capability-alongside-inference-cost"></a>
### Significantly Advancing the Performance–Efficiency Pareto Frontier
<p align="center">
<a href="./assets/minicpmv4.7/performance_efficiency_frontier.png"><img src="./assets/minicpmv4.7/performance_efficiency_frontier.png" alt="MiniCPM-V 4.7 35B-A3B and 1B: average evaluation score versus inference time. Stars and circles denote non-reasoning mode; diamonds denote reasoning mode." width="760"></a>
</p>
**Overall Performance (35B-A3B, Instruct)**
**Vision-Language**
<p align="center">
<a href="./assets/minicpmv4.7/overall_performance_35ba3b_instruct.png"><img src="./assets/minicpmv4.7/overall_performance_35ba3b_instruct.png" alt="Vision-Language performance comparison for MiniCPM-V 4.7 35B-A3B in Instruct mode" width="90%"></a>
</p>
**Language**
<p align="center">
<a href="./assets/minicpmv4.7/overall_performance_35ba3b_language.png"><img src="./assets/minicpmv4.7/overall_performance_35ba3b_language.png" alt="Language performance comparison for MiniCPM-V 4.7 35B-A3B in Instruct mode" width="90%"></a>
</p>
**Overall Performance (1B, Instruct)**
<p align="center">
<a href="./assets/minicpmv4.7/overall_performance_1b_instruct.png"><img src="./assets/minicpmv4.7/overall_performance_1b_instruct.png" alt="Full performance comparison for MiniCPM-V 4.7 1B in Instruct mode" width="90%"></a>
</p>
### Smooth On-Device MoE Inference without Compromising Performance
Running an MoE model on a device is only the starting point. **The real challenge is to preserve model capability while delivering smooth generation.** Storing expert weights on SSD can overcome memory capacity limits, but frequent loading stalls slow generation unless loading and computation overlap effectively. Restricting the number of experts or fixing an expert subset can speed up inference at the cost of model capability.
To address this trade-off, MiniCPM-V 4.7 introduces [Lookahead Expert Scheduling](https://github.com/OpenBMB/llama.cpp-omni/blob/main/docs/les/lookahead-expert-scheduling.md), delivering a **5× speedup on real devices without sacrificing model capability**. Cross-layer routing prediction preloads experts likely to be needed by the next layer and prioritizes computation for experts selected by the actual router that are already loaded, substantially reducing loading stalls. Weight quantization and an optimized storage layout also reduce the storage footprint and make scattered reads more efficient, further accelerating loading. The recording below compares generation speed before and after inference optimization:
<a name="see-it-in-action"></a>
<p align="center">
<a href="./assets/minicpmv4.7/moe_speed_comparison_en.gif"><img src="./assets/minicpmv4.7/moe_speed_comparison_en.gif" alt="MiniCPM-V 4.7 35B-A3B on-device inference efficiency: after optimization on the left, before optimization on the right." width="640"></a>
</p>
### Model Weights
MiniCPM-V 4.7 is available in **35B-A3B and 1B** variants. Each row lists the model weights alongside its corresponding **DSpark speculative decoding heads**, available in **BF16 and GGUF**.
| Model | Description | Model weights | DSpark heads |
|:---|:---|:---|:---|
| MiniCPM‑V 4.7<br>**35B-A3B** | The first open-source MoE model to run smoothly on-device without compromising performance, raising the bar for on-device performance. | [🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-35B-A3B) **BF16**<br>[🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-FP8) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-35B-A3B-FP8) **FP8**<br>[🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-int4) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-35B-A3B-int4) **int4**<br>[🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-AWQ) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-35B-A3B-AWQ) **AWQ**<br>[🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-GPTQ) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-35B-A3B-GPTQ) **GPTQ**<br>[🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-35B-A3B-gguf) **GGUF** | [🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-DSpark) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-35B-A3B-DSpark) **BF16**<br>[🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-DSpark-gguf) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-35B-A3B-DSpark-gguf) **GGUF** |
| MiniCPM‑V 4.7<br>**1B** | Combines stronger capabilities with extremely high efficiency, advancing the performance–efficiency Pareto frontier for multimodal large language models. | [🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-1B) **BF16**<br>[🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-int4) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-1B-int4) **int4**<br>[🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-AWQ) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-1B-AWQ) **AWQ**<br>[🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-GPTQ) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-1B-GPTQ) **GPTQ**<br>[🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-gguf) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-1B-gguf) **GGUF** | [🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-DSpark) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-1B-DSpark) **BF16**<br>[🤗](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-DSpark-gguf) [<img src="./assets/modelscope_logo.png" width="20px"></img>](https://modelscope.cn/models/OpenBMB/MiniCPM-V-4.7-1B-DSpark-gguf) **GGUF** |
The 1B int4 variant uses bitsandbytes; AWQ and GPTQ use W4A16 quantization. Quantized variants reduce memory usage; GGUF weights support CPU inference.
Also available on [Ollama](https://ollama.com/openbmb/minicpm-v4.7).
## MiniCPM-o 4.5
MiniCPM-o 4.5 is the latest model in the MiniCPM-o series, a **9B end-to-end omnimodal model**. It can continuously receive video and audio while generating speech and text, enabling real-time interactions that see, hear, and speak simultaneously.
**Model highlights:**
- **🔥 Vision capability approaching Gemini 2.5 Flash.** It scores 77.6 on average across 8 popular OpenCompass benchmarks. **With only 9B parameters, its visual understanding surpasses GPT-4o and Gemini 2.0 Pro**, and a single model supports both instruct and thinking modes.
- **🎙 Strong speech capability.** It supports bilingual real-time speech conversation with configurable voices in English and Chinese, making conversations **more natural, expressive, and stable**. A simple reference audio clip is enough for **voice cloning and role play**, and its voice cloning performance surpasses strong TTS tools such as CosyVoice2.
- **🎬 Full-duplex, proactive live interaction.** It process real-time, continuous video and audio input streams simultaneously while generating concurrent text and speech output streams in an end-to-end fashion, without mutual blocking. This **allows MiniCPM-o 4.5 to see, hear, and speak simultaneously**, enabling smooth, real-time omnimodal conversations. Beyond responding to requests, the model can also interact proactively, for example by initiating reminders or comments based on its continuous understanding of the scene.
### News
- **2026-05-17** · Released [API access](./docs/api.md) for MiniCPM-o 4.5.
<p align="center">
<a href="https://www.youtube.com/watch?v=6UzC-O1Q-1U"><img src="./assets/minicpmo4_5/video_play.png" alt="Watch the MiniCPM-o 4.5 full-duplex multimodal interaction demo on YouTube." width="420"></a>
</p>
<!-- [Live demo](https://minicpmo45.modelbest.cn) · [Web Demo Deployment](./docs/minicpm_o4dot5_en.md#web-demo-deployment) · [Examples](./docs/minicpm_o4dot5_en.md#examples) · [Limitations](./docs/minicpm_o4dot5_en.md#limitations) · [MiniCPM-o 4.5 evaluation](./docs/minicpm_o4dot5_en.md#evaluation) -->
<a name="model-zoo"></a>
<a name="choose-your-model"></a>
## Model overview
<table>
<thead>
<tr>
<th align="left" width="180" nowrap="nowrap"></th>
<th align="left"><b>MiniCPM-V 4.7</b></th>
<th align="left"><b>MiniCPM-o 4.5</b></th>
</tr>
</thead>
<tbody>
<tr>
<td nowrap="nowrap"><b>Models</b></td>
<td>35B-A3B MoE · 1B</td>
<td>9B end-to-end omnimodal model</td>
</tr>
<tr>
<td nowrap="nowrap"><b>Core capabilities</b></td>
<td>Image, multi-image, document and video understanding</td>
<td>Visual understanding, speech understanding and generation</td>
</tr>
<tr>
<td nowrap="nowrap"><b>Technical features</b></td>
<td>Mixed 4×/16× visual token compression; smooth on-device inference without compromising performance</td>
<td>Full-duplex multimodal live streaming, receiving audio and video while generating speech and text</td>
</tr>
<tr>
<td nowrap="nowrap"><b>Model guides</b></td>
<td><a href="./docs/minicpm_v4dot7_en.md">MiniCPM-V 4.7 model guide</a></td>
<td><a href="./docs/minicpm_o4dot5_en.md">MiniCPM-o 4.5 model guide</a></td>
</tr>
<tr>
<td nowrap="nowrap"><b>Examples</b></td>
<td><a href="./docs/minicpm_v4dot7_en.md#examples">Examples</a></td>
<td><a href="./docs/minicpm_o4dot5_en.md#examples">Examples</a></td>
</tr>
<tr>
<td nowrap="nowrap"><b>Evaluation</b></td>
<td><a href="./docs/minicpm_v4dot7_en.md#evaluation">Evaluation</a></td>
<td><a href="./docs/minicpm_o4dot5_en.md#evaluation">Evaluation</a></td>
</tr>
</tbody>
</table>
## Get started
<table>
<thead>
<tr>
<th align="left"><b>MiniCPM-V 4.7</b></th>
<th align="left"><b>MiniCPM-o 4.5</b></th>
</tr>
</thead>
<tbody>
<tr>
<td align="center"><a href="#model-weights">Model weights</a></td>
<td align="center"><a href="./docs/minicpm_o4dot5_en.md#model-weights">Model weights</a></td>
</tr>
<tr>
<td><a href="./docs/minicpm_v4dot7_en.md#inference-with-transformers">HF</a> · <a href="https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/deployment/vllm/minicpm-v4_7_vllm.md">vLLM</a> · <a href="https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/deployment/sglang/minicpm-v4_7_sglang.md">SGLang</a> · <a href="https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/deployment/ollama/minicpm-v4_7_ollama.md">Ollama</a> · <a href="https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/deployment/llama.cpp/minicpm-v4_7_llamacpp.md">llama.cpp</a></td>
<td><a href="./docs/minicpm_o4dot5_en.md#inference-using-hugging-face-transformers">HF</a> · <a href="https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/deployment/vllm/minicpm-o4_5_vllm.md">vLLM</a> · <a href="https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/deployment/sglang/minicpm-o4_5_sglang.md">SGLang</a> · <a href="https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/deployment/ollama/minicpm-o4_5_ollama.md">Ollama</a> · <a href="https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/deployment/llama.cpp/minicpm-o4_5_llamacpp.md">llama.cpp</a></td>
</tr>
<tr>
<td align="center"><a href="./docs/deployment_en.md#training-llama-factory-swift">Fine-tuning</a></td>
<td align="center"><a href="./docs/minicpm_o4dot5_en.md#use-minicpm-o-45-in-other-inference-and-training-frameworks">Fine-tuning</a></td>
</tr>
<tr>
<td align="center"><a href="https://huggingface.co/spaces/openbmb/MiniCPM-V-4.7-Demo">Live demo</a></td>
<td align="center"><a href="https://minicpmo45.modelbest.cn">Live demo</a></td>
</tr>
<tr>
<td align="center"><a href="./docs/minicpm_v4dot7_en.md#on-device-deployment">Local deployment</a></td>
<td align="center"><a href="./docs/minicpm_o4dot5_en.md#web-demo-deployment">Local deployment</a></td>
</tr>
<tr>
<td colspan="2" align="center"><a href="./docs/api.md">API</a></td>
</tr>
<tr>
<td colspan="2" align="center"><a href="https://github.com/OpenSQZ/MiniCPM-V-CookBook">Cookbook</a></td>
</tr>
</tbody>
</table>
## Earlier updates
- **2026-06-25** · MiniCPM-V 4.6 joined the [official Ollama model library](https://ollama.com/library/minicpm-v4.6).
- **2026-05-17** · Released [API access](./docs/api.md) for MiniCPM-V 4.6.
- **2026-05-11** · Released [MiniCPM-V 4.6](./docs/minicpm_v4dot6_en.md), with mixed 4×/16× visual token compression and mobile deployment support.
### Community milestones
- **2025-01-19** · MiniCPM-o reached No. 1 on GitHub Trending and No. 2 on Hugging Face Trending.
- **2024-05-23** · MiniCPM-V reached No. 1 on both GitHub Trending and Hugging Face Trending.
## Built with MiniCPM
| Project | What you can explore |
|:---|:---|
| [OpenAvatarChat](https://github.com/HumanAIGC-Engineering/OpenAvatarChat) | Interactive digital-human conversations on a single PC |
| [Ollama-OCR](https://github.com/imanoop7/Ollama-OCR) | Extract text from images and PDFs with local vision-language models |
| [comfyui_LLM_party](https://github.com/heshengtao/comfyui_LLM_party) | Connect language and vision models to ComfyUI workflows |
Community projects may support different model generations. Check each project's model list. [More community projects](./docs/project_en.md#awesome-work-using-minicpm-v--o)
<a name="docs--research"></a>
## Project resources
| Category | Links |
|:---|:---|
| **Model documentation** | [MiniCPM-V 4.7](./docs/minicpm_v4dot7_en.md) · [MiniCPM-o 4.5](./docs/minicpm_o4dot5_en.md) |
| **Deployment & development** | [Deployment & training](./docs/deployment_en.md) · [Cookbook](https://github.com/OpenSQZ/MiniCPM-V-CookBook) · [Documentation site](https://minicpm-o.readthedocs.io/en/latest/index.html) |
| **Papers** | [Technical reports & related papers](./docs/project_en.md#technical-reports-and-key-techniques-papers) |
| **Versions & updates** | [Earlier models](./docs/project_en.md#legacy-models) · [Full release history](./docs/changelog_en.md) |
| **Community** | [Discord](https://discord.com/invite/7q3ry8Ny8K) · [WeChat](./docs/wechat.md) · [Feishu / Lark](./assets/feishu_qrcode.png) · [MiniCPM Wiki](https://modelbest.feishu.cn/wiki/UtWxwcERfiRIpIkBOjuc3h9tn1D?fromScene=spaceOverview) |
| **License & usage** | [Apache-2.0](./LICENSE) · [Model license](./docs/project_en.md#model-license) · [Usage statement](./docs/project_en.md#statement) |
Developed by [THUNLP](https://nlp.csai.tsinghua.edu.cn/) and [ModelBest](https://modelbest.cn/). Code and model weights are released under Apache-2.0.
## Citation
If you find our model/code/paper helpful, please consider citing our papers 📝 and staring us ⭐️！
```bibtex
@misc{cui2026minicpmo45realtimefullduplex,
title={MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction},
author={Junbo Cui and Bokai Xu and Chongyi Wang and Tianyu Yu and Weiyue Sun and Yingjing Xu and Tianran Wang and Zhihui He and Wenshuo Ma and Tianchi Cai and others},
year={2026},
url={https://arxiv.org/abs/2604.27393},
}
@proceedings{yu2025minicpmv45cookingefficient,
title={MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe},
author={Tianyu Yu and Zefan Wang and Chongyi Wang and Fuwei Huang and Wenshuo Ma and Zhihui He and Tianchi Cai and Weize Chen and Yuxiang Huang and Yuanqian Zhao and others},
year={2025},
url={https://arxiv.org/abs/2509.18154},
}
@article{yao2024minicpm,
title={MiniCPM-V: A GPT-4V Level MLLM on Your Phone},
author={Yao, Yuan and Yu, Tianyu and Zhang, Ao and Wang, Chongyi and Cui, Junbo and Zhu, Hongji and Cai, Tianchi and Li, Haoyu and Zhao, Weilin and He, Zhihui and others},
journal={arXiv preprint arXiv:2408.01800},
year={2024}
}
```