MiniCPM5-2B (LiteRT-LM)

This repository hosts the LiteRT-LM (LiteRT formerly known as TensorFlow Lite) version of MiniCPM5-2B, optimized for fully on-device inference on mobile and edge hardware.

---

Available Models

• minicpm_wi4c_wi8_afp32.litertlm: This model features mixed INT4/INT8 weight-only quantization with FP32 activations (afp32). MLP projections use channelwise INT4 with Hadamard rotation; all remaining weights (attention, embedding, and lmhead) use channelwise INT8.
• minicpm_wi8_afp32.litertlm: This model features weight-only INT8 quantization (wi8) with FP32 activations (afp32).

What is MiniCPM?

MiniCPM5-2B is part of the MiniCPM5 series from OpenBMB. It is a dense 2.5B-parameter Transformer built specifically for on-device, local, and resource-constrained deployment.

Highlights

• 🧠 Hybrid Reasoning — a single checkpoint serves as both a fast assistant and a deliberate reasoner via a built-in <think> template (enable_thinking).
• 🛠️ Tool use — native tool-calling support in the chat template.
• 📏 Long context — native 131,072-token context length.
• 📱 Built for the edge — compact footprint designed for local assistants, coding agents, and tool-use workflows.

Model Information

Item	Value
Type	Causal Language Model
Architecture	Standard LlamaForCausalLM
Parameters	2,516,756,480 (~2.5B)
Non-Embedding Parameters	1,981,982,720
Layers	42
Attention Heads (GQA)	16 (Q) / 2 (KV), head dim 128
Hidden / Intermediate Size	2048 / 6144
Vocabulary Size	130,560
Context Length	131,072

---

Use the model

Edge Gallery App (Android)

1. Get the App: Install the app from Google Play or download the latest APK from the GitHub releases page.
2. Importing the Model: Navigate to the Model manager within the app and click the "+" (plus) icon in the bottom-right corner. Two options will appear:
   • Import from HF (Recommended): Select this option, and a dialog box will appear showing an example Hugging Face model URL. Enter the HF link for the desired .litertlm model and click submit. The model will then appear in your list, and you can proceed to download it (a Hugging Face account login is required).
   • From local model file: First, download the .litertlm model directly to your Android device, OR download it to your computer and push it via ADB (e.g., adb push minicpm_wi4c_wi8_afp32.litertlm /sdcard/Download/). Then, select this option, choose the downloaded file from your storage, configure your preferred parameters, and tap "Import".

For full details on importing models and other features, see the Edge Gallery App Wiki.

To build the demo app from source, please follow the instructions from the GitHub repository.

Try It (Desktop/CLI)

Install uv and run the model directly from the LiteRT-LM command line:

uv tool install litert-lm
uvx litert-lm run --from-huggingface-repo=litert-community/MiniCPM5-2B minicpm_wi4c_wi8_afp32.litertlm --prompt="What is the capital of France?"

Links

• 🤗 Original model (BF16): openbmb/MiniCPM5-2B
• 🤗 Smaller sibling: litert-community/MiniCPM5-1B
• 📦 GitHub: OpenBMB/MiniCPM
• 🛠️ LiteRT docs: ai.google.dev/edge/litert

---

License

Released under the Apache-2.0 License, consistent with the upstream openbmb/MiniCPM5-2B.

Citation

@article{minicpm4,
  title={MiniCPM4: Ultra-efficient LLMs on end devices},
  author={MiniCPM, Team},
  journal={arXiv preprint arXiv:2506.07900},
  year={2025}
}