MiniCPM5-2B (LiteRT-LM)

This repository hosts a LiteRT-LM (LiteRT, formerly TensorFlow Lite) conversion of openbmb/MiniCPM5-2B, optimized for fully on-device inference on mobile and edge hardware.

The model was re-authored with ai-edge-torch (Llama-architecture decoder, since MiniCPM5-2B ships as a standard LlamaForCausalLM), verified for numerical parity against the original HF checkpoint (cosine similarity 0.999995 on last-token logits), converted to a multi-signature (prefill_8/64/128/256/512/1024 + decode) .tflite with INT8 dynamic weight-only quantization, and packaged into .litertlm with litert-lm-builder.

Available Files

• MiniCPM5-2B_wi8_afp32.litertlm: dynamic weight-only INT8 quantization (wi8) with FP32 activations (afp32). KV cache buffer length 1280 tokens.

Model Information

Item	Value
Type	Causal Language Model
Architecture	Standard LlamaForCausalLM
Layers	42
Hidden size	2048
Attention Heads (GQA)	16 (Q) / 2 (KV), head_dim 128
Vocab size	130,560
Context Length (native)	131,072
Exported KV cache length	1,280

Verification

• Numerical parity: re-authored ai-edge-torch model vs. original HF transformers forward pass — cosine similarity 0.999995 on last-token logits (identical top-5 token predictions).
• End-to-end generation: verified with litert-lm run, including correct <think>...</think> reasoning-channel parsing and coherent completions (factual QA + code generation tested).

Use the model

Desktop/CLI

Install uv and run the model directly from the LiteRT-LM command line:

uv tool install litert-lm
uvx litert-lm run --from-huggingface-repo=luispoveda93/MiniCPM5-2B-litert-lm MiniCPM5-2B_wi8_afp32.litertlm --prompt="What is the capital of France?"

Edge Gallery App (Android)

1. Install the app from Google Play or the GitHub releases page.
2. In Model manager, tap + → Import from HF, paste this repo's .litertlm URL, and download.

Links

• 🤗 Original model (BF16): openbmb/MiniCPM5-2B
• 📦 GitHub: OpenBMB/MiniCPM
• 🛠️ LiteRT docs: ai.google.dev/edge/litert
• Reference conversion for the smaller checkpoint: litert-community/MiniCPM5-1B

License

Released under the Apache-2.0 License, consistent with the upstream openbmb/MiniCPM5-2B.

Citation

@article{minicpm4,
  title={MiniCPM4: Ultra-efficient LLMs on end devices},
  author={MiniCPM, Team},
  journal={arXiv preprint arXiv:2506.07900},
  year={2025}
}