julianmb

/

MiniCPM5-2B-NPU2

like 4

Same error here on Linux (Strix Halo, FLM v1.0.1/v1.0.2/v1.0.4) — it's the model's tokenizer_config.json, not your setup.

It declares bos_token/eos_token/pad_token as strings but ships no bos_token_id/eos_token_id/pad_token_id. FLM reads the ids from that file, not from config.json (which does carry 0/1/1). v1.0.1/v1.0.2 report it as the unhelpful type must be number, but is null; v1.0.4 says it outright: bos_token is set in tokenizer_config.json but bos_token_id is missing or not an integer.

Add these three keys to tokenizer_config.json and it loads:

"bos_token_id": 0,
"eos_token_id": [1, 130073],
"pad_token_id": 1

eos_token_id must be an array — FLM rejects a scalar with eos_token_id is missing or not an array. 130073 is <|im_end|>, which is what the shipped chat_template.jinja actually ends turns with. (Values come from the repo's own config.json: bos 0, eos 1, pad 1.)

Heads-up on what comes next: after that fix it loads cleanly for me, but every inference then returns runlist failed execution (ERT_CMD_STATE_TIMEOUT) on Linux — I've filed that separately. You're on Windows, so if it actually generates for you after this fix, please say so — that would pin the second problem to the Linux driver stack and would be genuinely useful to know.

Thanks for reporting this! @Platano78 is spot-on. FastFlowLM's C++ loader expects explicit integer IDs in tokenizer_config.json.

I have just pushed a fix directly to this repository adding:

"bos_token_id": 0,
"eos_token_id": [1, 130073],
"pad_token_id": 1

How to update your local setup:

1. If you cloned the repository via git into your models directory:

   cd ~/.config/flm/models/MiniCPM5-2B-NPU2   # (or D:\models\flm\models\MiniCPM5-2B-NPU2 on Windows)
   git pull
   

2. Or manually add the three keys above into your local tokenizer_config.json (ensure eos_token_id is an array [1, 130073], where 130073 is <|im_end|>).
3. Re-run flm serve minicpm5:2b.

The model card has also been updated with a Troubleshooting & FAQ section covering this.