无限循环。。。
Thanks for the report! We can't reproduce this from the screenshot alone — could you add:
The input — the full prompt / raw conversation text (most important)
Inference framework and version (vLLM / llama.cpp / transformers ...)
Sampling params (temperature, top_p, repetition_penalty, ...)
Whether you're using the original weights or a quantized build
That would let us reproduce and dig in much faster. Thanks!
Weights: Quantized build — official openbmb/MiniCPM5-2B-GGUF, Q8_0 (2,679,710,688 bytes, downloaded via hf-mirror 2026-09-10)
Framework & version: llama.cpp fork XHToken/llama.cpp @ commit 4a3635c (built 2026-09-04, CUDA 12.8, llama-server built-in HTTP API; binary self-reports 0.1.2-dev build 4a3635c). Not mainline llama.cpp, not vLLM.
Launch flags:
llama-server -m MiniCPM5-2B-Q8_0.gguf -a MiniCPM5-2B \
--host 0.0.0.0 --port 8893 -c 131072 \
-ctk q8_0 -ctv q8_0 -fa on -ngl 999
Hardware: NVIDIA RTX A6000 48GB (full offload), host inference.
Sampling params: Client (ZCode IDE agent) defaults for creative story continuation — temperature ~0.7–1.0, top_p ~0.95, no repetition/frequency penalty set (exact client values to be confirmed on my side). enable_thinking not explicitly passed (template default).
Input: Chinese creative story continuation request (long-form open-ended generation; protagonist “小李”， motif “小铃铛”). Full raw prompt available from the reporter if needed.
Symptom: After several hundred tokens of coherent story text, output degenerates into an endless repetition loop — the phrase “树上挂着铃铛，” repeated hundreds of times until max_tokens cap (see attached screenshot). Task category: Chinese creative long-form generation. Structured short tasks (extraction/summarization/tool-calls) on the same deployment did NOT show this.
Thanks for the detailed report. This issue is likely related to the default sampling behavior of the llama.cpp deployment being used.
Unlike SGLang, which defaults to min_p=0, this llama.cpp deployment defaults to min_p=0.05. That setting removes tokens whose probability is below 5% of the highest-probability token, which can inadvertently filter out tokens that would help the model escape a repetition loop.
Please try setting temperature=1.0, top_p=0.95, min_p=0.0 and see whether the repetition issue is resolved.