dokoomae/LFM3-1.7B-think-lang-fix-patch

LFM3-1.7B-think-lang-fix — patch bundle

Compact, reproducible rank-1 weight patch (497 KB) that turns LiquidAI/LFM3-1.7B into dokoomae/LFM3-1.7B-think-lang-fix: the model's <think …> reasoning then follows the prompt language (Chinese prompt → Chinese reasoning, English prompt → English reasoning). No training, no gradients.

Contents

File	Description
thinklang_rank1_patch.safetensors	Per-layer rank-1 factors: g[2048] + proj[7680] + coeff for 13 layers
patch_meta.json	Axis tokens, layer lists, γ coefficients, target tensor formula
apply_patch.py	Merge script: applies the patch to a local copy of the base model
technical_report_zh.pdf	Full technical report (Chinese): diagnosis, layer-wise data, results

Quick start

pip install torch safetensors
python3 apply_patch.py \
  --model-dir /path/to/LFM3-1.7B \
  --out-dir  /path/to/LFM3-1.7B-fixed

Output directory is a drop-in replacement for the original (same config/tokenizer/chat template), loadable with transformers or vLLM via trust_remote_code=True.

What it edits

13 tensors model.layers.{l}.feed_forward.w2.weight (l ∈ {5,6,7,11,12,13,16,18,19,20,21,22,23}):

w2_new = w2 + coeff · outer(g, proj)      # fp32 math, cast back to bf16

• 11 layers: coeff = −1.2 — projects out the "English-opener prior" their MLPs write into the residual stream
• layers 19 & 23: coeff = +1.5 — amplifies the correct prompt-following writers 2.5× along the same axis

All other 273 tensors are untouched. See the technical report for the full diagnosis (root cause: layers 18–22 MLPs indiscriminately write an English-CoT prior; layer 23 MLP is the correct language-follower but too weak).

Results (25-prompt battery, official sampling settings)

Prompt language	Baseline	Patched
Chinese	3/12 (25%)	12/12 (100%)
English	8/8 (100%)	8/8 (100%)
Malay	1/3 (33%)	2/3 (67%)
Japanese	0/2 (0%)	2/2 (100%)

The patch itself is licensed Apache-2.0; the base model remains under the Liquid AI Community License.