WordStitch-4B v0.2 Alpha

Experimental but usable. WordStitch-4B is a 4B language model fine-tuned for Chinese-Pinyin lexical rescue in English and Chinese/English/Pinyin code-mixed text.

When an English word is hard to spell, type the Pinyin of its Chinese meaning. WordStitch attempts to recover the intended word from context and rewrites the complete sentence as natural English. No <pinyin> tags or special delimiters are required.

I forgot my yusan.
        ↓
    WordStitch
        ↓
I forgot my umbrella.

More examples:

Tomorrow I need to go yiyuan.
→ Tomorrow I need to go to the hospital.

I want to take ditie to xuexiao.
→ I want to take the subway to school.

wo想take ditie去school
→ I want to take the subway to school.

Project code, training scripts, and documentation: github.com/heavry/WordStitch

Files

• repository root and adapter/: selected checkpoint-500 LoRA adapter, tokenizer, and chat template (about 142MB; the subdirectory mirror keeps downloads organized)
• WordStitch-4B-Q4_K_M.gguf: llama.cpp/Ollama deployment artifact, 2,708,803,936 bytes
• evaluation/: frozen inputs, outputs, reviews, summaries, and freeze manifest
• docs/: provenance, redistribution policy, and license review
• Modelfile: Ollama configuration

The 9.10GB merged BF16 checkpoint is intentionally not distributed. The immediate rodrigomt safetensors conversion used for training identifies its base model but does not expose an explicit license field or LICENSE file. See docs/LICENSE_REVIEW.md.

Evaluation

The same frozen 100-example WordStitch evaluation set was used before and after LoRA training. Checkpoint 500 was selected by validation loss before the frozen set was scored.

Category	Base BF16	LoRA BF16	Mac Q4_K_M
Chinese input	10/10	9/10	9/10
Standard Pinyin rescue	5/10	10/10	9/10
Initial Pinyin	4/10	9/10	9/10
Final Pinyin	6/10	9/10	9/10
Multiple rescues	2/10	8/10	7/10
Consecutive Pinyin	1/10	6/10	5/10
Chinese/English/Pinyin	6/10	10/10	10/10
Mild Pinyin typo	5/10	10/10	9/10
Contextual ambiguity	7/10	9/10	9/10
Long mixed input	5/10	9/10	9/10
WordStitch total	51/100	89/100	85/100

This means 89/100 semantic passes on the project's 100-example frozen evaluation set; Q4_K_M scored 85/100. Evaluation was reviewed by an AI evaluator (Astra), not an independent human benchmark. The benchmark is small and is not an external certification.

General QA remained 19/20 before and after LoRA. Normal English preservation remained 20/20. These are small retention checks and do not establish broad capability preservation.

Known limitations

WordStitch-4B is experimental. It performs well on common single-token Pinyin lexical rescue, but consecutive Pinyin sequences, multiple simultaneous replacements, ambiguous transliterations, uncommon vocabulary, and quantization remain challenging. It can choose the wrong word, change correct English, or produce awkward grammar. bingxiang was observed becoming “window,” and hamburger examples sometimes omitted the article “a.” Typo performance on the frozen set must not be interpreted as universal 100% reliability.

llama.cpp

hf download heavry/WordStitch-4B \
  WordStitch-4B-Q4_K_M.gguf \
  --local-dir .

llama-cli \
  -m WordStitch-4B-Q4_K_M.gguf \
  --system-prompt "Rewrite the user input as natural, complete English. The input may mix English, Chinese, toneless Mandarin pinyin, and mildly misspelled pinyin. Recover missing English words using the sentence context. Output only the final English text, without explanations or alternatives." \
  -p "I forgot my yusan." \
  --temp 0

For an OpenAI-compatible API, including a Grok Bot VM:

hf download heavry/WordStitch-4B WordStitch-4B-Q4_K_M.gguf --local-dir ./models/wordstitch

llama-server \
  -m ./models/wordstitch/WordStitch-4B-Q4_K_M.gguf \
  --host 127.0.0.1 \
  --port 8080 \
  -c 2048 \
  -ngl 99 \
  --jinja \
  --chat-template-kwargs '{"enable_thinking":false}' \
  --reasoning-budget 0

Send the WordStitch instruction as the system message and the mixed sentence as the user message. Keep temperature at 0 for concise rewriting. Put authentication in front of the server before exposing it outside a trusted host.

Ollama

hf download heavry/WordStitch-4B WordStitch-4B-Q4_K_M.gguf --local-dir .
ollama create wordstitch -f Modelfile
ollama run wordstitch "I forgot my yusan."

LoRA adapter

Load adapter/ with PEFT on the exact training base revision:

from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
from peft import PeftModel

base_id = "rodrigomt/Qwen3.5-4B-Uncensored-Aggressive"
adapter_id = "heavry/WordStitch-4B"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = Qwen3_5ForConditionalGeneration.from_pretrained(base_id, dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)

Use the system instruction shown above and disable thinking in the chat template.

Training summary

• 9,710 training rows; 576 group-isolated validation rows
• BF16 LoRA, rank 16, alpha 32, dropout 0.05
• 30,474,240 trainable parameters
• 600 steps; checkpoint 500 selected
• best validation loss: 0.192294
• effective batch size 16; maximum length 384; learning rate 5e-5

The complete mixed-source training corpus is withheld. Published materials include aggregate manifests, generation and cleaning scripts in GitHub, attribution, and the frozen evaluation. See docs/DATA.md.

License and provenance

Project-authored code and documentation are Apache-2.0. Qwen/Qwen3.5-4B and the HauhauCS aggressive GGUF are marked Apache-2.0. The rodrigomt safetensors conversion used for training lacks explicit license metadata, so the merged BF16 checkpoint is withheld and that uncertainty is disclosed. Model artifact use remains subject to upstream terms. See LICENSE, NOTICE, and docs/LICENSE_REVIEW.md. This is a provenance record, not legal advice.