CantoneseLLM-v2.0-8B-Thinking-Chat-Vector-Merged

This is an intermediate checkpoint, released as a baseline. It is an 8B dense model built from Qwen3-8B-Base by continual pre-training on 784M tokens of Cantonese and Hong Kong text, then adding the chat vector taken from Qwen3-8B.

It has Hong Kong knowledge and instruction following, acquired at zero training cost for the second of those — and it scores above its own donor on HKCanto-Eval, which the 30B-A3B merge did not manage. What it does not have is Cantonese reasoning: it thinks in Simplified Chinese. Installing that behaviour took three further stages, and the result is CantoneseLLM-v2.0-8B-Thinking.

Use this model if you want a Cantonese-knowledgeable 8B instruct model and do not care what language the chain-of-thought is in, or you are reproducing the paper's ablations. Otherwise use the final model.

📄 Paper: CantoneseLLM v2: Reasoning in a Low-Resource Language (arXiv:2609.06970)

🧪 Evaluation Benchmark: hon9kon9ize/hkeval2025

What the chat vector does, and what it does not

The merge is one operation and requires no instruction data:

Δ_chat = θ_instruct − θ_base            # Qwen3-8B − Qwen3-8B-Base
θ_merged = θ_cpt + Δ_chat               # added to the continually pre-trained checkpoint

It installs instruction following and alignment for free. It also carries over the donor's reasoning language and script, which continual pre-training on Traditional Chinese and Cantonese does not override. On a translated GSM8K probe, under a system prompt that explicitly asks for Hong Kong Cantonese, this model reasons entirely in Simplified Chinese:

嗯，用户问的是Janet每天赚多少钱。首先，我需要理清楚整个流程。题目说Janet的鸭子每天生16只蛋。

(Well, what the user is asking is how much Janet earns each day. First, I need to work through the whole process. The problem says Janet's ducks lay 16 eggs a day.)

The script leaks into the answer as well. Counting eggs, this checkpoint writes 只 where Hong Kong Cantonese requires the classifier 隻 zek3 — 只 can only be read zi2, which is a different word. On this single probe the paper counts six such classifier errors. The final RLVR model produces none.

Trace length over eight fixed probes (mean tokens between the reasoning tags):

Checkpoint	Mean CoT tokens	Median	CoT-to-answer ratio
Qwen3-8B (donor)	491	502	2.94
This model	468	512	3.27
CantoneseLLM-v2.0-8B-Thinking (final)	232	237	1.09

This model reasons at roughly the donor's length. The length was never the problem; the language and the script were.

---

Benchmark results

HKCanto-Eval (Cheng et al., 2025), reasoning mode on:

Model	MMLU	CantoMMLU	Cultural	Linguistic	Academic & Prof.	Avg.
Qwen3-8B (donor)	81.08	76.68	57.94	39.50	81.61	67.36
This model	80.04	76.96	66.67	41.50	82.36	69.51
CantoneseLLM-v2.0-8B-Thinking (final)	73.86	69.73	58.33	36.00	76.08	62.80

The merge gains 2.15 points over the donor (+3.18%) — the only merge in this work that improved on the model it took the chat vector from. The gain is almost entirely Cultural, 57.94 → 66.67, an 8.73-point jump that is the continual pre-training showing through in the category most about Hong Kong. Linguistic also gains 2.00 and Academic & Professional 0.75, while English MMLU costs 1.04.

This model is the highest-scoring 8B checkpoint in the whole pipeline — 6.71 points above the final RLVR model. That is not a reason to prefer it. The benchmark is multiple-choice and does not measure the language of the chain-of-thought, which is the entire difference between the two checkpoints. Selecting on this average gets you a model that thinks in Simplified Chinese.

---

Artefacts released with this work

The CPT corpus itself is not released, but its two largest public constituents are.

	What it is
🌐 Traditional-Chinese-Common-Crawl-by-year	Traditional Chinese extracted from all 111 Common Crawl snapshots, released per snapshot across thirteen years
🇭🇰 Cantonese-Web-Data	The Cantonese subset of the above, filtered with CantoneseDetect and globally deduplicated to 477,298 unique documents
🏋️ cantonese-nemo-gym-environments	The six NeMo-Gym environments and the multiplicative language reward. Used by the final models, not by this checkpoint; MIT licensed
📊 Nemotron-3-Nano-RL-Training-Blend-STEM-Yue-Translated	The RLVR prompt corpus, a three-way parallel en/ yue/ zh-hk/ translation of NVIDIA's Nemotron-3-Nano RL blend. Also used by the final models only

Other models in this release

All four checkpoints are in the CantoneseLLM v2.0 collection.

Model	What it is
CantoneseLLM-v2.0-8B-Thinking-Chat-Vector-Merged	this model — CPT + chat-vector merge at 8B, before SFT, DPO and RLVR
CantoneseLLM-v2.0-8B-Thinking	Built from this checkpoint through SFT, DPO and two-stage RLVR. Reasons in Cantonese, and scores lower here for that trade
CantoneseLLM-v2.0-30B-A3B-Thinking	The flagship. Full five-stage pipeline at 30B-A3B, and it recovers far more of the SFT regression than the 8B does
CantoneseLLM-v2.0-30B-A3B-Thinking-Chat-Vector-Merged	The same intermediate stage at 30B-A3B, where the inherited reasoning language is Written Chinese rather than Simplified

---

Usage

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "hon9kon9ize/CantoneseLLM-v2.0-8B-Thinking-Chat-Vector-Merged"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")

SYSTEM = "你係CantoneseLLM，一個由Hon9Kon9ize開發嘅語言模型，請使用香港嘅廣東話回答用家問題"

messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "小明有 5 個蘋果，佢俾咗 2 個朋友，每人 1 個，跟住又買多 3 個。佢而家有幾多個蘋果？"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=2048, temperature=0.6, top_p=0.95)
print(tokenizer.decode(out[0][len(inputs.input_ids[0]):], skip_special_tokens=True))

vLLM

The stock command works — no special flags are required:

vllm serve hon9kon9ize/CantoneseLLM-v2.0-8B-Thinking-Chat-Vector-Merged

Add --tensor-parallel-size N to shard across N GPUs, and --reasoning-parser qwen3 if you want the reasoning block returned separately as reasoning_content rather than inline in content (parser names vary by vLLM version).

OpenAI-compatible API

The served endpoint speaks the OpenAI protocol, so the official client works unchanged:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

SYSTEM = "你係CantoneseLLM，一個由Hon9Kon9ize開發嘅語言模型，請使用香港嘅廣東話回答用家問題"

resp = client.chat.completions.create(
    model="hon9kon9ize/CantoneseLLM-v2.0-8B-Thinking-Chat-Vector-Merged",
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "小明有 5 個蘋果，佢俾咗 2 個朋友，每人 1 個，跟住又買多 3 個。佢而家有幾多個蘋果？"},
    ],
    temperature=0.6,
    top_p=0.95,
    max_tokens=2048,
)

msg = resp.choices[0].message
print(getattr(msg, "reasoning_content", None) or "")  # populated with --reasoning-parser
print(msg.content)

Or with curl:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hon9kon9ize/CantoneseLLM-v2.0-8B-Thinking-Chat-Vector-Merged",
    "messages": [
      {"role": "system", "content": "你係CantoneseLLM，一個由Hon9Kon9ize開發嘅語言模型，請使用香港嘅廣東話回答用家問題"},
      {"role": "user", "content": "點解香港嘅雨季集中喺五月到九月？"}
    ],
    "temperature": 0.6, "top_p": 0.95, "max_tokens": 2048
  }'

Sampling: temperature 0.6, top-p 0.95 — the settings used for every evaluation reported above and in the paper. Avoid greedy decoding.

System prompt: the Cantonese system prompt above was used throughout evaluation. Behaviour with other system prompts, or with none, is not characterised.

Chat template and thinking mode are inherited from the donor. The chat vector came from Qwen3-8B, a hybrid model, so this checkpoint keeps the stock Qwen3 thinking / non-thinking switch. All evaluation above was run with reasoning mode on.

---

Training pipeline

Stage	What it installed	Compute
Continual pre-training	Hong Kong knowledge and Cantonese lexis. 784M tokens over 568K rows, 530 steps (2.19B tokens seen, 2.79 passes), LR 3×10⁻⁵, on 64 TPU v6e chips (Google TRC) via MaxText	103 TPU chip-hours
Chat-vector merge	Instruction following and alignment. Δ = Qwen3-8B − Qwen3-8B-Base, added to the CPT checkpoint	0

That is the whole pipeline for this checkpoint — no SFT, no DPO, no RLVR. The corpus was 32.9% Common Crawl (5.4% of the total in Cantonese), 20.9% English replay from Nemotron pre-training data, and 17.1% web fiction, which is the only source where Cantonese is sustained over tens of thousands of tokens.

At 8B the CPT ran at a single learning rate of 3.0×10⁻⁵, with 520, 530 and 540 steps compared to probe the effect of the final stretch of training. The 530-step checkpoint was selected.

---

Risks & Limitations

• It reasons in Simplified Chinese. This is the defining limitation and the reason the rest of the pipeline exists. The script leaks into answers through the 只 / 隻 conflation described above. If you need Cantonese chain-of-thought, or correct Traditional Chinese classifiers, use the final model.
• Instruction following is inherited, not trained. It arrives through weight arithmetic from the donor and was never fine-tuned on Cantonese instruction data. Behaviour outside the donor's own distribution is not characterised.
• No translation, data-curation or judging ability. Those were installed at SFT, which this checkpoint precedes.
• CPT was 784M tokens — small by continual-pre-training standards (Taiwan-LLM used 35.1B, Swallow and SEA-LION 200B each). Hong Kong knowledge is real but thin, and the corpus leans on web fiction for long-form Cantonese.
• Alignment is whatever the chat vector carried over. No additional safety tuning was performed at any stage.
• Long context is inherited and untested — the CPT sequence length was 4,096 packed. Behaviour beyond that is whatever the Qwen3 base provides.
• Standard LLM caveats apply: it will hallucinate.

---

Citation

@misc{cantonesellm_v2,
      title={CantoneseLLM v2: Reasoning in a Low-Resource Language},
      author={Tsz Chung Cheng and Chung Shing Cheng and Chaak Ming Lau and Cheuk Hei Chong},
      year={2026},
      eprint={2609.06970},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2609.06970},
}

The chat vector method is due to Huang et al., Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages, ACL 2024, pp. 10943–10959.

Acknowledgements

Continual pre-training (CPT) was carried out on Cloud TPUs (Tensor Processing Units) from Google's TPU Research Cloud with MaxText. Post-training was carried out on computer resources offered under the category of General Projects by Research Institute for Information Technology, Kyushu University. Usage fee and cost of data-curation costs with proprietary APIs were covered by Votee AI

The RLVR stage builds on NVIDIA's NeMo-RL and NeMo-Gym, and on the Nemotron post-training and RL datasets.