johnbean393/chiboard-1-t2-preview-0712

Chiboard-1 T2 Preview (2026-07-12)

This repository contains a public preview of Chiboard-1 T2, a 1.2B-parameter Chinese pinyin input-method teacher model preference-tuned from johnbean393/chiboard-1-t1.

This checkpoint did not pass the frozen production acceptance gate. It is published for evaluation and research, not as a replacement for T1. T2 made strong gains on hard ambiguity and modest gains on revision behavior, but regressed on the general plain-input guardrails. T1 remains the recommended production baseline.

What changed

T2 was trained for one epoch on 150,000 preference pairs using DPO with a chosen-answer NLL anchor:

Setting	Value
Base model	johnbean393/chiboard-1-t1
Training pairs	150,000
DPO beta	0.10
RPO alpha	1.0
Learning rate	1e-6
Microbatch	16
Gradient accumulation	8
Effective batch	128
Max sequence length	768
Max prompt length	512
Seed	20260711
Training runtime on one H100	28m 51s

Post-training preference diagnostics on 5,000 rows:

• Reward accuracy: 61.5%
• Preference gain: 0.2227
• Chosen NLL/token: 0.09539 (reference: 0.10111)

Frozen evaluation

The candidate was compared with immutable T1 and S1 baselines using paired evaluation rows and 2,000 bootstrap samples.

Comparison	Baseline	T2	Delta	95% CI	Result
Plain exact match vs T1	62.748%	62.408%	-0.341 pp	[-0.611, -0.062] pp	Failed
Plain CER vs T1	0.116030	0.116505	+0.000475	[+0.000171, +0.000796]	Failed
Revision exact match vs T1	63.616%	63.821%	+0.205 pp	[-0.037, +0.428] pp	Passed
Revision CER vs T1	0.055451	0.054807	-0.000643	[-0.001120, -0.000160]	Passed
Hard-ambiguity exact match vs S1	52.509%	59.962%	+7.453 pp	[+6.561, +8.355] pp	Passed
Hard-ambiguity CER vs S1	0.136857	0.122874	-0.013983	[-0.015165, -0.012860]	Passed

The release gate rejected the candidate for statistically supported plain exact-match and CER regressions. This preview is useful for studying the tradeoff between ambiguity handling and general behavior.

Prompt format

Inputs use the following serialized structure:

{committed_context}<|reserved_6|>{raw_pinyin}<|reserved_7|>{provisional_display}<|reserved_8|>

For an empty context and display, nihao becomes:

<|reserved_6|>nihao<|reserved_7|><|reserved_8|>

Usage

The checkpoint was produced with Transformers 4.55.4. Some newer Transformers versions may not recognize the serialized TokenizersBackend class name. The fallback below loads the same fast tokenizer while neutralizing the incompatible metadata field.

import torch
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    PreTrainedTokenizerFast,
)

model_id = "johnbean393/chiboard-1-t2-preview-0712"

try:
    tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
except (ValueError, AttributeError):
    tokenizer = PreTrainedTokenizerFast.from_pretrained(
        model_id,
        extra_special_tokens={},
    )

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

committed_context = ""
raw_pinyin = "nihao"
provisional_display = ""
prompt = (
    f"{committed_context}<|reserved_6|>{raw_pinyin}"
    f"<|reserved_7|>{provisional_display}<|reserved_8|>"
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=64,
        do_sample=False,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id,
    )

generated = output[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(generated, skip_special_tokens=True))

Limitations and intended use

• Preview/research checkpoint; not production-approved.
• General plain-input quality is slightly worse than T1 under the frozen test.
• The largest gain is concentrated in targeted hard-ambiguity cases.
• Chinese pinyin IME behavior is the intended domain; unrelated general-purpose text generation has not been validated.
• Long inputs can be computationally expensive even though the architecture advertises a large context window.

Immutable lineage

Component	Revision
T1 base model	4086865d8813a01909a420579ee7b15821bf80b0
DPO dataset	6811f2dcdd3a770591e0b878d6c57b60865e59e7
Plain evaluation dataset	54f9a912fce5f60df4918f83b3d9f0b5595f47b4
Revision evaluation dataset	62ffeead7a381d4d88d640a44531e3f3cbb9996e
Tokenizer SHA-256	516b5b72266074897f80b3bee95d1b8b74497dfc5e58d1be289db107c47b7c99

The repository also includes lineage.json, training_summary.json, and the frozen evaluation/acceptance.json decision artifact.