hayamimi-punct-ja-4class

Japanese punctuation restoration (、 。 ？ ！) for hayamimi, exported to ONNX. Token classification head fine-tuned from sbintuitions/modernbert-ja-30m (MIT) on Japanese web text from HuggingFaceFW/fineweb-2 jpn_Jpan (ODC-By 1.0). Weights are MIT; if you redistribute them, credit FineWeb-2 (ODC-By attribution).

日本語の句読点復元モデル（読点・句点・？・！）。hayamimi の PunctuatorJa4Class が読む形式そのままで置いてあります。

Files

file	what	size
punct_4class.onnx	fp32 export	147 MB
quantized_ort/punct_4class.int8.onnx	dynamic int8 (recommended for CPU)	37 MB
hf/tokenizer.json, hf/tokenizer_config.json, hf/config.json	fast tokenizer + config used at inference
hf/model.safetensors	PyTorch weights, for further fine-tuning	147 MB
SHA256SUMS	checksums of the above

Results (FLEURS ja, n=250, punctuation-position F1)

model	P	R	F1	latency / line (CPU, 6 threads)
shipped mojicast-punct (fp32, 、。 only)	0.872	0.482	0.621	44 ms
this model, fp32	0.841	0.939	0.887	4.6 ms
this model, int8	0.858	0.920	0.888	3.4 ms

？ recall 1.00 on a question set. Decode gates (hayamimi defaults): on speech the ！ head fires on emphatic narration as confidently as on real exclamations, so PunctuatorJa4Class never emits ！ by default (exclaim_threshold=1.01) and gates 、 at 0.8 (comma_threshold=0.8); with those gates the model also beats the shipped one on sparse-punctuation TV captions (F1 0.64 vs 0.62). Pass exclaim_threshold=0.9 for dense text where ！ matters. The table above is plain argmax. This model was trained on densely punctuated web prose; measure on your own material before adopting it for captions (the gated defaults above are what the hayamimi CLI uses). Full record: docs/eval/punct_retrain.md in the hayamimi repo.

Usage (hayamimi)

python scripts/download_models.py --punct-4class     # fetches this repo into models/punct-ja-4class-permissive/
python scripts/realtime_transcribe.py --punct-model 4class

Or directly:

from punct_ja import PunctuatorJa4Class
p = PunctuatorJa4Class()   # models/punct-ja-4class-permissive by default
p.restore("明日の会議は午後三時から始まります資料の準備をお願いします")

Inputs are expected NFKC-normalised (the repo's ja_text_norm.safe_nfkc, which keeps fullwidth ？ and ！). The restorer only inserts marks; it never changes the input characters.

Training

scripts/make_punct_trainset.py → scripts/train_punct_ja.py → scripts/export_punct_4class.py in the hayamimi repo (RTX 3080 Ti, minutes).