genmai-310M

A Japanese kana-kanji conversion model: it converts a reading (katakana) into Japanese text, optionally conditioned on the text before the cursor. It is the AI converter of the macOS input method genmai, which runs it on device with Core ML (model-coreml.zip).

• Architecture: GPT-2, character-level vocabulary, 24 layers, 1024 hidden, 310M parameters
• Smaller variant (90M, distilled from this model, about twice as fast): subdiox/genmai-90M
• Base model: Miwa-Keita/zenz-v2.5-medium (CC BY-SA 4.0)

Files

Releases are git tags (v1.0.0); use a tag to pin a version. main is the latest release.

file	contents
model.safetensors	fp32 weights in the Hugging Face GPT-2 layout (1.2 GB), for transformers
config.json	GPT-2 config; the prompt format is recorded as "genmai_prompt_format": "zenzlc"
tokenizer.json, tokenizer_config.json	the tokenizer
model-coreml.zip	what the genmai input method runs: the model as stateful Core ML packages for the Apple Neural Engine (283 MB; see below): genmai_0.mlpackage (layers 0–11) and genmai_1.mlpackage (layers 12–23 and the head), int8 weights, and genmai_coreml.json (static shapes: format genmai-coreml-1, prompt_slots 256, output_steps 64, rows 5, prefill_chunk 32, chunks 2)

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("subdiox/genmai-310M", revision="v1.0.0")
tok = AutoTokenizer.from_pretrained("subdiox/genmai-310M", revision="v1.0.0")
x = tok("<s>\uee00" + "ハハハハガイタイラシイ" + "\uee01", return_tensors="pt", add_special_tokens=False)
y = m.generate(**x, num_beams=5, num_return_sequences=5, length_penalty=0.0, max_new_tokens=64)
print([tok.decode(s[x.input_ids.shape[1]:], skip_special_tokens=True) for s in y])  # 母は歯が痛いらしい, ...

Rank beams by the plain sum of log-probabilities (length_penalty=0.0): with transformers' default length normalisation, long degenerate outputs such as ハハハハガイタイらしい can win.

Prompt format (zenzlc)

Left context first (up to the last 500 characters), then the reading, then the output:

<s>[U+EE02]{left context}[U+EE00]{reading in katakana}[U+EE01]{output}</s>

[U+EE0x] are the private-use marker characters U+EE02 (context), U+EE00 (input) and U+EE01 (output). Without context the [U+EE02]{left context} part is omitted. Spaces are written as U+3000 and newlines are removed. config.json records the format name (genmai_prompt_format). Decode with beam search (genmai uses 5 beams) and stop at EOS; long readings are split at 、。 into chunks of at most 40 characters, each decoded after the previous output.

Core ML (Apple Neural Engine)

model-coreml.zip holds the same model converted with coremltools 9 (macOS 15 / iOS 18 or later) for the genmai input method's Core ML decoder. A package with all 24 layers does not load on the Neural Engine, so the model is split into two packages; each has two functions that share the package's states (per layer, keys and values in fp16):

• prefill: 32 prompt tokens written to prompt slots start ..< start + 32 (mask (32, 256) additive). Inputs of genmai_0: ids, pos (1, 32) int32, start (1,) int32, mask; of genmai_1: hidden (1, 32, 1024) fp16 (the output of genmai_0), start, mask. Outputs: hidden (genmai_0), logp (32, 6000) (genmai_1).
• decode: one token for each of 5 beams. Inputs of genmai_0: tok (5, 1) int32, pos (1,) int32; of genmai_1: hidden (5, 1, 1024) fp16; both: wsel (320, 5) / keep (320, 1), pmask (1, 256) / smask (5, 320) fp16 additive. Beam r's token of step t is stored at output slot t · 5 + r (wsel is the one-hot placement, keep = 1 − its row sums); each beam attends to the prompt slots in pmask and to its ancestors' slots in smask.

The Neural Engine computes in fp16 (at most 65504), which this model exceeds: its residual stream reaches about 81k and one attention head of the second layer has logits up to about 500k. The packages are reparametrized exactly to stay in range: the residual stream is scaled by 0.25 (embeddings and the attention / MLP output projections × 0.25, layer norm epsilon × 1/16; the head uses the unscaled embeddings), and attention logits are computed / 256 (folded into the query weights), the row maximum subtracted and the result × 256 before the softmax. Linear weights are int8 (symmetric, per channel), the rest fp16. The conversion script is Tools/coreml/genmai_coreml.py of genmai-ime (--chunks 2 --residual-scale 0.25 --score-scale 256).

On an M4 Pro (5 beams): top-1 accuracy (exact) 94.1 / 93.0 / 78.6 % on everyday sentences / AJIMEE / hard sentences (the weights in 8 bits on the GPU: 93.1 / 92.5 / 78.6 %, top-1 identical on 96–98.5 % of the items); process footprint 80–140 MB; a whole everyday sentence converts in a median 113 ms, the input method's conversion key (which converts while you type) answers in a median 30 / 39 ms (everyday sentences / AJIMEE). The first load compiles the model for the Neural Engine (about 20 s; macOS caches the result).

Training

Fine-tuned from zenz-v2.5-medium in several stages on (reading, text) pairs:

1. ~6M automatically read sentences from Japanese Wikipedia, FineWeb-2 (Japanese), Aozora Bunko and Diet minutes (readings by UniDic with corrections learned from reviewed samples);
2. long left context (up to 500 characters);
3. reading/surface pairs of the Mozc dictionary (dictionary_oss) and sentences containing rare dictionary words;
4. curated data: hand-checked items, synthetic sentences and homophone sets generated with Claude, and LLM-judged corrections, with replay of the earlier data;
5. training with an auxiliary next-character language-model loss on the real text before the cursor (so the model learns how words are used in running text, not only the conversions).

The release is a single model after one more round of stage 5 on ~1M further pairs.

Evaluation

Top-1 accuracy, beam size 5, bf16; candidates whose reading does not match the input are dropped. AJIMEE, everyday sentences and ime_v1 dev are judged leniently (orthographic variants accepted); homophones = word-level accuracy on held-out homophone sentences; hard = 28 sentences written to stress an IME (exact match up to orthography).

set	genmai-310M (bf16)	Zenzai (zenz-v3.2-small, azooKey settings)
AJIMEE-Bench (200)	95.0 %	85.5 %
everyday sentences (101)	98.0 %	97.0 %
ime_v1 dev (544)	93.6 %	–
homophone sentences (1,607)	93.1 %	–
average of the four	94.9 %	–
hard IME sentences (28)	22 / 28	–

Limitations

• Readings that repeat one sound (きしゃのきしゃがきしゃできしゃした) tend to map every occurrence to the same word or drop part of the reading.
• Rare proper nouns and fine homophone distinctions (特徴/特長, 追及/追求) are still error-prone.
• The model copies the left context strongly: a wrong conversion earlier in the context can be repeated.

License and attribution

CC BY-SA 4.0. The training data include Wikipedia (CC BY-SA), FineWeb-2 (ODC-By), Aozora Bunko, the National Diet minutes, and the Mozc dictionary (BSD-3-Clause, Copyright Google Inc.; contains data derived from IPAdic, Copyright Nara Institute of Science and Technology).