VimeML Tiny Japanese GPT v4.2

A 15,070,848-parameter Japanese decoder-only language model for local contextual candidate reranking and short within-sentence continuation. This release contains the selected V4.2 epoch 20 / step 1,081,942 FP32 inference bundle, its INT8 block32, FP32-compute Core ML stateless package, and a matching chunked KV-cache Core ML package.

Status: deployed default. V4.2 is the default reranking and next-word model of the Vime input method; V2.1 ships alongside it as the fallback. V4.2 is not a demonstrated accuracy improvement over V2.1 (see Evaluation).

This is a custom PyTorch model, not a Transformers AutoModel integration or an instruction/chat model. SentencePiece, kana retrieval, candidate scoring and search run outside the Core ML graph. The architecture, loader and scoring code are included under source/; no trust_remote_code or training checkpoint is required.

Files and identity

Path	Contents
`inference/`	Unique FP32 state_dict, config, tokenizer, token manifest and provenance; no optimizer
`coreml/ios18-int8-block32/`	Stateless Core ML `.mlpackage` and manifest
`coreml/ios18-int8-block32-kvchunk/`	Chunked KV-cache Core ML `.mlpackage` and manifest
`source/`	Frozen VimeML Python source, scripts, configurations and requirements
`infer.py`	FP32 generation and candidate-scoring wrapper using the V4 bundle loader
`evaluation/summary.json`	Aggregate quality, alignment and measurement results; no benchmark texts or traces
`RELEASE.json`, `SHA256SUMS.txt`, `verify_release.py`	Release inventory and optional byte verification

Source: Voltline/VimeML, commit 7f4e87f030f67e5e4a75aed8e795aeda9ce49023.
FP32 inference manifest: 05ba14f33f28aa18317adfe007d4ac2a355d38ce8636b294ff37468b2209a41b.
Core ML manifests: stateless 0c3b6f3b809aaf3dd2f9ae18626c363f72dd34c59953dea48fe865ebbde9d6da, chunked KV fd4bbb6c31fa2f71b2c9be6ab5feb0bbb03b7f51216907c7ff271171d26d1b75.
Tokenizer: 296a2bdf4f4862be8bda848ed1cd261f0b0fd43064181fad5a513c6ce73081b5.
Original selected training checkpoint: 0e83ea7417b9816b37183044468343b151f4e610da0764c31f1f780c5fa5796d; the optimizer-bearing checkpoint is not distributed.

These identities are SHA256 digests. Original model manifests remain unchanged; their historical paths describe provenance rather than required installation locations. The release inventory uses the same format as V1 and V2.1.

Architecture and tokenization

Pre-Norm RMSNorm/SwiGLU, bias-free Linear layers, learned absolute positions and causal attention. Vocabulary 8,192; context 128; width 384; 7 layers; 6 heads of dimension 64; SwiGLU width 960; dropout 0; RMSNorm epsilon 1e-5. Input embedding and LM head share one parameter.

SentencePiece 0.2.1 Unigram trained on a shared train-only sample: identity normalization, whitespace preservation, byte fallback and add_dummy_prefix=false. PAD/UNK/BOS/EOS IDs are 0/1/2/3. IDs 4–6 are user-defined markers <CTX>, <READ>, <OUT> from an earlier reading-conditioned experiment; V4.2 never trains on them. Reject them in input and exclude them from generation (the loader does the latter). V4 token IDs differ from V1 and V2; model and tokenizer must be updated together.

Quick start: downloaded files

Python≥3.11. Install the included requirements after selecting a suitable PyTorch build. FP32 inference does not invoke Apple runtime services.

python -m pip install -r source/requirements.txt
python verify_release.py
python infer.py --prompt "今日は雨が降っているので、" --max-new-tokens 8
python infer.py --prompt "雨が降っているので、" --candidate "傘を持っていく" --candidate "橋を渡っていく"

The candidate command ranks supplied strings; it does not retrieve dictionary candidates or perform kana conversion. The diagnostic wrapper raises on invalid or overlong inputs; the application retains dictionary order when a whole candidate pool cannot be scored.

The V4 loader reads weights with torch.load(..., weights_only=True), restores the shared head, and checks the model configuration, parameter count and tokenizer contract. verify_release.py checks the complete inventory and sizes without running a model; --hashes additionally checks all file bytes against the recorded digests. Checksums are not a digital signature. Keep source/, the tokenizer and inference files together.

Candidate scoring

Encode context + candidate jointly with BOS. Find the common token prefix of the context and all candidate sequences, then sum full-vocabulary log-softmax over the remaining suffix. Do not append candidate-final EOS; ties keep original order. This suffix likelihood is a scoring proxy, not exact string conditional probability. Oversized or invalid pools fall back as a whole, and recall failures stay in evaluation denominators. Deployment uses pure LM scores, with AzooKey retrieving kana candidates.

Training

Random initialization (seed 1219827974), 20 epochs, 1,081,942 updates, batch 512, BF16 with FP32 weights, fused AdamW (β 0.9/0.95, weight decay 0.1, clip 1.0), cosine learning rate 6e-4→6e-5 after 1,000 warmup updates, and a 30% prefix crop on eligible windows. Total exposure is 12,104,389,994 supervised next-token targets on one NVIDIA L20C GPU.

Web text: the frozen V1/V2 Japanese sentence corpus from nine shards of FineWeb2-Edu Japanese and Tatoeba, re-encoded with the 8K tokenizer: 25,185,368 training sentences, excluding 97 known validation overlaps.
Dialogue: independent utterances from RealPersonaChat and the Multi-Relational Multi-Party Chat Corpus (477,334 training turns, dialogue-level splits), sampled to about 5% of supervised targets per epoch, which repeats the dialogue pool several times.

The objective is ordinary unmarked next-token loss. Epoch 20 was selected because it has the best complete validation LM BPC, before any IME evaluation. The test split and the historical 1,000-case blind pool were not used.

On the original V1/V2 validation text, FP32 LM BPC is 3.045152, versus 3.080191 for V2.1 measured the same way; chat validation BPC is 3.122060. BPC is total NLL including EOS divided by Unicode characters×ln2. Token losses across tokenizers are not compared directly.

Evaluation

Fixed AzooKey N-best pools are reranked without adding correct answers. The Standard IME tracks use WRIME and JMultiWOZ cases with frozen AI-expert-reviewed labels, not native-speaker gold; Legacy is an exposed historical web subset with draft labels; AJIMEE JWTD_v2 is public and has been used for error analysis.

Pool	AzooKey Top-1	V4.2 FP32 Top-1 / Top-5	V4.2 INT8 Top-1 / Top-5	V2.1 FP32 / INT8 Top-1	Pool coverage
Standard development	453/728	523 / 624	522 / 623	505 / 506	630/728
Fixed reference	467/725	541 / 643	—	535 / —	650/725
Legacy regression	317/500	370 / 442	371 / 442	374 / 375	443/500
AJIMEE	87/200	139 / 161	139 / 161	144 / 144	162/200
Historical development	111/137	123 / 136	123 / 136	122 / 122	136/137

The 725-case reference split had already been consumed by an earlier comparison; V4.2 was registered under plan 81512f147d3749ef9a36e61470fce173 before scoring and evaluated once, so it is a fixed reference, not a fresh blind test. Against V2.1 it is +6 (40 improvements, 34 regressions; exploratory exact McNemar p=0.56). The Standard development gain (+18, p=0.047) has been reused across experiments and does not replicate at the same size on the reference. Legacy (−4) and AJIMEE (−5) trail V2.1 without statistical significance.

About 61% of all Top-1 flips against V2.1 are notation variants of the same word (kana/kanji/okurigana), where the frozen references list one form; V4.2 leans toward hiragana. AJIMEE losses are mainly encyclopedic homophone errors. Exact-match Top-1 is therefore sensitive to orthographic style. No labels were changed in response to model outputs.

Core ML variants

Weight-only symmetric INT8, block size 32; FP32 computation; CPU_ONLY; minimum iOS 18. No activation quantization or INT8 arithmetic is claimed. 37 matrices are compressed; RMSNorm weights, masks and a converter-inserted head bias constant remain FP32. The stateless package is 17,164,271 bytes, versus 60,467,999 bytes uncompressed (about 71.6% smaller).

Stateless: input_ids INT32 [1,T], 1≤T≤128, right PAD → logits FLOAT32 [1,T,8192].
Chunked KV (17,193,424 bytes): input_ids INT32 [1,T], position INT32 [1] (current cache length, position+T≤128), key_cache/value_cache FLOAT32 (7,1,6,128,64) → logits for the new tokens plus updated caches. The caller owns cache snapshots and rollback.

xcrun coremlcompiler compile coreml/ios18-int8-block32/model.mlpackage compiled-int8 --platform ios --deployment-target 18.0

Uncompressed conversions pass frozen logits alignment (maximum absolute error 5.2e-5 stateless and chunked). INT8 fails strict FP32 logits alignment (maximum 0.477, mean 0.0495 at T=128; V2.1 INT8 reached 1.009 and 0.1085) at unchanged atol/rtol 3e-4; padding and future-token invariants are exact. The chunked INT8 graph matches the stateless INT8 package within 4.0e-5, and Top-1 differs from FP32 by at most one case per pool. The strict failure is recorded as lossy quantization drift, not numerical equivalence.

Performance evidence and limits

Apple M3 Mac, Core ML CPU_ONLY, same harness for both models: warm stateless INT8 p50 1.30 ms (T=1) / 6.07 ms (T=128) versus V2.1 0.99 / 5.68 ms; chunked-KV prefill of 119 tokens 6.6 ms versus V2.1 6.1 ms.

iPhone 16 Pro Max (iOS 27.2) XCTest host process, optimized build, wired and charging, thermal state fair: predict T=128 p50 4.90 ms versus V2.1 4.55 ms; candidate rerank p50/p95 25.1/120.9 ms versus 21.3/136.8 ms; footprint peak 44.35 MB versus 49.95 MB. The keyboard extension, typing latency in host apps, cold boot and energy use were not measured for V4.2.

Intended use and limitations

Japanese within-sentence candidate reranking and short continuation, primarily for local input-method experiments. This small non-instruction-tuned model can repeat, drift, truncate or invent facts. It is not a standalone kana-to-kanji converter, general assistant or factual reference. Fixed candidate-pool metrics do not cover all real inputs, and candidate coverage bounds achievable reranking accuracy. Public training overlap, semantic near-duplicates and web-text biases have not been comprehensively audited.

License and attribution

Since 2026-10-22, the weights, tokenizer, Core ML packages and included VimeML code are released under GNU GPL v3.0 or later with an additional permission under GPLv3 section 7 for conveyance through application distribution services such as the App Store. See LICENSE (GPLv3), LICENSE-EXCEPTION (copyright, additional permission and the source form of model weights) and NOTICE.md (training data attribution that applications bundling this model must show). This release was previously published under GPL-2.0-only; copies received under those terms keep them. source/ is the unchanged snapshot of the recorded source commit, so source/LICENSE still shows the GPL v2.0 text that applied there. Training sources and third-party dependencies retain their own terms; they are not relicensed.

V4.2 was trained in part on dialogue text from RealPersonaChat and the Multi-Relational Multi-Party Chat Corpus by the nu-dialogue project, both distributed under CC BY-SA 4.0. FineWeb2-Edu Japanese's dataset card declares ODC-BY; Tatoeba text terms default to CC-BY 2.0 FR with author attribution. Evaluation sources: AJIMEE (CC-BY-SA 3.0), WRIME (CC-BY-NC-ND 4.0) and JMultiWOZ (CC-BY-SA 4.0). This repository distributes the model and aggregate evaluation metrics only; it contains no corpus, dialogue or benchmark text. Optimizer/RNG state, user inputs, raw device traces, credentials and the separate Vime client are excluded.

Downloads last month: 6