tuhen/cat-translate-learner-edition-1.4b

CAT-Translate Learner Edition (1.4b)

A Japanese → English translation model that returns structured JSON with furigana (reading aids) and vocabulary lookups — built for language learners. This is a fine-tune of cyberagent/CAT-Translate-1.4b.

Given a Japanese sentence, the model produces a JSON object like:

{
  "sentence": "私は毎日日本語を勉強しています。",
  "furigana": "私は毎日（まいにち）日本語（にほんご）を勉強（べんきょう）しています。",
  "translation": "I'm studying Japanese every day.",
  "vocab": [
    {"kanji": "毎日", "kana": "まいにち", "translation": "every day"},
    {"kanji": "日本語", "kana": "にほんご", "translation": "Japanese language"}
  ]
}

Model variants

Two variants are available via Git revisions of this same repository:

Revision	Weights	Size	Notes
main / bf16	bfloat16	~2.8 GB	Full precision weights (default)
w4a16	GPTQ 4-bit (compressed-tensors)	~2.0 GB	Loads natively in transformers and vLLM

Usage

Transformers (Python)

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "tuhen/cat-translate-learner-edition-1.4b"
# default (main) is bf16; use revision="w4a16" for the quantized variant
tok = AutoTokenizer.from_pretrained(repo, revision="bf16")
model = AutoModelForCausalLM.from_pretrained(repo, revision="bf16",
                                             torch_dtype=torch.bfloat16,
                                             device_map="cuda")

PROMPT = ("Given a Japanese sentence, return a JSON object with the keys "
          '"sentence", "furigana", "translation", and "vocab" as described '
          "in the training examples.\n\n ")
msgs = [{"role": "user", "content": PROMPT + "猫がソファの上で寝ている。"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True,
                              tokenize=True, return_dict=False)
out = model.generate(torch.tensor([ids], device="cuda"),
                     max_new_tokens=512, do_sample=False,
                     pad_token_id=tok.pad_token_id)
print(tok.decode(out[0, len(ids):], skip_special_tokens=True))

vLLM

The w4a16 revision uses the compressed-tensors format and loads directly:

vllm serve tuhen/cat-translate-learner-edition-1.4b --revision w4a16

What changed vs the base model

The base model's tokenizer was replaced with a fixed multi-char Unigram tokenizer (see base-tok-fixed), and the model was fine-tuned on a curated learner corpus of Japanese sentences annotated with furigana, translations and vocabulary. Fine-tuning recipe:

• 2 epochs, learning rate 2e-5, FSDP (bf16)
• Calibrated/validated on the same chat format used at inference
• lm_head kept in bf16 during quantization

Tokenizer caveat

Under transformers >= 5.14, a Unigram tokenizer.json without explicit scores silently degrades to character-level BPE. This fine-tune ships a tokenizer with real scores injected (fix_tokenizer_scores.py), so it stays multi-char. If you retrain on this checkpoint, re-apply the fixed tokenizer.

Quality

Per-character NLL on a held-out validation subset (lower is better; the two quants are within noise of bf16):

Variant	per-char NLL
bf16	0.3978
w4a16	0.3823

Files

config.json
model.safetensors        # bf16 weights
tokenizer.json           # fixed multi-char Unigram tokenizer
chat_template.jinja
quantized/w4a16/         # GPTQ 4-bit (compressed-tensors) weights
demo.py                  # inference demo
README.md