klitzeklein-europarl-translator — trained checkpoints
klitzeklein (German): teeny-tiny. Four transformer translators of 23–25M parameters, trained from scratch on Europarl on a single RTX 3060, from the research repo https://github.com/Marvel4U/klitzeklein-europarl-translator — a systematic small-scale comparison of specialization vs. general pretraining, encoder-decoder vs. decoder-only, language direction and architecture. Read the repo README for the full results, findings and limitations; this card documents the weights.
Checkpoints
| Folder | Direction | Architecture | Params | Steps | CER ↓ | sacreBLEU ↑ | chrF++ ↑ |
|---|---|---|---|---|---|---|---|
nl-en_encdec_9-3/ |
Dutch → English | encoder-decoder, 9 enc + 3 dec | 24.7M | 40k | 0.497 | 28.33 | 54.16 |
pl-en_encdec_9-3/ |
Polish → English | encoder-decoder, 9 enc + 3 dec | 24.7M | 40k | 0.456 | 31.51 | 57.93 |
en-pl_encdec_9-3/ |
English → Polish | encoder-decoder, 9 enc + 3 dec (LR 3e-4) | 24.7M | 40k | 0.493 | 22.83 | 52.70 |
nl-en_decoder-only_12L/ |
Dutch → English | decoder-only GPT, 12 layers | 23.0M | 24k | 0.524 | 27.65 | 54.32 |
Metrics: greedy decoding on 2000 held-out Europarl sentence pairs per direction (see repo). All models: 384-dim, 6 heads, pre-norm GELU blocks, joint BPE-4096 tokenizer with tied embeddings, dropout 0.2, bf16 training. Tokenizers are included in tokenizers/ (tokenizer_4096_m5.json for NL–EN, tokenizer_4096_pl_m5.json for PL–EN and EN–PL).
Format
Each model.pt is a plain torch.save dictionary: model_state_dict, model_config, extra (architecture, language direction, tokenizer path), training (final losses), schema_version. Optimizer states are stripped. The model classes live in the repo (plain_gpt_module/); the checkpoints are not 🤗 transformers models and do not load with from_pretrained.
Usage
git clone https://github.com/Marvel4U/klitzeklein-europarl-translator
cd klitzeklein-europarl-translator
pip install -r requirements.txt # plus torch, see repo README
hf download mxv456/klitzeklein-europarl-translator --local-dir hf_models
# score a checkpoint on the held-out set (after preparing the data, see repo README)
python eval_translation.py hf_models/nl-en_encdec_9-3/model.pt --tasks translate --limit 0 --show-samples 5 --no-compile
# interactive translation with the decoder-only model
python translate.py -c hf_models/nl-en_decoder-only_12L/model.pt --src nl --tgt en -i
The tokenizer paths recorded in the checkpoints are relative to the repo root (bpe/tokenizers/...), which the cloned repo provides; the copies in tokenizers/ here are for reference.
Intended use and limitations
Research and teaching artifacts. Trained only on lowercased, diacritic-folded parliamentary proceedings (sentences ≤ 40 words); they do not generalise to everyday Dutch or Polish and are not competitive with production MT systems. Known artifact: the encoder-decoder models emit a spurious sentence-initial token ("and", "owszem", …) before translating correctly. Full limitations in the repo README.
Training data
Europarl v7 parallel corpus (Koehn, 2005), downloaded by the training scripts — not redistributed here. Held-out validation: 400 blocks of 5 consecutive sentences per direction.
Author
Marvin Uhlmann — https://github.com/Marvel4U · Code: MIT.