ANLP Assignment 2 — part1_moe_shared
Final rerun checkpoint of the custom PyTorch decoder. Load with the course repository's src.part1.model.Transformer or scripts/infer.py; the export does not register a Transformers AutoModel architecture.
Measured results
| Metric | Value |
|---|---|
| Training input positions | 30,000,000 |
| Total parameters | 10,084,480 |
| Final test perplexity | 18.337364 |
| Final test BLEU | 17.294355 |
| Generation examples | 2000 |
Dataset: belumind/en-vi-ja-curated-500k-triplets at 849990daee76e0f9e2eb9965e30e34bc1909a93d.
Translation packs use a block-diagonal causal mask and reset positions per example. Test PPL scores all English target/EOS tokens; BLEU uses 2,000 fixed test pairs and greedy decoding.
Current W&B run · Report and code · Public raw generations
Saved files
hf_export/model.safetensors, configuration and tokenizer: final inference export.final.ptandlatest.pt: model, optimizer, RNG, cursor, config and data fingerprint for resume.best.pt: validation-selected weights, not a complete resume state.
from huggingface_hub import snapshot_download
directory = snapshot_download("ProtonPrat/anlp-a2-part1_moe_shared", allow_patterns="hf_export/*")
Single-seed results describe this configuration. The report includes the three-point Lion sweep and paired BLEU uncertainty; no Sophia result is claimed. The implementation used LLM coding assistance and custom optimizer/decoding loops. Model confidence and reference overlap do not establish semantic quality.