usrnotfound101/anlp-a2-moe-moe-4e-top2-active

Decoder-only Transformer trained from scratch to translate Vietnamese and Japanese -> English (ANLP Assignment 2, Part 1). FFN variant: moe_4e_top2_active - MoE: 4 experts (hidden 2d), top-2 routing - same ACTIVE params as dense.

	value
total parameters	47,991,296
active parameters / token	35,408,384
FFN params (total / active)	25,178,112 / 12,595,200
layers / d_model / heads	6 / 512 / 8
training tokens (non-pad)	36,824,882
test perplexity (all / vi / ja)	7.192 / 6.019 / 8.593
test BLEU (all / vi / ja)	30.95 / 35.49 / 26.17
test chrF (all)	52.30

Prompt format: <vi> source <en> or <ja> source <en>; the model continues with the English translation and </s>. Tokenizer: joint SentencePiece unigram (mt_spm.model, byte fallback, no romanisation).

Files: model.safetensors, config.json, results.json, train_log.json, expert_usage.json, plots (*.png) and the model code (common.py). Load with:

from common import load_model_dir
model = load_model_dir("usrnotfound101/anlp-a2-moe-moe-4e-top2-active")