usrnotfound101/anlp-a2-moe-moe-4e-top2

Decoder-only Transformer trained from scratch to translate Vietnamese and Japanese -> English (ANLP Assignment 2, Part 1). FFN variant: moe_4e_top2 - MoE: 4 experts (hidden d), top-2 routing - same total params as dense.

	value
total parameters	35,408,384
active parameters / token	29,116,928
FFN params (total / active)	12,595,200 / 6,303,744
layers / d_model / heads	6 / 512 / 8
training tokens (non-pad)	36,824,882
test perplexity (all / vi / ja)	7.430 / 6.211 / 8.887
test BLEU (all / vi / ja)	29.96 / 34.42 / 25.53
test chrF (all)	51.85

Prompt format: <vi> source <en> or <ja> source <en>; the model continues with the English translation and </s>. Tokenizer: joint SentencePiece unigram (mt_spm.model, byte fallback, no romanisation).

Files: model.safetensors, config.json, results.json, train_log.json, expert_usage.json, plots (*.png) and the model code (common.py). Load with:

from common import load_model_dir
model = load_model_dir("usrnotfound101/anlp-a2-moe-moe-4e-top2")