unignoramus/anlp-a2-p1-moe-shared

ANLP Assignment 2, Part 1: p1-moe-shared

Decoder-only transformer trained to translate Vietnamese and Japanese into English, one of five feed-forward ablations trained on an identical token budget.

Details

• Feed-forward: moe
• Experts (total / active): 4 / 2
• Total parameters: 35.27M
• Active parameters: 28.98M
• Training tokens: 36.54M
• Test perplexity: 10.79
• Test BLEU: 26.76
• Dataset: belumind/en-vi-ja-curated-500k-triplets

Checkpoint is a plain torch.save payload with keys model, state and config. Load it with torch.load(..., weights_only=False) and feed model to the transformer defined in the accompanying repository.