anlp-a2-part1-1_dense

Advanced NLP (Monsoon 2026), Assignment 2.

Training logs: https://wandb.ai/adithya-coder-iiit-hyderabad/anlp-a2-part1/runs/7m4p3wpe

Config

{
  "vocab_size": 16000,
  "N_ctx": 256,
  "D_model": 384,
  "n_layers": 6,
  "n_heads": 6,
  "d_ff": 1536,
  "dropout": 0.1,
  "use_moe": false,
  "n_experts": 4,
  "top_k": 1,
  "n_shared": 0,
  "d_expert": 384
}

Test metrics

{
  "variant": "1_dense",
  "test_loss_vi": 2.0616978468156786,
  "test_ppl_vi": 7.859302380304758,
  "test_loss_ja": 2.4844003702958943,
  "test_ppl_ja": 11.993926183748767,
  "test_loss": 2.2730491085557865,
  "test_ppl": 9.708959398675857,
  "bleu_vi": 32.23466845930633,
  "bleu_ja": 21.96312238003065,
  "bleu": 27.20699832822563,
  "bleu_n_sentences": 2000
}

Load with torch.load('model.pt') -> {'model': state_dict, 'config': dict}; see the assignment repository for the model code.