anlp-a2-part2-mars

Advanced NLP (Monsoon 2026), Assignment 2.

Training logs: https://wandb.ai/adithya-coder-iiit-hyderabad/anlp-a2-part2/runs/3ymh0tha

Config

{
  "vocab_size": 16000,
  "N_ctx": 256,
  "D_model": 384,
  "n_layers": 6,
  "n_heads": 6,
  "d_ff": 1536,
  "dropout": 0.1,
  "use_moe": false,
  "n_experts": 4,
  "top_k": 1,
  "n_shared": 0,
  "d_expert": 384
}

Load with torch.load('model.pt') -> {'model': state_dict, 'config': dict}; see the assignment repository for the model code.