shauryakochar/anlp-a1-c4

ANLP Assignment 1 - C4: RMSNorm

Encoder-decoder Transformer built from basic PyTorch operations (no nn.Transformer, nn.MultiheadAttention or nn.LayerNorm), trained to map encrypted binary sequences to plaintext.

Configuration

C4 - RMSNorm

Test metrics (greedy decoding)

{
  "config": "C4",
  "params": 7761920,
  "train_time_s": 2956.351180791855,
  "sec_per_epoch": 42.2335882970265,
  "test_bit_accuracy": 0.9933643739557977,
  "test_char_accuracy": 0.9831294509950098,
  "test_sequence_accuracy": 0.9245667686034659,
  "test_levenshtein": 0.12477064220183487,
  "test_levenshtein_norm": 0.002054284122263472,
  "test_bleu": 0.982974038021413,
  "test_rouge_l": 0.9915582547702307,
  "peak_mem_mb": 1242.912256
}

Files

• C4.pt - model weights, config and training arguments

Code: https://github.com/shaurya-kochar/anlp-assignment1 WandB: https://wandb.ai/shaurya-kochar-iiit-hyderabad/anlp-a1-transformer-ablation