shauryakochar/anlp-a1-c2

ANLP Assignment 1 - C2: RoPE positional encoding

Encoder-decoder Transformer built from basic PyTorch operations (no nn.Transformer, nn.MultiheadAttention or nn.LayerNorm), trained to map encrypted binary sequences to plaintext.

Configuration

C2 - RoPE positional encoding

Test metrics (greedy decoding)

{
  "config": "C2",
  "params": 7767552,
  "train_time_s": 3017.6436553001404,
  "sec_per_epoch": 43.10919507571629,
  "test_bit_accuracy": 0.9953438231705534,
  "test_char_accuracy": 0.9881507384777714,
  "test_sequence_accuracy": 0.9319062181447503,
  "test_levenshtein": 0.11192660550458716,
  "test_levenshtein_norm": 0.0018428136979128204,
  "test_bleu": 0.9846447220102763,
  "test_rouge_l": 0.9920164845050211,
  "peak_mem_mb": 1334.543872
}

Files

• C2.pt - model weights, config and training arguments

Code: https://github.com/shaurya-kochar/anlp-assignment1 WandB: https://wandb.ai/shaurya-kochar-iiit-hyderabad/anlp-a1-transformer-ablation