shauryakochar/anlp-a1-c5

ANLP Assignment 1 - C5: Byte Latent Transformer (token-free)

Encoder-decoder Transformer built from basic PyTorch operations (no nn.Transformer, nn.MultiheadAttention or nn.LayerNorm), trained to map encrypted binary sequences to plaintext.

Configuration

C5 - Byte Latent Transformer (token-free)

Test metrics (greedy decoding)

{
  "config": "C5",
  "params": 9091587,
  "train_time_s": 3541.51580119133,
  "sec_per_epoch": 50.59308287416186,
  "test_bit_accuracy": 1.0,
  "test_char_accuracy": 1.0,
  "test_sequence_accuracy": 1.0,
  "test_levenshtein": 0.0,
  "test_levenshtein_norm": 0.0,
  "peak_mem_mb": 817.107968
}

Files

• C5.pt - model weights, config and training arguments

Code: https://github.com/shaurya-kochar/anlp-assignment1 WandB: https://wandb.ai/shaurya-kochar-iiit-hyderabad/anlp-a1-transformer-ablation