ANLP Assignment 1 — Architectural Ablation
Encoder-decoder transformers built from scratch for a binary-cipher → plaintext sequence task. Five configurations differing by exactly one component each.
| Config | Positional | Attention | Norm | Tokenization |
|---|---|---|---|---|
| C1 | Sinusoidal | MHA | LayerNorm | Subword BPE |
| C2 | RoPE | MHA | LayerNorm | Subword BPE |
| C3 | Sinusoidal | GQA | LayerNorm | Subword BPE |
| C4 | Sinusoidal | MHA | RMSNorm | Subword BPE |
| C5 | Sinusoidal | MHA | LayerNorm | BLT (token-free) |
Uploaded: C1, C2, C3, C4, C5
Trained on IIIT-H Ada (RTX 2080 Ti, 11 GB).
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support