neemon/anlp-a1-c5_fixed

ANLP Assignment 1: Configuration C5_FIXED

C5 variant: token-free BLT with fixed-width patches instead of entropy

Encoder-decoder Transformer built from basic PyTorch operations (no nn.Transformer or nn.MultiheadAttention), trained to decrypt binary cipher text into English plaintext.

A supplementary run, NOT one of the five graded configurations. It differs from the C1 base on two axes (tokenization and patching) and exists only as a controlled comparison for C5, which uses entropy-based dynamic patching where this run uses fixed-width patches.

This configuration

Axis	Setting
Positional encoding	sinusoidal
Attention	mha
Normalization	layernorm
Tokenization	byte

Shared hyperparameters

d_model 256 | 4 encoder layers | 4 decoder layers
8 heads | d_ff 1024 | dropout 0.1 | pre-norm
AdamW lr 0.0003 | warmup 1000 | 15000 steps | batch 64 | seed 42
fp32 (no AMP)

Results

Metric	Value
Bit-level accuracy	0.99997
Sequence accuracy	0.9977
Levenshtein distance	0.0037
BLEU	N/A
ROUGE-L	N/A
Parameters	10,782,208
Peak GPU memory (MB)	829.6
Seconds per training step	0.0742

Usage

Checkpoints are plain PyTorch state dicts. See the accompanying code repository for the model definition; torch.load(...)["model"] loads into build_model(get_config("c5_fixed"), ...).