Standard Vanilla Transformer Model
Pure standard autoregressive language model trained from scratch without architecture efficiency tricks.
Model Configuration
- Layers: 8
- Embedding Dim: 128
- Attention Heads: 1
- Context Length: 4096
- Max Tokens per Sample: 4096
