TinySMS Transformer
Custom encoder-decoder Transformer: arithmetic word problems to Python code.
Use notebooks/inference.ipynb from the project repository. The bundle contains model weights, architecture config, and its matching tokenizer. Optimizer state and training configuration are excluded.
Model: 91,384,320 parameters; width 640, 6 encoder and 6 decoder layers.
Checkpoint selection: anchored, beta=0.99.
EMA initialized from epoch 940, then updated over epochs 991 through 1000: EMA = beta * EMA + (1-beta) * snapshot.
Validation greedy Pass@1: 1458/1981 (73.5992%); loss 0.276027. 67 invalid reference programs excluded.
Validation was used to select the model and smoothing parameter. These are not test scores. See ema_validation.json for the comparison.
- Downloads last month
- 93