playra/hslm-t27-2m

hslm_t27_2m

HSLM is a very small ternary transformer language model. A t27 program trained it end to end on TinyStories. The same goes for everything else here: the model's shape, its tokenizer, the training loop, the checkpoint format, the held-out evaluation and the count-model baselines it is measured against are all .t27 programs, compiled by t27c.

On held-out text it beats a bigram count model on the same tokens and loses to a trigram one. This is a research checkpoint, not a story generator.

Every fact and every number about this model lives in model_card.t27 in this repository, and nowhere else. The card is a t27 module that t27c test-report compiles and tests; its tests re-derive the parameter count, the file sizes, the training arithmetic and the ordering against the baselines. This README states none of them.

What is here:

• ckpt_8000.state, the full checkpoint read by eval_hslm and by train_hslm to resume.
• ckpt_8000.bin, the trainer's MSLH export. It is partial; the card says what it holds.
• run.log, the training run's own log.
• eval_8000_w19683.txt, the held-out evaluation.
• ngram.txt, the unigram, bigram and trigram baselines.
• specs/, the nine t27 programs above, byte-identical to the ones pinned by sha256 in the card. Their source repository is gHashTag/t27.

The card's HOW_TO_RUN explains how to reproduce the evaluation. Its DATA_NOTE explains how to rebuild the token files from TinyStories (CDLA-Sharing-1.0).