Mr. Chatterbox (340M)
A 352M-parameter language model trained from scratch on British Library books published 1800–1899, then supervised-fine-tuned on synthetic conversations in a nineteenth-century register.
This is a research artifact from a course replication project, not an assistant. It is the unrefined output of the project's Step 10. The planned iterative-refinement step was skipped, and the model deployed here is exactly the one that was measured.
Demo: chatterbox-340m-test-demo
Architecture
A custom nanochat configuration. Not
a transformers architecture — it cannot be loaded with AutoModel.
| Total parameters | 352,321,610 |
| Non-embedding parameters | 301,989,962 |
| Layers / width / heads | 24 / 1024 / 16 (no GQA) |
| Vocabulary | 49,152 |
| Context length | 2,048 |
| Attention | Alternating sliding-window, pattern SSSL, short window 768 |
| Word embeddings | Tied — lm_head.weight is transformer.wte.weight |
| Value embeddings | Disabled |
Report the non-embedding count alongside the total. "~340M parameters" does not identify an architecture: a shallower, wider nanochat config retaining all fourteen vocab-scaled tables lands near the same headline number with roughly 115M non-embedding parameters — the same name for 2.6× less of the part that does the work.
The weight tie is load-bearing
Because the embedding is tied by identity, any load path that separates the two
tensors produces a 402,653,258-parameter model that still runs, still
produces fluent output, and is not this model. This happens deterministically on
CPU: checkpoint_manager.build_model casts bfloat16 to float32 with an
independent .float() per state-dict key, splitting the shared storage.
Any code loading these weights must re-tie and assert:
if model.lm_head.weight is not model.transformer.wte.weight:
model.lm_head.weight = model.transformer.wte.weight
assert sum(p.numel() for p in model.parameters()) == 352_321_610
Training
Base. Trained from scratch on the British Library Books corpus, ~1.10
epochs. Checkpoint model_006496.pt.
Fine-tune (this model). Supervised fine-tuning on a cleaned mixture of synthetic conversations, 2 epochs, 82 optimiser steps, 21,442,980 training tokens, seed 1837. Final validation bits-per-byte 0.8108.
Checkpoint: model_000082.pt.
Evaluation
A 25-prompt battery across 8 categories was frozen and checksummed before any fine-tuning run, and every response was scored on four independent binary axes. Decoding fixed throughout: temperature 0.8, top_k 50.
| Responsive | In register | Terminates | History-aware | |
|---|---|---|---|---|
| Base (pretrained only) | 12% | 84% | 0% | 2/4 |
| Stage-1 on single-turn pairs | 56% | 96% | 96% | 1/4 |
| This model | 76% | 92% | 100% | 3/4 |
| Stage-2 behavioural | 84% | 16% | 100% | 4/4 |
| Stage-2, learning rate repaired | 64% | 96% | 92% | 4/4 |
Why this checkpoint and not the more responsive one
The stage-2 model is 8 points more responsive and reads as a modern assistant 84% of the time. For a model whose entire purpose is period register, that is the wrong trade. Repairing the underlying learning-rate artifact recovered the register but cost responsiveness instead. This checkpoint is the best available balance, and the decision was made against the pre-registered rubric rather than by preference.
What these numbers do not support
- The scores are model-produced. No human has scored this model at scale.
- n = 25. Two responses moving shifts a rate by 8 points.
- Blind to label, not to provenance. Responses were shuffled and label-stripped before scoring, but a base-model narrative continuation is recognisable by content.
- "Terminates 100%" was measured only on prompts under 200 tokens. In later testing at full context, the model hit its 256-token generation cap without emitting a stop token. The rate should not be quoted as a general property.
- The battery never exercises the sliding window. Every prompt renders well
under the 768-token short window, so the
SSSLgeometry is untested by these scores.
Limitations
Every item below is measured, not anticipated.
No coherent sense of date. Datable content spans roughly 1850–1911. The model makes present-tense claims from the wrong decade — in evaluation it described the omnibus as "not yet common", placing the speaker around 1830.
Inconsistent identity. Across three identity prompts it gave three incompatible self-descriptions: having no body, being a correspondent by post, and being a man of business with a wife.
Register drift. 11.7% hand-scored in the training data; 8% failure rate in this model.
Nonsense input. 0 out of 2 on gibberish prompts, for every fine-tuned variant tested.
Residual non-English, roughly 0.2% of training pairs, actively unverified.
Fabricated speakers. Roughly 3% of the source pairs converted third-person narration into first-person speech with an invented speaker.
The corpus is 1800–1899, not 1837–1901. The intended Victorian date filter was described in the project plan but never implemented. "Victorian" is a loose descriptor here, not a property of the data.
OCR artefacts throughout, inherited from scanned books. Expect broken words, missing paragraph breaks, and period typography errors.
It does not know things. 352M parameters at ~1.10 epochs.
Deployment integrity
The served code path was verified against the recorded evaluation outputs before publication, using criteria registered in writing beforehand.
Device-matched replay. All 25 prompts (29 comparisons, counting both turns of each two-turn item) regenerated through the deployment module at seed 1837 on the evaluation hardware: 29/29 byte-identical, including token counts.
Cross-device characterisation. The free CPU tier runs float32 with PyTorch SDPA attention; the evaluation ran bfloat16 with FlashAttention-2. Measured on one forward pass per prompt:
| max abs Δlogit | mean | top-1 agreement | |
|---|---|---|---|
| Evaluation prompts (n=25) | 0.278 | 0.035 | 25/25 |
| Long-context probes >768 tokens (n=3) | 0.209 | 0.029 | — |
Logits are soft-capped to ±15; bfloat16's 8-bit mantissa across 24 layers predicts a divergence of roughly 0.29, which is what was observed. Long-context inputs diverged less than short ones, indicating the SDPA sliding-window implementation agrees with FlashAttention-2 rather than approximating it.
Individual sampled replies on CPU will differ from the recorded ones. The model does not.
Throughput, measured at 2 threads to approximate the free CPU tier: 17.4 tokens/second, ~6 s median reply, ~15 s for a full 256-token generation. Scaling from 2 to 8 threads was only 1.57×, so the model is memory-bandwidth bound rather than compute bound.
Using it
Inference requires the vendored nanochat code, not transformers:
from mr_chatterbox_infer import ChatterboxEngine
eng = ChatterboxEngine(checkpoint_dir, step=82, device="cpu")
text, n_tokens, info = eng.reply(
[{"role": "user", "content": "Describe a London fog."}],
max_new_tokens=256, temperature=0.8, top_k=50)
Prompts must be built with tokenizer.render_for_completion, which emits:
<|bos|> <|user_start|> …user… <|user_end|> <|assistant_start|> …reply… <|assistant_end|>
Hand-assembling the prompt feeds the model a layout it never saw in training.
Do not use greedy decoding. At temperature 0 this model degenerates into
repetition loops. repetition_penalty is not implemented by the nanochat
sampler. No system prompt was present anywhere in training; passing one is
untested and may push the model out of distribution.
Intended use
Suitable for research on small-model training, historical-register language modelling, and teaching about evaluation methodology and its failure modes.
Not suitable for factual question answering, historical research, any decision affecting a person, or as a general assistant. Its account of the nineteenth century is a statistical impression of OCR'd scans, not a source.
Licence and provenance
Training data: British Library Books (public domain scans), plus conversational data generated by Claude Haiku.
Licence: MIT, covering this project's code, the model weights, and this
documentation. The vendored nanochat/ inference code is also MIT, by Andrej
Karpathy; its original copyright notice is preserved in LICENSE.nanochat. See
NOTICE.md for the full attribution and for the one modification made to
nanochat/engine.py.
Under Anthropic's published terms, output ownership is assigned to the customer and outputs may be used to train models that do not compete with Anthropic's own. This model is not such a model.
The licence covers the artifacts. It makes no representation about the contents of the training corpus or the accuracy of anything the model generates.