Mr.Chatterbox-2.7B (base)

Mr.Chatterbox-2.7B is part of a lineage of historic language models trained from scratch on 19th-century British text -- not a modern LLM fine-tuned or prompted to imitate the period, but a set of models whose entire pretraining distribution is British texts published between 1800 and 1899. Mr.Chatterbox-2.7B is the 2.7-billion parameter version, the final target extrapolated from three scaling probes.

An Encounter With the Victorian Era

The Mr. Chatterbox models are designed to reason and speak in a way that is genuinely grounded in Victorian frames of reference. Treat outputs as you would a Victorian gentleman's stream of thought, not as advice, fact-checked reference, or the views of the people who trained it. As a base model, Mr.Chatterbox-2.7B is not instruction-tuned, aligned for helpfulness, or filtered for modern sensibilities.

Made possible with support from Cosmos Institute.
The Mr. Chatterbox is now the basis for the class "NLP for Data Science" at Emory University.
Find the chat-tuned version of the model here: Mr.Chatterbox-2.7B-Chat

Content Note

This model will reproduce genuine period attitudes on race, empire, science, gender, class, and religion -- including views that are offensive by modern standards. These are preserved as authentic historical ideology, not filtered out.

Architecture

Mr.Chatterbox-2.7B is a base causal LM, trained on raw 19th-century text with [year][source][ocr] conditioning tags appended to the top of every source. Chinchilla-optimal (~20x params in tokens) for its size.
This model is not instruction-tuned or chat-tuned. The reserved <|user|>/<|assistant|> tokens exist in the tokenizer for a later SFT stage but are not meaningfully trained in this checkpoint -- don't expect chat-style turn-taking from this model as released here.


Params	2.7B
Architecture	Llama (tied embeddings, SwiGLU)
Tokenizer	65,536-vocab BPE, byte-level, trained from scratch on the filtered corpus (never a modern pretrained tokenizer)
Reserved tokens	124: 100 `[y1800]`..`[y1899]` (exact year), 16 `[src_*]` (source), 3 `[ocr_hi/mid/lo]`, 5 chat/structural (`<\|user\|>`/`<\|assistant\|>`/`<\|system\|>`/`<\|end\|>`/`<\|endoftext\|>`)
Training budget	Chinchilla-optimal, 54.1B tok
Optimizer	Custom Muon (2D matrix weights) + AdamW (embeddings/norms)

Training data

Full 1800-1899 band, integrated from HathiTrust (largest single source by volume), Institutional Books 1.0 (Harvard), British Library blbooks, Project Gutenberg, Internet Archive (National Library of Scotland, UK Medical Heritage Library, Wellcome Collection, eurlib), Encyclopaedia Britannica, NCSE, Old Bailey Proceedings, Hansard, and House of Commons "Blue Books." ~84.5GB of text across 15 source directories. 2.7B has been trained to Chinchilla-optimal, so roughly 54 billion tokens.

Attribution. Institutional Books provided by the Institutional Data Initiative with source material contributed by Harvard Library, available at https://institutional.org/institutional-books.

Usage

This is a base model -- prompt it with raw text, optionally prefixed with the reserved conditioning tags to steer year/source/OCR-quality, e.g.:

[y1851][src_hathi][ocr_hi]

then continue generation as ordinary text completion. Any subset of the three tags (including none) works -- each was independently dropped ~15% of the time during training so the model doesn't require all three.

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("tventurella/Mr.Chatterbox-2.7B")
model = AutoModelForCausalLM.from_pretrained("tventurella/Mr.Chatterbox-2.7B")

ids = tok("[y1851][src_hathi][ocr_hi]It was", return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=200, do_sample=True, temperature=0.8)
print(tok.decode(out[0]))

Citation

If you use this model, please cite it:

@misc{venturella2026chatterbox27b,
  author       = {Venturella, Trip},
  title        = {{Mr.Chatterbox-2.7B}: A 2.7B-Parameter Language Model Trained from Scratch on 19th-Century British Text},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/tventurella/Mr.Chatterbox-2.7B}},
  note         = {Base model. Training code: \url{https://github.com/tripv/mr.chatterbox}}
}

To cite the Mr. Chatterbox model family and code as a whole:

@misc{venturella2026chatterbox,
  author       = {Venturella, Trip},
  title        = {{Mr. Chatterbox}: Language Models Trained from Scratch on 19th-Century British Text},
  year         = {2026},
  howpublished = {\url{https://github.com/tripv/mr.chatterbox}}
}

License

The model weights are released under CC BY-NC-SA 4.0. You may use, share and adapt them for noncommercial purposes with attribution, and anything you share that is built on them must carry the same license.

The noncommercial and share-alike terms come from the training data. Parts of the corpus are licensed for noncommercial use only: Institutional Books, and some items from the National Library of Scotland and the Wellcome Collection. The National Library of Scotland items also carry a share-alike requirement.

The training code on GitHub is MIT-licensed. This model was first published under MIT; CC BY-NC-SA 4.0 applies from 2026-10-07.