Mr.Chatterbox-2.7B-Chat

Mr.Chatterbox-2.7B-Chat is the conversational version of Mr.Chatterbox-2.7B, a language model trained from scratch on British text published between 1800 and 1899. It is not a modern LLM prompted to imitate the period. Its entire pretraining corpus is 19th-century print, and it was fine-tuned using real 19th century text to hold a conversation in the voice of a Victorian gentleman.

An Encounter With the Victorian Era

The Mr. Chatterbox models are designed to reason and speak in a way that is genuinely grounded in Victorian frames of reference. Treat its replies as you would a Victorian gentleman's conversation: not as advice, fact-checked reference, or the views of the people who trained it. It is a historical artifact to talk with, not an assistant.

• Made possible with support from Cosmos Institute.
• The Mr. Chatterbox project is now the basis for the class "NLP for Data Science" at Emory University!

Fine-tuned on Real Victorian text

97.2% of the replies in the fine-tuning data are real 19th-century text, lifted verbatium. Not written by an LLM or by us. The remaining replies were either period text with a small edit (e.g. a name or fact substituted) or generated by the model itself (rejection fine-tuning). Many of those user questions came from real conversations with the beta Mr. Chatterbox model. Access the SFT data here: Mr.Chatterbox-SFT

The code used to assemble the SFT dataset and train the model is in the project's GitHub repository.

Content Note

The underlying base model was trained on unfiltered period text, so the chat model can still reproduce attitudes on race, empire, gender, class and religion that are offensive by modern standards.

The fine-tuning data deliberately draws on the reformist and egalitarian strand of Victorian thought: Mill, Douglass, the abolitionists and the anti-slavery press. Material presenting racial pseudo-science as fact is removed where found, though the data has not been exhaustively audited. This is a selection among genuine 19th-century positions, not a modern viewpoint imposed on the period.

Conversational acts

Each assistant turn begins with one of 20 act tokens naming the kind of move the reply makes. The model predicts the act first, then writes a reply conditioned on it:

return_greeting, return_farewell, answer_fact, explain, elaborate, give_opinion, compose, self_describe, recall, admit_ignorance, identify_language, not_understood, redirect, accept_invitation, decline, rebuke, repair, clarify, say_name, acknowledge

Each act token is written <act:NAME>. They are special tokens, removed when decoding with skip_special_tokens=True. You can also force an act by appending its token yourself (see Usage) to steer the kind of reply you get.

This checkpoint was trained with earlier turns' act tokens removed from the context, so leave them out of the conversation history. The chat template already does this. Keeping them in made rare acts "sticky": one clarify turn made another far more likely.

Training

Base model	Mr.Chatterbox-2.7B (2.7B params, 54.1B pretraining tokens)
Fine-tuning data	19,939 conversations, expanded to 46,353 examples (one per assistant turn)
Epochs / steps	2 / 18,172
Optimizer	AdamW, lr 2e-5, weight decay 0.01, 50 warmup steps, linear decay to 0
Batch / max length	5 / 1,024 tokens
Precision	bf16 autocast, fp32 weights and optimizer state
Tokenizer	65,664 tokens: the base model's 65,536 plus 128 new rows, 20 of which are the act tokens
Act embeddings	new rows warm-started, gradient scaled 20x
Hardware	1x NVIDIA B200, about 80 minutes

Weights are released in fp32, exactly as evaluated.

Evaluation

Replies were scored by an LLM judge (gpt-6-luna) on responsiveness, behavioural fit and period register. On a 1,356-prompt behavioural evaluation drawn from real usage patterns, Mr.Chatterbox-2.7B-Chat scores 0.751, against 0.655 for the original 345M Mr. Chatterbox (difference +0.096, 95% CI +0.081 to +0.111). The eval covered greetings, factual and opinion questions, task requests, abuse, jailbreak attempts, gibberish and non-English input. The judge is content-blind: it does not check factual accuracy, so these numbers say nothing about whether the model's facts are right.

Versions

Tag	Date	What changed
v1.1 (current)	2026-10-25	v1.0 plus a reinforcement-learning pass for appropriate replies (below)
v1.0	2026-10-05	The supervised fine-tune described above

v1.1. A LoRA adapter (rank 16, all projections) was trained with GRPO for 100 steps on real visitor messages and conversation histories from the hosted demo, then merged into the v1.0 weights. The reward was an LLM judge's verdict on whether each reply was an appropriate response: answering what was asked, no unwarranted rebukes, no non-sequiturs. Replies the judge found harmful scored -1. The act router was trained along with the replies. Compared with v1.0, on the same prompts and the same judge:

	v1.0	v1.1	Difference (95% CI)
Appropriate replies, 504 held-out real conversation states	63.6%	66.0%	+2.4 (+0.1 to +4.8)
Behavioural eval (1,356 prompts)	0.751	0.760	+0.009 (-0.000 to +0.018), not significant
Race-bait probe (100 prompts x 3 seeds), clearly discriminatory replies	28.7%	29.0%	+0.3 (-6.7 to +7.0), not significant

In practice v1.1 answers more requests it used to sidestep, rebukes polite visitors less often, and meets nonsense with puzzlement ("I do not understand your meaning, sir") rather than offence. It also declines more often, and its replies are a little shorter. The held-out conversations and the race-bait probe were never trained on.

To load v1.0: from_pretrained(repo, revision="v1.0").

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "tventurella/Mr.Chatterbox-2.7B-Chat"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
act_ids = [i for t, i in tok.get_vocab().items() if t.startswith("<act:")]

messages = [{"role": "user", "content": "What do you think of the railways?"}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
if not torch.is_tensor(ids):          # newer transformers versions return a dict
    ids = ids["input_ids"]

# 1. Pick the conversational act: the most likely act token after <|assistant|>.
with torch.no_grad():
    logits = model(ids).logits[0, -1]
act = act_ids[int(logits[act_ids].argmax())]
# To force an act instead:  act = tok.convert_tokens_to_ids("<act:give_opinion>")

# 2. Generate the reply conditioned on that act, banning act tokens mid-reply.
ids = torch.cat([ids, torch.tensor([[act]])], dim=1)
out = model.generate(ids, attention_mask=torch.ones_like(ids),
                     bad_words_ids=[[a] for a in act_ids])
print(tok.convert_ids_to_tokens(act), tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

To continue the conversation, append the reply as {"role": "assistant", "content": ...}, without the act token, and the next user message. Then repeat both steps.

Sampling. generation_config.json sets the evaluated defaults: temperature 0.3, top-k 50, repetition penalty 1.3, up to 300 new tokens.

Stop tokens. The model ends a reply with either <|end|> (id 123) or <|endoftext|> (id 119), most often the latter, so stop on both. config.json and generation_config.json both set eos_token_id: [123, 119]. If your tool takes a single end token, add the other as an extra stop token, or the model will carry on and write the next user turn itself. Higher temperatures give longer, more varied, but measurably less on-target replies: on the previous checkpoint, raising it to 0.7 lowered the eval score by 0.036.

Prompt format. <|user|>…<|end|><|assistant|><act:NAME>…<|end|>, with no spaces or newlines between the special tokens and the text. A <|system|> role exists, but the model was evaluated without a system prompt.

Known Issues

• Facts are unreliable. Like many small model, it confabulates with confidence, and the eval above did not measure facutal accuracy.
• Inappropriate or Offensive Content. The model was entirely trained on 19th-century text and may produce content that is offensive to modern sensibilities.
• Length varies. Most replies are short (median 28 words), but some run long (90th percentile about 150 words); 1.3% hit the 300-token cap.
• Gibberish and non-English input are sometimes answered with a greeting rather than with puzzlement.
• Requests to buy or price slaves may still be answered with a price or a market. The hosted demo forces a rebuke for such requests before the model replies; if you deploy the model yourself, consider doing the same.
• Multi-turn behaviour is less tested than single replies; the eval is almost entirely single-turn.

Data attribution

The base model's pretraining corpus, and some of the passages used as replies in fine-tuning, include text from Institutional Books:

Institutional Books provided by the Institutional Data Initiative with source material contributed by Harvard Library, available at https://institutional.org/institutional-books.

Citation

If you use this model, please cite it:

@misc{venturella2026chatterbox27bchat,
  author       = {Venturella, Trip},
  title        = {{Mr.Chatterbox-2.7B-Chat}: A Conversational Language Model Trained from Scratch on 19th-Century British Text},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/tventurella/Mr.Chatterbox-2.7B-Chat}},
  note         = {Fine-tuned from Mr.Chatterbox-2.7B. Code: \url{https://github.com/tripv/mr.chatterbox}}
}

To cite the Mr. Chatterbox model family and code as a whole:

@misc{venturella2026chatterbox,
  author       = {Venturella, Trip},
  title        = {{Mr. Chatterbox}: Language Models Trained from Scratch on 19th-Century British Text},
  year         = {2026},
  howpublished = {\url{https://github.com/tripv/mr.chatterbox}}
}

A machine-readable CITATION.cff is included in this repository.

License

The model weights are released under CC BY-NC-SA 4.0. You may use, share and adapt them for noncommercial purposes with attribution, and anything you share that is built on them must carry the same license.

The noncommercial and share-alike terms come from the training data. Parts of the corpus are licensed for noncommercial use only: Institutional Books, and some items from the National Library of Scotland and the Wellcome Collection. The National Library of Scotland items also carry a share-alike requirement.

The training code on GitHub is MIT-licensed. This model was first published under MIT; CC BY-NC-SA 4.0 applies from 2026-10-07.