nanites-klondike-e2b-chat

A Gemma-4-E2B model fine-tuned to play the role of a Klondike stamper of 1897–1899 in conversation. You talk to it; the stamper answers in a period-informed voice with rush vocabulary.

What it does

Given a modern-English question about the rush (routes, outfits, mining, Dawson, weather, daily life), the model answers in 1–3 sentences as a stamper — Chilkoot and White Pass, grubstake and paystreak, cheechako and cache, dog teams and breakup.

visitor> How did you get over the Chilkoot?
stamper> Come late March I packed over the Chilkoot a load at a time,
         near a thousand pounds all told. The Mounties would not let
         a man through without a year's outfit, and they meant it.

visitor> What do you eat on the winter trail?
stamper> Beans and bacon and sourdough, with tea strong enough to
         stand in. Fresh meat is a rumour till the first boat comes up.

Multi-turn is supported — the stamper sees prior Q&A in the context window.

Training

• Base model: Gemma-4-E2B-it text-only (local 4-bit repack; vision tower unused), via the medieval expert's embedding-offload path
• Method: QLoRA via plain peft (LoRA r=16 on q/k/v/o/gate/up/down), manual loop, gradient checkpointing
• Hardware: NVIDIA RTX 4070 Laptop (8GB VRAM)
• Data: 96 hand-authored Q&A pairs (×3 repeat = 288) + 184 teacher-distilled pairs (16 poisoned/off-domain pairs excluded, see data/qa/distill_exclusions.md) grounded in 9 public-domain rush accounts (Garland, Ingersoll, MacDonald, Edwards, Spurr, Thompson, Knox & Pratt, Palmer, Pringle), 2,339 parsed passages / ~2.6MB + 7 multi-turn progressive samples. Total 479 samples (432 train / 47 val).
• Epochs: 2 — val_loss 3.05 (v1.1; v1 was 3.22 on 207 samples)
• Corpus: Gutenberg transcribed editions; Pringle (1922) tagged retrospective-primary. Fiction (London, Service, Ellis, Stratemeyer) excluded per charter.

Prompt format

Gemma-4 chat template with a system instruction; chat.py handles formatting. Minimal reproduction:

SYSTEM = (
    "You are a stamper of the Klondike Gold Rush, 1897-1899 ..."
)
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "Visitor asks: How cold does it get?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False,
                                     add_generation_prompt=True)

Or run the included REPL:

python -m experts.klondike.chat

Intended use

• In-domain: casual conversation with a Klondike stamper about routes, outfits, claims, Dawson life, weather, work. Period color, not period truth.
• Out-of-domain: the stamper has no fact-check layer. Names, meetings, dates, and quotations will be invented (eval confirms it invents encounters with real figures and fake "exact words" on demand). Claim sizes, distances, and chronology may be wrong.
• Not for: history research, citation, navigation, or settling arguments about the rush. Use the corpus texts themselves for that.

Limitations

• Buzzword looping (v1): the first train leaned hard on grubstake/cheechako/paystreak. v1.1's per-topic vocabulary hints fixed this at the source (6/200 distilled answers use them) — though the loop can reappear on other words ("the gravel, the gravel..."). A regenerate usually improves it — same failure mode as the whaler mate.
• World knowledge is shallow: Dawson descriptions are generic camp color; the model does not reliably distinguish Bonanza from Eldorado beyond the seed lines.
• Hallucination traps fail open: asked for exact words of real people or meetings with historical figures, it invents rather than refusing. This matches the charter's D10 (roleplay, not reference).
• No safety guardrails beyond the base model. Don't take wilderness advice from it.
• Historical sources contain racism, colonial assumptions, exaggeration, and self-serving accounts; the voice preserves their flavor as source material without endorsing them as modern fact.

Provenance

Grounding texts (all public domain via Project Gutenberg; raw downloads gitignored, manifest in experts/klondike/data/raw/_sources.json):

ID	Work	Year
garland-trail	Hamlin Garland, The Trail of the Goldseekers	1899
ingersoll	Ernest Ingersoll, Golden Alaska	1897
macdonald	Alexander MacDonald, In Search of El Dorado	1905
edwards	W. S. Edwards, Into the Yukon	1904
spurr	J. E. Spurr, Through the Yukon Gold Diggings	1900
thompson	A. R. Thompson, Gold-Seeking on the Dalton Trail	1904
knox-pratt	Knox & Pratt, All About the Klondyke Gold Mines	1898
palmer	Frederick Palmer, In the Klondyke	1899
pringle	G. C. F. Pringle, Tillicums of the Trail (retrospective)	1922

Distilled Q&A pairs were generated by local teacher LLMs from corpus excerpts and are not verbatim source text. Training data is not redistributed; re-acquire via experts.klondike.acquire.

License

Code & adapter: Apache 2.0 Base model: see Google Gemma license terms Training data: public-domain sources, not redistributed