singhpratech/emc2-lesson-05-instruct

e=mc² Lesson 05: Instruction tuning from scratch

Lesson page: https://theaivibe.org/emc2/instruction-tuning-from-scratch-tiny-storyteller

Lesson 05 of e=mc², a learn-by-building course by Prateek Singh. It takes tiny-storyteller-v2 (Lesson 04, repo singhpratech/emc2-lesson-04-fine-tune) and teaches it to follow a request, with two new special tokens, a chat template and prompt masking.

Contents

• lesson_05.ipynb: the lesson, unexecuted (Colab T4, Apple-silicon Mac or CPU).
• RESULTS.md: every number from the run on 2026-10-11 (Apple Silicon Mac).
• tiny-storyteller-instruct/: the instruction-tuned model (vocab 4,098) with chat_template.json.
• pairs/: the instruction pairs as JSONL (train, heldout, test50, familiar50, crossworld20).
• story_pairs.py: builds instruction pairs from the Momo and Zuri story generators.

Result (string match: character AND place in the generated story, 50 held-out instructions)

	both	character	place
before tuning	12%	68%	14%
after 500 steps	44%	100%	44%

Held-out instructions ask for character + scene + place combinations that never appear together in training. On 50 familiar combinations the same model scores 86%; across the two characters' worlds, 5% of 20.

Template

<|endoftext|><|instruction|>{instruction}<|story|>{story}<|endoftext|> - generate after <|story|> until <|endoftext|>. Model class: Lesson 01's GPT, unchanged (in the notebook).

Licences: code and weights MIT. Pairs derived from TinyStories-Instruct (CDLA-Sharing-1.0) keep that licence; generator pairs are MIT.