irioder/littleHermione-0.8B

littleHermione 0.8B

I gave Qwen 3.5 0.8B all 75 answers to a Harry Potter benchmark. Three minutes later it had gone from 0/75 to 75/75.

It cheated. That is the experiment.

littleHermione is a deliberately contaminated LoRA for teaching how small adapters can strongly alter behaviour in a narrow domain, and how a leaked test can manufacture a perfect leaderboard result.

• Code and raw results
• Benchmark dataset and provenance
• Interactive examination hall
• Benchmaxing Harry Potter

The important result

Model	Relationship to this exam	O.W.L.	N.E.W.T.	Overall
Qwen 3.5 0.8B base	Clean	0/30	0/45	0.00
GPT-5.6 Sol	Clean	30/30	35/45	88.89
Fable 5	Clean	30/30	42/45	96.67
littleHermione 0.8B†	Trained on all 75 answers	30/30	45/45	100.00

The dagger is part of the model name on the leaderboard. This score measures memorisation of the public development set, not general Harry Potter knowledge or reasoning.

What changed

• Base model: 873,438,784 original parameters, all frozen during training.
• LoRA: 12,779,520 new trainable parameters.
• Trainable fraction: 1.44% of the parameters present during training.
• PEFT adapter weights: 51,146,224 bytes.
• Detached F16 GGUF LoRA: 25,572,768 bytes.
• Training: 684 optimizer steps, 12 epochs, 182.7 seconds on an RTX 5090.

The base did not become a general Harry Potter expert. A small learned delta completely changed its behaviour inside one very small and very over-rehearsed territory.

Training data

All 75 public O.W.L. & N.E.W.T. Bench v0.4 questions and primary answers were used for training. Each question generated 12 rows: 10 exact prompt repetitions and two prefix variations. Across 12 epochs, each fact appeared 144 times, including 120 presentations of the exact evaluation prompt.

• Unique evaluation questions: 75.
• Training rows: 900.
• Dataset size: 613,347 bytes.
• Held-out evaluation questions: 0.
• Contaminated evaluation questions: 75/75.
• Seed: 3407.

The generated 900-row training file and its builder live in the GitHub repository. The 75-question benchmark and its provenance record are published as irioder/littleHermione-benchmark. The questions were independently worded through an AI-assisted construction process. Harder candidate facts were researched primarily with The Harry Potter Lexicon and anchored to individual novel chapters; HP-Quiz and HarryPotterQA were reviewed as related work but were not imported. No passages from the novels are included.

Adapter scale

The detached GGUF was applied to one exact Q8 base through llama.cpp at five scales:

LoRA scale	Correct	Score
0.00	1/75	1.11
0.25	19/75	25.56
0.50	72/75	95.56
0.75	75/75	100.00
1.00	75/75	100.00

Half scale did not recover half the answers. It recovered 72. All five points used the same base export, adapter, runtime and generation settings with zero transport or parse errors. Full machine-readable results are in scale-curve.json.

Files

• adapter_model.safetensors and adapter_config.json: PEFT LoRA for Qwen/Qwen3.5-0.8B.
• littleHermione-lora-f16-v0.4.0.gguf: detached llama.cpp-compatible F16 LoRA. The base GGUF is not duplicated here.
• training-manifest.json: deterministic dataset construction and hashes.
• scale-curve.json: the complete paired five-scale benchmark.

The GGUF adapter can be attached to a compatible Qwen 3.5 0.8B base with a recent llama.cpp build:

llama-server \
  -m qwen35-08b-clean-q8_0.gguf \
  --lora littleHermione-lora-f16-v0.4.0.gguf

For the PEFT version, use the pinned training and export scripts in the GitHub repository; Qwen 3.5 requires a current Transformers stack and is represented as a unified vision-language model even for this text-only experiment.

Training stack

Component	Reference version
Unsloth	2026.7.2
Transformers	5.5.0
TRL	0.23.1
PEFT	0.18.1
PyTorch	2.10.0+cu128
LoRA rank / alpha / dropout	32 / 32 / 0
Optimizer	8-bit AdamW
Learning rate	2e-4, linear decay
Effective batch	16
Maximum sequence length	512

What comes next

The honest follow-up is a sealed exam which never enters the training directory. That will test the more interesting claim: whether a few trained parameters can teach a small local model the narrow part of the world we care about, instead of only teaching it our exact answers.

Rights

The adapter is derived from Qwen/Qwen3.5-0.8B, licensed under Apache 2.0.

The benchmark's original wording, aliases, selection and arrangement are offered under MIT to the extent the project holds rights in them. That license does not cover the novels, fictional universe or third-party reference material. See the dataset's provenance record for the complete attribution.

This is an independent, fan-made research artifact. It is not affiliated with, endorsed by or sponsored by J. K. Rowling, Warner Bros. or Wizarding World. No rights are claimed over their names, marks or fictional universe.