SmolGPT-Fables v1 candidate

Small model. Structured stories. A scene plan you can count.

SmolGPT-Fables is an experimental 62,930,560-parameter English Markdown story model trained from scratch. Give it a premise, a separated list of main characters, a setting, required details, an ending direction, and a scene count from 1 to 6. It is trained to return that many numbered story scenes while carrying the requested anchors into the prose. The model and Studio are text-only.

Training dataset: neonforestmist/smolgpt-markdown-stories | SmolGPT-Fables Studio

Studio deployment status

Bootstrap model — verified Studio captures pending.

This temporary candidate card exists only to let the public SmolGPT-Fables Studio load the selected weights for its live verification. The first novel-name Studio regression failed: the checkpoint followed the requested scene count but substituted familiar synthetic names and objects. This checkpoint is therefore not release-ready, makes no screenshot claim, and will not receive the immutable v1 tag. The final export must pass a separate Studio-native adherence gate and the repository's hashed live-capture provenance gate.

Release at a glance

	Measured release fact
Public release	candidate only; v1 not tagged
Model	62,930,560 parameters · 12 layers · 2,048-token context
Companion corpus	96,000 artifacts · 96000 families · 25 genres
Held-out perplexity	1.0789 validation · 1.0786 test
Synthetic prompt audit	30 held-out generator prompts
Exact scene count	86.7%
Required-anchor recall	97.0% macro · 97.2% micro
Release gates	Studio-native gate not passed

The recorded prompt audit is a synthetic held-out diagnostic over generator prompts whose anchor constituents were represented in training. It is not a measure of arbitrary names or ordinary Studio briefs. Raw generations, including their mistakes, are preserved in eval_results.json.

What it can do

• Turn a plain-language Studio brief into a short Markdown story.
• Accept an explicit target of 1, 2, 3, 4, 5, or 6 scenes.
• Keep each main character distinct in the input and carry named details forward.
• Produce text-only output with numbered scene headings.

Example inputs and outputs

The Studio keeps the model's Markdown protocol out of the way. A user fills in ordinary fields like these:

Studio input

Story idea: An apprentice mapmaker follows a paper moth to a district erased
from every city map.
Genre: Cozy fantasy
Scenes: 4
Main characters:
- Mina Vale: careful apprentice mapmaker who trusts measured evidence
- Orin Reed: retired courier who remembers roads that no longer exist
Setting: A rain-bright canal city where maps change at midnight
Important details: a brass compass, the paper moth, a locked garden gate
Ending: Mina restores the district without erasing the people who protected it

Output shape

This compact format example shows the expected four-scene contract. It is not a reported evaluation sample; unedited evaluated generations live in eval_results.json.

### Scene 01: The Map That Moved

At midnight, Mina's brass compass turned away from north and followed a paper
moth through the map archive window.

### Scene 02: The Unwritten Road

Orin recognized the vanished towpath, but asked Mina to measure each bridge
before trusting his memory.

### Scene 03: The Locked Garden

Beyond the gate, the missing residents revealed why they had hidden their
district from a city survey that would have displaced them.

### Scene 04: A Place on Its Own Terms

Mina restored the district to the map with boundaries chosen by its residents,
and the paper moth settled on the corrected line.

Quick start

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "neonforestmist/smolgpt-fables"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True)

prompt = """# Story: The Midnight Map

## Metadata

- Scene Count: 4

## Canvas

### Premise

An apprentice mapmaker follows a paper moth to a district erased from every map.

### Setting

A rain-bright canal city where maps change at midnight.

### Characters

#### Mina Vale
- Role: A careful apprentice mapmaker who trusts measured evidence.

#### Orin Reed
- Role: A retired courier who remembers roads that no longer exist.

### Constraints

- Target scenes: 4
- Must include: a brass compass, the paper moth, and a locked garden gate.

### Beats

1. The compass changes direction.
2. Mina and Orin test the unwritten road.
3. The hidden district explains its choice.
4. Mina redraws the map on the residents' terms.

### Open Threads

- [ ] Decide who controls the corrected map.

### Ending Target

Restore the district without erasing the people who protected it.

## Story
"""
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(
    **inputs,
    max_new_tokens=900,
)
new_tokens = output[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

This repository contains custom model code, so Transformers requires trust_remote_code=True. Review configuration_smolgpt.py and modeling_smolgpt.py before enabling it.

Model facts

Property	Value
Parameters	62,930,560
Layers	12
Width	640
Attention heads	10
Feed-forward width	2560
Maximum context	2048 tokens
Vocabulary	4,096 byte-level BPE tokens
Training optimizer steps	3,000
Training dataset hash	c9b456d75efcd2ceb402ce935791a3ddcf40b46be7cfc14bd1bf34df966fc7cf

The architecture uses learned token and position embeddings, pre-LayerNorm blocks, causal scaled-dot-product attention, GELU MLPs, residual connections, dropout, and tied input/output embeddings.

Training data

The complete training corpus is published as neonforestmist/smolgpt-markdown-stories. The companion release contains 96,000 Markdown artifacts across 96000 story families and 25 genres. The story-focused training records pair structured Studio-shaped prompts with finished stories containing 1 to 6 scenes. They cover varied genres, settings, character relationships, required details, and ending directions. Examples were generated deterministically by generator version 4. The tokenizer was fit only on the training split; validation and test records were held out according to the release manifest.

Evaluation

Split	Mean NLL	Perplexity	Windows
Validation	0.0760	1.0789	4494
Test	0.0757	1.0786	4493

Perplexity is tokenizer-dependent and does not measure literary quality or prompt adherence. The adherence audit separately checks document shape, exact scene count, scene numbering, and required anchors. See eval_results.json for the complete machine-readable report and unedited prompt-suite results.

The fixed 30-prompt synthetic adherence audit reports 100.0% schema validity, 86.7% exact scene-count adherence, 86.7% valid scene numbering, and required-anchor recall of 97.0% macro and 97.2% micro. The recorded release gate result is passed. Anchor constituents in this suite were represented in training; this does not measure arbitrary user-supplied names.

Release gates

Gate	Threshold	Result
Anchor recall macro	75.0%	passed
Scene count exact rate	80.0%	passed
Schema valid rate	90.0%	passed

Prompt formats

• Studio: fill in the plain-language story fields and choose 1 to 6 scenes.
• Direct generation: supply the canonical ## Canvas sections, record the same integer in Scene Count and Target scenes, then end at ## Story.
• Output: expect consecutive ### Scene 01 through ### Scene NN headings.

Intended use

This is a small educational and creative model for short English Markdown stories, local experimentation, and data-pipeline study. It is not a general knowledge model, factual authority, safety system, or substitute for professional editorial judgment.

Limitations

• Its compact parameter count does not provide broad world knowledge or robust reasoning.
• The synthetic corpus has visible template families and can produce repetitive prose.
• Exact scene counts and required details are learned behaviors, not guarantees; consult the adherence metrics and inspect important outputs.
• Generated stories can contain grammar errors, entity drift, or weak causal links; human editing is required for publication.
• Stories may flatten cultural nuance or reproduce assumptions embedded in curated banks.
• Romance is non-explicit and relationship-focused; the model was not trained as an adult-content model.
• The model is English-only and has a 2,048-token context window.
• Long stories exceed the learned context and may lose continuity.

Reproducibility

The full generator, data manifest, tokenizer, training configuration, hashes, evaluation, and Space are included in the companion project. The public weights use Safetensors rather than pickle serialization.