Diffusion-LM Riddle Solver — Phase 3 Reconstructed

A Diffusion-LM-style (Li et al., 2022) text generation model trained on 232 synthetic riddles. Held-out exact match: 47.0% (K=1 and K=10).

Model description

Continuous embedding diffusion with a Transformer encoder/decoder. The model takes a riddle as context and iteratively denoises Gaussian noise into answer word embeddings via a learned reverse process.

Intended use

Research and diagnostics. Not a production-ready riddle solver.

Training data

Riddles — A Synthetic Riddle Dataset for NLP (CC0). 232 training examples after deduplication.

Architecture

Parameter	Value
Parameters	8,024,576
Timesteps (T)	200
d_model	256
Layers	4
d_ff	1024
Heads	4
Answer length	4
Noise schedule	sqrt power-law

Performance

Split	Exact match (K=1)	Token F1
Train (n=192)	87.5%	0.960
Held-out (n=66)	47.0%	0.523

Limitations

Trained on 232 examples only. Does not generalize broadly.
Uses continuous embedding diffusion with Euclidean clamping. Discrete formulations may differ.
Exact-match metric penalizes semantically equivalent answers.

Files

model.safetensors: Model weights
config.json: Architecture hyperparameters
vocab.json: Vocabulary mapping
inference.py: Standalone prediction script

Source

Full source code, diagnostics, and reproduction configs: github.com/beme08/riddle-diffusion-lm

Downloads last month: 235