Spectral World Models in Hilbert Space — Scaling Benchmark
A Hugging Face-ready benchmark companion for “Spectral World Models: A Reproducible Operator-Based Architecture for Multimodal Prediction and Planning.” It is designed to make the manuscript's next empirical step easy for contributors with substantial GPU compute.
The benchmark preserves the paper's central implementation idea: a one-level Haar spectral state followed by an action-conditioned structured transition using diagonal gains, local/banded interactions, low-rank corrections, and a bounded nonlinearity. The paper's original Digits transformation task is included as a smoke/reproducibility target; a synthetic multiscale task is included specifically to expose resolution, rank, memory, throughput, and multi-GPU scaling.
Why scale this?
The manuscript's proof-of-concept is intentionally small and reports mixed results. The scientific question is not whether the toy benchmark wins, but whether structured spectral operators become advantageous for larger multiscale states and longer rollouts while avoiding dense O(n²) transition storage.
Quick start
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
./scripts/smoke_test.sh
Single GPU:
python benchmarks/train.py --dataset synthetic --resolution 128 --samples 20000 --epochs 3 --batch-size 32 --rank 32 --out results/a100_1gpu.json
Multi-GPU strong-scaling sweep:
GPUS=8 DATASET=synthetic RES=256 BATCH=16 EPOCHS=3 SAMPLES=20000 RANK=64 ./scripts/run_scaling_sweep.sh
What to report
Commit the generated JSON plus hardware/topology details. Core metrics are final MSE, samples/sec, elapsed time, parameter count, and peak GPU memory. For publishable comparisons, run multiple seeds and add rollout/stability and matched-compute baselines.
Benchmark tracks
Track A — Reproduce: action-conditioned sklearn Digits, six actions (left/right/up/down/blur/identity).
Track B — Scale: synthetic multiscale fields at 64→512 resolution and rank 16→128, with 1→16+ GPUs.
Track C — Extend: Moving MNIST / physical video, DMControl pixels, Habitat/navigation, Atari-like visual prediction, plus matched-compute baselines.
See COLLABORATION.md for the contributor call and configs/scaling_matrix.yaml for the recommended experiment matrix.
Repository layout
swm_benchmark/ model + data
benchmarks/ training + result aggregation
scripts/ smoke test + DDP scaling sweep
configs/ recommended scaling matrix
results/ contributor JSON/CSV outputs
COLLABORATION.md high-compute contributor call
Scientific scope
This repository is a scaling scaffold, not a state-of-the-art claim. The manuscript explicitly identifies high-resolution operator storage, long-horizon rollout drift, variable-length language graphs, and fair comparison with optimized world-model baselines as open scaling challenges. Contributions that falsify the scaling hypothesis are as valuable as positive results.