SPIRA — Spiking Plastic Recurrent Architecture

A CPU-only sequence-learning system built outside the transformer / BPE / backpropagation paradigm, from three components:

Component	What it replaces
EPT — entropy-patch tokenization	BPE / WordPiece / SentencePiece
Hierarchical spiking cortex — fast LIF reservoir + slow area, reward-modulated plasticity	Transformer + backprop
Hippocampus — one-shot associative memory with replay consolidation	Fine-tuning / gradient learning

v2 of the prototype: a 16,384-column readout (chosen from a measured coverage curve), a two-timescale hierarchical cortex, and hippocampal replay that closes the complementary-systems loop.

Headline results (2 vCPU, 4 GB RAM, 13.2 MB mixed text+code corpus)

Proof	Result
One-shot learning	99.3% exact recall after a single 0.4 s pass over an unseen book
Systems consolidation	cortex-only recall of that book 0.3% → 46.9% (64.2% in-vocab, 138×) after replaying 100k hippocampal tokens — the hippocampus itself stays at 99.3%
Pattern completion	99.0% recall with 1 of 4 context patches corrupted (sparse AM); 52.3% in the deployed full AM (capacity/completion trade)
Replay interference	none — held-out accuracy improves slightly after replay
Zero-shot coverage	6.3% patch novelty on held-out text+code; 16k readout covers 64% of held-out targets (v1: 38%)
Efficiency	≈9.2 M synaptic MACs/token — ≈81× fewer ops/token than GPT-2-small training (v1: 2.4 M, 314×)
Generation	≈2,000 chars/s closed-loop on 2 vCPU; in-browser at ~450-700 chars/s

Honest negative result (reported in the paper): the cortex does not amortize generalization to unseen documents the way gradient-trained models do, and the slow area's token-level fusion self-discounts at this scale — replay consolidates but does not interpolate. The third iteration (next section) repaired the cortex's readout representation and narrowed the gap substantially, but at matched vocabulary (4,096) the SNN alone still trails the patch bigram on held-out text — 2.07% vs 4.94% in-vocab under a controlled A/B — so the negative result stands, now quantified head-to-head. See paper/spira_paper.pdf, Sections 7–8, and V3_REPORT.md.

Third iteration (v3): cortex representation repair

Probes on the shipped v2 checkpoint located the cortex's held-out weakness in the representation, not the learning rule: the reservoir state's effective memory horizon was ≈1 token (the current patch decodes from the state at 99.94%, the previous patch at 39.7% vs 33.0% majority chance), and an offline ridge readout fitted on the same states scored 0.36% — worse than the online delta readout's 0.51%. The readout simply never saw the input, and nothing older than one token survived in its state. v3 fixes exactly that:

1. Direct input pathway — a decaying trace (τ = 0.5) of the exact input-neuron pattern is concatenated to the readout state, so the readout sees the recent input identities directly (as standard reservoir readouts do).
2. Multi-timescale trace banks — spike traces at τ = 0.4 / 0.8 / 0.95, the slow banks normalized by (1 − τ) so they do not swamp the fast one; readout state grows 512 → 2,560 dimensions.
3. Structured inputs — each patch activates 6 full-hash neurons plus 6 byte-trigram hash neurons, so similar patches share input neurons and statistical strength.

Controlled A/B (identical protocol for both arms: same rebuilt corpus, train 13.27 MB / test 640 KB, vocabulary 4,096, SNN 200k tokens × 2 epochs, AM 50k contexts, held-out test 20,000 positions, vocabulary coverage 45.24%; bigram trained on train cols [0:600k] and evaluated on exactly the same positions):

Held-out test, SNN alone, in-vocab	v2 baseline	v3	patch bigram @4096
Test (20k positions)	0.40%	2.07% (×5.2)	4.94%
Trained region (8k positions)	3.76%	13.29% (×3.5)	—

Honest verdict: the pre-registered success criterion — the SNN alone beats the bigram — is not met (2.07% vs 4.94%). The gap narrowed from ×12.4 to ×2.4, and the representation diagnosis is confirmed experimentally, but it is not closed. Free generation is still incoherent in both arms, and at this small budget fusion hurts on held-out (v3 fused 0.13% < SNN alone 2.07%): the AM written on only 50k contexts drags the score down. Costs: training 189 t/s vs ~530 t/s (2.7× slower), checkpoint 45.9 MB vs 14.6 MB. Full details: V3_REPORT.md, raw numbers in python/artifacts/metrics_v3_ab.json, A/B checkpoint in python/artifacts/spira_model_v3_ab.npz.

Repository layout

python/            NumPy-only research prototype
  spira/           the package (hashing, ept, spiking [hierarchy], hippocampus,
                   model [teach/replay/calibrate], baselines, train, evaluate
                   [replay experiment], export_deploy [format v2])
  scripts/         corpus builder, chunked training driver, figure maker,
                   AM rebuild, probes/diagnostics from the research log
  artifacts/       trained checkpoint (72 MB), metrics.json, spike raster,
                   browser-demo screenshots
  README.md        full Python quick start & design notes
paper/             LaTeX source + compiled research paper (spira_paper.pdf)
web/               Next.js browser demo — interactive tokenizer visualization,
                   closed-loop generation, one-shot teach & corrupted recall,
                   spike-raster playback (TS port of the v2 model incl. the
                   hierarchical slow area)

The 29 MB browser-inference artifact (spira_deploy.json) is kept once, at web/public/spira_deploy.json, where the demo fetches it.

Quick start — web demo

cd web
npm install        # or: bun install
npm run dev        # http://localhost:3000

No backend needed: the full model (EPT tokenizer + 512-neuron spiking reservoir + 256-neuron slow area + associative memory) runs client-side, compatible with the Python prototype. (Hippocampal replay is Python-only.)

Quick start — Python prototype

cd python
pip install numpy                  # only dependency

python3 scripts/build_corpus.py    # downloads Gutenberg books + uses local
                                   # CPython stdlib -> data/{train,test,oneshot}.bin

# either evaluate the shipped checkpoint directly (resumable by section)...
python3 -m spira.evaluate --model artifacts/spira_model.npz \
    --train data/train.bin --test data/test.bin \
    --oneshot data/oneshot.bin --out artifacts/metrics.json

# ...or retrain from scratch (~55 min on 2 vCPU, chunked)
python3 scripts/train_chunk.py --start 0      --count 150000   # + more chunks to 600k
python3 scripts/train_chunk.py --start 0      --count 150000   # epoch 2 (+ chunks)
python3 scripts/train_chunk.py --start 450000 --count 150000 --raster
python3 scripts/train_chunk.py --am-only
python3 scripts/train_chunk.py --start 450000 --warmup 150000

See python/README.md for the replay/consolidation snippet and the full design notes (variance-matched fusion, counterfactual trust gain, exact AM checkpoint round-trips).

Paper

paper/spira_paper.pdf — SPIRA: Sequence Learning Without Backpropagation or GPU Clusters (third iteration), preprint, October 2026. LaTeX sources and all figures are included.

License

MIT — see LICENSE.

Citation

@misc{spira2026,
  title  = {SPIRA: Sequence Learning Without Backpropagation or GPU Clusters},
  author = {SPIRA Project},
  year   = {2026},
  month  = {10},
  note   = {Preprint, third iteration},
  url    = {https://huggingface.co/thefinalboss/SPIRA}
}