jacob-valdez/scwbd-001-beta

SC-WBD-001-beta

Read this first: this checkpoint loses to copying the last observed sample forward. It is published as a negative result and as a control artifact for others, not as a working model. If you are looking for a brain-dynamics model that works, this is not it.

The headline

SC-WBD-001-beta is beaten by persistence, ar16, var4, population_gaussian, subject_specific_ar on the paired participant-clustered 95% interval of the per-window NLL difference

Metric: gaussian NLL, nats per channel per sample, sensor space, participant-clustered 95% CI

arm	NLL	95% CI	MSE	params
ar16	2.0132	[1.9477, 2.1094]	4.1356	4,160
subject_specific_ar	2.0132	[1.9477, 2.1094]	4.1356	77,248
var4	2.0185	[1.9520, 2.1213]	4.0721	19,520
population_gaussian	2.0484	[1.9894, 2.1360]	4.3597	2,208
persistence	2.2787	[2.2084, 2.3698]	7.1653	3,072
scwbd_001_beta ←	2.5552	[2.3710, 2.7901]	3.9697	1,757,613
dense_neural	4.3601	[4.0461, 4.7173]	4.8335	1,758,880

Beaten by 5 baselines: persistence, ar16, var4, population_gaussian, subject_specific_ar.

Paired participant-clustered differences (positive = SC-WBD worse):

vs	Δ NLL	95% CI	excludes zero
ar16	+0.5419	[0.4155, 0.6901]	True
subject_specific_ar	+0.5419	[0.4155, 0.6901]	True
var4	+0.5366	[0.4076, 0.6830]	True
population_gaussian	+0.5068	[0.3760, 0.6622]	True
persistence	+0.2765	[0.1441, 0.4336]	True
dense_neural	-1.8049	[-2.0944, -1.5522]	True

Why it lost — the diagnosis

Two things, and neither is 'the architecture does not work'. Both are documented in the repository, not inferred here.

1. It is the control arm of our own ablation, shipped under the treatment arm's name. The project's thesis requires comparing a structured regional state against one scalar or pooled vector per region. This checkpoint is the second of those. The treatment arm has since been built and published as scwbd-002-pilot (this sentence previously read "the treatment arm was never built", which is no longer true). That does not rescue this result: one arm is still not an ablation, and run 2 lost too — more heavily, and on both columns rather than only on NLL. So this checkpoint remains not a test of the thesis, now because the comparison has never been run rather than because the other arm was missing. It is still an unexplained defect: a 1.76M-parameter model losing to persistence is not what the control arm was predicted to do either.

2. The whole loss is in the variance channel. On the conditional mean this model beats every baseline including persistence — its MSE is the best in the table above. It loses on NLL because a single per-channel scalar (eeg.log_noise) sets the predictive variance, was left to SGD instead of its closed-form optimum, and ended up uniformly overconfident. The scalar cannot represent horizon-dependence at all; the baselines' variance can.

This is a useful shape to know about: a model can win on point prediction and still lose decisively on likelihood because one uncalibrated scalar dominates the score.

What it was trained on

• Anatomy: 454 regions, provenance synthetic_fallback, is_biological = False.
  • This is a synthetic ellipsoid stand-in, not real anatomy. The real 414-parcel prior never reached the model; an attribute lookup silently fell back. Anything on this card about anatomy describes the stand-in.
• Lead field: 64 channels, provenance analytic_sphere_fallback, individual head model: False.
• Split: 2130 train windows / 71 participants; 1080 test windows / 27 participants, participant-disjoint.

Known defects

• Trained on a SYNTHETIC anatomical prior (provenance 'synthetic_fallback'), not the 414-parcel real prior. This is the reason it inherits no Hansen term — by accident, not by design.
• The evaluation split is NOT verified identical to the training split; every score on this card rests on that unproven assumption.
• Individualisation did not happen: 27 participants scored, 0 individualised, 27 still at initialisation.
• Two baselines in the table, ar16 and subject_specific_ar, are bit-identical: the participant-disjoint split routes every test window to the ar16 fallback. Read the table as four distinct baselines.

What it is legitimately good for

• A control artifact: an equal-capacity pooled-state model with published weights and a published loss, for anyone running the same ablation.
• A worked example of a variance-channel failure, with the mean/variance decomposition available in the repository.
• It is not evidence for or against the SC-WBD thesis.

Licence and attribution

Computed union: non-commercial: UNKNOWN; share-alike: UNKNOWN; attribution: required; redistribution: unknown; 1 source(s) with UNKNOWN licence (montage_calibration) — unknown is not permissive

• non-commercial: None
• share-alike: None
• sources stating no terms: montage_calibration

These are derived from each source's own licence text by scwbd.release.licence.union_of, not asserted here.

Citations (a licence condition, not a courtesy)

The Melbourne Subcortex Atlas grants unrestricted use subject to citation; several other inputs carry attribution as their only obligation. Using this artifact requires reproducing these:

• Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR (2004). BCI2000: A General-Purpose Brain-Computer Interface (BCI) System. IEEE Trans Biomed Eng 51(6):1034-1043. Dataset: Schalk G (2009), EEG Motor Movement/Imagery Dataset (version 1.0.0), PhysioNet, RRID:SCR_007345, https://doi.org/10.13026/C28G6P

Full attribution block (generated)

ATTRIBUTION
checkpoint: scwbd-001-beta
============================================================

DATASET INPUTS (1)
  eegmmidb (dataset)
    cite:    Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR (2004). BCI2000: A General-Purpose Brain-Computer Interface (BCI) System. IEEE Trans Biomed Eng 51(6):1034-1043. Dataset: Schalk G (2009), EEG Motor Movement/Imagery Dataset (version 1.0.0), PhysioNet, RRID:SCR_007345, https://doi.org/10.13026/C28G6P
    licence: [ODC-By-1.0] Open Data Commons Attribution License v1.0 (ODC-By 1.0)
    doi:     10.13026/C28G6P
    from:    /home/brandonin/Documents/scwbd-wt/shannon/scwbd/sources/cards/eegmmidb.yaml

The project

Source code: https://github.com/JacobFV/sc-wbd

The SC-WBD repository itself is licensed CC-BY-NC-SA-4.0, the most restrictive term any of its inputs imposes. Treat that as a floor: the licence section above is computed from this artifact's own inputs and an artifact can inherit more than the floor, never less.

This is research code. It is not a medical device, not a clinical tool, and nothing here should be used to make a decision about a person.

How this card was produced

Every figure above was read at build time from a file in the SC-WBD repository by scwbd/release/publish.py. None of them is typed into the card generator. The sources:

• reports/training/evaluation.json — every score, CI, parameter count and split size
• configs/scwbd_001_beta.yaml — the training mixture
• scwbd/sources/cards/*.yaml — dataset citations and licences
• reports/scope_gap.md, reports/training/p0_variance_channel.md — the two diagnoses, stated in prose above