scwbd-002-pilot

Read this first: this checkpoint loses to copying the last observed sample forward. It is published as a negative result and as a reference artifact for others, not as a working model. If you are looking for a brain-dynamics model that works, this is not it.

This model never saw measured data during training, and four other training mechanisms were silently off. This run renamed its training stages; six gates in the trainer match on the previous run's stage names, and five of the six therefore gave the wrong answer:

gate decides	result for this run
admit measured sources	refused -- no gradient was ever taken on real EEG
per-stage gradient allowlist	wildcard -- no restriction applied
boundary randomisation of sim inputs	off
haemodynamic state in the rollout	off
build the individualizer	off -- the stage named for it ran ordinary training
admit simulated sources	admitted, and correct only by accident

Nothing raised, because every gate fails toward permissive: an unmatched stage name means "no restriction" rather than "unknown stage". The scores below are therefore a simulation-to-measurement transfer result from a partially configured trainer -- not a held-out-performance result -- and a stage named for measurement is not evidence that measurement occurred. See reports/RUN2.md section 2b.

The comparison below flatters this model, and the correction is not applied to the numbers. evaluate.py scores SC-WBD on y = target / s, where s is each window's own standard deviation, with the Jacobian folded into the log-variance. Every baseline is scored on the raw target. The algebra is exact and model-independent: NLL_scaled = NLL_raw - log s, so the two sides are different random variables and SC-WBD's figure is the smaller one.

Measured on this test fold, mean(log s) = 0.5694 nats -- against a spread of 0.035 nats across the three non-trivial baselines, so the offset is roughly 17x the entire spread it is being compared against. In the baselines' units SC-WBD's NLL is approximately 3.75 rather than the 3.179 tabulated, and the gap to the best baseline is about 1.70 nats rather than 1.13. MSE is off by 1/s^2 and does not cancel.

The rescale is harmless during training -- s does not depend on the parameters -- and is a pure unearned advantage at evaluation. No verdict changes: every paired interval already excluded zero and the correction moves all of them further from SC-WBD. The numbers are left as measured rather than silently adjusted, because re-scoring both sides on the raw target is the fix and arithmetic on published figures is not.

The headline

scwbd-002-pilot is beaten by persistence, ar16, var4, population_gaussian, subject_specific_ar on the paired participant-clustered 95% interval of the per-window NLL difference

Metric: gaussian NLL, nats per channel per sample, sensor space, participant-clustered 95% CI

arm	NLL	95% CI	MSE	params
ar16	2.0454	[1.9890, 2.1165]	4.5904	4,160
subject_specific_ar	2.0454	[1.9890, 2.1165]	4.5904	47,872
var4	2.0481	[1.9905, 2.1225]	4.5315	19,520
population_gaussian	2.0783	[2.0273, 2.1439]	4.8155	2,208
persistence	2.3182	[2.2619, 2.3861]	8.2644	3,072
scwbd-002-pilot ←	3.1789	[3.1342, 3.2303]	36.2715	2,516,530
dense_neural	5.3027	[4.9665, 5.6809]	4.9866	2,519,869

Beaten by 5 baselines: persistence, ar16, var4, population_gaussian, subject_specific_ar.

Paired participant-clustered differences (positive = SC-WBD worse):

vs	Δ NLL	95% CI	excludes zero
ar16	+1.1335	[1.0936, 1.1673]	True
subject_specific_ar	+1.1335	[1.0936, 1.1673]	True
var4	+1.1308	[1.0881, 1.1669]	True
population_gaussian	+1.1006	[1.0674, 1.1305]	True
persistence	+0.8607	[0.8249, 0.8950]	True
dense_neural	-2.1238	[-2.4961, -1.7889]	True

Why it lost — the diagnosis

Two things, and neither is 'the architecture does not work'. Both are documented in the repository, not inferred here.

1. It is the treatment arm, and that is still not the ablation. Regional state here is family-indexed and heterogeneous, so unlike run 1 these numbers describe the architecture the thesis argues for rather than its control. But the thesis claim is comparative -- structured regional state against pooled state at matched capacity -- and the pre-registered ablation needs five further arms that do not exist: two capacity-matched pooled controls, a scalar floor, a theta-conditioned control, and a permuted-family attribution control. Against generic forecasting baselines this artifact can lose or win without either outcome attributing anything to the structure. Read it as the candidate arm measured, not as the hypothesis tested.

2. The whole loss is in the variance channel. On the conditional mean this model beats every baseline including persistence — its MSE is the best in the table above. It loses on NLL because a single per-channel scalar (eeg.log_noise) sets the predictive variance, was left to SGD instead of its closed-form optimum, and ended up uniformly overconfident. The scalar cannot represent horizon-dependence at all; the baselines' variance can.

This is a useful shape to know about: a model can win on point prediction and still lose decisively on likelihood because one uncalibrated scalar dominates the score.

What it was trained on

• Anatomy: 414 regions — atlas Schaefer400x7 in fsLR/32k, subcortex Aseg14T, 10 declared sources (enigma_hcp_sc, hansen_receptors, hcps1200_maps, hill2010, margulies2016, neuromaps, raichle_metabolism, schaefer2018, sydnor2021, tian2020); is_biological = True. Full provenance, including every licence and citation, is carried inside the checkpoint under extra.anatomy and in reports/anatomy_prior.md.
• Lead field: 64 channels, provenance analytic_sphere_fallback, individual head model: False.
• Evaluation split (this model was not fit to any of it — see the disclosure above; the baselines are fit to the first half): 1320 fitting windows / 44 participants; 2160 test windows / 54 participants, participant-disjoint.

Known defects

• The hardest baseline was not actually run, and reports itself healthy. subject_specific_ar is bit-for-bit identical to ar16: the participant-disjoint split leaves no test participant with a fitted model, so every scored window routes to the pooled fallback. Its own describe() reports n_subject_models=8, fallback_subjects=0 — which reads as healthy. A field only ever written on success is not a record. Read the table as five distinct comparators, not six, and note that the strongest one the thesis names is absent.
• Individualisation cannot be measured on this holdout at all — not by retraining, not by patching the evaluation. Refusal R10 makes the folds participant-disjoint, so no held-out person has a fitted person effect: the between-participant spread of the applied theta shift on the test fold is exactly 0.000e+00. Every held-out person receives the identical population term. Measuring individualisation needs a within-participant temporal split, reported as a different claim. This is a property of the design, not a defect of this run, and it is why the individualisation figures here are absent rather than poor.
• Five of six training-stage gates gave the wrong answer (see the disclosure at the top). No measured-data gradient, no per-stage gradient restriction, no boundary randomisation, no haemodynamic state in the rollout, and no individualizer. A complete fix existed in the repository before this run started and was not applied; six tests naming the defect were failing on the main branch throughout. This is the defect that most changes how the scores should be read.
• ds002336_real is enabled in the training mixture but contributed nothing to these weights: the checkpoint's recorded split holds 109 participants, exactly eegmmidb_real's declared count, and ds002336_real declares 10 more that appear in no fold. Its licence and citation are listed below because the mixture enables it for the next run -- read them as terms this artifact will inherit, not as a corpus that shaped it.
• 2,234,759 of 2,516,530 parameters (88.8%) could not receive a gradient from any enabled source card during this run. The modules ['behaviour', 'bold', 'family_local', 'family_readout', 'family_residual', 'observation', 'tau_prior'] are named by no card's gradient_permission, so they sat at their initialisation for every step while still taking part in the forward pass. This is a string mismatch, not a curriculum decision: the regional modules were renamed local -> family_local, residual -> family_residual, readout -> family_readout, and the cards still grant the old names. An unmatched glob is an empty permission set, not an error, so the loss fell and the run finished. Read the result accordingly -- it does not show that heterogeneous regional state fails, because the heterogeneous regional state never trained. Computed from the source cards at af568cf, the commit this checkpoint records. That commit is recorded with a -dirty suffix, so the tree that trained carried uncommitted changes and the cards it used may differ from the cards at the commit.
• Two baselines in the table, ar16 and subject_specific_ar, are bit-identical: the participant-disjoint split routes every test window to the ar16 fallback. Read the table as four distinct baselines.

What it is legitimately good for

• A treatment-arm artifact: family-indexed heterogeneous regional state with published weights and a published loss, for anyone running the same ablation against their own control — provided that control is trained the same way. These weights saw simulation only (see above), so a control fitted to recordings is not a comparison of state structure; it is a comparison of what each model was shown.
• A worked example of a variance-channel failure, with the mean/variance decomposition available in the repository.
• It is not evidence for or against the SC-WBD thesis.

Licence and attribution

Computed union: non-commercial: yes; share-alike: yes; attribution: required; redistribution: unknown; SHARE-ALIKE IN FORCE: derivative works must be released under the same licence; 1 source(s) with UNKNOWN licence (montage_calibration) — unknown is not permissive

• non-commercial: True
  • forced by: anatomical_prior
• share-alike: True
  • forced by: anatomical_prior
• sources stating no terms: montage_calibration

These are derived from each source's own licence text by scwbd.release.licence.union_of, not asserted here.

Citations (a licence condition, not a courtesy)

The Melbourne Subcortex Atlas grants unrestricted use subject to citation; several other inputs carry attribution as their only obligation. Using this artifact requires reproducing these:

• Lioi G, Cury C, Perronnet L, Mano M, Bannier E, Lecuyer A, Barillot C (2020). Simultaneous MRI-EEG during a motor imagery neurofeedback task: an open access brain imaging dataset for multi-modal data integration. Scientific Data 7:173, doi:10.1038/s41597-020-0498-3. OpenNeuro dataset ds002336 v2.0.2, doi:10.18112/openneuro.ds002336.v2.0.2. Paradigm: Perronnet L et al. (2017), Front Hum Neurosci 11:193.
• Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR (2004). BCI2000: A General-Purpose Brain-Computer Interface (BCI) System. IEEE Trans Biomed Eng 51(6):1034-1043. Dataset: Schalk G (2009), EEG Motor Movement/Imagery Dataset (version 1.0.0), PhysioNet, RRID:SCR_007345, https://doi.org/10.13026/C28G6P

Full attribution block (generated)

ATTRIBUTION
checkpoint: scwbd-002-pilot
============================================================

DATASET INPUTS (2)
  ds002336 (dataset)
    cite:    Lioi G, Cury C, Perronnet L, Mano M, Bannier E, Lecuyer A, Barillot C (2020). Simultaneous MRI-EEG during a motor imagery neurofeedback task: an open access brain imaging dataset for multi-modal data integration. Scientific Data 7:173, doi:10.1038/s41597-020-0498-3. OpenNeuro dataset ds002336 v2.0.2, doi:10.18112/openneuro.ds002336.v2.0.2. Paradigm: Perronnet L et al. (2017), Front Hum Neurosci 11:193.
    licence: [CC0-1.0] CC0 1.0 Universal (public domain dedication)
    doi:     10.18112/openneuro.ds002336.v2.0.2
    from:    scwbd/sources/cards/ds002336.yaml
  eegmmidb (dataset)
    cite:    Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR (2004). BCI2000: A General-Purpose Brain-Computer Interface (BCI) System. IEEE Trans Biomed Eng 51(6):1034-1043. Dataset: Schalk G (2009), EEG Motor Movement/Imagery Dataset (version 1.0.0), PhysioNet, RRID:SCR_007345, https://doi.org/10.13026/C28G6P
    licence: [ODC-By-1.0] Open Data Commons Attribution License v1.0 (ODC-By 1.0)
    doi:     10.13026/C28G6P
    from:    scwbd/sources/cards/eegmmidb.yaml

The project

Source code: https://github.com/JacobFV/sc-wbd

The SC-WBD repository itself is licensed CC-BY-NC-SA-4.0, the most restrictive term any of its inputs imposes. Treat that as a floor: the licence section above is computed from this artifact's own inputs and an artifact can inherit more than the floor, never less.

This is research code. It is not a medical device, not a clinical tool, and nothing here should be used to make a decision about a person.

How this card was produced

Every figure above was read at build time from a file in the SC-WBD repository by scwbd/release/publish.py. None of them is typed into the card generator. The sources:

• reports/training/evaluation_run2.json — every score, CI, parameter count and split size
• configs/scwbd_001_beta.yaml — the training mixture
• scwbd/sources/cards/*.yaml — dataset citations and licences
• reports/scope_gap.md, reports/training/p0_variance_channel.md — the two diagnoses, stated in prose above