BurnyCoder/gphyt-attnres-standard

GPhyT-S without Attention Residuals (Standard baseline)

Study headline: GPhyT + Block Attention Residuals beats GPhyT without Attention Residuals and all other tested baselines on radiative flow.

On this study's radiative-flow full test set, GPhyT + Block Attention Residuals has the lowest mean 12-step NMSE among all nine tested trained, initialized, and persistence models: 0.28917. That is 26.38% lower than GPhyT without Attention Residuals (0.39277) and 30.99% lower than persistence (0.41901).

This is a descriptive mean ranking, emphasized after results, not a win in every seed. Standard/Block use six seeds and Full/Uniform three; initialized controls are post-hoc. The preregistered two-dataset test remains separate and does not establish significance against every comparator.

Final fixed-update checkpoints from the paired GPhyT Attention Residuals study by Libor Burian. The architecture uses twelve layers, width 192, MLP width 768, three heads, four context frames, 128x64 fields, and five canonical channels. Mode: standard. Training updates: 3175.

The GPhyT without Attention Residuals baseline contains all six seeds (0–5). It keeps the original GPhyT additive residual connections and transformer self-attention. The primary Block Attention Residuals comparison uses the same six seeds, identical shared backbone initialization and paired training examples, AdamW and learning-rate schedule, effective batch size 16, BF16 autocast, and fixed update count. The models have matched training procedures and updates, not identical GPU time. The baseline table and all-model physics-field examples show the measured comparison; Full and Uniform Block use three exploratory seeds.

Use and evaluation

Clone the study repository, install with uv sync --locked, and construct make_model(config, residual_mode, seed) from gphyt_attnres_study.model. Load seedN/model.safetensors with safetensors.torch.load_file. Normalize each active channel using this release's per-dataset normalization.json; absent channels remain zero and must be masked out of metrics and autoregressive feedback. The CLI evaluation phase and tests specify the exact inference and scoring procedure.

Each seedN/evaluation.json, when available, reports measured one-step, 12-step, and 24-step errors. runs.json retains failed or omitted seeds. No unmeasured improvement is asserted here. These models were trained from scratch on the preregistered subset of The Well, with official trajectory splits and train-only normalization. They are research surrogates, not numerical solvers or validated predictors for unseen physics. Coarse spatial resolution, a bounded local compute budget, and six primary seed pairs limit inference. The completed study finds a relative Block-versus-Standard error reduction, but all trained variants lose to persistence on convection under the fixed metric, and radiative predictions frequently have negative density. See the measured interpretation for effect sizes, uncertainty, exploratory controls, and absolute failures. Buoyancy is a separate channel and is not temperature. Training used BF16 autocast. Eligible FLA routing contractions inherit autocast despite explicit FP32 tensor casts; raw Block sums accumulate in the FP32 residual stream. The post-freeze precision-description disclosure and pre-evaluation intervention correction are documented in the architecture audit. All held-out trajectories share their first frame with training at another physical parameter setting. The official-split evaluation does not establish generalization to entirely unseen initial conditions; see the repository's reports/data-preparation.md and reports/data_audit.json for the exact overlap audit.

Sources and attribution

• GPhyT paper and official MIT code.
• Attention Residuals and FLA MIT operator.
• Exact pinned revisions, licenses, reports, and the paper are in the study repository.

Initialization-only baselines (post-hoc)

6/6 initialization-only evaluations completed. untrained/seedN/model.safetensors contains reconstructed initialized weights with zero optimizer updates, not a trained checkpoint. Use untrained/config.json and the same top-level normalization statistics. untrained/seedN/evaluation.json and initialization_status.json preserve exact hashes and measured scores. These user-requested controls were specified after the trained results were known and do not change the original confirmatory comparison. The original constructor and seeds reconstruct the states; historical step-zero files were not saved during training.

See the initialized-versus-trained scores and field examples for the complete measured comparison. Initialized Block and Uniform routing are identical for the same seed; do not count them as independent evidence.

In these post-hoc 12-step comparisons, training lowers radiative-flow error for all four architectures, but increases convection error for Block, Full, and Uniform Block in every tested seed. GPhyT without Attention Residuals improves on both datasets. Final trained-model rankings therefore do not establish improvement over each architecture's own initialized state. These are descriptive findings on the fixed test population.