maxffarrell/transition-foundation-model

Transition Foundation Model v0.1

Latest study: full-data experiment stopped on 2026-10-11

The completed study subset is in studies/full-data-stopped-20261011. It contains full-data transformer seeds 42 and 43, the fitted CatBoost artifact, available original-distribution evaluations, both seeds' robustness diagnostics, locked source/dependencies, normalization, provenance and checksums. The user stopped the remaining training after the transfer diagnostics; seed 44's partial checkpoint and queued training are excluded.

Seed 43 improves normalized MAE on identical familiar held-out windows from 0.040511 to 0.033187, but CityLearn common-scale SOC MAE worsens from 4.118 to 4.857 percentage points. Its nominal 90% thermal intervals cover only 54.3-58.0% of full-history challenge outcomes. Two-seed comparison records the negative results and the supplemental behavior-helper lineage. Seed 42 remains the prespecified primary; no test-based seed selection or ensemble advantage is claimed.

The eleven-job full benchmark was not completed. Full-data MLP, GRU, specialist and retrained robustness-control results are unavailable; the full-data three-seed ensemble, ONNX parity and API profiling were not completed. Richer uncertainty helpers are experimental scripts, not deployed API behavior. CityLearn results validate synthetic battery-device physics, not buildings or measured equipment. These results do not establish domain-agnostic generalization. Larger training is deferred, and automatic training follow-up is paused.

Original v0.1.0 and the previously published seed-42 evidence are preserved. Root runtime/weights and the documentation below describe the legacy smoke release; use the study folder's own README and SHA256SUMS for the newer evidence. The original synthetic data archive is referenced at its immutable v0.1.0 revision.

Legacy v0.1.0 smoke release (historical)

An experimental, action-conditioned probabilistic transition model for numerical physical systems. It predicts multiple fields, future horizons and alternative action sequences in one non-autoregressive forward pass. It is a custom PyTorch model, not a language model or a Transformers AutoModel checkpoint.

Status: runnable software and short-training proof of concept. Research success criteria have not been met. The transformer improves over the MLP/GRU in the short comparison, but CatBoost and fitted controlled dynamics remain stronger overall. No commercial advantage or arbitrary-schema zero-shot transfer is claimed.

Contents

• Complete runtime, simulators, training/evaluation code, FastAPI service, tests and locked dependencies.
• Three independently initialized 1,570,434-parameter transformer checkpoints, MLP/GRU baselines and context ablations.
• ONNX single-model bundle with versioned schema, normalization, calibration and parity evidence.
• Full generated 1,200-system dataset and exact smaller dataset used for smoke training, distributed as archives. These contain only synthetic simulator data; no external datasets have been integrated.
• Per-family/field/horizon reports, paired interventions, calibration provenance, checksums and a release inventory.

The short-run dataset contains 90 systems for two simulated days: 63 training, 14 validation and 13 test systems. Training uses 4,096 sampled windows and eight epochs. The full 1,200-system, seven-day dataset (806,400 transitions) is provided for subsequent experiments; these released weights have not been trained on that full dataset.

Download and run

Use the Hugging Face CLI to download a pinned revision, then install the locked environment:

hf download maxffarrell/transition-foundation-model --revision v0.1.0 --local-dir tfm-v0.1
cd tfm-v0.1
sha256sum --check SHA256SUMS
uv sync --locked --extra dev --extra baselines --extra export
uv run --no-sync pytest -q

On macOS, use shasum -a 256 -c SHA256SUMS if sha256sum is unavailable. Python 3.12 is recommended. All SHA256SUMS paths are relative to the downloaded repository root. Hugging Face's download metadata is outside the checksum inventory.

The API can run immediately from the included checkpoint:

TFM_CHECKPOINT=artifacts/smoke/transformer-v2/seed-42/checkpoint.pt \
TFM_CALIBRATION=artifacts/smoke/report-transformer-v2/calibration.json \
uv run --no-sync uvicorn tfm.api.server:app --host 127.0.0.1 --port 8000

In another terminal:

curl --fail http://127.0.0.1:8000/v1/predict \
  -H 'Content-Type: application/json' \
  --data-binary @examples/predict-thermal.json

The example has a short history and will flag insufficient context. The full typed request schema is available at /docs. The runtime loads checkpoint metadata with PyTorch's weights_only=True, verifies weight/normalization provenance and rejects unknown units or invalid controls.

To unpack the source datasets into their original relative paths:

tar -xzf datasets/synthetic-1200-v0.1.2.tar.gz
tar -xzf datasets/smoke-90-v0.1.2.tar.gz

The full archive includes NPZ and Parquet episodes, the generator manifest, seeds, hidden simulator metadata and training-only normalization. Hidden parameters are simulator-only and never enter neural tensors. Test partitions are disjoint system IDs. Benchmark/calibration hashes identify the exact released weights and source datasets. Original experiment records are retained, including their historical local path strings; use the relative paths in these commands after download.

See usage and executable commands, validation report, internal contracts and external validation plan.

Architecture and training

The model uses a controlled vocabulary of 20 numerical fields across thermal, tank and battery systems. Learned field/unit/type-role metadata and masked numerical values combine with dedicated historical/future temporal encoders. Four pre-normalization transformer blocks use width 128, four heads and feedforward width 512. Shared Gaussian heads predict standardized changes and clamped log standard deviations. Query-specific causal prefixes prevent actions after a requested horizon from influencing that horizon.

The default history is 24 observations, future plans have eight 15-minute steps, and output horizons are 15/30/60/120 minutes. The 96-observation variant has 2,750,082 parameters. Physical-unit outputs require the separately versioned normalization and decoding logic; ONNX alone is insufficient.

Training uses FP32, Gaussian NLL over observed targets, AdamW at 0.0003, cosine decay after warmup and gradient clipping at 1.0. Model selection, calibration and test systems remain separate. Transformer ensemble seeds are 42/43/44. Default full-study training commands use up to 50 epochs and 150,000 windows; that study has not been run.

Results

Field/horizon-balanced MAE divided by each field's training standard deviation, on the same 512 held-out windows; lower is better:

Model	Battery	Tank	Thermal
Transformer	0.24644	0.20519	0.08072
Three-member ensemble	0.24397	0.20475	0.08037
MLP	0.26557	0.21290	0.09717
GRU	0.26920	0.22744	0.10626
CatBoost	0.17793	0.16986	0.08333
Fitted controlled affine dynamics	0.19645	0.19399	0.05348
Persistence	0.27418	0.20509	0.13506

These statistics average standardized field/horizon rows, not unlike raw physical units. Physical-unit errors and Gaussian NLL remain in the original reports. Counterfactual effect errors also fail to establish an overall transformer advantage; only 30 snapshots were evaluated. The 24-hour context did not improve the short matched-anchor ablation. Independently trained specialist and adaptation-speed comparisons remain unrun.

Uncertainty and limitations

Nominal 90% interval coverage for the single transformer is 93.49% battery, 84.19% tank and 92.61% thermal. Thermal coverage applies to only 73.01% of observed queries. The small reserved thermal calibration partition contains only one-node systems, leaving mass_temp without a calibration entry. Its intervals are null; Gaussian dispersion is uncalibrated. Ensemble coverage is 93.82% / 86.18% / 92.53%, with the same thermal gap.

Calibration uses reserved systems, but correlated windows and small calibration counts do not establish a system-level coverage guarantee. Interval width matters and is reported. Prediction conditions on supplied external paths; uncertainty in those forecasts is not modeled. Short context, shifted regimes or a changed configuration can invalidate observed calibration. OOD detection is only a current-state six-standard-deviation heuristic.

Version 0.1 supports controlled numerical schemas, continuous states/actions and fixed 15-minute sampling. Unknown field names require explicit mapping. It does not support arbitrary text/images, discrete-event workflows or arbitrary-name zero-shot transfer. Live CityLearn/BOPTEST/Minari evaluation, unseen configurations/families, full-scale training and independently trained specialist comparisons have not been performed. Native collectors are included with contract fixtures, not external performance claims.

Single-query API p95 was 3.04 ms on CPU and 5.22 ms on MPS, including in-process validation/preparation/inference/serialization, excluding network sockets. Actual localhost endpoints returned HTTP 200. ONNX Runtime parity passed. The pre-release implementation suite had 72 passing tests with Ruff lint/format checks; see the validation report for timing scope and known exporter warnings.

Provenance and license

Original source, weights and synthetic trajectories are released under MIT. Dependencies retain their own licenses and are installed separately. There are no pretrained embeddings, third-party model weights or external training datasets in this release. Generator version is 0.1.2; manifests and run records retain exact system IDs and seeds. release.json and SHA256SUMS identify included bytes. This is an experimental research artifact, not validated control software for real equipment.