methane-plume-compact

A compact U-Net that segments methane plumes in AVIRIS-NG airborne hyperspectral tiles. It takes the mag1c methane-enhancement product and three visible bands, and it was trained from scratch on the STARCOP dataset. It has 123,649 parameters and takes 506 KB as fp32 ONNX or 175 KB as int8, and it runs in about 18 ms per 512 × 512 tile on one Apple M4 CPU thread. This repository holds five training seeds. The evaluation is self-run on the STARCOP test split and is not an official ranking. Under the claim rule fixed before the test split was read, the result is not distinguishable from the paper's mag1c-only model, and it is not claimed to be better. The weights are CC BY-NC 4.0 (non-commercial use only) because the training data is. Code, pre-registration, test log, results and scorer: https://github.com/sankalpsthakur/methane-plume-compact

Result

• Protocol: STARCOP AVIRIS test split (342 tiles), paper-text protocol: strong and weak plumes split by emission rate, a tile counts as positive above 10 predicted pixels, AUPRC = average precision. Five seeds, self-evaluated on 2026-10-17.
• Result: this 123,649-parameter U-Net reaches weak-plume F1 54.47 ± 2.08. The paper's mag1c-only row is 47.57 ± 4.17.
• Verdict: the paired tile-bootstrap interval against the released mag1c-only model includes zero (+5.71 [−2.5, +19.2]). Under the pre-registered rule, the result is therefore not distinguishable from the published row.

Six qualifications go with that result:

1. More false-alarm tiles. Under the same scorer, the tile-level false-positive rate is worse than both released models:

   • 50.51 ± 1.65 % here, against 36.00 % for the released mag1c-only model (paired +14.51 [+8.3, +20.6]);
   • against 40.57 % for the released mag1c+RGB model (paired +9.94 [+4.5, +15.2]);
   • level with the paper's mag1c-only mean (52.11 ± 10.98).

   The weak-F1 gain comes with more false-alarm tiles.

2. One released run. The released mag1c-only model is one training run. The tile bootstrap does not capture its seed-to-seed noise (the paper reports a standard deviation of 4.17).

3. The control carries no claim. The control row below retrains the released STARCOP architecture (6,629,233 parameters) from scratch with a paper-like recipe (the released config's Adam optimiser and plateau schedule at learning rate 1e-4) in this pipeline (window sampler, loss weights, mixed precision, scorer) for 5 of the paper's 15 epochs: 22,215 steps of batch 32, against 52,450 for the small model, on the training tiles outside the two validation dates. It is one run, listed for reference.

   • What it suggests: trained with this pipeline, the released architecture reaches the same weak F1 as the small model (54.42 [45.5, 63.2] against 54.47). That suggests the gain over the released runs comes from the training pipeline rather than from the small architecture, but it rests on one control run.

4. Validation and test numbers are not comparable. Validation numbers come from easier held-out dates (weak F1 78.06 for this configuration).

5. Second evaluation of this architecture on the same test split.

   • On 2026-10-07 the same architecture was trained on CPU with 0.128 M windows of batch 16 and scored on these 342 tiles: weak F1 42.70 ± 2.30, below the target.
   • This GPU run was planned, with the same target and claim rule, before that evaluation; the plan precedes it in the history of the author's private working repository (the public code repository is a fresh export of it).
   • It changed only the training budget: 13 times the windows, batch 32 and mixed precision.

6. No third look. A third, validation-gated wave (Amendment 6: the same architecture for 15 epochs, 78,675 steps of batch 32, seed 0) failed its gate on 2026-10-19 (UTC).

   • Its validation composite was 61.918 against 63.925 for the 10-epoch run (-2.007; the gate needed +2.0).
   • The test split was therefore not read a third time, and the result above stands unchanged.
   • The gate decision is results/gpu_runs/gpu-gate3/gate_decision.json in the code repository; the rule is in protocol/PREREGISTRATION.md, Amendments 6 and 6a.

Model description

• Architecture. A plain U-Net with five encoder levels of 4, 8, 16, 32 and 64 channels and two 3 × 3 conv–BatchNorm–ReLU blocks per level.
  • The first convolution has stride 2, so every level runs at half resolution.
  • The decoder uses nearest-neighbour × 2 upsampling with skip concatenation.
  • Full-resolution detail returns through bilinear upsampling, a 4-channel full-resolution feature computed from the input, and a 3 × 3 output convolution.
  • 123,649 parameters, 53.6 times fewer than the 6,629,233 of the released STARCOP U-Net with a MobileNetV2 encoder.
• Five seeds (0 to 4): same architecture, recipe and data, different random seeds. They are not ensembled.
  • Seed 0 comes first by index only. It is also the best of the five on weak F1 (57.46).
  • The expected performance is the five-seed mean, not seed 0.
• ONNX: opset 17, static input shape.
  • fp32 is exported from PyTorch with BatchNorm folded into the convolutions. It matches PyTorch to within 2.1e-5 in the logits, and every test metric agrees to 0.01.
  • int8 uses static QDQ quantisation in onnxruntime: signed 8-bit weights (per channel) and activations, with MinMax calibration on 48 training tiles. No test data was used for calibration.

Input and output

	name	type and shape	content
input	x	float32, (1, 4, 512, 512)	channels in this order: mag1c, TOA_AVIRIS_640nm, TOA_AVIRIS_550nm, TOA_AVIRIS_460nm, each normalised as clip(raw / factor, 0, 2) with factors 1750, 60, 60, 60
output	logit	float32, (1, 1, 512, 512)	per-pixel plume logit, before the sigmoid

• Raw values come from the STARCOP tile files of the same names (mag1c.tif, TOA_AVIRIS_640nm.tif, ...). The factors are the constants of the official STARCOP code.
• Pixel rule: a pixel is plume when sigmoid(logit) > 0.5, that is, when logit > 0.
• Tile rule: two rules are in use, and they give very different false-alarm rates (see Evaluation).
  • The paper's text counts a tile as positive when it has more than 10 plume pixels.
  • The official code requires more than 640 at 512 × 512 (10·H·W/64²).
• No-data: no rule is defined. The STARCOP tiles were used as stored, and clip passes NaN through, so inputs must be finite.
• Shape: fixed at one 512 × 512 tile per call.

import numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download

path = hf_hub_download("sankalpsthakur/methane-plume-compact", "weights/gf_d1_s0_fp32.onnx")
sess = ort.InferenceSession(path, providers=["CPUExecutionProvider"])

FACTORS = (1750.0, 60.0, 60.0, 60.0)   # mag1c, TOA 640 nm, 550 nm, 460 nm

def predict(bands):
    """bands: four 512 x 512 arrays as stored in the tile files, in the order above."""
    x = np.stack([np.clip(np.asarray(b, np.float32) / f, 0, 2) for b, f in zip(bands, FACTORS)])[None]
    logit = sess.run(["logit"], {"x": x})[0][0, 0]
    mask = logit > 0                    # sigmoid(logit) > 0.5
    return mask, int(mask.sum()) > 10   # paper-text tile rule; the official code uses > 640

Training

• Data: the 3,425 tiles of the STARCOP training split (AVIRIS-NG). No test tile was used for training, selection or calibration.
• Recipe:
  • 128 × 128 windows (stride 64), with ±32-pixel jitter and random flips and 90° rotations. Windows are drawn so that, on average, half contain more than 40 plume pixels.
  • Loss: binary cross-entropy on the logits, weighted per pixel by the dataset's weight_mag1c map.
  • Optimiser: AdamW (learning rate 2e-3, no weight decay), a 200-step warm-up, then cosine decay to 2 % of the peak.
  • Batch 32, 52,450 steps (1.68 M windows): 10 epochs of the paper's sampler, where the paper trains for 15.
  • Mixed precision, on Kaggle with 2 × Tesla T4.
• Selection: this configuration (d1) was chosen from three candidates on validation dates held out from training (2019-10-22 and 2019-10-23). The rule was fixed in advance: fewest parameters among the candidates within 2.0 points of the best validation score. The configuration was then frozen, and each seed was retrained on all training tiles.
• Pre-registration: the target, the claim rule, the seeds, the step budget and the compute caps were all committed before any training run. The step budget was set from a synthetic throughput probe, by a rule fixed in advance.
  • In this run, each seed was scored on the test split once per format (PyTorch, ONNX fp32, ONNX int8). The earlier CPU run is qualification 5 under Result.
  • No result was discarded, and no test number changed a configuration.

Evaluation

Self-evaluated on the STARCOP AVIRIS test split (342 tiles) on 2026-10-17. For this model, values are the five-seed mean ± standard deviation, with the 95 % tile-bootstrap interval of the mean in brackets. The other rows are single runs or the paper's published values.

Paper-text protocol:

model	F1 strong	F1 weak	tile FPR (> 10 px)	AUPRC
this model, fp32 (5 seeds)	82.78 ± 2.21 [70.8, 89.3]	54.47 ± 2.08 [30.1, 78.2]	50.51 ± 1.65 [43.7, 57.1]	55.94 ± 8.08 [39.1, 74.9]
this model, int8 (5 seeds)	82.85 ± 1.97 [70.8, 89.1]	55.46 ± 2.57 [31.3, 78.6]	50.06 ± 2.51 [43.3, 56.6]	55.50 ± 7.50 [38.3, 74.7]
released mag1c-only, 1 run, same scorer	66.74	48.76	36.00	40.91
published mag1c-only (paper, Table 2, 5 runs)	74.15 ± 6.10	47.57 ± 4.17	52.11 ± 10.98	49.41 ± 5.49
released mag1c+RGB (run R3), 1 run, same scorer	83.08	42.34	40.57	50.60
published mag1c+RGB (paper, Table 2, 5 runs)	81.96 ± 3.71	43.42 ± 5.72	43.66 ± 7.36	51.99 ± 2.76
control: released architecture, paper-like recipe, 5 of 15 epochs, 1 run	71.83 [57.8, 81.5]	54.42 [45.5, 63.2]	46.29 [38.8, 54.0]	56.18 [41.6, 70.9]

Paired differences: this model's seed mean minus the released run, on the same 342 tiles, with 95 % tile-bootstrap intervals.

reference	F1 strong	F1 weak	tile FPR (> 10 px)	AUPRC
released mag1c-only (the comparator of the claim rule)	+16.04 [+6.6, +25.6]	+5.71 [−2.5, +19.2]	+14.51 [+8.3, +20.6]	+15.03 [+2.3, +24.1]
released mag1c+RGB (run R3)	−0.30 [−5.7, +4.0]	+12.13 [+3.8, +25.3]	+9.94 [+4.5, +15.2]	+5.34 [−7.0, +16.8]

A positive FPR difference means more false-alarm tiles. The claim rule compares against the mag1c-only model, so no claim is made from the weak-F1 difference against R3.

Official-code variants of the same predictions differ from the paper-text protocol in three ways:

• plumes are split by label size: strong above 1,000 labelled pixels, weak from 1 to 1,000;
• a tile counts as positive above 640 pixels;
• AUPRC is computed over the official code's 16 thresholds.

These are point estimates; no paired interval was pre-registered for them.

model	F1 strong	F1 weak	tile FPR (> 640 px)	AUPRC
this model, fp32 (5 seeds)	83.91 ± 2.15	49.88 ± 2.24	11.31 ± 0.63	56.16 ± 8.24
released mag1c-only	66.74	47.51	9.14	39.00
released mag1c+RGB (R3)	84.23	39.02	12.57	37.15

Per seed, for the ONNX files in this repository (fp32 / int8):

seed	F1 strong	F1 weak	tile FPR (> 10 px)	AUPRC
0	86.24 / 86.16	57.46 / 58.30	49.14 / 49.14	68.87 / 67.58
1	83.06 / 82.86	53.81 / 53.68	50.86 / 52.00	54.99 / 54.43
2	81.11 / 81.76	53.28 / 54.71	49.14 / 49.14	47.32 / 47.70
3	82.91 / 82.38	55.60 / 58.01	50.29 / 46.86	56.82 / 56.33
4	80.59 / 81.07	52.19 / 52.60	53.14 / 53.14	51.70 / 51.48

Definitions:

• Pixel F1 is taken at p > 0.5 and pooled over the tiles of each group. In the paper-text protocol, strong plumes are 1,000 kg/h or more and weak plumes are below 1,000 kg/h.
• Tile FPR is the share of plume-free test tiles that the model flags.
• AUPRC is the average precision over the pixels of all tiles.

The scorer re-implements the aggregation of the official run_validation (spaceml-org/STARCOP). On the released mag1c+RGB checkpoint it matches the official code exactly on three metrics and within 4e-7 on AUPRC.

Size and speed

	this model	released STARCOP U-Net (MobileNetV2 encoder, mag1c+RGB)
parameters	123,649	6,629,233
ONNX fp32	506,025 B	26,497 KB
ONNX int8	174,804 B	7,074 KB
Apple M4, 1 thread, batch 1, median, fp32 / int8	18.41 ms / 18.76 ms	not measured
Kaggle CPU (Intel Xeon @ 2.00 GHz), 1 thread, batch 1, median, fp32 / int8	32 to 34 ms / 54 to 58 ms	not measured

• int8 is not faster than fp32 on either CPU. Its gain is size.
• M4 measurement: seed 0, at a 1-minute load average below 4. Repeated measurements of the same architecture varied by about 15 %.
• Kaggle figures: the range over the five seeds.
• Released model sizes: its checkpoint exported with the same code.

Intended use and limits

• Intended use: research and non-commercial work on compact methane-plume segmentation, benchmarking on STARCOP-format AVIRIS-NG tiles, and edge-size baselines.
• Not for operational leak detection, regulatory reporting or emission quantification. The model outputs a pixel mask, not an emission rate, and it has no field validation.
• Sensor and product: trained only on AVIRIS-NG tiles with the mag1c product as prepared in STARCOP. Other instruments, retrievals and normalisations are untested.
• False alarms: the 10-pixel tile rule flags about half of the plume-free test tiles (50.51 %); the official 640-pixel rule flags 11.31 %. Pick the tile rule that suits the application.
• Small test set: 342 tiles with few weak plumes, so the intervals are wide. The seeds also differ, for example AUPRC from 47.3 to 68.9.
• Reproducibility: training is not bit-reproducible across hardware. The SHA-256 of every weight file is in SHA256SUMS.

Data, licence and attribution

• Weights: CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/). The training data is CC BY-NC 4.0, so these weights are for non-commercial use only.
• Dataset: STARCOP dataset: Semantic Segmentation of Methane Plumes with Hyperspectral Machine Learning Models, v1.0, by Vít Růžička, Gonzalo Mateo-Garcia, Luis Gómez-Chova, Anna Vaughan, Luis Guanter and Andrew Markham. Zenodo, 2023, https://doi.org/10.5281/zenodo.7863343, CC BY-NC 4.0.
  • Get the data from the Zenodo record. A Kaggle mirror of it carries an "MIT" label, which is not the dataset's licence.
• Paper: Růžička, V., Mateo-Garcia, G., Gómez-Chova, L., Vaughan, A., Guanter, L. and Markham, A. Semantic segmentation of methane plumes with hyperspectral machine learning models. Scientific Reports 13, 19999 (2023). https://doi.org/10.1038/s41598-023-44918-6
• Training from scratch: no released STARCOP checkpoint was used for initialisation or distillation. The released checkpoints were only scored, as references.
• Independence: this is independent work, not affiliated with or endorsed by the STARCOP authors.

Files

file	what
weights/gf_d1_s{0..4}_fp32.onnx	the five seeds, fp32, 506,025 B each
weights/gf_d1_s{0..4}_int8.onnx	the five seeds, int8, 174,804 B each
results/edge_report.json	per seed: sizes, test metrics of both ONNX files and Kaggle-CPU latency, as written by the training kernel
SHA256SUMS	SHA-256 of every weight file

The training, scoring and protocol code, the pre-registration and the test log are at https://github.com/sankalpsthakur/methane-plume-compact (code Apache-2.0; its STARCOP-derived files CC BY-NC 4.0).

Citation

If you use these weights, please cite the STARCOP paper and dataset:

@article{ruzicka2023starcop,
  title   = {Semantic segmentation of methane plumes with hyperspectral machine learning models},
  author  = {R{\r{u}}{\v{z}}i{\v{c}}ka, V{\'\i}t and Mateo-Garcia, Gonzalo and G{\'o}mez-Chova, Luis and Vaughan, Anna and Guanter, Luis and Markham, Andrew},
  journal = {Scientific Reports},
  volume  = {13},
  pages   = {19999},
  year    = {2023},
  doi     = {10.1038/s41598-023-44918-6}
}

@misc{starcop2023dataset,
  title     = {{STARCOP} dataset: Semantic Segmentation of Methane Plumes with Hyperspectral Machine Learning Models},
  author    = {R{\r{u}}{\v{z}}i{\v{c}}ka, V{\'\i}t and Mateo-Garcia, Gonzalo and G{\'o}mez-Chova, Luis and Vaughan, Anna and Guanter, Luis and Markham, Andrew},
  publisher = {Zenodo},
  version   = {v1.0},
  year      = {2023},
  doi       = {10.5281/zenodo.7863343}
}