None00000/Solar-Filament-None00000

Prompted Recurrent Refinement for Solar Filament Instance Segmentation

Solution of Phạm Đức Công and Lê Anh Đức (FPT University, Hanoi, Vietnam) to the Solar Filament Segmentation Challenge 2026 (IEEE BigData Cup 2026; MAGFiLO annotations on GONG H-alpha full-disk images; metric: Panoptic Quality).

The repository contains the inference package, the trained weights, an end-to-end notebook, the training code of every learned stage, and the evidence behind each design decision.

	Panoptic Quality
Final pipeline, 5-fold out-of-fold on the 707 labelled images	0.4405
Final pipeline, public leaderboard (about half of the 180 test images)	0.39
Baseline (semantic segmentation + connected components), older split, see §2	0.40 – 0.41
Reference: one human annotator scored against another (296 multi-annotated images)	0.34

Contents

1. The task
2. Baseline: semantic segmentation + connected components
3. Final pipeline: proposal → filter → refiner
4. Results
5. Quick start
6. Reproducibility
7. Training
8. Repository layout
9. Known limitations
10. License, data and acknowledgements

1. The task

A training image (20220710085152Th.jpeg) and its expert annotation. Left: the input. Middle: the expected output, one mask per filament (11 here, each in its own colour). Right: zoom on the window marked in yellow.

Input. One full-disk H-alpha image of the Sun from the GONG network: 2048 × 2048 pixels, 8-bit greyscale (JPEG).

Output. A set of binary masks, one mask per filament, each the size of the image. They are submitted as COCO run-length encodings, one row per filament.

Data. 707 labelled training images (2011–2022) and 180 test images. A training image was annotated by one, two or three experts (411 / 145 / 151 images). That gives 1154 annotation records and 8199 filament masks in total.

Metric: Panoptic Quality (PQ), computed against every annotation record separately. Take one image and one annotation record, with ground-truth masks $G_1, \dots, G_m$ and predicted masks $P_1, \dots, P_n$.

1. Compute the intersection over union of every pair:

$$\mathrm{I}\mathrm{o}\mathrm{U}(G_i,P_j)=\frac{\mathrm{\mid }{G}_i\cap P_j\mathrm{\mid }}{\mathrm{\mid }{G}_i\cup P_j\mathrm{\mid }}\backslash mathrm\{IoU\}(G\_i,\; P\_j)\; =\; \backslash frac\{|G\_i\; \backslash cap\; P\_j|\}\{|G\_i\; \backslash cup\; P\_j|\}$$

2. Classify the masks:
   • a pair with $\mathrm{IoU} > 0.5$ is a true positive (TP);
   • a prediction that does not reach $\mathrm{IoU} > 0.5$ with any ground-truth mask is a false positive (FP);
   • a ground-truth mask that no prediction reaches is a false negative (FN).
3. Add up TP, FP, FN and the IoU of the true-positive pairs over all (image, record) pairs of the evaluation set.
4. Compute

$$\mathrm{P}\mathrm{Q}=\frac{\sum _{(G,P)\in \mathrm{T}\mathrm{P}}\mathrm{I}\mathrm{o}\mathrm{U}(G,P)}{\mathrm{\mid }\mathrm{T}\mathrm{P}\mathrm{\mid }+\frac{1}{2}\mathrm{\mid }\mathrm{F}\mathrm{P}\mathrm{\mid }+\frac{1}{2}\mathrm{\mid }\mathrm{F}\mathrm{N}\mathrm{\mid }}=\underset{\text{SQ: segmentation quality}}{\underbrace{\frac{\sum _{(G,P)\in \mathrm{T}\mathrm{P}}\mathrm{I}\mathrm{o}\mathrm{U}(G,P)}{\mathrm{\mid }\mathrm{T}\mathrm{P}\mathrm{\mid }}}}\times \underset{\text{RQ: recognition quality}}{\underbrace{\frac{\mathrm{\mid }\mathrm{T}\mathrm{P}\mathrm{\mid }}{\mathrm{\mid }\mathrm{T}\mathrm{P}\mathrm{\mid }+\frac{1}{2}\mathrm{\mid }\mathrm{F}\mathrm{P}\mathrm{\mid }+\frac{1}{2}\mathrm{\mid }\mathrm{F}\mathrm{N}\mathrm{\mid }}}}\backslash mathrm\{PQ\}\; =\; \backslash frac\{\backslash sum\_\{(G,P)\; \backslash in\; \backslash mathrm\{TP\}\}\; \backslash mathrm\{IoU\}(G,\; P)\}\{|\backslash mathrm\{TP\}|\; +\; \backslash frac\{1\}\{2\}|\backslash mathrm\{FP\}|\; +\; \backslash frac\{1\}\{2\}|\backslash mathrm\{FN\}|\}\; =\; \backslash underbrace\{\backslash frac\{\backslash sum\_\{(G,P)\; \backslash in\; \backslash mathrm\{TP\}\}\; \backslash mathrm\{IoU\}(G,\; P)\}\{|\backslash mathrm\{TP\}|\}\}\_\{\backslash text\{SQ:\; segmentation\; quality\}\}\; \backslash times\; \backslash underbrace\{\backslash frac\{|\backslash mathrm\{TP\}|\}\{|\backslash mathrm\{TP\}|\; +\; \backslash frac\{1\}\{2\}|\backslash mathrm\{FP\}|\; +\; \backslash frac\{1\}\{2\}|\backslash mathrm\{FN\}|\}\}\_\{\backslash text\{RQ:\; recognition\; quality\}\}$$

SQ is the mean IoU of the matched pairs; RQ is an F1-like detection score.

The organisers' rule counts every pair above 0.5. Our strict variant first matches ground truth and predictions one to one (Hungarian). We report both; for this pipeline they agree within 0.0003, because our masks never overlap.

What this rewards. Exactly one mask per physical filament, with a good outline.

• A filament broken into two pieces usually loses its TP and adds FPs, because neither piece reaches IoU 0.5.
• Two filaments joined into one mask usually lose both TPs and add one FP.
• A mask that covers only part of a filament (IoU ≤ 0.5) counts as both an FN and an FP.

Experts also disagree with each other: scoring one annotator's masks against another's gives PQ 0.34.

2. Baseline: semantic segmentation + connected components

Our first line of work treated the problem as binary semantic segmentation and read the instances off the mask.

2.1 Why semantic segmentation is a sound formulation here

There is a single class (filament), and the filaments of one image are almost always far apart. If a model predicted the union of all filament masks perfectly, splitting that union into 8-connected components would give the instances back. We checked this on all labelled data (scripts/semantic_oracle.py, output in docs/figures/semantic_oracle.json):

1. For each of the 1154 annotation records, merge all its filament masks into one binary mask: the ideal output of a semantic model.
2. Split that mask back into instances with 8-connected components (area ≥ 8 px).
3. Score these instances against the record's original masks.

Ideal semantic mask → connected components (707 images, 8199 filaments)	PQ	TP	FP	FN
Organisers' rule	0.9989	8190	2	9
Strict one-to-one	0.9989	8189	2	10

Why it works:

• 99.6 % of the filament masks are a single connected piece.
• The distance from a filament to its nearest neighbour in the same annotation has a median of 140 px. 95 % of the filaments are more than 18 px away from any other.
• Only 20 of the 8096 filaments that have a neighbour actually touch it, so only those get merged by the union.

So the formulation itself costs at most 0.1 % of PQ: the limit is how well the semantic mask is predicted, not the step from pixels to instances.

2.2 Data split

The baselines were developed before the final 5-fold protocol existed. They use one split of the 707 labelled images by capture day (seed 42), so images taken on the same day, often minutes apart and nearly identical, never sit on both sides. The validation part is divided once more: a small subset picks the epoch, the rest only reports.

Part	Images	Use
train	209 (about 30 %)	training crops of 768 px, built per annotation record: one crop centred on each filament, four random crops containing filaments and three crops with no filament (4910 crops)
validation	498 (about 70 %)
↳ quick74	74 of the 498	scored after every epoch to choose the checkpoint
↳ audit424	the other 424	report only, never used to choose anything

The split lists are in splits/split209.json.

2.3 Common setup

All four baselines use the same frozen backbone and data split. Optimisation and post-processing follow each documented recipe; this compares practical systems with similar trainable parameter counts.

Backbone	DINOv3 ConvNeXt-B, pre-trained with self-supervision, frozen. With 209 training images only the head is trained. The convolutional backbone gives a 4-level feature pyramid (strides 4, 8, 16, 32).
Input	768 × 768 crops; the grey image is repeated to 3 channels and normalised with ImageNet statistics.
Augmentation	D4: the 8 symmetries of a square, i.e. a rotation by 0°, 90°, 180° or 270°, with or without a mirror flip. The Sun has no preferred orientation in these images, so all 8 versions are valid training examples.
Schedule	20 epochs, AdamW, seed 42.
Checkpoint	Every 5 epochs (5, 10, 15, 20) the model segments the quick74 images in full and is scored with the organisers' rule; the best epoch is kept. We also report the last epoch, which involves no choice at all.
Inference	3 × 3 overlapping tiles of 768 px (offsets 0, 640, 1280 on each axis) cover the 2048 × 2048 image.

2.4 The four heads

Head	Output	Trainable parameters	Loss	Specific settings
Mask2Former	instances directly: 100 queries, each a mask and a filament / no-object score	2.4 M	Hungarian set loss: class 2, mask BCE 5, Dice 5, no-object 0.1	pixel decoder 3 layers, transformer decoder 6 layers, width 128; lr 1e-4, weight decay 1e-4, effective batch 64, extra augmentation (small rotations, brightness, contrast, gamma). Tiles are merged when two masks overlap with IoU > 0.5 or one contains the other by > 70 %.
FPN	filament probability	1.6 M	BCE + soft Dice	feature pyramid, width 128; lr 3e-4, 100 warm-up updates, batch 8
U-Net (U0)	filament probability	1.8 M	BCE + soft Dice	3-level decoder (width 128) plus half- and full-resolution branches on the raw image for thin structures; same optimiser as FPN
DDP-mini	filament probability	2.117 M	BCE + soft Dice	FPN neck (width 128) + 3-layer deformable-attention decoder (4 heads, FFN 512, time embedding 512), our smaller binary adaptation of DDP: the mask is refined from noise in 3 DDIM steps with a fixed noise seed, so inference is deterministic; same optimiser as FPN

For the three semantic heads the instances come from the probability map:

flowchart LR
    A["H-alpha image<br/>2048 × 2048"] --> B["semantic head on 9 tiles<br/>(frozen ConvNeXt-B)"]
    B -->|"tiles blended with<br/>a Hann window"| C["threshold 0.5"]
    C --> D["8-connected components<br/>area ≥ 128 px"]
    D --> E["one instance<br/>per component"]

2.5 Results

The four 20-epoch runs are in progress under the setup above (training/baselines/). PQ is computed on full 2048 images, per annotation record.

Head	Epoch chosen on quick74	quick74 (chooses the epoch)	audit424, chosen epoch	audit424, last epoch	all 498, chosen epoch	strict one-to-one, audit424
Mask2Former	pending
FPN	pending
U-Net (U0)	pending
DDP-mini	pending

The audit424 column is the fair comparison. The quick74 score is optimistic because it was used to pick the epoch.

The original DDP head (11.564 M parameters, width 256, 6 layers) is a historical reference. DDP-mini keeps the diffusion objective, noise schedule and three inference steps. U0/FPN resumed after epoch 5; earlier epoch 1-4 scores remain in history but do not participate in checkpoint selection.

2.6 Why we moved on

The baseline plateaued, and the reason is structural rather than a matter of tuning:

• With a predicted mask, connected components are fragile. §2.1 shows that the step is lossless for a perfect mask, but a thin, low-contrast section of a real prediction splits one filament into several components, and a bridge of false pixels joins two. Both failure modes are exactly what PQ punishes, and the semantic model has no notion of an instance with which to avoid them.
• Nothing we changed in the semantic model moved the score: longer training, multi-scale and contrast augmentation, backbone fine-tuning (full and LoRA) and coverage / clDice / instance-aware losses; none ended above the baseline.
• Its false positives are confident (median probability about 0.95), so no threshold removes them. Filtering the components with a separate classifier gained only +0.005, because the remaining errors are errors of shape (split, merged or partial masks) that a keep/drop decision cannot repair.

What the baseline did give us is a reliable answer to "where are the dark filament-like pixels". The final pipeline keeps that ability, moves it to the role of a proposal generator, and adds a model that decides instance by instance.

3. Final pipeline: proposal → filter → refiner

The pipeline separates "where could a filament be" (cheap, tuned for recall) from "what exactly is this one filament" (expensive, one decision per candidate).

flowchart LR
    A["H-alpha image<br/>2048 × 2048"] --> B["1 · PROPOSAL<br/>frozen DINOv3 ViT-S+/16<br/>+ 0.43 M head, 9 tiles"]
    B -->|"probability map"| C["2 · FILTER<br/>components at p ≥ 0.05<br/>LightGBM on 32 features, keep ≥ 0.5"]
    C -->|"one crop + anchor<br/>per candidate"| D["3 · REFINER<br/>DINOv3 ViT-B/16 + recurrent decoder<br/>3 rounds → mask + presence"]
    D --> E["4 · POST-PROCESSING<br/>native-resolution resampling<br/>greedy de-duplication"]
    E --> F["pixel-disjoint instances<br/>submission.csv"]

All constants live in filament_pq/config.py; a stage-by-stage specification is in docs/PIPELINE.md.

Stage 1 — Proposal: a high-recall probability map

Goal	Lose (almost) nothing: a filament missed here can never be recovered.
Model	Frozen DINOv3 ViT-S+/16 (28.7 M); features of blocks 3, 6, 9, 12. A 0.43 M-parameter head (lateral 1 × 1 projections, top-down fusion, and a half-resolution raw-image branch that restores thin structures) outputs a per-pixel probability.
Inference	9 tiles of 768 px at offsets {0, 640, 1280}², blended with a Hann window; mean of the fold heads.
Training	One head per fold; BCE (positive weight 4) + Tversky (α = 0.2 on FP, β = 0.8 on FN), 4 epochs, last epoch kept.
Code	filament_pq/proposal.py, training in training/proposal/

At the threshold 0.05 the out-of-fold map has pixel recall 0.908 at precision 0.295. The low precision is intended: precision is the job of the next two stages.

Stage 2 — Filter: from pixels to candidate filaments

Goal	Remove most false regions cheaply, before the expensive refiner runs.
Candidates	8-connected components of the map at p ≥ 0.05 with at least 8 px.
Model	LightGBM (250 trees, 15 leaves, class-balanced), one per fold, on 32 interpretable features of a component: shape and size, probability statistics, darkness against Gaussian backgrounds, black-hat responses, thickness, elongation and contrast with its surroundings. A component is kept when the mean score is ≥ 0.5.
Output	For every kept component, a crop (bounding box + 128 px of context, native resolution) and the component itself as the anchor, i.e. the prompt that tells the refiner which filament is wanted.
Code	filament_pq/components.py, filament_pq/crops.py, training in training/components/

Stage 3 — Refiner: one filament per prompt

Goal	Segment exactly the filament the anchor points to, and say whether it exists at all.
Input	The crop letterboxed to 512 × 512 with six channels: image, proposal probability, anchor mask, signed distance to the anchor, current mask estimate, letterbox validity.
Model	DINOv3 ViT-B/16 (blocks 1–8 frozen, blocks 9–12 fine-tuned) fused at stride 16 with a convolutional stem over the prompt channels; four global self-attention blocks, so the two ends of a long filament can inform each other; a U-Net decoder back to full resolution (6.5 M decoder parameters).
Recurrence	Three rounds; each round receives the previous mask as its "current" channel and corrects it. Round 3 is used.
Presence	logit(LightGBM score) + a learned residual from pooled tokens. Prompts with presence < 0.5 are discarded, which removes false candidates that survived stage 2.
Training	One refiner per held-out fold on the crops of the other four folds; the target of a crop is, per annotation record, the filament that overlaps the anchor most. Prompt corruption, D4 + photometric + scale augmentation; BCE + Lovász hinge + Skeleton Recall on the centre line; 30 epochs, last checkpoint.
Code	filament_pq/refiner/, training in training/refiner/

Each prediction is one instance by construction: the fragmentation and merging of the baseline cannot happen inside a prompt, and the presence output gives a per-instance handle on false positives.

Stage 4 — Post-processing: back to the full image

1. Average the probabilities of the 5 fold refiners × 8 dihedral transforms (D4 test-time augmentation).
2. Resample the probability bilinearly to the crop's native size, then threshold at 0.5 (thresholding at 512 and upsampling the binary mask gives staircase borders on small crops).
3. Keep the connected component that overlaps the anchor most (≥ 128 px).
4. Accept candidates greedily in descending presence: drop a candidate whose IoU with an accepted instance exceeds 0.5, otherwise remove the pixels already taken.

The output is a set of pixel-disjoint instances, written as COCO RLE by filament_pq/submission.py. Both scorers are reported separately; their observed difference is small. Overlapping ground-truth annotations can still make the matching rules differ.

What each stage keeps (out-of-fold, 707 images, 8199 ground-truth filaments)

After	Candidates	Ground-truth filaments still reachable
Stage 1, components at p ≥ 0.05	16 721	8138 touched by a component (99.3 %)
Stage 2, LightGBM ≥ 0.5	5493	7282 touched by an anchor (88.8 %); 7794 inside a crop window (95.1 %)
Stages 3–4, final instances	–	Legacy: 5101 matched with IoU > 0.5 (TP); 2512 FP, 3098 FN

4. Results

Protocol. Five folds split by observation date (141/141/142/141/142 images, seed 42, splits/fold_assignment.json), so images taken minutes apart never sit on both sides of a split. Every image is predicted only by models that did not see it. Scores are computed on full 2048 images, per original annotation record (annotators are never merged), and include ground truth lost before the refiner. TP, FP, FN and the IoU sum are pooled over all records before PQ is computed.

Setting (out-of-fold, 707 images)	PQ, organisers' rule	PQ, strict one-to-one
Refiner, threshold at 512 + nearest upsampling, no TTA	0.4227	0.4227
+ bilinear resampling to native resolution, threshold 0.5	0.4363	0.4363
+ D4 test-time augmentation (submitted)	0.4405	0.4405
Same choice made inside a nested leave-one-fold-out	0.4396	0.4395

• Submitted setting: SQ 0.683, RQ 0.645; per fold 0.4295 / 0.4300 / 0.4472 / 0.4521 / 0.4454.
• The organisers' self-evaluation notebook, run on our out-of-fold predictions in submission format, returns 0.440524.
• Public leaderboard: 0.39.
• The baseline of §2 reached 0.4085 under the strict rule on a different protocol, so the gap to 0.4405 is indicative only; before the post-processing changes the refiner alone was at 0.4227.

Ablations, the per-epoch training history, the error audit and the negative results are in docs/RESULTS.md; the reasoning behind each choice is in docs/DESIGN.md.

5. Quick start

git lfs install
git clone https://github.com/Cdshf01230/Solar-Filament-None00000.git
cd Solar-Filament-None00000
pip install -r requirements.txt
python scripts/predict.py --images /path/to/test_images --weights weights --out submission.csv

Alternatively open notebooks/pipeline.ipynb. It loads the weights, runs every stage on a test image with visualisations, reproduces submission.csv for the whole test set and evaluates out-of-fold on labelled images.

• Data. Download the competition data from Kaggle and point --images to the folder of test images. The data are not redistributed here.
• Hugging Face mirror: None00000/Solar-Filament-None00000.
• Weights. Everything is in weights/ (about 1.1 GB, Git LFS): 2 DINOv3 backbones, 5 proposal heads, 5 LightGBM models, 5 refiners. Nothing is downloaded at run time. See weights/MODEL_CARD.md.
• Hardware. One CUDA GPU with 16 GB. For the 180 test images: about 25 min for stages 1–2, then about 15 min per fold refiner on an L4 (about 70 min per fold on a T4), with D4 TTA. No re-training is needed.
• Environment. Python 3.13; exact package versions are pinned in requirements.txt.

6. Reproducibility

• On 9 November 2026, scripts/predict.py reproduced the submitted CSV byte for byte on an NVIDIA L4 (tests/test_regression.py against reference/submission.csv). Other GPU types can differ in the last floating-point bits, which may flip a few pixels at threshold boundaries.
• Every threshold is a fixed constant in filament_pq/config.py; none was tuned on test data.
• Seeds are fixed (42) for the proposal heads, LightGBM (deterministic=True) and the refiner, whose prompt sampling and augmentation use stateless generators seeded by (seed, epoch, sample index).
• The released refiner files contain only the decoder and the four fine-tuned ViT blocks. The export (scripts/export_weights.py) asserts that every other backbone tensor is bit-identical to the public DINOv3 weights.

Fresh training from scratch has not been repeated to establish bitwise reproducibility. The recovered release weights passed CPU strict-load and finite checks for all 15 learned models; see docs/REPRODUCIBILITY.md.

7. Training

Inference needs none of this. The code under training/ is the code the released models were trained with; recipes and hyper-parameters are in docs/TRAINING.md.

Stage	Trainable part	Data per model	Schedule
1 · Proposal head	0.43 M head (backbone frozen)	9 tiles of 768 px per annotation record, one fold (about 141 images)	4 epochs, AdamW 3e-4
2 · LightGBM filter	250 trees	components of one fold, out-of-fold proposal maps	CPU, deterministic
3 · Refiner	6.5 M decoder + ViT-B blocks 9–12	crops of four folds (about 4400 crops / 566 images)	30 epochs, AdamW, layer-wise LR decay 0.65, drop path 0.1; Kaggle 2 × T4 (folds 0, 2, 3, 4) and Colab L4 (fold 1)

8. Repository layout

Path	Content
filament_pq/	Inference package: proposal.py, components.py, crops.py, refiner/, postprocess.py, submission.py, metrics.py, pipeline.py, config.py
scripts/	predict.py (images → submission.csv), export_weights.py, export_lightgbm.py, make_splits.py, make_figures.py
notebooks/pipeline.ipynb	End-to-end walkthrough and reproduction
weights/	Model weights (Git LFS) and their model card
splits/	The fixed date-disjoint folds and the five hold-out splits
training/	Training code of the three learned stages, and of the baseline decoder (training/proposal/u0_multihead/ddp_model.py)
docs/	PIPELINE · DESIGN · RESULTS · TRAINING
report/	Source of the 4-page technical report
tests/, reference/	Regression test and the submitted predictions it compares against

9. Known limitations

From the out-of-fold error audit (docs/RESULTS.md):

• Incomplete masks (558 FN): the refiner paints only part of a filament although its anchor already covers most of it.
• Filaments never prompted (402 lost before the refiner, 514 only inside crop padding): the component was filtered out or missed.
• Two filaments under one anchor (481 FN): the refiner outputs one instance per prompt. A zero-shot attempt to extract a second instance did not help (0.4409 against 0.4405); a trained second output is the natural next step.
• All reported validation numbers are cross-validation estimates on the labelled images; the public leaderboard score (0.39) is lower than the out-of-fold score (0.44).

10. License, data and acknowledgements

• Code: MIT (LICENSE).
• Pre-trained backbones: the DINOv3 weights in weights/ are redistributed under the DINOv3 License (Meta).
• Data: MAGFiLO, Kaggle edition 2026. Training uses only the 707 labelled competition images; inference reads only the test images.
• Acknowledgement: this work uses GONG data obtained by the NSO Integrated Synoptic Program, managed by the National Solar Observatory, which is operated by the Association of Universities for Research in Astronomy (AURA), Inc. under a cooperative agreement with the National Science Foundation.

References: MAGFiLO (Ahmadzadeh et al., Scientific Data, 2024); DINOv3 (Siméoni et al., 2025); DDP (Ji et al., ICCV 2023); Panoptic Quality (Kirillov et al., CVPR 2019); LightGBM (Ke et al., NeurIPS 2017); Lovász hinge (Berman et al., CVPR 2018); Skeleton Recall loss (Kirchhoff et al., ECCV 2024).