mskayacioglu/gz-decals-morphology-challenger

GZ DECaLS Morphology Challenger

This repository contains the frozen weights and calibration artifacts for a three-member EfficientNetV2-S ensemble trained to predict the ten-question Galaxy Zoo DECaLS GZD-5 morphology decision tree.

The model produces calibrated vote-fraction estimates, calibrated decision confidence, and Dirichlet posterior concentrations for 34 answers across ten conditional questions. It is a custom PyTorch model and is not directly compatible with the standard Transformers pipeline API.

Model summary

• Architecture: torchvision EfficientNetV2-S
• Members: three exponential-moving-average checkpoints
• Parameters per member: 20,221,042
• Input: galaxy image, converted to grayscale
• Model crop: 288×288
• Outputs: 34 Dirichlet concentration parameters
• Ensemble: equal mixture of three members and eight dihedral test-time views
• Mixture components: 24
• Calibration: per-question vector scaling, decision temperature scaling, and concentration scaling

Each member maps its 34 logits to positive concentrations using:

alpha = 1 + 99 * sigmoid(logit)

Intended use

The model is intended for research inference on galaxy images comparable to the Galaxy Zoo DECaLS cutouts used during training. Appropriate uses include morphology catalogue construction, sample filtering, uncertainty-aware downstream analysis, and reproducibility studies.

This model is not intended for non-astronomical images, human-facing high-stakes decisions, or use without respecting the conditional Galaxy Zoo question tree.

Preprocessing

The frozen deployment preprocessing is:

1. Decode the source image as RGB.
2. Average the RGB channels to grayscale.
3. Resize to 300×300 with OpenCV INTER_AREA.
4. Round and quantize to uint8.
5. Center-crop to 288×288.
6. Apply the eight registered rotations/reflections.
7. Repeat the grayscale plane to three channels.
8. Apply ImageNet mean and standard-deviation normalization.

The exact configuration is stored in deployment_config.json.

Evaluation

The registered test population contains 49,917 galaxies. Evaluation required at least 34 root-question votes and a question relevance ratio of at least 0.5.

Metric	Published 2021 reference	Challenger
Macro accuracy	0.87973	0.89782
Macro weighted F1	0.87356	0.89634
Macro top-label ECE	Not reported	0.01616
Maximum question ECE	Not reported	0.02935
Credible coverage error	Not reported	0.01594

The 95% galaxy-level bootstrap intervals were:

• macro accuracy: [0.89390, 0.90178];
• macro weighted F1: [0.89243, 0.90048].

Weighted F1 improved on all ten questions. Frozen A100 inference processed 49,917 galaxies in 606.7 seconds, including all 24 ensemble/view components.

Important evaluation limitation

The benchmark gate passed, but this result is not a pristine independent confirmation. The project test split had been opened during an earlier discarded experiment before the challenger evaluation. Challenger training and calibration did not use test labels, so there is no direct train-test leakage into the published weights. A new external sample or untouched holdout is still required for a strict confirmatory claim.

No additional model selection or tuning should be performed against the reported test result.

Download

hf download mskayacioglu/gz-decals-morphology-challenger \
  --local-dir ./challenger_efficientnetv2s_v1
cd challenger_efficientnetv2s_v1
shasum -a 256 -c SHA256SUMS

For the Google Colab workflow, download the repository contents into:

/content/drive/MyDrive/gz-decals-morphology/models/challenger_efficientnetv2s_v1/

Then run 08_inference_challenger.ipynb from the source repository.

Loading one member

The .pt files are compact inference payloads containing a state dictionary and the metadata required by the project notebooks.

from pathlib import Path

import torch
from torch import nn
from torchvision.models import efficientnet_v2_s

model_dir = Path("challenger_efficientnetv2s_v1")
payload = torch.load(
    model_dir / "challenger_member_seed_20260731.pt",
    map_location="cpu",
    weights_only=False,
)

model = efficientnet_v2_s(weights=None)
model.classifier[-1] = nn.Linear(
    model.classifier[-1].in_features,
    len(payload["label_columns"]),
)
model.load_state_dict(payload["model_state"])
model.eval()

PyTorch .pt files use pickle-based serialization. Only load weights obtained from this repository or another source you trust. Verify SHA256SUMS before loading.

Full ensemble inference also requires the moment-matching and calibration procedure implemented in the project inference notebook. A single member is not equivalent to the published ensemble.

Repository contents

• challenger_member_seed_*.pt: three frozen EMA inference payloads
• challenger_calibration_v1.json: frozen calibration transformations
• challenger_ensemble_contract_v1.json: ensemble membership and component definition
• challenger_finalist_contract_v1.json: final deployment freeze
• challenger_final_result_contract_v1.json: final benchmark status
• challenger_preregistration_v1.json: registered challenger protocol
• benchmark_contract_v2.json: frozen evaluation definition and target values
• challenger_final_test_report_v1.json: final metrics and uncertainty report
• prediction_freeze_v1.json: label-free prediction freeze record
• deployment_config.json: portable preprocessing and output configuration
• label_columns.json: ordered 34-output label vector
• question_schema.parquet: question-to-output slices
• hub_manifest_v1.json: portable file mapping and canonical weight hashes
• SHA256SUMS: release integrity manifest

The original contracts are preserved byte-for-byte. Their model_path fields record the original Colab Drive location; portable consumers should use the corresponding filename from hub_manifest_v1.json.

Source code

Training, calibration, evaluation, and deployment notebooks are maintained at:

https://github.com/mskayacioglu/gz-decals-morphology

Frozen source revision: a0d6f0b86d6c3d42d886494947ee88963d060de3.

Data and citation

Training used the Galaxy Zoo DECaLS release:

• Dataset: https://doi.org/10.5281/zenodo.4573248
• ArXiv preprint: https://arxiv.org/abs/2102.08414
• Paper: Walmsley et al., Monthly Notices of the Royal Astronomical Society, 509(3), 3966–3988, https://doi.org/10.1093/mnras/stab2093

Please cite the original Galaxy Zoo DECaLS paper and dataset when using this model.

License

The model release is provided under the Creative Commons Attribution 4.0 International license. See LICENSE.md. The original GZD-5 dataset is also distributed under CC BY 4.0 and retains its own attribution requirements.