nasa-ibm-ai4science/IMP-Segmentation-NASA-IBM-Lunar-Foundation-Model

NASA-IBM LFM — Irregular Mare Patch Segmentation

Fine-tuned IMP segmentation checkpoint for the NASA-IBM Lunar Foundation Model (NASA-IBM LFM).

Binary semantic segmentation of irregular mare patches (IMPs) in meter-scale LROC NAC imagery. IMPs are rare, morphologically diverse volcanic landforms usually distinguished by subtle boundary expressions rather than strong visual signatures. IMPs complement impact-feature benchmarks for assessing lunar volcanic history.

The released checkpoint uses a frozen encoder with a trained UNet decoder — the adaptation strategy that performed best on this benchmark. Test IoU₁ 0.5709 ± 0.0114, F1₁ 0.7268 ± 0.0093, ahead of full fine-tuning (0.5693 ± 0.0114) and of the best ImageNet-pretrained baseline (ConvNeXt-V2-B, 0.5687 ± 0.0181). Though the metrics of these three models sit inside one another's seed spread and should be read as comparable rather than ranked. With only 100 training tiles, the reduced trainable-parameter count of a frozen encoder acts as a useful regularizer.

The decisive signal on this benchmark is elsewhere: an architecturally identical random-init control collapses to IoU₁ 0.3142 ± 0.0746, so lunar pretraining is essential in this low-data regime, not merely helpful.

📄 Full method and tables: paper · 💻 Fine-tuning code and configs: GitHub · 🧠 Pretrained backbone: nasa-ibm-lunar-fm

Qualitative IMP segmentation predictions on four examples. Green overlays mark ground-truth IMP extents; colored overlays show model predictions. Predictions are broadly similar across models, with the NASA-IBM LFM generally identifying slightly larger IMP regions than the baselines, which may lead to more false positives, but in the second example lets it capture IMP regions that none of the baselines detect. Colors in the top-row maps are for visualization only and carry no meaning.

Checkpoint at a glance

Science theme	Volcanic history
Task	Binary semantic segmentation (background vs. Irregular Mare Patch)
Input	Single-channel LROC NAC, 256 × 256 px at ~1 m/px
Splits (train/val/test)	100 / 20 / 10 image–mask pairs
Backbone	NASA-IBM LFM ViT-B, FlexiViT patch size 8
Adaptation	Frozen encoder; UNet decoder + auxiliary head trained
Loss	Dice
Test IoU₁ / F1₁	0.5709 ± 0.0114 / 0.7268 ± 0.0093
License	Apache-2.0

Because the encoder is frozen, the released artifact is small and only includes the decoder and head weights, and it composes with the unmodified pretrained backbone.

Repository contents

imp_segmentation/   UNet decoder + auxiliary head weights, TerraTorch config

The pretrained backbone is not duplicated here — pull it from nasa-ibm-lunar-fm and point the config's backbone checkpoint field at it.

Benchmark dataset

Ships as part of SomBench.

Built from 2,623 polygon annotations (Hargitai et al., 2025) paired with corresponding LROC NAC images. A subset of annotations was manually adjusted to correct misalignment caused by NAC pointing uncertainty — typically tens of meters in uncontrolled frames — and roughly 100 polygons exceeding the 256 m patch size were excluded. Both limitations point to the value of future benchmarks derived from controlled NAC products such as NAC PHO.

Metrics are reported on the positive (IMP) class only: intersection-over-union (IoU₁) and F1 score (F1₁).

Intended use

Delineating irregular mare patches in meter-scale NAC imagery, and as a starting point for fine-tuning on your own IMP or subtle-boundary landform labels. IMP extents feed studies of late-stage lunar volcanism and thermal history. The model was not validated for operational use.

Usage

Runs through TerraTorch, configured from YAML. The companion repo adds the SomBench benchmark datamodules, backbone wrappers, support for registering new input modalities into the pretrained backbone, and FlexiViT patch-embedding interpolation.

git clone https://github.com/NASA-IMPACT/NASA-IBM-Lunar-Foundation-Model && cd $_
pip install -e .

# reproduce the fine-tune
terratorch fit  --config configs/finetune/imp_segmentation_frozen.yaml

# evaluate a released checkpoint
terratorch test --config configs/finetune/imp_segmentation_frozen.yaml \
                --ckpt_path imp_segmentation/checkpoint.ckpt

from huggingface_hub import snapshot_download

local = snapshot_download("nasa-ibm-ai4science/nasa-ibm-lunar-fm-imp")
# then set the backbone checkpoint path in the YAML to the downloaded nasa-ibm-lunar-fm backbone

Exact registry names and full hyperparameters are in the GitHub repo.

Evaluation

All backbones are fine-tuned through TerraTorch with the data loaders, splits, augmentations, loss, and evaluation metric held fixed; only encoder weights and initialization differ. Values are mean ± standard deviation over 5 random seeds; bold is the best mean per column, underline the second best. Differences smaller than the reported seed spread should not be read as a ranking.

Baselines are ResNet-50, ViT-B MAE, ConvNeXt-B, ConvNeXt-V2-B, SwinV2-B, and DaViT-B initialized from public ImageNet-1k/22k weights; two task-specific segmentation architectures assembled through the standard segmentation_models_pytorch factory (DeepLabV3+ / ResNet-50 and SegFormer / MiT-B2, using their native heads and a single unified learning rate); and two from-scratch controls — a random-init ResNet-50 and an architecturally identical random-init copy of our model, which isolates the contribution of lunar pretraining from that of architecture.

Model	IoU₁ ↑	F1₁ ↑
ResNet-50 (ImageNet)	0.4731 ± 0.0114	0.6423 ± 0.0105
ResNet-50 (random init)	0.3496 ± 0.0323	0.5175 ± 0.0355
ViT-B MAE (ImageNet)	0.5541 ± 0.0369	0.7125 ± 0.0298
ConvNeXt-B (IN22k)	0.5374 ± 0.0197	0.6989 ± 0.0168
ConvNeXtV2-B (IN22k)	0.5687 ± 0.0181	0.7249 ± 0.0146
SwinV2-B (ImageNet)	0.5555 ± 0.0159	0.7141 ± 0.0132
DaViT-B (ImageNet)	0.5623 ± 0.0055	0.7198 ± 0.0045
DeepLabV3+ / ResNet-50	0.5154 ± 0.0180	0.6801 ± 0.0157
SegFormer / MiT-B2	0.5475 ± 0.0099	0.7076 ± 0.0083
NASA-IBM LFM (ps8, random init)	0.3142 ± 0.0746	0.4742 ± 0.0892
NASA-IBM LFM (ps8, full FT)	0.5693 ± 0.0114	0.7255 ± 0.0093
NASA-IBM LFM (ps8, LoRA)	0.5593 ± 0.0137	0.7173 ± 0.0113
NASA-IBM LFM (ps8, frozen)	0.5709 ± 0.0114	0.7268 ± 0.0093

What to read from this table

• The NASA-IBM LFM variants occupy the top positions, but the leaders are statistically comparable. Frozen (0.5709), full FT (0.5693), and ConvNeXt-V2-B (0.5687) all fall within each other's seed spread — we report the ordering, not a claim of a winner.
• Pretraining is essential here, and the control proves it. The random-init copy of our model shares the architecture and tokenization scheme without lunar pretraining and collapses to IoU₁ 0.3142 ± 0.0746 — below every ImageNet-pretrained baseline, and with the widest variance in the table. On 100 training tiles there is no substitute for a pretrained representation.
• Frozen wins where craters punished it. This is the one benchmark in the paper where freezing the encoder is the best configuration; on crater detection the same strategy falls below every baseline. The difference is data volume — with 100 tiles the reduced trainable-parameter count regularizes.
• LoRA remains competitive (0.5593 ± 0.0137) at a small fraction of the trainable parameters, and is the better default if you plan to adapt to a larger IMP-like dataset.

Training configuration

Framework	TerraTorch, YAML-configured
Decoder	UNet, plus an auxiliary segmentation head (weight 1.0)
Loss	Dice
Backbone patch size	8 (FlexiViT interpolation of the 16 × 16 pretrained patch embedding)
Normalization	per-dataset statistics computed over the training split after masking no-data pixels
Augmentation	D4 group transforms (rotations + flips), random brightness/contrast jitter
Optimizer	AdamW, cosine schedule, 500 warm-up steps
LR (pretrained)	backbone 1e-4, head 3e-4, layer-wise LR decay 0.85
LR (ImageNet baselines, from-scratch controls)	flat 1e-4
LR (DeepLabV3+, SegFormer)	single unified learning rate, native segmentation heads
Schedule	batch size 32, fixed budget of 600 epochs (no early stopping)
Seeds	5 per configuration

Optimizer recipes are deliberately not shared across backbone families: our pretrained model uses layer-wise LR decay with a lower encoder LR to avoid pushing pretrained representations away early in training, while ImageNet baselines and from-scratch controls use flat learning rates conventional for their architectures.

Limitations

• 10 test tiles. The single largest caveat. Every conclusion drawn from the ordering of the top rows is bounded by this.
• Subtle, contested boundaries. IMPs are defined by weak boundary expressions; annotation noise in the source catalog is real enough that part of it needed manual realignment.
• Uncontrolled NAC frames carry pointing uncertainty of tens of meters. Controlled products (NAC PHO) would make a stronger benchmark.
• Large IMPs excluded — polygons exceeding 256 m were dropped when building the benchmark.

Citation

@article{fraccaro2026lfm,
  title  = {Multimodal-Multiresolution Foundation Model for Lunar Remote Sensing},
  author = {Fraccaro, Paolo and Nyirjesy, Gabby and Szwarcman, Daniela and Patil, Himanshu
            and Gaur, Vishal and Lal, Rohit and Slank, Rachel A. and Dawson, Geoffrey
            and Debary, Hiyam and Dionelis, Nikolaos and Barker, Michael K. and Annex, Andrew
            and Viswanathan, Vishnu and Morse, Zachary and Schaefer, Ethan I. and Kumar, Ankur
            and Watson, Campbell D. and Dawson-Rigas, Rebekah I. and Maskey, Manil
            and Roy, Sujit and Ramachandran, Rahul and Bernab\'e-Moreno, Juan},
  year   = {2026}
}

@misc{sombench2026collection,
  author = {Patil, Himanshu and Nyirjesy, Gabby and Slank, Rachel A. and Gaur, Vishal
          and Szwarcman, Daniela and Fraccaro, Paolo and Dionelis, Nikolaos and Barker, Michael K.
          and Annex, Andrew and Viswanathan, Vishnu and Morse, Zachary and Schaefer, Ethan I.
          and Debary, Hiyam and Kumar, Ankur and Lal, Rohit and Dawson, Geoffrey
          and Watson, Campbell and Dawson-Rigas, Rebekah I. and Maskey, Manil
          and Bernab\'e-Moreno, Juan and Ramachandran, Rahul and Roy, Sujit},
  title        = {{SomBench}: Benchmark Dataset for Advancing Machine Learning in Lunar Science},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/collections/nasa-ibm-ai4science/lunar-fm-ml-ready-benchmark-dataset-sombench}}
}

@article{hargitai2025clusters,
  title   = {Clusters of irregular mare patches on the Moon},
  author  = {Hargitai, Henrik and others},
  year    = {2025}
}

Please also cite TerraMind (Jakubik et al., 2025), TerraTorch (Gomes et al., 2025), and FlexiViT (Beyer et al., 2023).

Acknowledgments

Supported by NASA under Award No. 80MSFC25M0084. Data products courtesy of LROC.

Contact: Sujit Roy (sujit.roy@nasa.gov) · Paolo Fraccaro (paolo.fraccaro@ibm.com)