OoshadhaSam/solar-irradiance-visual-backbone-benchmark

Controlled Visual-Backbone Benchmark for Solar Irradiance Forecasting

Inference-only weights for A Controlled Visual-Backbone Benchmark for Multimodal Short-Term Solar Irradiance Forecasting, accepted at MERCon 2026.

• Project page: https://oshadha345.github.io/irradiance_benchmark/
• Code: https://github.com/Oshadha345/irradiance_benchmark
• Checkpoints: https://huggingface.co/OoshadhaSam/solar-irradiance-visual-backbone-benchmark
• Paper: https://arxiv.org/abs/2607.23633
• PDF: https://arxiv.org/pdf/2607.23633

Repository Contents

This repository contains the 38 learned checkpoints reported in the camera-ready benchmark:

• 18 Folsom visual-backbone models and one temporal-only diagnostic.
• 18 NREL visual-backbone models and one temporal-only diagnostic.
• Sanitized inference configurations, exact training configurations, test metrics, efficiency reports, and checksums.

Smart persistence is an analytical baseline and therefore has no learned checkpoint.

checkpoints/
├── folsom/
│   ├── convnext-tiny/
│   ├── ...
│   └── temporal-only/
└── nrel/
    ├── convnext-tiny/
    ├── ...
    └── temporal-only/

Each model directory contains:

File	Purpose
model.safetensors	Inference-only PyTorch state dictionary
config.yaml	Portable inference configuration with external weight paths disabled
training_config.yaml	Camera-ready training configuration with machine-specific paths redacted
test_metrics.json	Strict test-set metrics
efficiency_report.json	Parameters, FLOPs, and FPS where available
best_val_metrics.json	Metrics associated with checkpoint selection
erf_overlay.png	Effective receptive field output where available

Benchmark Protocol

Axis	Fixed setting
Input	Circularly masked 224 × 224 RGB sky image
Weather history	40 timesteps × 7 channels
Target	10-minute-ahead clear-sky index, transformed to GHI
Visual descriptor	Four projected stages concatenated to 1024-D
Temporal descriptor	Single-layer LSTM, 128-D
Head	Concatenation, LayerNorm, Linear, GELU, Dropout 0.3, Linear
Optimization	AdamW, Huber loss, batch size 32, 8 epochs, seed 42
Efficiency	Batch size 4, 10 warm-up and 50 timed iterations

Headline Results

Dataset	Model	RMSE (W/m²)	Forecast skill (%)
Folsom	Smart persistence	81.37	0.00
Folsom	Temporal-only	69.51	14.57
Folsom	VMamba Small	65.39	19.64
Folsom	Swin Base	65.50	19.50
NREL	Smart persistence	17.48	0.00
NREL	Temporal-only	21.33	-22.00
NREL	Swin Tiny	23.76	-35.94

NREL has only 313 matched strict-test samples and is provided as a low-data stress test. It should not be interpreted as a statistically resolved ranking of visual architectures.

Loading a Checkpoint

Install this codebase and the dependencies required by the selected backbone, then download one model directory:

from huggingface_hub import snapshot_download

local_dir = snapshot_download(
    repo_id="OoshadhaSam/solar-irradiance-visual-backbone-benchmark",
    allow_patterns=["checkpoints/folsom/vmamba-small/*"],
)

Load the portable configuration and safetensors state dictionary:

from pathlib import Path

import yaml
from safetensors.torch import load_file

from models import build_model

model_dir = Path(local_dir) / "checkpoints/folsom/vmamba-small"
config = yaml.safe_load((model_dir / "config.yaml").read_text())
model = build_model(config)
model.load_state_dict(load_file(model_dir / "model.safetensors"), strict=True)
model.eval()

Dataset loaders and preprocessing are maintained in the associated GitHub repository.

Intended Use

These checkpoints support reproduction and analysis of the controlled MERCon 2026 benchmark. They are intended for research on short-term solar irradiance forecasting, visual-backbone comparison, efficiency analysis, and receptive-field interpretation.

Permitted use is limited to non-commercial research and evaluation under the checkpoint license below. Commercial use requires the user to obtain any necessary permissions from the benchmark authors and applicable upstream licensors, including NVIDIA for MambaVision-derived checkpoints.

Training Data

The release contains no dataset files. Models were trained with chronological, site-specific splits:

Dataset	Train	Validation	Test	Test period
Folsom	385,115	47,598	224,022	2016
NREL	580	64	313	2020

Users must obtain the datasets independently and comply with their respective access and usage terms. The associated code repository contains the preprocessing and split definitions needed for reproduction.

Limitations

• The models were evaluated at one fixed multimodal operating point, not individually tuned to each backbone.
• The reported runs use a single fixed seed.
• NREL is a small matched-sample stress test.
• Dataset access and preprocessing remain the responsibility of the user.
• The checkpoints are not validated as operational safety-critical forecasting systems.

Citation

@misc{samarakoon2026controlledvisualbackbonebenchmarkmultimodal,
  title         = {A Controlled Visual-Backbone Benchmark for Multimodal Short-Term Solar Irradiance Forecasting},
  author        = {Oshadha Samarakoon and Dushan Herath and Ishara Ranmandala and Dilshara Herath and Roshan Godaliyadda and Parakrama Ekanayake and Vijitha Herath},
  year          = {2026},
  eprint        = {2607.23633},
  archivePrefix = {arXiv},
  primaryClass  = {eess.IV},
  url           = {https://arxiv.org/abs/2607.23633}
}

License

The checkpoint artifacts in this repository are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license (CC BY-NC-SA 4.0). This license requires attribution, prohibits commercial use, and requires adaptations to be shared under the same or a compatible license.

This checkpoint license does not relicense source code, datasets, papers, third-party software, or upstream pretrained models. Those materials remain governed by their respective licenses. See CHECKPOINT_LICENSE.md for the attribution notice, modification statement, scope, and upstream lineage.

Upstream Attribution

Backbone lineage	Upstream implementation or weights	Upstream terms
ConvNeXt	facebookresearch/ConvNeXt, accessed through timm	Upstream repository and weight terms
Swin Transformer	microsoft/Swin-Transformer, accessed through timm	Upstream repository and weight terms
VMamba	MzeroMiko/VMamba	MIT
Spatial-Mamba	EdwardChasel/Spatial-Mamba	Apache-2.0
MambaVision	NVlabs/MambaVision and NVIDIA checkpoints	Code: NVIDIA Source Code License-NC; pretrained weights: CC BY-NC-SA 4.0
PyTorch image models	huggingface/pytorch-image-models	Apache-2.0

The released models modify their initialization checkpoints through supervised fine-tuning for single-horizon solar irradiance regression, replacement of the original classification interface, and integration with the benchmark's temporal LSTM and regression head.