Sunmj/IRecon-MCM

IRecon MCM 128

Inference-only modulation and conditioning module for registered SWIR, MWIR, and LWIR fusion.

Architecture

• 12 input channels: noisy latent, frozen base noise prediction, three bands, and object mask.
• 96 hidden channels, six time-conditioned residual blocks.
• 8 outputs: four noise residual channels and four spatial gates.
• 1,371,080 trainable parameters.
• Frozen Stable Diffusion 1.5 inpainting base, 10-step DDIM inference at 128 x 128.

Training provenance

• Object-disjoint Objaverse simulation.
• Direct online-noise training from a randomly initialized MCM; no clean pretraining is contained in this checkpoint.
• Selected checkpoint: cumulative step 145,000, training seed 3409.
• Validation masked image MAE at selection: 0.1384654571.
• Loss: diffusion noise MSE + 0.1 image Smooth-L1 + 0.05 first-gradient loss.
• Online perturbation: band-dependent global noise and local spherical-region noise.
• Global noise sigma (SWIR/MWIR/LWIR): 0.08 / 0.025 / 0.01.
• Local spherical-region sigma (SWIR/MWIR/LWIR): 0.20 / 0.10 / 0.05.

The original checkpoint also contained optimizer and scaler state. The released file strips those states and stores only the MCM state dict and non-identifying inference metadata.

Evaluation boundary

The thesis reports an MAE of 0.21288, PSNR of 13.9054 dB, and SSIM of 0.54345 for one validation-selected checkpoint averaged over four inference perturbation/sampling seeds. Independent training-seed results are reported separately. The model improves simulated intensity restoration over fixed linear fusion but does not yet improve the downstream GSO pose/mesh initializer over noisy SWIR. It has not been calibrated on a real three-band sensor.

Required base model

Supply a legally obtained Stable Diffusion 1.5 inpainting Diffusers directory. This adapter does not include base-model weights.

File integrity

SHA256: 573516143fb7a3c59256c449f0d2b34aa56be6945e3c9ba8a0cc307d2cd68739

Repository: github.com/ss00sxt/IRecon