Degradation-Modeled Multipath Diffusion for Tunable Metalens Photography
Authors: name: Jianing Zhang; name: Jiayi Zhu; name: Feiyu Ji; name: Xiaokang Yang; name: Xiaoyun Yuan
Published: 2025-06-28 04:48:37

Abstract: Metalenses offer significant potential for ultra-compact computational
imaging but face challenges from complex optical degradation and computational
restoration difficulties. Existing methods typically rely on precise optical
calibration or massive paired datasets, which are non-trivial for real-world
imaging systems. Furthermore, a lack of control over the inference process
often results in undesirable hallucinated artifacts. We introduce
Degradation-Modeled Multipath Diffusion for tunable metalens photography,
leveraging powerful natural image priors from pretrained models instead of
large datasets. Our framework uses positive, neutral, and negative-prompt paths
to balance high-frequency detail generation, structural fidelity, and
suppression of metalens-specific degradation, alongside pseudo data
augmentation. A tunable decoder enables controlled trade-offs between fidelity
and perceptual quality. Additionally, a spatially varying degradation-aware
attention (SVDA) module adaptively models complex optical and sensor-induced
degradation. Finally, we design and build a millimeter-scale MetaCamera for
real-world validation. Extensive results show that our approach outperforms
state-of-the-art methods, achieving high-fidelity and sharp image
reconstruction. More materials: https://dmdiff.github.io/.