Denoising as Adaptation: Noise-Space Domain Adaptation for Image
  Restoration
Authors: name: Kang Liao; name: Zongsheng Yue; name: Zhouxia Wang; name: Chen Change Loy
Published: 2024-06-26 17:40:30

Abstract: Although learning-based image restoration methods have made significant
progress, they still struggle with limited generalization to real-world
scenarios due to the substantial domain gap caused by training on synthetic
data. Existing methods address this issue by improving data synthesis
pipelines, estimating degradation kernels, employing deep internal learning,
and performing domain adaptation and regularization. Previous domain adaptation
methods have sought to bridge the domain gap by learning domain-invariant
knowledge in either feature or pixel space. However, these techniques often
struggle to extend to low-level vision tasks within a stable and compact
framework. In this paper, we show that it is possible to perform domain
adaptation via the noise space using diffusion models. In particular, by
leveraging the unique property of how auxiliary conditional inputs influence
the multi-step denoising process, we derive a meaningful diffusion loss that
guides the restoration model in progressively aligning both restored synthetic
and real-world outputs with a target clean distribution. We refer to this
method as denoising as adaptation. To prevent shortcuts during joint training,
we present crucial strategies such as channel-shuffling layer and
residual-swapping contrastive learning in the diffusion model. They implicitly
blur the boundaries between conditioned synthetic and real data and prevent the
reliance of the model on easily distinguishable features. Experimental results
on three classical image restoration tasks, namely denoising, deblurring, and
deraining, demonstrate the effectiveness of the proposed method.