Restoration by Generation with Constrained Priors
Authors: name: Zheng Ding; name: Xuaner Zhang; name: Zhuowen Tu; name: Zhihao Xia
Published: 2023-12-28 17:50:54

Abstract: The inherent generative power of denoising diffusion models makes them
well-suited for image restoration tasks where the objective is to find the
optimal high-quality image within the generative space that closely resembles
the input image. We propose a method to adapt a pretrained diffusion model for
image restoration by simply adding noise to the input image to be restored and
then denoise. Our method is based on the observation that the space of a
generative model needs to be constrained. We impose this constraint by
finetuning the generative model with a set of anchor images that capture the
characteristics of the input image. With the constrained space, we can then
leverage the sampling strategy used for generation to do image restoration. We
evaluate against previous methods and show superior performances on multiple
real-world restoration datasets in preserving identity and image quality. We
also demonstrate an important and practical application on personalized
restoration, where we use a personal album as the anchor images to constrain
the generative space. This approach allows us to produce results that
accurately preserve high-frequency details, which previous works are unable to
do. Project webpage: https://gen2res.github.io.