Paper: ODEdit: Blind Face Restoration through Ordinary Differential Equations (ICLR-2024)

Reviewer comment (Experiments):
SDEdit can solve plenty of image editing and synthesis tasks. However, the proposed ODEdit is only for face restoration. Can the authors explain the reasons?

Author response:
The current bottleneck of ODEdit lies in the instability of DDIM inversion. The knowledge that an image represents a face acts as a strong prior, which significantly stabilizes the DDIM inversion, thereby facilitating the application of our method. However, when applied to varied-domain images such as those in ImageNet, the inversion process proved too unstable even before performing any update steps, preventing a fair assessment of the method's performance.

Reviewer comment (Theory):
The advantage that replacing SDE with ODE is not clear. Why should the injection of extra noise be circumvented? Are there any disadvantages when using ODE?

Author response:
In fact, injecting random noise into an input increases the complexity of the denoising task, especially in extreme cases. For instance, if the input is already clean, adding noise to further clean it paradoxically introduces additional stochasticity. A major challenge with SDEdit arises when the input is so severely damaged that a significant amount of noise must be injected to generate outputs that are within the expected distribution. In such cases, fidelity to the original image may be completely lost and such scenarios are the ones we mostly address. Some papers address this issue by adding extra guidance when using SDEdit for denoising. However, we propose a guidance-free method. There are certainly drawbacks to using ODEs. Primarily, the quality of images generated using SDEs appears to be superior. Additionally, our method heavily depends on the accuracy of ODE inversion; inaccuracies in this process can lead to failure, as explained previously. Finally, our method is slower as it requires to perform multiple steps to invert the image, whereas SDEdit simply injects noise.

Reviewer comment (Theory):
It is not clear how can we get Algorithm 1.

Author response:
We have included more details and insights about algorithm 1.

Reviewer comment (Experiments):
The experiments lack some details and some comparisons are unfair and incomplete.

Author response:
Our method is designed to work on 64x64 images, which limits its applicability. Since we propose a blind restoration method and are comparing it with other methods that also claim to be blind, the type of corruption should not be a concern. Therefore, it's reasonable to compare blind restoration methods across various types of corruption. However, it's important to note that some methods in our benchmark specifically incorporate super resolution and blur during training. Consequently, we expect these methods to perform better with these specific types of noise. Our goal is to use corruptions that we believe were not present during training, in order to accurately assess the real robustness of these restoration approaches. We aim to test a wide range of corruptions to evaluate the versatility and robustness of the methods, rather than their exceptional performance in specific scenarios. Additionally, we have included an example of restoration on a real, old, damaged image in the supplementary material, despite the fact that this specific sort of monochromatic damaged images is a known limit of our approach.

Reviewer comment (Experiments):
What is the testing dataset? The authors seem not to describe it in the manuscript.

Author response:
Our method is designed to work on 64x64 images, which limits its applicability. Since we propose a blind restoration method and are comparing it with other methods that also claim to be blind, the type of corruption should not be a concern. Therefore, it's reasonable to compare blind restoration methods across various types of corruption. However, it's important to note that some methods in our benchmark specifically incorporate super resolution and blur during training. Consequently, we expect these methods to perform better with these specific types of noise. Our goal is to use corruptions that we believe were not present during training, in order to accurately assess the real robustness of these restoration approaches. We aim to test a wide range of corruptions to evaluate the versatility and robustness of the methods, rather than their exceptional performance in specific scenarios. Additionally, we have included an example of restoration on a real, old, damaged image in the supplementary material, despite the fact that this specific sort of monochromatic damaged images is a known limit of our approach.

Reviewer comment (Experiments):
Why not compare different methods on real degraded face images?

Author response:
Our method is designed to work on 64x64 images, which limits its applicability. Since we propose a blind restoration method and are comparing it with other methods that also claim to be blind, the type of corruption should not be a concern. Therefore, it's reasonable to compare blind restoration methods across various types of corruption. However, it's important to note that some methods in our benchmark specifically incorporate super resolution and blur during training. Consequently, we expect these methods to perform better with these specific types of noise. Our goal is to use corruptions that we believe were not present during training, in order to accurately assess the real robustness of these restoration approaches. We aim to test a wide range of corruptions to evaluate the versatility and robustness of the methods, rather than their exceptional performance in specific scenarios. Additionally, we have included an example of restoration on a real, old, damaged image in the supplementary material, despite the fact that this specific sort of monochromatic damaged images is a known limit of our approach.

Reviewer comment (Experiments):
The synthetic degradations are based on (Hendrycks & Dietterich, 2019) which is not considered in existing methods e.g. CodeFormer. This makes the comparison unfair. Why not compare different methods based on the testing set of CodeFormer?

Author response:
Our method is designed to work on 64x64 images, which limits its applicability. Since we propose a blind restoration method and are comparing it with other methods that also claim to be blind, the type of corruption should not be a concern. Therefore, it's reasonable to compare blind restoration methods across various types of corruption. However, it's important to note that some methods in our benchmark specifically incorporate super resolution and blur during training. Consequently, we expect these methods to perform better with these specific types of noise. Our goal is to use corruptions that we believe were not present during training, in order to accurately assess the real robustness of these restoration approaches. We aim to test a wide range of corruptions to evaluate the versatility and robustness of the methods, rather than their exceptional performance in specific scenarios. Additionally, we have included an example of restoration on a real, old, damaged image in the supplementary material, despite the fact that this specific sort of monochromatic damaged images is a known limit of our approach.

Reviewer comment (Evaluation):
Face ﬁdelity is not only reflected in reconstruction metrics (PSNR, SSIM) but also in face recognition. Maybe the authors should provide some face recognition metrics to further verify the degree of face fidelity.

Author response:
Thank you for your comment. It is indeed ideal to reconstruct a face that can be recognized as the same individual as in the source  image by face recognition algorithms. However, since our focus is on cases with a high degree of corruption, the faces in the source images are often unrecognizable. In many instances, face recognition algorithms even fail to detect a face in the source images, let alone provide meaningful landmarks. In such cases of high corruptions, comparing directly with the target is not our goal per se. We strive to generate a realistic image as close as possible to the source image.  
As a result, in cases of severely corrupted inputs, generating a realistic, plausible face already constitutes an important step.

Reviewer comment (Experiments):
Compared with existing BFR methods, the proposed method is an unsupervised method without any prior. Then, how does this method perform on real-world face images?

Author response:
Thank you for your comment. We have included an example of an old, damaged image in our appendix. However, the prior model we trained was based on a relatively small number of faces, all of which are centered, aligned, and have pleasing colorations. Consequently, the prior we are using is somewhat limited in directly addressing real-world scenarios. Therefore, our examples should be viewed more as proofs of concept rather than as a definitive model.

Reviewer comment (Experiments):
The experimental part is insufficient and lacks analysis. The experimental results can only verify the superiority of the proposed method but cannot verify its robustness and adaptability. The authors should provide more convincing experiments and detailed analysis.

Author response:
We emphasize that our method performs more consistently across a wide range of different corruptions. This consistency is what we refer to as robustness, as opposed to excelling in handling a specific type of corruption. Indeed, when considering one type of corruption at a time, there are methods that outperform ours. However, these methods often fail more dramatically when confronted with different types of corruption. It is this versatility that we specifically aim to achieve.

Reviewer comment (Reproducibility):
The implementation details are missing. The authors should provide the degradation process of this paper in detail.

Author response:
We have included additional details in the paper.

Reviewer comment (Evaluation):
As seen in Table 1, the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) are worse than previous works.

Author response:
Our method aims to make as few changes as possible to the source image in order to enhance its realism. Consequently, all pixel-based metrics remain close to those of the corrupted image, while the FID and LPIPS metrics show significant improvement. This is a direct result of our approach.

Reviewer comment (Evaluation):
Results are not clear: the base method leads to blurry reconstructions.

Author response:
A: Our method works with 64x64 images, whereas other methods perform super resolution in 512x512. As a result, ODEdit outputs do seem low resolution compared to others because they are. However, ODEdit is not simply producing blurry or averaged reconstruction. We proposed adding a super resolution model post-ODEdit to compare improvements more fairly. Furthermore, other models simply fail to work on severe corruption altogether, while ODEdit makes them work again. We chose to focus on 64x64 images because the ODE solver we used (DDIM) tends to be more and more unstable when moving to higher resolution. However, provided that DDIM inversion works, we added some results we obtained in 256x256 in Appendix that show a significantly higher degree of details. We also added in Appendix example comparing plugging blurred images, as suggested, to plugging ODEdit outputs to super resolution models. Finally, to combine the methods, we only need to input low-resolution ODEdit outputs to other approaches.

Reviewer comment (Reproducibility):
**Algorithm 1** — This is the core of the method. It's unclear what does it mean  ODE_SOLVER( . ). Is this running one step of DDIM Eq(9)? This is used twice in the algorithm. The rationale of the algorithm is not clear or explained.

Author response:
A: We clarified in the paper what is meant by ODE_solver. In our practical case, it consists indeed of running one step of DDIM. It is used twice because it is used backward and forward, to revert the image, and generate a new one, as an ODE can be run both ways. We added more explanations regarding algorithm 1.

Reviewer comment (Evaluation):
**Results of the base methods are blurry** — This method doesn't seem to be doing a good job. Better results are obtained when plugging the method with another prior (CodeFormer, GFPGAN, etc). So in the end, it's more like with this ODEdit you can get a blurry reconstruction (Average) and then from there another prior is enforced to get a better reconstruction. So in the end, where is the gain coming from? Also it's not clear how this method is applied in conjunction with other methods.

Author response:
A: Our method works with 64x64 images, whereas other methods perform super resolution in 512x512. As a result, ODEdit outputs do seem low resolution compared to others because they are. However, ODEdit is not simply producing blurry or averaged reconstruction. We proposed adding a super resolution model post-ODEdit to compare improvements more fairly. Furthermore, other models simply fail to work on severe corruption altogether, while ODEdit makes them work again. We chose to focus on 64x64 images because the ODE solver we used (DDIM) tends to be more and more unstable when moving to higher resolution. However, provided that DDIM inversion works, we added some results we obtained in 256x256 in Appendix that show a significantly higher degree of details. We also added in Appendix example comparing plugging blurred images, as suggested, to plugging ODEdit outputs to super resolution models. Finally, to combine the methods, we only need to input low-resolution ODEdit outputs to other approaches.