Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents a robust GAN-Inversion (RGI) method for mask-free image inpainting and unsupervised pixel-wise anomaly detection. Furthermore, they further propose a relaxed RGI method for achieving better results. Promising experimental results on image inpainting and anomaly detection by using the proposed method are shown with comparison with some previous methods.
strength_and_weaknesses: Strengths:
1. This paper resents a robust GAN Inversion (RGI) method and a relaxed RGI (R-RGI) method for mask-free image inpainting and anomaly localization problems.
2. This paper presents two theorems for robust RGI that provides the theoretical foundation for the optimization formulation of the proposed algorithm.
3. Experimental results by applying the proposed RGI and R-RGI methods to two application problems on some public datasets are shown with promising results.

Weaknesses:
1. The proposed RGI or R-RGI methods require to solve the large optimization problems during the inference. There is no discussion on the computational cost involved in solving the large optimization problems during the inference. It could be very computationally infeasible for practical application.

2. Another limitation with the GAN inversion approach is that it requires a large number of normal images to train a GAN model for the specific task. In many cases, there are not enough numbers of normal images for training the GAN models. In their experiments on image inpainting, the datasets used in the experiments are CelebA, Standard cars, and LSUN bedroom all contain very large numbers of images for the GAN model training. But for MVTec and BTAD datasets, the numbers of training images are not a lot. There is no discussion on the numbers of images used in the model training for the implementation on anomaly detection experiments. There should be more discussion on the issue of training data requirement for the proposed method.

3. The experiments on pixel-wise anomaly detection are only performed on their synthetic defect dataset based on BTAD dataset. The experimental evaluation is not sufficient to demonstrate the proposed RGI or R-RGI achieve SOTA performance for anomaly detection. They should perform experiments with the standard protocol on the commonly used datasets, such as MVTec and BTAD datasets and include experimental comparison with some recent SOTA methods.

4. The discussion in section 3.3 claims the proposed RGI method is connected to robust statistics and the loss function in eq. 3 can be simplified by using the robust error function, which avoids introducing M in the loss function. However, there is no justification of this claim. If this is the case, is the resulting optimization problem also simpler to solve? 
clarity,_quality,_novelty_and_reproducibility: The formulation of the proposed RGI and R-RGI methods is clear, but the implementation details are not clear enough and the limitations of the proposed methods are not discussed in the paper.

I have concerns with the quality of the paper in terms of the presentation and experimental justification.

The novelty of the paper seems solid mainly because this paper presents new extended formulation for GAN inversion with nice theoretical proofs. 

This paper is not easy to reproduce since it doesn't provide implementation details on the GAN model training amd the network architecture of the GAN model used in the experiments.
summary_of_the_review: In summary, the paper has nice novelty in tern of the proposed new algorithms for solving the mask-free semantic inpainting and pixel-level anomaly detection problems. The proposed methods are accompanied by two theorems for robust GAN-Inversion. However, there are some concerns with the paper in terms of lack of discussion of the critical  limitations for the GAN-Inversion approach and the insufficient experimental justification, especially on the anomaly detection experiment.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors proposed a robust GAN-inversion method to cope with restoring images with unknown corruption and detecting unknown defects. They addressed the problem by involving the mask in the optimization process, and encouraging it to be sparse. Besides, to further close the GAN gap, they proposed to finetune the GAN generators while adapting new datasets. The authors showed the experimental results on both mask-free inpainting and unsupervised defect detection tasks, and claim it outperforms SOTA, and can be applied to other SOTA framework easily. 
strength_and_weaknesses: Strength:
- The paper is well-written and easy to follow. The two tasks are closely related and the proposed methods make sense.
- The two-stage optimization further closes the GAN gaps. The generalization of the method is intuitive.

Weaknesses:
- The idea is simple, effective and intuitive, but the problem setting is not that practical. 
    (1) Center block is too easy to be overfitted, and it cannot reveal the advantages of the mask learning. 
    (2) Random missing is also not practical enough, and a simple denoising method may be more effective in resolving the problem.
    (3) Synthetic defects are limited, and cannot be proven to be generalized to real tasks. 
- The sparsity assumption is limited. Similar ideas can also be found in paper [1], while the authors of that inpainting paper proposed to jointly optimize the mask and reconstruction loss during training.  
- It would be interesting to study whether we can learn the regional GAN-inversion by matching partial generated images with the ground truth. For example, whether we can optimize the inversion network to only extract the foreground objects or salient objects. It may be an interesting extension of this work. 
- Finetunning the generator may be not that practical for real applications. 

[1] Zeng, Yu, et al. "High-resolution image inpainting with iterative confidence feedback and guided upsampling." European conference on computer vision. Springer, Cham, 2020.
clarity,_quality,_novelty_and_reproducibility: The writing quality is good. 
summary_of_the_review: The major concerns of this paper:
- The synthetic dataset (for both tasks) fail to convince the reviewer that the proposed method is effective enough to recover the unknown corruption in the wild. If the author claimed only synthetic dataset can be used due to the lack of clean images for training GANs, then there are additional concerns on how the proposed methods can be generalized to more real-world tasks. Or the problems are not about the synthetic dataset, but the synthetic data (center block / random pixel missing) in this paper is not that realistic and diverse to reveal the advantages of the proposed method.
- The evaluation is poor, especially for the mask-free inpainting tasks. Baselines are weak, and the metrics are not advanced to demonstrate the visual quality. For GAN-based generation, at least LPIPS/FID should also be reported. 
- More interesting topics like partial GAN-inversion should be explored to prove the robustness.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the GAN-Inversion problem. In particular, the GAN-inversion problem when the image suffers from missing or some pixels are no longer reliable. The proposed method is very simple and straightforward. A mask is introduced to the inversion optimization, and the mask has a constraint on the 0-norm. The results seem to be good, and this simple method works. The author also provided theoretical proof for this method.

strength_and_weaknesses: Strength:
The problem is important. The solution is simple, and the results are good.
It's actually surprising that such a simple method can work. As far as I know, before the stylegan came along, this kind of method was very difficult to work.

Weaknesses:
1. The trade-off parameter $\lambda$ seems to be an important value, but the article does not show the sensitivity of the method to lambda.
2. The proof may not make much sense. This paper provides some theoretical guarantees. I briefly checked the proof. Its biggest problem is that some assumptions basically do not appear in real-world applications. But the final experimental result was good. Probably this does not experimentally affect the performance of the proposed method. But on the other hand, this reduces the contribution of this paper, because it seems that this guarantee is not important.
clarity,_quality,_novelty_and_reproducibility: The content of the article is relatively easy to understand. The method is not complicated and should be easily implemented.
summary_of_the_review: My rating is based on: 1) there are no major technical omissions, 2) the problem studied is important, and 3) the proposed method works. But since 1) the method is simple, 2) the proof is of limited value. I don't rate it too high.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper presents a Robust GAN-inversion method for restoring gross corrupted images. They also extend RGI to Relaxed RGI for generator fine-tuning to mitigate the gap between the GAN learned manifold and the true image manifold. Their result achieves sota performance in two applications: mask-free semantic inpating and unsupervised pixelwise anomaly detection.
strength_and_weaknesses: Strength: 1. Paper is well written and easy to understand.
2. The research problem is important and has practical value.
3. Their proposed method seems novel.

Weakness: The experiment seems not thorough, For example, In Table 1, I'm curious about why they only compare limited cases in the experiment. I think the method should compare with more previous works and datasets to demonstrate its efficiency. Besides, I think the paper should add an ablation study to demonstrate the method.
clarity,_quality,_novelty_and_reproducibility: The paper's idea is novel, but the experiement is not thorough, they presented limited cases in the paper.
summary_of_the_review: I think the paper's idea is novel, and the topic of this paper is also very important, I think the paper should add more experiments to demonstrate their method.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a GAN-Inversion method to solve the image restoration problems. From the results shown in this paper, this author emphazie a lot on the image inpainting problem, and also show the potential in solving the anomaly detection problems. 
strength_and_weaknesses: The strength and weakness of this problem are all very obvious. 
1. By using the GAN-Inversion, the author achieves better results on non-mask image inpainting scenarios, which help the classical image inpainting problems overcome a challenging obstacle. 
2. I know that for image inpainting problems, the guys always overfit their module some specified scenarios, and show very promising results on some test data with similar properties. Yeh etal is just like this. I am not sure whether this method is still in this way. If they still solve this problem in this manner and show poor generalization ability to other scenarios, I donot think the GAN-Inversion is a big plus. 
3. To relive my concern, please provide more details in the rebuttal about the details of training data, for example, if you want to get the result on CELEBA dataset, do you need to use similar data to train your network for the generative image prior?
4. Is it ok to show more results beyond current established dataset? 
5. Is it ok to provide more details about Figure 2? From the figure, this work still relies on some kinds of mask to achieve the results, though this mask is not explicitly obtained. However, considering the examples in this figure, where the mask is with significant differences to its surrounding regions, it may not be very difficult to estimate such a mask. 
6. Image inpainting has already been a rat race. Many methods have been proposed. Why do you only compare with so limited methods?
clarity,_quality,_novelty_and_reproducibility: 1. We all agree that image inpainting is not a new problem. 
2. We all agree that gan-inversion technique is not a new method. 
3. Thus using 2 to solve 1 is actually an increanmental improvement. 
4. For clairty, its Figure 2 makes me feel that this method has some shortages. Please address it. 
5. I can not evaluate the reproducibility. 
summary_of_the_review: Well. I am also very busy recently, thus I may not have enough time to go through every detail of this paper. However, based on my experiences in the past several years, this paper still needs some time to be better refined.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents a robust GAN-Inversion (RGI) method for mask-free image inpainting and unsupervised pixel-wise anomaly detection. The paper also presents a relaxed RGI method for better results. The paper includes experimental results for image inpainting and anomaly detection, showing promising results compared to previous methods. Reviewers think the method is overall simple and effective. However, the paper lacks discussion of the computational cost involved in solving the large optimization problems required for the proposed methods (in responses the authors admitted the computational cost can be an issue for online inference), and the experiments can also benefit from more analysis (such as for the domain-gap in anomaly detection). Other potential limitations of the paper include the need for a large number of normal images to train the GAN model, and the lack of discussion on the number of images used for training in the anomaly detection experiments. Overall, this is a borderline paper, but reviewers think it may bring some additional useful discussions to the community, so I'm leaning towards acceptance.
justification_for_why_not_higher_score: It is a borderline paper.
justification_for_why_not_lower_score: Some reviewers expressed concerns on the novelty of the work, and the limitations of the method mentioned above.
Decision: Accept: poster