Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work presents an approach to modify face memorability as a facial attribute using Generative Adversarial Networks (GANs). They essentially propose “memorability modification vector.” Their approach first determines the hyperplane and we can move the latent vector of each image, in the positive or negative direction of the normal vector of that hyperplane. They control the distance of the latent vector from the separating hyperplane to manipulate the memorability of that image. They conducted experiments with StyleGAN and StylyeGAN2 to demonstrate effectivness of their apporaching using FID and KID metrics. 

main_review: Strength 

+ They tackle an interesting problem “What makes an image memorable?”, where the problem is well defined. This work is the first experiment about ‘memorability modification’ that could be applied in various applications. 

+ Their approach solves the problem by introducing the concept of distance between the latent vector or the extended latent vector and the normal vector of the hyperplane.

+ The paper is easy to follow and presented the experimental results with StyleGAN and StyleGAN2. 

Weakness

- Writing can be sigificantly improved. There are several typos and mistakes in writing (e.g., As the result, → As a result).

- Experiments with more datasets are needed because only two StyeGANs are used for evaluation. More SOTA GAN generated images need to be evaluated to provide the effectiveness and validity of this approach.

- Although they tackle an interesting problem, the proposed method is rather simple, and it is a combination of existing methods. Therefore, the novelty is rather weak.

 
summary_of_the_review: Although this work tackles an interesting problem, the novelty is weak. Also, more experiments with different GAN datasets would be helpful. It is limited to StyleGANs. Also, the writing should be improved. Reproducibility is not certain as they did not provide the code.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes to find the hyperplane to separate images with different memorability in the latent-vector spaces of StyleGAN1 and StyleGAN2, then they can move the latent-vectors along the normal vector of the hyperplane to control the memorability of the generated images. A SENet50 trained on the US 10k Face Database with memorability labels is used to find the hyperplane. The experimental results show that the proposed method can change the memorability of images measured by the scores of the SENet50, and this method can be extended to modify real facial images.
main_review: Strengths:
1. The experimental results show that the proposed method does change the memorability of images.
2. The proposed method can be applied to real facial images.

Weaknesses:
1. Are there experimental comparisons with other existing methods that change image memorability? Can those methods be applied to the images generated by StyleGAN? What're the advantages of the proposed method over the existing methods?
2. When choosing VGG16, ResNet50, or SENet50, why the accuracy of them on the US 10k Face Database with real labels is not considered?
3. The accuracy when using the extended latent space (w) is reported in Table 2. It's better to report the accuracy when using the latent space (z) in the appendix.
4. Fig 4, 5, 6, and 8 lack analysis, such as what can be proved by these figures.  The top-left numbers on the facial images are not explained.
summary_of_the_review: The effectiveness of the proposed method is validated by experiments, while the superiority (over existing methods) is not. The experimental results need more detailed descriptions.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper addresses the problem face image memorability. The proposal is based upon the identification in Style GAN of a hyper axis in the latent space where movement is correlated with memorability
main_review: As pros:
 - basic observation (latent space analysis with effect on memorability) is very interesting. it is strong enough to make the paper outstanding. However the paper develops ideas previously published as mentioned in prior work
- technical bases is solidly built
- the evaluation is rather strong. The observation, which again is very interesting, needs to be proven by experimental evaluation and the paper does a good job. Furthermore, all the examples  showed in the paper do confirm the basic idea and developments
- the appendix is useful

As cons:
 - the paper is limited in scope as it addresses only the problem of face memorability
- the experimentation is limited to the statistic of UsFaces database. It is not extended with an analysis over gender, age, race.
- the evaluation does not compare the proposed method with prior art mentioned on pager 3 in paragraph "Modifying Image Memorability."

Small issues:
 - the amount of images used in experimentation is "hidden" in the text. The text should me be made more clear to enhance the quantities used. Also some of the quantities should be added in the figure and tables captions
summary_of_the_review: Overall the idea of of the paper is nice but the evaluation is barely convincing.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a method to control the memorability of face images in the latent subspace of GANs. The basic approach is similar to InterFaceGAN [Shen et al., 2020]. The authors first train a memorability assessor network on the 10k US Adult Faces Database and used the assessor to provide memorability scores on generated face images from StyleGANs pre-trained on the FFHQ dataset. They then find the hyperplane in the StyleGAN latent space to separate low-memorable and high-memorable images according to the memorability score. The memorability of faces is modified by changing the distance from the hyperplane. They demonstrated that their method can successfully control the memorability scores of generated images while keeping the realism as face images and show some qualitative examples and discussions about memorability-modified face images.
main_review: ## Strengths

- The proposed approach is reasonably designed to modify face memorability.

- The paper shows many example images, and we can observe some interesting cases of appearance changes with respect to memorability scores.

## Weaknesses

- Technical novelty is not clear. As discussed in the paper, the underlying approach based on the attribute-separating hyperplane is mostly equivalent to InterFaceGAN. Shen et al. already demonstrated their approach on face image generation tasks, and this work extends it to memorability modification. The concept of image memorability modification for generated faces has been also discussed in [Goetschalckx et al., 2019], and the contribution is not significant from the technical perspective.

- Experimental validation is not sufficient. Essentially, this paper provides qualitative results and quantitative evaluation in terms of face realness (FID and KID metrics) and memorability (assessor score) without any comparison with baseline approaches. There are some missing aspects such as memorability assessment with human participants and comparison with existing memorability modification approaches. In both senses, it would be required to demonstrate the difference from [Goetschalckx et al., 2019] by showing that the proposed method can control human memory performance on face images better.

- While the paper claims that the proposed technique aims at modifying memorability without affecting identity and other facial attributes, there are some cases in the examples whose identity clearly changed after memorability modification. Compared to other relatively objective facial attributes, it is not a trivial task to judge if the facial identity was maintained during the modification and this would require quantitative evaluation by either machine or human evaluator.
summary_of_the_review: The approach is reasonable and has the potential to lead to new findings in face memorability studies. However, given that the technical novelty is relatively incremental and the experimental results obtained at this point are still in the early stages, the research contribution is not sufficient.
review_title: null

Meta-review: The reviewers raised a number of major concerns including the limited novelty of the proposed, inadequate motivation of the design choices and, most importantly, insufficient and unconvincing experimental evaluation presented. The authors’ rebuttal addressed some of the reviewers’ questions but failed to alleviate all reviewers’ concerns. Hence, I cannot suggest this paper for presentation at ICLR.
Decision: Reject