Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper studied the inconspicuous and transferable adversarial attacks on face recognition models. Different from previous works that consider $L_p$ norm perturbations, this paper introduces the framework of adversarial attributes, which generates noise on the attribute space of face images based on StyleGAN. An importance-aware attribute selection approach is proposed to ensure the stealthiness and attacking performance. The experiments on various face models show the effectiveness.
strengths_and_weaknesses: Strengths:

+ A new method for face attack is proposed, which crafts adversarial noises on the attribute space of face images. The general idea is reasonable and well realized.
+ The framework of face attribute attack is illustrated clearly with some novel techniques, such as important-aware attribute selection and multi-objective optimization.
+ The experiments on typical and robust face recognition models show the effectiveness of the proposed method.

Weaknesses:

- A significant drawback of the proposed method is that it is a digital-world attack method, which can hardly be implemented in the physical world.
- The notations used in this paper are unclear. The paper uses $v_i$ to denote the vicinity appro vector, but what is the exact formulation of $v_i$. I guess $v_i=z_i + n_i$. But why do you adopt the $L_2$ norm of $v_i$ in the stealthy loss in Eq. (5)? I guess it should be the $L_2$ norm of $n_i$?
- How to train the whole framework? Do the StyleGAN models keep fixed or need finetuning? How do you choose the training data?
- The semantics of adversarial face images seem to be inconsistent to the original ones. For the third example in Figure 6, I can hardly recognize the attacker image and adversarial image as the same person.
- The paper did not compare with a state-of-the-art patch attack method [33], which is also based on StyleGAN.
questions: 1. Explain the utility of the proposed method, especially the implementation in the physical world.
2. Clarify the notations and training details.
3. Discuss the semantic inconsistency between real images and the adversarial images.
4. Compare with the SOTA face attacks.
limitations: The authors have discussed the limitations and negative societal impact of their work.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents a new attack based on styleGAN to manipulate face image attributes and achieve impersonation. Compared to existing methods including gradient-based, patch-based and stealth-based ones, the proposed method aims to result in less visible artifact (to ensure the adversarial perturbation is imperceptible). The proposed method is specific to face recognition and impersonation attack. Experimental evaluations with typical face recognition model and defensively trained face recognition model demonstrate the effectiveness of the proposed method.
strengths_and_weaknesses: # Strengths

1. The proposed method is clearly-motivated, intuitive and effective as demonstrated by experimental results.

# Weaknesses

1. [important; problem setting and evaluation] Is identity information really disentangled from the attribute dimension? Is editing facial attribute really leaving identity information intact? Compared to existing methods, a fair setting should be generating person image with identity information intact while attributes can be changed. If identity information is changed to any extent, it would be much more easier to impersonate. In this regard only several qualitative example is not enough, because the given examples (figure 1 and figure 6) give me a feeling that the resulting person image is not the original identity... especially the third row in figure 6. If the attribute editing is not purely editing attributes, then the comparison in table 1 and table 2 in fact involves unfair comparison, because the other methods like gradient methods do not manipulate identity.

2. [limitation] The proposed method is deeply dependant on the underlying face generation model (style GAN) in terms of both imperceptibility of modification artifacts (quality of generated image), as well as attribute manipulation. Meanwhile, when a high-quality generator is not available in a different domain (like natural images form imagenet), this method will be invalid in that domain.
questions: My two major concerns about the paper is written in weaknesses. The attribute editing process is not guaranteed to be orthogonal to identity information. Meanwhile, the proposed method is highly dependent on face generator (styleGAN) and not very flexible to be extended to other domains.

---

The authors have justified the mentioned problems. Some of my original concerns persist, but they largely stem from the underlying GAN.
After rebuttal, I raised the score from 4 to 5.
limitations: Limitations not clearly elaborated.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors have proposed to produce inconspicuous and transferable adversarial attacks on face recognition systems. Instead of perturbing the pixel intensity, the authors propose to semantically perturb the facial attributes such as smiling, eyeglass, mustache, blurry, and pale skin. Although adversarially manipulating facial semantics or attributes is not new, in this paper, the authors do bring to the table something new: (1) importance-aware attribute selection strategy that can select and update one particular attribute noise vector that leads to the largest degradation of adversarial loss in each step, and (2) multi-objective optimization that can balance the stealthiness and attacking strength. Experimentally, the proposed method is benchmarked against several types of adversarial attacks including gradient-based noise attacks, patch-based attacks, and stealthy-based attack methods. The evaluation is carried out on both basic face recognition models, as well as robust ones with adversarial training. The proposed method performs favorably compared to the baselines in terms of both the direct attack success rate as well as black-box transferability. 

strengths_and_weaknesses: Strengths:

(1) The paper is well written. The presentation and organization of various components is very clear. The experiments are thoroughly carried out with adequate baselines as a comparison. 

(2) Although adversarial perturbation based on semantic manipulation is not new, the authors have managed to incorporate two new components (importance-aware attribute selection and multi-objective optimization to balance the two conflicting losses) to improve upon prior semantic-based adversarial face perturbations. 

(3) The experimental results are pretty strong, in favor of the proposed method, in terms of the performance on both the basic FR, robust FR, as well as on black-box transferability tasks. 

Weaknesses:

This is overall a solid paper. At this review stage, I do not seem to spot apparent weaknesses in this submission. 


Minor point: on line 206, there is a typo: steady-based should be stealthy-based. 

questions: I do not have questions. 

limitations: Yes.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes an adversarial attack method, Adversarial Attributes, against face recognition. To improve attack transferability, this paper generates attacks on the high-level semantics by injecting the adversary into the edited latent vectors in several attributes. To balance stealthies and attacking strength, an importance-aware attribute selection and the multi-objective optimization strategy are introduced. Experiments carried out on FFHQ and CelebA-HQ datasets demonstrate the effectiveness of the proposed method.
strengths_and_weaknesses: Strengths:

1. The idea of generating attacks through perturbing on the high-level semantics (facial attributes) is interesting and novel and facilitates attack transferability.

2. Proposed Adversarial Attributes Perturbation has some technical novelties with clear figure illustration. Importance-Aware Attribute Selection and Multi-Objective Optimization is reasonable with theoretical support.

3. Evaluation is reasonably thorough and both promising qualitative and quantitative results are claimed. Ablation study also demonstrates the effectiveness of each module in the proposed attack method.

Weaknesses：

From Visualizations in this paper, e.g. Fig. 6, it seems textures of adversarial faces generated by the proposed attack methods are smoothed and lost. This may be one of the drawbacks of the proposed method.

questions: 1. Which equation corresponds to the vicinity appro vector vi? How to define the vicinity appro vector vi statistically?

2. What is the distribution of values of balanced weights w1 and w2? Could you show more results for these two balanced weights?

3. Why most of textures of adversarial faces generated by the proposed attack methods are smoothed and lost?

limitations: Yes
review_title: null

Meta-review: metareview: This paper studies adversarial attacks on facial recognition systems. The key contribution is that, instead of directly manipulating pixel space, this paper proposed to perturb the facial attributes for generating inconspicuous and transferable adversarial examples. The initial concerns are mostly about requiring 1) more ablations/comparisons, and 2) clarifications on experiment details and visualization (especially Figure 6). 

Most concerns are well addressed in the rebuttal, and 3 (out of 4) reviewers agree to accept this paper. The reviewer RXXP is still (slightly) concerned about the novelty contribution and rates it as a borderline case. Given its effectiveness and comprehensive analysis, the AC agrees that this paper has its own merits and will be of interest to the general NeurIPS community, therefore recommend accepting it. 

In the final version, the authors should include all the clarifications and the additional empirical results provided in the rebuttal.


confidence: Certain
Decision: Accept