Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to utilize semantic information for defending against adversarial attacks. The motivation seems interesting. 
strength_and_weaknesses: Strengths:
- The motivation for applying semantic information in defending against adversarial attacks seems interesting.

Weaknesses:
- There lacks the SOTA defending models in comparison, such as [A],[B]. The performance gain looks limited compared to the comparison methods (not the latest ones).
- It is not clear how these semantic features come.

[A] “Learnable boundary guided adversarial training.” CVPR 2021.
[B] “Adversarial feature desensitization.” NeurIPS, 2021.

clarity,_quality,_novelty_and_reproducibility: Applying semantic information in defending against adversarial attacks seems novel.
summary_of_the_review: The motivation for applying semantic information in defending adversarial attacks seems interesting. However, the effectiveness of the proposed method is not well justified.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose to improve the adversarial robustness of image classification models by adding constraints between visual representations and semantic word vectors. The authors first demonstrate the correlation between visual representations and semantic word vectors and how robust features are aligned with semantic word vectors. Then they propose an adversarial framework (SCARL) to learn robust visual representations through regularizing the structure of visual representations and that of semantic word vectors. As the result, the authors demonstrate the proposed framework improves the model's robustness compared to current approaches.

strength_and_weaknesses: Strength: 
1. The intuition and analysis of the correlation between robust visual representations and semantic word vectors are interesting and meaningful. This opens another lens for understanding the adversarial robustness of vision models and robustness of visual representations.
2. The paper is well-written and the proposed methods are clearly formulated.
3. The theoretical analysis for the lower bound is helpful. But some comparisons between the results w/ and w/o utilizing this estimated lower bound would be appreciated.
Weakness: 
1. The similarity analysis in Figure 2 is not well interpreted. How such similarity metrics are normalized is crucial for interpreting the results.
2. The claim in caption of Figure 1 that FGSM is always less robust than standard AT lacks supporting evidences.
3. Comparison with existing methods can be improved. Some of the existing methods can trade off accuracy over robustness (.e.g. TRADES). It might be fairer if the authors also include results of TRADES with different settings.
4. There is a significant performance gap on natural accuracy of models trained with SCARL in CIFAR-10. That is an interesting phenomenon that might be due to limited richness of semantic information in the visual representations. The authors should discuss more on this and other potential limitations.

Some minor things:
P3: nor-robust -> non-robust
Fig 2,3 visible margins of the images. They are also redundant in the appendix.
Other suggestions:
	There is a line of research exploring how semantic word vectors improve the generalization/robustness of the vision models [1] might also be related. Some discussion of differences and future directions would be interesting.

[1] Grounded Language-Image Pre-training

clarity,_quality,_novelty_and_reproducibility: This paper did not provide the code. Hope they can release the code. 
summary_of_the_review: See above.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new adversarial training scheme that leverages the semantic information from language embeddings to improve the robustness of the model to adversarial attacks. The proposed method is evaluated against various adversarial attacks and compared to multiple methods, showing competitive results.
strength_and_weaknesses: Strengths:
1. The paper is well organized. The writing is clear, and it is very easy to follow.
2. The idea of maintaining the semantic information of the class labels during adversarial training is sound.
3. The proposed method is evaluated on benchmark datasets with various adversarial attacks. The performance is generally competitive compared to prior arts. It also conducts extensive ablative studies on the components of the method.
4. The paper provides clear details on reproducing the method.

Weakness:
1. The improvements over previous methods are not convincing enough. The margins are small, especially considering that the proposed method requires more information, i.e., a pretrained word embedding. This might require more resources in real-world applications than other methods, making it less attractive.
clarity,_quality,_novelty_and_reproducibility: Clarity: The paper is well-written and easy to follow.

Quality & Novelty: The paper seems novel and has some potential. But the empirical results are not convincing enough to show significant advantages over existing works.

Reproducibility: The paper contains enough information to reproduce the method.
summary_of_the_review: This paper proposed a novel adversarial training method that enforces the alignment between the visual feature and the label semantics from a pretrained word embedding to improve the robustness of adversarial attacks. It shows some potential, but I am not very convinced by the results, considering it requires much more resources than other methods that do not rely on external information. Therefore I am more inclined to rejection.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper introduces an approach for adversarial robustness using semantic information to assist model training. Specifically, the authors propose Semantic Constraint Adversarial Robust Learning (SCARL) and demonstrate the impact of their approach on model robustness on multiple benchmarks. 

Strengths
--------------

-The idea of applying semantic information in defending against adversarial attacks is well-motivated and interesting (Reviewer Xby6,  Reviewer FuyA )
-The paper is well-written  and organized ( Reviewer FuyA and Reviewer ARcW)
-Sound theoretical analysis ( Reviewer FuyA)
- The performance is generally competitive compared to prior arts (Reviewer ARcW)


Weaknesses
---------------
-The performance gain compared to SOTA is limited; small margins (all reviewers)
- Experiments with TRADE were requested by Reviewer FuyA, but the improvement is also not very large. 
- Additional minor issues with Fig2 and missing discussions that were partially addressed in the author response period. 


The reviewers recommended 
-5: marginally below the acceptance threshold	
-6: marginally above the acceptance threshold	
-5: marginally below the acceptance threshold

justification_for_why_not_higher_score: All the reviewers agreed that the paper covers an interesting problem, the use of semantic information to approach adversarial robustness. However, there is a consensus that the paper is not yet ready for publication in its current version. 

PC members suggest the following improvement, which may strengthen the paper's future version.
- show stronger performance compared to the state-of-the-art. There could be better ways to explore leveraging semantic information to make the gap clearer or probably use other/additional sources of semantic information. 
- experimentation on additional datasets with larger number of classes and hopefully show how the use of semantic information can help as the data/ vocabulary of classes increases.
-Explore how the semantic information may benefit low-resource classes, which has few/no examples 
justification_for_why_not_lower_score:  N/A.
Decision: Reject