Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper investigate the effectiveness of backdoor attacks against semi-supervised learning, a setting in which the learner has access to a large number of unlabeled data. The paper shows effective attacks against such a learning scheme.
strength_and_weaknesses: **Strength**

The paper is technically sound in developing its attack techniques.

**Weakness**

Presentation is this paper's main weakness. On top of typical English language problems, this paper fails to deliver/highlight the thesis in its introduction.

Example 1. In intro, it says "In this paper we analyze the impact of backdoor data poisoning attacks on semi-supervised learning
methods to highlight a vulnerability of these methods that practitioners should be aware of when
considering the security of their models."  1) Could you, in succinct language, describe what this vulnerability is, and 2) what are "these methods" referring to? The paper primarily examines FixMatch. What are the other methods?

Example 2. In contribution, it says "We analyze the unique dynamics of data poisoning during semi-supervised training and
identify characteristics of attacks that are important for attack success." Could you explain what characteristics you have identified? The paper has a very dense experiment discussion section, in which useful insights are buried in large chunk of text. Could you highlight them in pinpoints?

Moreover, in Section 2.3, the authors have mentioned other work attacking semi-supervised learning. What is the novelty of this paper then? The related work section is not only about listing literatures but more importantly distinguishing your work from theirs. 

Language-wise, there are many unnecessarily long sentences. For example on the metric used on Page 5, "Second is the attack success rate which is the percentage of non-target samples from the test set which are predicted as the target class when triggers are added to them." There are two "which" and a "when" in one running sentence with no comma... Please use short sentences if you couldn't master long ones yet. Paper writing is about clearly conveying your idea with not ambiguity. 

Given the poor quality of presentation, I'm afraid that I can't tell the contribution of the paper clearly. 
clarity,_quality,_novelty_and_reproducibility: Clarity is a big issue of this paper. I've listed my concerns in the previous section.
summary_of_the_review: Overall, this paper has some solid experiment work. However, its presentation hinders me from fully understanding the contribution and novelty. Unfortunately, I do not think it meets the ICLR standard.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors present a backdoor data poisoning attack targeting semi-supervised learning classifiers. For this, the authors rely on a clean-label backdoor attack proposed by Turner et al. where the malicious points injected not only contain the trigger, but also an adversarial perturbation, which is necessary to achieve a high attack success rate. The experiments show the effectiveness of the attack in CIFAR-10 dataset against FixMatch, a recent method for training semi-supervised learning classifiers. 
strength_and_weaknesses: Strengths:
+ Backdoor poisoning attacks in semi-supervised learning have not been explored in the research literature and it can pose a significant risk in some application domains. Although poisoning attacks have been proposed in this context, it is not the case for backdoors. 
+ The background is well covered and the organization of the paper is good. 

Weaknesses:
+ Although the context of application is novel, the proposed attack is a straightforward application of the backdoor attack proposed by Turner et al. in the context of supervised learning, which really limits the practical novelty of the paper. 
+ The attack requires to train a separate model to craft the adversarial perturbations needed for the backdoor attack. From the paper, it is unclear what are the settings for doing this and this can have an impact on the threat model. For example, it does not really make sense if the authors use a supervised learning scheme to train the model and craft the adversarial examples (knowing all the labels of the complete dataset) and the defender only has access to a few labels and uses semi-supervised learning. It looks like the model for the adversary is quite strong (even if the attack is black/grey box. 
+ The attack just targets one semi-supervised learning algorithm, FixMatch. It would be necessary to provide a more comprehensive evaluation against different types of semi-supervised learning algorithms. See for example the targeted attack provided by Carlini et al 2021. 
+ The experimental evaluation is limited: 1) Only CIFAR dataset is used. 2) No defenses are considered (e.g. defenses against poisoning attacks in semi-supervised learning). 3) Only one semi-supervised learning method. 
+ The use of untargeted attacks for generating the adversarial perturbations looks a bit odd. It seems that this is causing the low effectiveness for attacks with higher perturbations. It is unclear why the authors do not use targeted attacks instead to increase the effectiveness of the backdoor attacks as the perturbation increases. 

clarity,_quality,_novelty_and_reproducibility: Overall, the paper is well organized and written. 
In terms of the novelty, as mentioned before, backdoors attacks against semi-supervised learning algorithms are somewhat novel, but the proposed attack is just a straightforward application of the attack proposed by Turner et al. against supervised learning. 
There are certain parts of the experimental settings that are not really clear, as mentioned before (generation of adversarial perturbations). In this sense, what would happen if the attacker just has a limited access to label data points? How do this impact the attack effectiveness? 

summary_of_the_review: The topic addressed in the paper is interesting and novel. However, the method used for creating the attack is basically the direct application of a previous attack. There are also some aspects in the threat model / adversary capabilities that are unclear. The application of the attack by generating untargeted adversarial perturbations is also a bit strange, which explains the counterintuitive experimental results where the effectiveness of the attack is reduced when the adversarial perturbation is bigger. Finally, the experimental evaluation falls short: only one semi-supervised learning algorithm considered, no analysis of defenses, and only one dataset. 
In summary, I think that the research direction is quite interesting, but the paper requires more work and a deeper analysis to be more convincing.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: The paper discusses vulnerabilities of the SSL methods to back-door attacks. The attacks are carried by poisoning unlabelled data. The SSL method of choice is FixMatch, while the analysis is carried on the CIFAR 10 dataset.
strength_and_weaknesses: Strength:
 - none

Weaknesses:
 - The paper, in my view, fails to claim any valid contribution. The idea to study adversarial attacks on SSL  method is not new as showed on section 2.3. There is no thorough discussion of those methods and what this paper brings in addition. Into more detail:
 * the way to carry the attack is given by prior art (Turner et al. 2019)
 * the fact that SSL methods are sensitive to attacks it has been stated before.
 * the amount of attack efficiency compared to previous works in this direction is not quantified. No comparison is carried
 * the discussion is restricted to CIFAR 10 database.
In contrast the paper by Feng et al  "Unlabeled Backdoor Poisoning in Semi-Supervised Learning" ICME 2022 discusses the problems in conjunction with two databases (CIFAR 10 and CIFAR 100), uses 2 architectures compared to 1 here. It compares different ways to attack and shows to severity of each.That paper is published into a conference that is less visible than ICLR. Expectations are higher here

The only potential contribution is that this paper tried to adapt the attack to the augmentation, yet particular and clear lessons are not drawn from this.
clarity,_quality,_novelty_and_reproducibility: The paper is reasonable clear and reproducible. In my view lacks any novelty, given the findings from Feng et al. 
summary_of_the_review: In summary, the contribution, if exists, it is not at the level required by a conference such as ICLR.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper investigates the vulnerabilities of semi-supervised learning methods to backdoor data poisoning attacks on the unlabeled samples and shows that a simple poisoning attack using adversarially perturbed samples is highly effective. While topic is important and valuable to the community, the study is mainly limited to applying Fixmatch for semi-supervised training on Cifar10. Furthermore, the way perturbation based attacks are generated may not be very realistic for semi-supervised learning. Finally, the effectiveness of existing defenses is not studied. It would be very helpful for the community to know whether such vulnerabilities can be alleviated using existing techniques. 
justification_for_why_not_higher_score: Explained in part 1.
Decision: Reject