Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper studies clean-label availability attacks against unsupervised contrastive learning. The authors first observe that previous poisons, designed to compromise supervised learning, fail to harm contrastive learning algorithms, such as SimCLR and BYOL. Then, the authors propose a new attack method tailored to compromise contrastive learning. Surprisingly, the proposed attack is not only effective for contrastive learning, but also for supervised learning. 
strength_and_weaknesses: Strengths:

- This paper is the first to study availability attacks against unsupervised contrastive learning.
- A dual branch scheme is proposed to mitigate the optimization obstacles and thus boost the attack performance.
- Experiments provide several novel observations:
  - It is an interesting observation that sample-wise CP mostly performs better, while class-wise CP can transfer better.
  - While almost all previous poisons in this area are linearly separable, the proposed poison is not. This would make availability attacks more undetectable.
  - A very simple data augmentation method based on Matrix Completion is proposed to mitigate poisons. The authors find that this simple method can perform better than adversarial training.

Weaknesses:

- Given that Matrix Completion is very effective in defending against the proposed attack, it is natural then to ask whether this simple method can defend against previous poisons, such as unlearnable examples and adversarial poisoning.
- Table 8 tests the defense power of AdvCL (i.e. a type of adversarial training) against the proposed poisons. Is the perturbation radius of adversarial training $\epsilon=8/255$? Prior work (Tao et al., 2021, Figure 5) has shown that adversarial training with small $\epsilon$ could perform better. Is this also true in the context of unsupervised contrastive learning? By the way, adversarial training was first identified by Tao et al as a promising defense against those poisons. Thus, the last sentence of the introduction needs to polishing.
- There are availability attacks that can work by injecting dirty labels into the training data (Biggio et al., 2012). That is to say, not all availability attacks are imperceptible. Thus, the first sentence of the introduction needs to polishing.
clarity,_quality,_novelty_and_reproducibility: The paper is clearly written, well-organized, and easy to follow. 
summary_of_the_review: The paper explains why previous poisons fail and shows how to effectively compromise contrastive learning. This paper observes many novel empirical findings. These constitute a thorough study of clean-label availability attacks against unsupervised contrastive learning.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper considers an adversarial setting for contrastive learning: the attacker is allowed to add imperceptible perturbation to all training samples of CL, which will reduce CL's effectiveness as a feature extractor. Experiments show that downstream task's performance will be lowered by the poisoning.
strength_and_weaknesses: **Strength**

This paper is technically sound. The experiments considers a wide range of baselines. I appreciate the effort.

**Weaknesses**

As the authors have self-identified in the limitation, the attack setting seems to be slightly artificial: the attacker has control over *all* training data is a very strong assumption... The attack mechanism is not too surprisingly novel either. I feel that the paper has presented much empirical evidence of what effect the attack is able to bring, but fails to provide in-depth analysis of its implication. For example, on Page 8, the authors have shown that noise in SL attack is sometimes linear separable, while that in CL attack is not. It is a very interesting observation. However, does that mean linear separable noise is easier to detect? or, what does this linear (non)separability suggest? In future edition or work, the authors may dig further the implication of such phenomena to contrastive learning or more general learning problem. Such insights could be more helpful than the attack/defense itself, given that the adversarial setting is not very common in practice.


clarity,_quality,_novelty_and_reproducibility: The paper is clear in describing settings, technical details and empirical finding. Its novelty and impact is somewhat hindered by the less realistic attack setting.
summary_of_the_review: Overall, this is a paper with solid empirical evaluation but slightly lacking novelty. To me, it is on the borderline.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper poisons unlabeled training data of contrastive learning (CL) to reduce the test accuracy of linear probing. The imperceptible poisons are iteratively optimized to minimize the CL objective function. After the poisons are generated, the authors add them to the training set and run the target CL algorithm from scratch. When the entire training set is poisoned, there is a clear drop in the linear probing accuracy, e.g., from ~90% to ~50% on CIFAR-10.


The poison generation proceeds as follows. At each iteration, the model parameters are first updated to minimize CL loss and then the poisons are also updated to minimize CL loss. This follows the basic idea in Huang et al (2021) although Huang et al (2021) minimize classification instead of CL loss. One worth mentioning difference is the authors show data augmentations of common CL algorithms are differentiable. They take the gradients of data augmentation into consideration when updating the poisons.

The authors analyze why the poisons work. They show the poisons work by shortcutting the instance-wise CL task, whose objective is to produce similar representations of two different augmented views. They show the poisoned embeddings of two augmented views are close while the clean embeddings are far.

strength_and_weaknesses: **Strength**

1. This paper gives the first attack against contrastive learning. It is interesting to know that one can find shortcuts for the instance-wise CL task.

2. Several interesting tricks, e.g., letting the gradient back through the momentum encoder and differentiable data augmentations, are proposed to improve the attack.

3. The authors show several defenses, including a newly proposed matrix completion, that can be used to defend against contrastive poisoning.

**Weaknesses**

1. How important is using differentiable data augmentations? I do not see an ablation study on this. Will the poisons be useless without this trick?

2. In Figure 4 you explain why contrastive poisoning works but only use class-wise poisons. It would be better if you could include the results of sample-wise poisons.

3. Most of the experiments assume the attacker has full access to the training data. Although most of previous works also use this assumption, it does not change the fact that it would probably be unrealistic.

4. The explanation of why CP works is that it provides shortcuts for the model to produce similar representations of different augmented views. If this is true, it is possible to directly optimize the poison so that it remains unchanged when the data augmentation is applied. For example, a circle is invariant to rotation. This data/model independent approach is more efficient than the optimization procedure.

clarity,_quality,_novelty_and_reproducibility: Please see Strength And Weaknesses. I hope the authors would reply to my first two concerns to further improve this work.

The paper is well written in general. There is a missing reference in Appendix C.2.
summary_of_the_review: Given that this paper is the first to show it is possible to design shortcuts for instance-wise CL task, I recommend acceptance.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper studies indiscriminate poisoning attacks on contrastive learning. Existing attack strategies developed for supervised learning are shown to be ineffective on contrastive learning. A novel strategy is thus proposed that not only is shown to be highly effective on contrastive learning but also impacts supervised learning models. A new defense measure is proposed based on data augmentation.

All reviewers and the AC agree that this paper makes several interesting and novel contributions. As contrastive learning becomes increasingly popular in many fields, it is very important to study how vulnerable various approaches are and to understand how various attacks could harm the resulting models. The paper shed light on several of theses aspects and the authors did a good job at addressing the reviewers comments. 

The weakness of the paper are adequately discussed by the authors and we strongly encourage them to pursue this line of work and perhaps design poisoning attacks that are even more efficient w.r.t to the fraction of poisoned data.

justification_for_why_not_higher_score: The paper provides novel and interesting contributions worthy of a spotlight presentation. However these might not groundbreaking enough to warrant an oral presentation. 
justification_for_why_not_lower_score: This paper is the first to study indiscriminate poisoning on contrastive learning. 
Given the increasing use of contrastive learning and the rich insights provided by this paper, the contributions are worthy of a spotlight presentation.
Decision: Accept: notable-top-25%