Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work investigates how pre-training distributions induce robustness in image-text models. They use Contrastive Language-Image Pre-training (CLIP) as a case study and evaluate it with six publicly available datasets (i.e YFCC, LAION, Conceptual Captions, WIT, RedCaps, and Shutterstock). The overall results and ablation studies show that the robustness of CLIP and other pre-trained image-text models does not depend on the quantity of the training data/data sources, but rather depends on the data quality. In addition, this work shows that mixing/combining multiple data sources/training data only dilutes the robustness of the best individual data source instead of yielding better models. Finally, the authors call for further study into dataset design, citing that web-crawling is not the effective way to build a pre-training dataset for robust generalization. 
strengths_and_weaknesses: Strengths:
1. The paper is well written, clear, and easier to understand and follow.
2. The studied problem is very interesting and important to the NeurIPS community, given that web-based data sources/training data are widely used in pre-training large models.
3. The experimental and research methodology used clearly support the central claim of the paper.

Weaknesses:
1. This work does not provide the details on how the data sources were combined during input mixing in Section 5.1. Were the samples selected randomly or controlled? If randomly did you try selecting using different random seeds? How diverse were the selected samples? This also applies when selecting the subset of the used datasets in Section 3. It would be better to clarify this, since different samples may contribute differently during pre-training?
2. It would also be more helpful to the community if you would discuss/recommend better dataset design approaches for robust generalization. As the paper title touches on both dataset design and robustness.
questions: Please refer to the weaknesses section.
limitations: Please refer to the weaknesses section.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper investigates the effects of training data distributions for vision-language pretraining. Specifically, the authors train CLIP on six publicly available datasets (image-text pairs) individually or collectively and then measure models’ robustness on ImageNet variants. In the experiments, the robustness against distribution shifts is shown as a linear trend (line) of accuracy on the base ImageNet data and its variants. Each line is compared with the y= x line (the perfect model). The empirical results suggest that 1) no single training data provides consistent out-of-distribution performance and 2) combining datasets (each has its own strength) might hurt robustness of the model. The authors also analyze the results of ensembling model outputs trained on different datasets. Lastly, the authors provide theoretical analysis on the universal line only determined by the training and two test datasets (independent from model architectures).
strengths_and_weaknesses: [Strengths]
- This paper investigates the robustness of models trained on different training data configurations. This is a nice step towards understanding the data efficiency of contrastive training.
- Overall, this paper is well-written and clearly explains the motivation, experiments, and analysis.

[Weaknesses]
My main concern is the discrepancy in scale between the experimental setting and the current SOTA models (e.g., CLIP, ALIGN, MURAL, CoCA etc.). Therefore, I’m not sure if the conclusions of this paper hold for other vision-language models trained with contrastive loss.

- Vision-language pre-training is usually done using noisy massive image-text pairs (e.g., ALIGN uses 1.8B examples). In the experiments, the size of training data (up to 15M examples) is quite different from the training setup of SOTA models. 
- Although the number of negatives (i.e., batch size if using batch negatives) is an important factor of contrastive learning, it’s unclear if the authors performed an initial investigation for it (batch size of 128 sounds relatively small). 
- The robustness of the models could be different in other tasks. One of the major tasks (for both pre-training and test) is image-text retrieval, but evaluation is done by a unimodal task only (ImageNet). 

questions: - Have you considered other tasks (e.g., image2text/text2image retrieval)? Or, do you have preliminary results on other tasks?
- How did you decide the hyperparameters for CLIP (shown in Appendix B)? 

limitations: As I mentioned in the weaknesses, the empirical results are drawn from the limited data setting, i.e., there is a significant gap between the experimental setting and the training setup of SOTA models. Also, the robustness against distribution shifts is investigated using a unimodal task (ImageNet).
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies the role of training data on the robustness of large image-text models like CLIP and ALIGN.
The paper is structured into two main parts — empirical analysis, and theoretical discussion.
The empirical analysis involves training CLIP models using six different datasets containing web-curated image-text pairs, and report their robustness on existing testbeds covering ImageNet classes.
The second half of the paper presents a theoretical discussion considering a binary classification setup, to explain empirical results.
Taken together, the paper argues that naive scaling of training data is not sufficient to improve robustness -- careful consideration during data curation is critical to endow the trained models with the desired robustness.
strengths_and_weaknesses: I think this paper, in its current form, already matches the quality of a typical publication at NeurIPS conference. I strongly recommend acceptance. Below I justify my rating by discussing the main strengths of the paper along each of the suggested axes:

**Originality:**
Large pre-trained models like CLIP and ALIGN are becoming increasingly popular in the vision community.
While most works make progress by scaling up data and models (e.g. ALIGN, BASIC) or by adding dense supervision (e.g. SLIP, DeCLIP, FILIP), this paper focuses on the impact of training data source/distribution on certain downstream tasks.
Experiments study CLIP's robustness conditioned on its training data, and theoretical discussion attempts to explain results obtained using noisy, internet-scale data.
To the best of my knowledge, this exploration using large pre-trained models at this level of detail is novel.

Side note: Findings of this paper match with a recent paper — [Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)](https://arxiv.org/abs/2205.01397). However, the experiments in this paper are different and the theoretical discussion is new. Moreover, this paper appeared on arxiv very close to the NeurIPS deadline so I would consider it a concurrent work. It may be worth citing this paper later in the camera-ready.

**Quality:**
Section 1 (Introduction) lists three precise questions explored in the paper (Line 41—44).
The experiments and theoretical explanations are well designed to concretely answer these questions.
Authors do a good job at defining minimal experimental setup and executing it thoroughly.
They begin with a common setting of training CLIP models using one/more of six existing image-text datasets, and answer each question by showing trends among a subset of trained CLIP models. In all experiments, the authors have trained multiple models when subsampling training data to capture the variance in subsampling and training runs.

**Clarity:**
The writing and presentation of this paper is excellent.
All technical details for empirical analysis are well-stated and easy to follow.
Both parts of the paper — empirical and theoretical — are broken down into sub-sections for the reader to easily find answers to each question asked in the introduction. All plots are neat and figure captions are self-contained.
Some plots often have 10 or more colors in the legend — the authors have used color-blind friendly palette and differently shaped markers to improve accessibility.
The supplementary includes some basic statistics of data used in the experiments (caption lengths, image resolution, qualitative examples, etc.)

**Significance:**
In this "scaling era", we are seeing rapid progress with large models like CLIP, ALIGN, etc. that are often trained on in-house data containing millions to billions of training instances.
The quality and concept distribution of training data plays a huge role in the success of these models.
Unfortunately due to its unprecedented scale, the training data is becoming increasingly black box. The general consensus in the field states that adding more training data will keep pushing the model performance. This paper presents some contrary evidence in the context of model robustness, which is rather not obvious. Hence, I think this is a timely contribution that will force practitioners and researchers to think more in-depth about the training data distribution when scaling CLIP-like models.

questions: The experimental setup makes heavy use of the recent LAION dataset.
A recent audit of the LAION dataset found several ethical issues with the dataset, e.g. verifiably pornographic imagery (especially of marginalized communities).
See "[Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://arxiv.org/abs/2110.01963)" for a more comprehensive discussion.

The authors have applied basic filtering to avoid NSFW content, which is a great step -- I suggest citing this dataset audit paper and making a direct note in the paper to ensure that future works also apply similar filtering to minimize over-amplification of problematic imagery.
I also suggest releasing all trained models to aid follow-up works and increase the potential impact on the community.
limitations: **Discussion about limitations and societal impact is adequate (Section 7).**
I concur with the authors' assessment – this paper focuses on analysis and understanding of CLIP (and similar models), it is fair to say that its societal impacts are not very different/more than CLIP itself.
review_title: null

Meta-review: metareview: This paper studies the effect of pre-training data on the robustness of pre-trained vision-language models such as CLIP. Both empirical and theoretical results are provided to show that simple scaling may not always improving robustness. This is a timely results that shed light on how to perform better pre-training to improve robustness to distribution shifts. All reviewers agree that the work is technically solid and the contribution is significant.
confidence: Certain
Decision: Accept