SlowGuess/NeurIPS_2023_OCR huggingface dataset:
the abstract section of this paper, we first encounter the following claim: "We show that the scale of our corpus can make up for its noise and leads to state-of-the-art representations even with such a simple learning scheme". The demonstration of this claim appears later in "Section 6.2. Pre-training Datasets" where the authors state that "To understand better how data size scaling wins over the increased noise, we further randomly sample  $3M$ ,  $6M$ , and  $12M$  ALIGN training data and compare them with the cleaned CC-3M data on  $B7 + BERT$ -base model. Table 10 (sic) shows that while the ALIGN data performs much worse than CC data with the same size ( $3M$ ), the model quality trained on  $6M$  and  $12M$  ALIGN data rapidly catches up. Despite being noisy, ALIGN data outperforms Conceptual Captions with only  $4x$  size." We note that these experiments (or similar ones) have not been replicated elsewhere to check if these scaling ratios presented ipse dixit in these contexts indeed hold at all.
E The tactical template: Fuzzy main section meets non-existent appendices
What unites the marquee projects of Dall-E, Parti, and Imagen is the near-same tactical template deployed when it comes to (non)declaring the training dataset information. The template runs something like this:
Step-1: Allocate a small nondescript subsection of the main section of the paper covering only the bare minimum details about the number of samples in the training dataset with cross-references to other similar black box datasets such as JFT. This coincidentally happens to be Section 4.1 in both Parti and Imagen papers (See Fig A2).
Step-2: Declare that somewhere in the succeeding sections titled on the lines of broader impacts or societal impacts are details about the 'potentially problematic' aspects of the dataset and the downstream risks while patronizingly citing previously published audit papers (such as [7] that have done the grunt work of exposing the gory details of such datasets. This happens to be Section 8 - Broader impacts in Parti and Section 6 for the Imagen model.
Step-3: Setting the reader up for a non-existent Appendix section that is not part of the main paper and does not contain any details about how the dataset is constructed and where the data is sourced from while noting the fact that it's not mandatory for the reviewers to even glance at the Appendix section in peer-reviewed avenues of publishing.
It is in this backdrop we observe that the authors of the BASIC model paper have not even addressed model safety and dataset auditing issues despite having trained their model on the largest image-text dataset ever assembled and presented a full-length 47-page paper with three revisions on ArXiv (See https://arxiv.org/abs/2111.10050).
4.1 Training Datasets
We train on a combination of image-text datasets for all Parti models. The data includes the publicly available LAION-400M dataset [43]; FIT400M, a filtered subset of the full 1.8 billion examples used to train the ALIGN model [9]; JFT-4B dataset [44], which has images with text annotation labels. For textual descriptions of JFT, we randomly switch between the original labels as text (concatenated if an image has multiple labels) or machine-generated captions from a SimVLM model [45]. We discuss the limitations of the data in Section 8. For all image inputs, we follow the DALL-E dVAE input processing (Section A.2. Training in [2]) for image tokenizer training and the DALL-E Transformer input processing (Section B.2. Training in [2]) for encoder-decoder training.
8 Broader Impacts
Bias and safety. Text-to-image generation models like GLIDE, DALL-E 2, Imagen, Make-a-Scene, CogView and Parti are all trained on large, often noisy, image-text datasets that are known to contain biases regarding people of different backgrounds. This is particularly highlighted in Birhane et al's [100] analysis of the LAION-400M dataset [43]: their study of the dataset surfaced many problems with respect to stereotyping, pornography, violence and more. Other biases include stereotypical representations of people described as lawyers, flight attendants, homemakers, and so on. Models trained on such data without mitigation strategies thus risk reflecting and scaling up the underlying problems. Our primary training data is selected and highly filtered to minimize the presence of NSFW content; however, we incorporated LAION-400M during finetuning with classifier-free guidance - this improved model performance but also led to generation of NSFW images in some contexts. Other biases include those introduced by the use of examples that primarily have English texts and may be biased to certain areas of the world. In informal testing, we have noticed, for example, that prompts mentioning wedding clothes seem to produce images biased towards stereotypically female and Western attire.
4.1 Training details
We train on a combination of internal datasets, with  $\approx 460\mathrm{M}$  image-text pairs, and the publicly available Laion dataset [61], with  $\approx 400\mathrm{M}$  image-text pairs. There are limitations in our training data, and we refer the reader to Section 6 for details. See Appendix F for more implementation details.
6 Conclusions, Limitations and Societal Impact
Imagen's training data was drawn from several pre-existing datasets of image and English alt-text pairs. A subset of this data was filtered to remove noise and undesirable content, such as pornographic imagery and toxic language. However, a recent audit of one of our data sources, LAION-400M [61], uncovered a wide range of inappropriate content including pornographic imagery, racist slurs, and harmful social stereotypes [4]. This finding informs our assessment that Imagen is not suitable for public use at this time and also demonstrates the value of rigorous dataset audits and comprehensive dataset documentation (e.g. [23, 45]) in informing consequent decisions about the model's appropriate and safe use. Imagen also relies on text encoders trained on uncurated web-scale data, and thus inherits the social biases and limitations of large language models [5, 3, 50].
Figure A2: The Google template used to (non)declare the training dataset information along with paper screenshots