Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper constructs a classifier of whether or not an image is offensive. This is operationalized by finding a dataset from the psychology community of a few thousand images and ordinal judgements of 'morality' from a study. Prediction of these judgements is predictably hard, in no small part because the data is small. To combat these issues, the paper uses CLIP and soft-prompt tuning. Soft-prompt tuning appears to be very effective for CLIP in this context reaching accuracies of over 95%, where baseline fine tuning only gets about 85%. 
main_review: The contribution of the paper is not wholely clear to this reviewer. 

First a short summary of the paper: (1) the paper picks a fairly obscure dataset for prediction (2) constructs a good classifier using prompt fine tuning and CLIP (cool!) (3) applies it to ImageNet and provides a few categories with images deemed immoral. 

Overall, the paper is pitched as a dataset analysis tool for problematic images. But little to no effort is put into presenting results of analysis, beyond listing a few categories in ImageNet that had a lot of 'immoral' judgements predicted. I am left essentially knowing that a classifier can be built to predict morality based on an obscure dataset.  

On a technical level, while the results of prompt tuning of CLIP in particular could be new, the application is a fairly obvious combination of existing ideas, although I am excited to know its possible. I would rather have seen a full exploration of prompt fine tuning in CLIP rather than a narrow application to this morality dataset.  
summary_of_the_review: The paper does not provide deep analysis of offensive content in any visual dataset. Technically, the paper combines currently trendy and exciting techniques in fairly straightforward ways to create a classifier from little data for morality judgements. This technical contribution is not fully explored, and in itself does not appear novel.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper described an interesting idea to leverage massively pre-trained models (specifically CLIP-based models) to infer offensiveness in images. The authors first gave detailed literature review regarding a recently raising concern about inappropriate images in computer vision datasets, and also performed several finetuning/probing baseline experiments to illustrate the feasibility to detect such inappropriate contents. Next the authors explored the possibility to mitigate the potential risk by utilizing the implicit knowledge learned by CLIP-based pre-
trained models. Lastly the authors choose ImageNet for a proof-of-concept validation and show that the proposed approach can discover previously neglected offensive images.
main_review: Strength:
1. This paper focused on an important aspect of fairness/dataset bias in modern deep learning models, and showed an interesting direction to reveal questionable instances from the dataset. As the authors advocate this approach if combined to the human-in-the-loop protocol for dataset collection, may help creating more responsible datasets while saving some tedious human efforts.
2. The authors give very detailed introduction, background and related work sections, highlighting the importance of the task.
3. The code and pre-trained checkpoint provided in the supplementary might be useful for future work.

Weakness:
1. The technical contribution of this paper is very limited. Although it's arguably as important to identify new question and explore new application of existing techniques, essentially what this paper did is just an extension of CLIP-based models. The baseline experiments are simply finetuning/linearly probing existing models (ResNet-50 and ViT) on a public dataset (SMID), where the offensiveness is more of a synonym of "immoral/blameworthy". The proposed work regarding CLIP is more like a specific zero-shot learning application (i.e. offensiveness), and the prompt tuning is also highly related to previous work. Maybe this work is more suitable for a dedicated conference/workshop rather than a generic top-tier AI conference like ICLR.
2. One important section that is missing from this paper is the analysis of precision/recall of the proposed work. In Table 1, the authors show precision/recall/F1 but this is on SMID dataset (if I'm understanding correctly), where the "ground truth" labels are inferred based on discretization of rating (<2.5 or >3.5). This is different from the true offensiveness detection. Specifically, since the authors already compared with Birhane & Prabhu (2021) with their hand surveyed image selection, how accurate does the proposed model match the performance of the human selection? In Page 9 the author mention that one image of Figure 5 (misogynistic image) is missed by Birhane & Prabhu, but I'm curious in a statistical level how well do they agree. Alternatively a user study might be useful, that within the 40,501 detected images how many are truly "offensive"? Right now it seems to me that the authors only show several methods effective on SMID dataset, but the key contribution of "inferring offensiveness" is built upon the assumption that SMID rating is a proxy for offensiveness.
3. The concerns about ethical issues are not fully addressed. See Ethics Concerns section for more comments.
4. Some minor issues need to be resolved: For example, Figure 4 caption is it "On the right" (not "On the left")? Footnote 4 on Page 5, "Oktober" -> "October". The format of citation is mistaken in multiple occurrences, such as Figure 3 caption "(Deng et al. 2009)", and second paragraph of page 6, in "CLIP (Radford et al. 2021)", "other similar models (Jia et al 2021)".
summary_of_the_review: In general I think it's a solid paper with proper experiments as proof-of-concept, but the lack of technical contribution and missing the actual analysis on the detection of offensive contents are major weakness of the submission. For top-tier AI conference like ICLR this paper is not good enough.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work investigates using CLIP for pre-training a model on the Socio-Moral Image Database (prior art) for the task of “offensiveness” detection for which they argue will assist in reducing bias in large models trained from large corpora like ImageNet.
main_review: To begin, I don’t think the work adequately defines what they are scoping/defining by "offensiveness". In Section 3, the work draws from some other works and focus in on “defamatory, false, inaccurate, abusive, indecent, obscene or menacing or otherwise offensive” as their scope. In what follows in the rest of the paper, there is an assumption that a large model trained on large data solves for this definition, when in fact, it’s like using an answer to “how” to a question about “what”. As a result, the problem statement of the paper is rather muddled since there’s essentially a “you know it when you see it” argument to the question of "what is offensive content?". Just to name a few other quandaries with problem scoping: to “whom” are we considering the content to be “offensive”? Since “offensiveness” is a concept that is a function of social norms, over what period of time are we considering or is the modeling a function somehow of time?

These questions have important design implications, for example, in the empirical experiments of this work, they discretize ratings in their Socio-Moral Image Database and threshold on <2.5 and >3.5 as offending and non-offending, respectively, but since it’s not clear how we are defining the “whom” of offensiveness, it’s unclear if these choices make sense [N.B. authors followed prior art with those thresholds, but that in itself is a choice by the authors]. To what extreme do we take “injustice anywhere is a threat to justice everywhere” (Martin Luther King Jr)? If it was found to be offensive to one rater? Do we define it at the level of a locale or people group sharing similar socio-cultural values? These are challenging philosophical questions even that I do not expect an ICLR paper to fully address, but a position on the scope needs to be made explicit, otherwise, the objective is unclear and it's difficult to even assess if this paper is making progress on the problem.

Further, I was confused by some of the reasoning in the presentation, which summarizing using quotes follows as:
	
> "...training data has been shown to have problematic characteristics resulting in models that encode e.g. stereotypical and derogatory
associations...  large, uncurated, Internet-based datasets encode e.g. dominant and hegemonic views..."
	
> "...we investigate modern vision PMs trained on large-scale datasets, in particular, the Contrastive Language-Image Pre-trained model (CLIP) (Radford et al., 2021) and argue that they themselves pave a way to mitigate the associated risks... they encode implicit knowledge to infer offensiveness in images overcoming previous issues, namely the lack of adequate and sufficient training data."
	
> "...we demonstrate that our approach can be utilized to annotate offensive images in vision datasets and, therefore, reliably assist the curation process of such datasets."
	
> "We illustrate our approach on the popular ImageNet-ILSVRC-2012 dataset and show that large computer vision datasets contain additional inappropriate content, which previous documentations had not detected."

The logic seems be that large models trained on large data have undesirable biases and to address this, they propose to use another large model trained on large data to filter out the undesirable biases because they believe a priori that it, like other such models is attuned to the undesirable biases and can be used to pre-train a supervised task to identify one dimension of these biases: “offensiveness”. There’s a weird “who watches the watchmen” problem here. It’s unclear if there is actual a reinforcing loop where model used for filtering (CLIP) has biases and as a result, any downstream tasks will just narrowly focus on its representing of those biases. I will grant that the authors acknowledge this in the Ethics Statement, but I actually think it’s far more central the methods proposed and there's a need to address it head-on versus just having it as an Ethics addendum.

Ultimately, perhaps most puzzling is that despite the argument that such a filtering technique could reduce harmful biases in large datasets like ImageNet1k, the impact of the qualitative examples provided in Section 4 is opaque. For example, are the examples given coming from a high-recall setting where we want to trigger on even very low softmax scores? Even if these are identified in ImageNet1k, do we have reason to believe the base rates are sufficiently high that a naively trained model would encode these biases? If so, why was this then not validated with experiments?
summary_of_the_review: Overall, the paper puts forward a number of interesting empirical experiments, but because the problem statement (“offensiveness”) to begin with is not well-scoped, it then becomes difficult to assess the contributions of the proposed methodology. In addition, the ultimate impact of the work is only shown qualitatively with example concepts (and occurrences) that it’s identified in ImageNet1k, but the results of these are not validated and its impact on say models trained on ImageNet1k without these “offensive” examples is not quantified.
review_title: null

Meta-review: The authors set out on an important question of whether abstract and culturally specific concepts like offensiveness can be detected in images. The novelty of this work comes in part from tackling this question and attempting to create a technology which can operationalize it.  However, despite the authors' insistence that offensiveness is not "just another label", in practice the work treats it very much that way and therefore does not present a compelling innovation either in modeling or in juxtaposition to other labeling tasks.  Known training and inspection techniques are used on existing representations and more powerful models with more training data generalize better. It is unclear what is novel in the approach or unique to offensiveness over other labels (including abstract ones).
Decision: Reject