Textual Training for the Hassle-Free Removal of Unwanted Visual Data: Case Studies on OOD and Hateful Image Detection

Authors: saehyung  lee
jisoo  mok
sangha  park
yongho  shin
dahuin  jung
sungroh  yoon
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-09 20:51:09

Abstract: In our study, we explore methods for detecting unwanted content lurking in visual datasets. We provide a theoretical analysis demonstrating that a model capable of successfully partitioning visual data can be obtained using only textual data. Based on the analysis, we propose Hassle-Free Textual Training (HFTT), a streamlined method capable of acquiring detectors for unwanted visual content, using only textual data in conjunction with pre-trained vision-language models. HFTT features an innovative objective function that significantly reduces the necessity for human involvement in data annotation. Furthermore, HFTT employs a clever textual data synthesis method, effectively emulating the integration of unknown visual data distribution into the training process at no extra cost. The unique characteristics of HFTT extend its utility beyond traditional out-of-distribution detection, making it applicable to tasks that address more abstract concepts. We complement our analyses with experiments in hateful image detection and out-of-distribution detection. Our codes are available at https://github.com/HFTT-anonymous/HFTT.

Main paper: 
 introduction: 
 We are currently in the midst of what is known as the large-scale AI era. The growth in both the size of deep neural networks and training datasets led to unparalleled achievements in a wide array of tasks [4, 43] . However, this transition to large-scale AI presents new, unforeseen challenges. Particularly, the recent reports on the biased behavior of large AI models raise significant concerns surrounding the continuously expanding size of training datasets without proper quality control and regulation [2, 41] . The massive scale of visual training datasets necessary to train large-scale models presents a challenge in curating data to ensure unbiased and safe datasets. This is primarily due to the impracticality of manually selecting and removing unwanted content from such an extensive collection of images. This issue of data curation has traditionally been addressed by: (i) creating a supervised dataset for a specific objective; (ii) training a model on this dataset; and then (iii) utilizing the model to develop a larger dataset [28] . However, this approach exploits considerable human labor and needs to be re-initiated from the beginning whenever there are changes in the training objective.Figure 1 : Overview of our proposed method. Task embeddings define the task to be performed. For example, in the case of hateful image detection, hate speeches would serve as task embeddings, while in OOD detection, the names of classes from the training distribution would be the task embeddings. Trainable embeddings are the only parameters that are trained in our method, defined in the joint embedding space. During the training phase, only textual data are used, and in the testing phase, these trained parameters are employed to classify images. Detailed explanations are provided in Sections 3.The field of out-of-distribution (OOD) detection, which aims identify OOD data that lie outside the training data distribution, can be considered a sub-branch of data curation research. Recent works in OOD detection utilize vision-language models (VLMs) [40, 29, 30] to take advantage of the rich and human-aligned representations learned by these models. For instance, Esmaeilpour et al. [11] augmented the pre-trained CLIP model [40] with an additional decoder module, trained on a vision-language dataset, for visual OOD detection. In a similar vein, Wang et al. [49] incorporated an extra "no" text encoder, trained on a vision-language dataset, into CLIP. These previous approaches, however, suffer from a significant limitation: They require a vast amount of additional vision-language data. Using data samples targeted for detection can improve sample efficiency, but this leads to the dilemma of needing to collect unwanted data for the purpose of removing them.In this work, we propose a novel method that no longer relies on additional visual data or a computationally expensive training process. We first outline our theoretical rationale, particularly demonstrating that with a successfully trained model on a bimodal dataset, like CLIP, one can obtain a classifier to partition data from one mode using solely the data from the other mode. Building upon our motivation, we propose a method called Hassle-Free Textual Training (HFTT). HFTT consists of a newly proposed loss and a clever textual data synthesis method, updating trainable parameters defined in the joint embedding space to improve the detection of undesirable visual content. Specifically, we decompose the weighted cross-entropy loss into a formula that includes a regularization term, which is tailored to our use. Additionally, to achieve higher detection accuracy, we introduce the concept of focal loss [33] . Moreover, our textual data synthesis method, which combines prompt templates and words, can effectively imitate the involvement of the entire visual data distribution in the training process. We illustrate an overview of our proposed method in Figure 1 .The proposed loss function brings considerable convenience in achieving our objective. To train an unwanted data detection model, it is necessary to define out-distribution for a given data distribution (in-distribution), which is not always straightforward due to vague boundaries between the two. For instance, in hateful content detection tasks [13] , the divide between what is hateful and what is not is influenced by various contexts, e.g., historical and social backgrounds. Our proposed loss eliminates the need for human labor to annotate out-distribution data because it does not involve a clearly defined set of out-distribution data. Furthermore, our textual data synthesis method requires no cost, employing a rule-based approach using only prompt templates and a set of words.Based on the principle that HFTT can detect out-distribution samples by merely defining the indistribution in natural language, we propose that this method can be extended to tasks beyond traditional OOD detection, including hateful image detection. Current OOD detection methods often fail in such extended tasks for two main reasons: firstly, they are based on the assumption of a distinct boundary between in-and out-distributions, which is unsuitable for tasks with abstract concepts.Secondly, methods requiring training images may lead to ethical concerns. Our proposed method, however, is not subject to these limitations.Through empirical validation, we verify that HFTT can enhance the performance of VLMs in identifying unwanted visual data, whether it be OOD instances or hateful images. Additionally, we demonstrate through feature visualization results that HFTT, despite not observing any visual data in the training phase, appears to have been trained as if it had. Furthermore, we provide various analyses of HFTT, including comparative results of using different textual data synthesis methods.In summary, our contributions are as follows: (i) We theoretically demonstrate how textual data can serve as a substitute for visual data in our scenario; (ii) We introduce a new loss function. Its use eliminates the need for labor in annotating out-distribution data; (iii) We propose a textual data synthesis method that can efficiently imitate the visual data distribution in our training; (iv) We empirically analyze HFTT, a method composed of the above proposals. Our experiments show that HFTT is effective in a range of scenarios, from traditional OOD detection to situations involving abstract concepts, like the identification of hateful images. 
related work: 
 Vision-language models. With the advancements in deep learning, tackling sophisticated tasks that demand an understanding of both vision and language modalities has become viable. The methodologies employed to encode image and text data exhibit notable distinctions owing to their inherent differences. Prominent within this domain are dual-stream models exemplified by CLIP [40] , ALIGN [23] , and FILIP [51] . These models employ separate encoders for text and image data, optimizing them through contrastive objectives to align semantically similar features across heterogeneous modalities. Primarily, VLMs integrate transformer-based encoders for text data, while a variety of architectures, encompassing convolutional neural networks [25, 15] and vision transformers [9] , are deployed for image encoding. The success of CLIP-like models has spurred numerous subsequent inquiries, with a focus on enhancing data efficiency and adaptability for diverse downstream tasks.-1 Out-of-distribution detection. Traditionally, OOD detection has evolved by defining posthoc OOD scores [18, 32, 27, 34] or formulating learning algorithms based on outlier exposure methods [26, 20, 10] . With the advancement of VLMs, methods for OOD detection that leverage both image and text embeddings have also progressed. Post-hoc OOD score methods based on VLM typically involve utilizing the OOD class name [12, 11] or defining OOD scores using the top similarity values between images and class names [37] . In the case of outlier exposure, which requires a training algorithm, some approaches employ prompt learning or fine-tune the image encoder [45] of models like CLIP. Transitioning from conventional methods to VLM-based approaches, none of these methods have attempted text-only training and subsequently applied their techniques to tasks such as hateful image detection.‖ -1 ‖ -1 ‖ -1‖ +1 ‖ + ‖ + 1 ‖ + ‖ A PhotoText-only training for vision tasks. Given the progress in VLMs, there have been numerous studies aimed at replacing images with textual representations in vision and multimodal tasks. Textual data presents the advantage of being easily collectible compared to visual data. Previous studies have demonstrated the effectiveness of using only textual information for various vision tasks, including image classification [38] , image captioning [31] , and multi-label image recognition [14] . Our work represents a pioneering effort in applying text-only supervision to unwanted visual data detection. 
method: 
 In this section, we propose a new textual training method for the convenient and successful removal of unwanted visual data. In Section 3.1, we theoretically demonstrate, through a motivating example, that when there is a well-trained model on a bimodal dataset, such as CLIP, it is possible to train a binary classifier successfully partitioning data from one mode using only data from the other. This theoretical revelation leads us to our novel loss function that allows hassle-free training of an unwanted visual data detector in Section 3.2. Lastly, in Section 3.3, we present our proposed method that includes a simple yet effective synthesis of textual data. The proposed method is executable even without access to the parameters of the backbone model, making it lightweight and applicable to black-box foundational models.We theoretically demonstrate that when there exists a well-trained bimodal model F : G × H -→ Z for a given bimodal data distribution, it is possible to train a classifier successfully partitioning data from one mode (H) using only the dataset from the other mode (G). To align with the operations of VLMs like CLIP in our scenario, we assume that the output vectors of F are normalized. We define a bimodal dataset D as follows:D = {(g i , h i , y i )} N i=1, where y u.a.r.∼ {-1, +1} and (g, h)i.i.d.∼ G y × H y .(g i , h i ) represents the input vectors from the two modes for a given data sample, where y i denotes the binary class of the i-th sample. We can partition the dataset D as follows:D = D -1 ∪ D +1 , where D y = {(g i , h i ) |y = y i }.We assume that samples belonging to the same class in the dataset D exhibit similar semantic patterns. Given F that successfully builds the joint embedding space for the bimodal data distribution, we can posit the following:u ⊤ +1 v +1 > u ⊤ +1 v -1 and u ⊤ -1 v +1 < u ⊤ -1 v -1 , where u y = E u∈Uy [u] , v y = E v∈Vy [v] , U y = {F (g i ) |g i ∈ D y }, and V y = {F (h i ) |h i ∈ D y }.U y and V y are class-conditional embedding sets for each of the two modes, respectively. For simplicity, we assume that the variances of the angular distributions relative to their mean vectors are equal for sets U -1 and U +1 , as well as for sets V -1 and V +1 .We investigate whether the cosine-similarity classifier θ ⋆ trained solely on the unimodal dataset (g i , y i ) N i=1 using F can successfully be applied to (h i , y i ) N i=1 . We establish the following theorem: Theorem 1. For the quadratic loss function L (u, y; θ) = 1 -yθ ⊤ u 2 , the optimal cosine-similarity classifier θ ⋆ that classifies sets U -1 and U +1 isarg min θ E u∈U-1 [L (u, -1; θ)] + E u∈U+1 [L (u, +1; θ)] = u +1 -u -1 ∥u +1 -u -1 ∥ .Proofs are in Appendix A. Theorem 1 demonstrates the optimal classifier θ ⋆ is orthogonal to u -1 + u +1 . We present an illustration in Figure 2 to enhance understanding of both the problem under investigation and the results of our analysis.Applying the classifier θ ⋆ trained to classify U -1 and U +1 to distinguish between V -1 and V +1 leads to the following: Corollary 1. The classifier θ ⋆ , with respect to V -1 and V +1 , satisfies the double inequalities ofE v∈V-1 θ ⋆⊤ v < 0 < E v∈V+1 θ ⋆⊤ v .This implies that we can successfully classify V -1 and V +1 by observing the cosine similarities with θ ⋆ . Motivated by these theoretical examples, we hypothesize that classifiers obtained solely using textual data can operate on visual data as well. Section 4 empirically demonstrates that the arguments developed based on our theoretical model can be applied to modern machine-learning settings.Our objective is to distinguish in-distribution data samples (D in ), conforming to given data distribution, from out-distribution data samples (D out ). The development of our new loss function begins with defining the binary cross-entropy loss L as follows:L(u, y) = - 1 + y 2 log p (u) - 1 -y 2 log (1 -p(u)) .We employ the notations introduced in Section 3.1. p(u) denotes the probability that the label of an embedding u is +1, where +1 signifies an out-distribution. With respect to datasets U -1 and U +1 , we minimizeEQUATIONWe introduce a hyper-parameter, λ ∈ [0, 1], to adjust the balance between in-distribution learning (the first term) and out-distribution learning (the second term). Equation ( 1) can be reformulated asEQUATIONThe second term can be understood as regularization for in-distribution learning. As λ approaches 0, in-distribution learning is more heavily impeded. Rather than employing the original regularization term, -u∈U-1 (1 -λ) L (u, -1), we propose changing it to u∈U-1(1 -λ) L (u, +1) .Before analyzing the significance of this modification to the objective function, we first examine the effects resulting from this change. Along with the modification, our objective function can be formulated as follows:EQUATIONTo minimize Eq. 2, it is imperative to distinguish between the in-distribution dataset and the outdistribution dataset. In-distribution data aligns with the objective of the given task, and any data not included in it becomes out-distribution data. However, in real-world scenarios, distinguishing between these distributions is not straightforward. For instance, if we consider U as the text embedding space, collecting out-distribution texts for a given set of in-distribution texts involves considerations such as homonyms, synonyms, and various forms of linguistic variations. Particularly, in tasks where the boundaries between in-distribution and out-distribution are ambiguous, as seen in challenges such as hate content detection [13] , constructing a dataset for Eq. 2 becomes difficult and requires considerable human labor. However, the utilization of Eq. 3 alleviates us from such challenges. In other words, the union of the two sets U -1 ∪ U +1 in Eq. 3 allows us to treat all data samples as out-distribution without the need to ponder their relationship with the in-distribution, providing a solution to the intricacies involved in dataset construction. The distinction between Eq. 2 and 3 becomes evident when comparing the gradient signals produced by the two different regularization terms. Gradients of the regularization terms in Eq. 2 and 3 can be computed as follows:(original) -EQUATIONu∈U-1∂L (u, +1) ∂p(u) = u∈U-1 -1 p(u) .The original regularization term weakly regularizes in-distribution samples that were sufficiently learned by the model (i.e., samples with low p(u)). However, the proposed regularization term does the exact opposite; it imposes stronger regularization on in-distribution samples with low p(u). In essence, our proposed regularization prevents the model from exhibiting high confidence in in-distribution samples and enforces the decision boundary to be created near the in-distribution.Recent studies show that the closer the decision boundary of the out-distribution data detector is to the in-distribution, the more effective the detector is at identifying various out-distribution data [26, 19, 10, 39] . Subsequent research efforts have been directed at obtaining out-distribution samples that reside close to the in-distribution while training a detector to bring its decision boundary closer to the in-distribution. For instance, Lee et al. [26] utilizes a generative adversarial network to acquire samples placed on the in-distribution boundary. Du et al. [10] models the in-distribution using a Gaussian distribution and samples embeddings from the low-likelihood regions of the defined Gaussian distribution. Likewise, we focus on training samples situated in the region close to the in-distribution by incorporating an additional focal loss.The focal loss was initially proposed to forcefully suppress the gradients for background pixels, which dominate the image, and intensify the learning signals from foreground pixels. Under our scenario, the in-distribution, like foreground pixels, tends to inhabit a small portion of the entire embedding space. In light of the similarity between the in-distribution and foreground pixels, we utilize the focal loss to restrain the loss from far out-distribution samples and amplify learning signals from samples near the in-distribution. The proposed loss can thus be defined as:Definition 1. Let B -1 = {x i } N i=1 and B = {x i } N i=1denote mini-batches that are respectively drawn from the specified in-distribution and the overall data distribution. Let L be the cross-entropy loss. Then, our proposed loss function isxi∈B-1 L (x i , -1) + (1 -λ) xj ∈B β j L (x j , +1) ; β j = N α j x k ∈B α k and α j = (1 -p (x j )) γ . (4) p (x)is the predictive probability of x belonging in out-distribution. γ ≥ 0 is treated as a hyperparameter of the focal loss. In Section C, we compare the results of using loss terms in Eqs. 2 and 3 and demonstrate the particular effectiveness of the focal loss.So far, we have assumed access to data sampled from the out-distribution. However, we may not always be able to anticipate the out-distribution in advance, and even if we can, sampling a subset of data that is representative of the entire distribution is not a straightforward problem in nature. In our scenario, we solely utilize textual data to learn an unwanted visual data detector. Therefore, the proposed scenario requires texts to define the in-distribution and a comprehensive corpus of textual data that can replace the entirety of visual data. VLMs, such as CLIP, obtained impressive zero-shot classification accuracy on diverse visual data benchmarks through the usage of prompts, e.g., "a photo of a {}." Inspired by the success of prompting in VLMs, we conjecture that all visual data can be expressed textually through prompts. This assumption allows the textual dataset to replace the unknown visual data distribution by integrating words associated with the visual data into prompts, drastically simplifying the process of textual data sampling in our method. One example of a prompt design utilized in our approach is: "This is a photo of a {}." To emulate the effect of using the entire visual data distribution, we adopt a word set 3 that includes approximately 370k English words. We report the results of using other prompt designs or textual data acquisition processes in Appendix C. While the optimization procedure in our method additionally requires in-distribution textual data according to Eq. 4, these can be obtained with minimal effort by creating arbitrary sentences or prompts related to the given task. Section 4 details how in-distribution textual data are attained for each experimental setting. Even though the task of unwanted visual data detection is a type of binary classification problem, learning a plain linear classifier in the embedding space of pre-trained VLMs through approaches like linear probing is not necessarily compatible with our task because the inand out-distributions are not expected to be linearly separable. To accurately estimate the probability that input x belongs in the out-distribution p (x), we must take advantage of the informative signals in the text encoder of pre-trained VLMs. In our method, p (x) is computed as follows:1. Obtaining embeddings {w in i } K i=1 for K texts that effectively represent in-distribution visual data is equivalent to defining the task. This process is akin to obtaining zero-shot classifiers using VLMs. We will refer to these text embeddings as task embeddings.Require: word set W, prompt templates P, in-distribution textual data G -1 , task embeddings{w in i } K i=1 , trainable embeddings {w out j } N j=1 , pre-trained model F , hyper-parameter λ 1: Initialize the trainable embeddings {w out j } N j=1 2: for mini-batches (B -1 , words) ∼ (G -1 , W) do 3:B ← word2data(words, P) # textual data synthesis (the entire data distribution) 4:Compute the proposed loss by Eq. 45:Update{w out j } N j=1# the incurred cost is negligible 6: end for 7: Output: out-distribution data detector F,{w in i } K i=1 , {w out j } N j=12. With a pre-trained vision-language model F and the set of N trainable embeddings {w out j } N j=1 defined in the joint embedding space, p (x) is obtained asN j=1 exp F (x) ⊤ w out j K i=1 exp F (x) ⊤ w in i + N j=1 exp F (x) ⊤ w out j .Our method minimizes the custom loss defined in Eq. 4 by learning {w out j } N j=1 only with textual data with the task embeddings {w in i } K i=1 and the model F kept frozen. Because the trainable embeddings are tuned in the output space of the backbone network, the proposed method results in little memory and computational cost. Furthermore, no need to access the parameters of the backbone network makes the proposed method extensible to black-box foundation models. The overall procedure of the proposed method is summarized in Algorithm 1. 
experimental setup: 
 We complement our analysis with case studies conducted on OOD and hateful image detection. For the OOD detection task, ImageNet-1k [7] is treated as in-distribution, and the following datasets are used as out-distribution data: iNaturalist [47] , SUN [50] , Places [52] , and Textures [5] . We specifically utilize OOD datasets that are carefully curated to be disjoint from ImageNet, as described in [22] . For hateful image detection, we utilize a dataset containing 892 Antisemitic/Islamophobic images and 420 phrases (Hate) [13] . The Hate dataset is a human-annotated dataset whose usage is limited to individuals with academic purposes to prevent its unethical and unregulated use.We adopt CLIP, the most extensively studied VLM, specifically using ViT-B/16 as the vision backbone. Unless specified otherwise, we set the batch size=256, learning rate=1.0, epoch=1, γ=1.0 (the focal loss hyper-parameter), λ=0, and N =10 (the number of trainable embeddings) for all experiments. Note that in the majority of scenarios, there is a substantial predominance of out-distribution textual data compared to in-distribution textual data, and our approach involves mini-batch sampling. Consequently, given the rarity of in-distribution data sampling, the training on in-distribution data remains largely unaffected even though λ = 0. All values presented in the tables of this paper are the average results over five runs. We conduct a comparative analysis of our approach against existing methods requiring in-distribution images, namely Mahalanobis [27] , MSP [18] , KNN [44] , and NPOS [46] . Additionally, we include methods that do not necessitate in-distribution data, Energy [34] , ZOC [11] , MaxLogit [21] , and MCM [36] , in our comparison. The evaluation is performed using OOD scores proposed by the aforementioned works, as well as the scores introduced in this paper (refer to p (x) in Sec 3.3). The calculation of Area Under the Receiver Operating Characteristic (AUROC) and False Positive Rate at 95% True Positive Rate (FPR95) is based on these scores.HFTT training and inference costs. The backbone model for HFTT remains untrained, while only trainable parameters (trainable embeddings) defined in the model output space are updated. Thus, the cost of this update process is almost equivalent to the forward propagation cost of the synthetic textual data. For the corpus (∼370k samples) and CLIP utilized in the experiments, the update process takes less than 2 minutes with a single V100 GPU. During inference time, the computation of HFTT involves obtaining cosine similarities between trainable embeddings and input embeddings. This cost amounts to 2×(batch size)×(embedding dimension)×(the number of trainable embeddings) FLOPS, which is negligible compared to the inference cost of the entire model. In OOD detection experiments, we utilize weights of zero-shot classifiers of pre-trained VLMs as task embeddings for HFTT. Indistribution textual data are obtained via combinations of various prompt templates and class names of in-distribution data, which, in our experimental setting, are equivalent to 1,000 Ima-geNet classes. We adopt the prompt set released by OpenAI for prompt ensembling 4 as prompt templates. The comparison results are reported in Table 1 . Despite the simple and lightweight nature of the proposed method, it outperforms even strong baselines that utilize images, on average.To understand how the trained embeddings provide the task embeddings with informative signals for identifying OOD data points, we visually analyze the joint embedding space of CLIP in Figure 3 . Even though visual OOD data were not involved in the training process, the trained embeddings are positioned on a sub-region of the embedding space inhabited by OOD data and thus can function as additional pointers for where OOD data lie in the joint embedding space. Therefore, this deliberate positioning of trained embeddings precludes the task embeddings from accidentally confusing out-distribution data as in-distribution by refining the decision boundary to intricately separate in-distribution and out-distribution regions. This visualization result can be attributed to two methodological characteristics that are unique to our method. First, our method directly optimizes the trainable embeddings and is not bounded by the modality gap between texts and images. Second, our method successfully places the trained embeddings on top of OOD data only through textual data; this consolidates that textual data can replace visual data, providing strong empirical support for the theory presented in Section 3.1. Together, these two aspects of HFTT yield the joint embedding space as illustrated in Figure 3 .In this task, the hateful data that contains offensive content against Muslims and Jews is treated as in-distribution data, whereas innocuous data void of such content is treated as out-distribution data. Consequently, embeddings of distinct phrases from a collection of offensive and hateful phrases, provided as part of the Hate dataset, are utilized as task embeddings. The entire set of offensive and hateful phrases is employed as in-distribution textual data.The Mahalanobis, MSP, KNN, and NPOS methods necessitate the construction of an in-distribution image dataset. Therefore, they should not be used as methods for unethical image detection tasks such as hate image detection, as doing so would require the construction of unethical image datasets, leading to numerous ethical problems such as direct or indirect leakage of sensitive information. In contrast, HFTT requires no usage of any image, thus it can be applied to any unethical image detection task without ethical concerns. To highlight the differences between traditional OOD detection methods and HFTT, we include two additional baselines [49, 24] and one extra dataset [3] .In Table 2 , we can observe that most OOD detection methods show significantly lower performance compared to HFTT. These results arise because OOD detection methods assume a classification problem with clear distinctions between classes. In tasks dealing with abstract concepts, the boundaries between data clusters within the in-distribution are ambiguous, which results in the underperformance of existing OOD detection methods. NegLabel shows different results compared to traditional OOD detection methods but still falls short of our proposed approach. We provide a further comparison of our method to CLIPN [49] and NegLabel [24] in Appendix B.To further study the generalizability of HFTT, we observe the effectiveness of HFTT in low-quality image detection [17] and within the medical image domain [6, 16, 48] . The findings demonstrate the potential extension of HFTT's applicability beyond conventional OOD detection tasks. The results of these experiments and an ablation study on hyper-parameters are provided in Appendices B and C.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: URL in Abstract and Section 4 Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. • The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
conclusion and limitation: 
 In this paper, we proposed a novel methodology for identifying undesirable content hidden within visual datasets. Close theoretical scrutiny of the joint embedding space of VLMs led to the development of HFTT, an efficient framework for training detectors to automatically identify unwanted visual content by leveraging solely textual data together with pre-trained VLMs. HFTT is comprised of a creative objective function that markedly diminishes human involvement in data annotation and the textual data synthesis technique in HFTT that can simulate the usage of unknown visual data distributions in the training process without additional cost. The distinctive attributes of HFTT broaden its applicability from a clearly-scoped OOD detection task to a far more general set of tasks that are more abstract. Because HFTT requires some type of VLM as the base model, its capabilities are bounded by the representative capacity of pre-trained VLMs. This dependency on pre-trained VLMs makes the use of HFTT in tasks that VLMs struggle with challenging.Impact Statements. This paper contributes to the growing field of data curation and selection research. As datasets for training large AI models expand without adequate safeguards, identifying unwanted data points, such as biased or offensive content, from training datasets is becoming crucial. We believe our work will make a positive contribution to this area, opening up new possibilities for the effortless removal of unwanted visual data. While our method could potentially be misused for content censorship, we believe the positive impact it provides significantly outweighs these concerns. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: This paper does not release any data or model. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.• Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: Section 4 and Appendix Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: URL in abstract and Section 4 Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Theorem 1. For the quadratic loss function L (u, y; θ) = 1 -yθ ⊤ u 2 , the optimal cosine-similarity classifier θ ⋆ that classifies sets U -1 and U +1 is arg minθ E u∈U-1 [L (u, -1; θ)] + E u∈U+1 [L (u, +1; θ)] = arg min θ 1 + θ ⊤ u -1 2 + 1 -θ ⊤ u +1 2 = u +1 -u -1 ∥u +1 -u -1 ∥ .Proof. Our assumption validates the following equations:θ ⊤ θ = u ⊤ u = v ⊤ v = 1, Σ +1 + Σ -1 = ϵI, ∥u +1 ∥ = ∥u -1 ∥, where Σ y and I denote the covariance matrix of U y and the identity matrix, respectively, and ϵ > 0 is a constant. Then, arg minθ E u∈U-1 [L (u, -1; θ)] + E u∈U+1 [L (u, +1; θ)] = arg min θ E u∈U-1 1 + θ ⊤ u 2 + E u∈U+1 1 -θ ⊤ u 2 = arg min θ E u∈U-1 1 + θ ⊤ u 2 + E u∈U+1 1 -θ ⊤ u 2 + θ ⊤ Σ +1 θ + θ ⊤ Σ -1 θ = arg min θ E u∈U-1 1 + θ ⊤ u 2 + E u∈U+1 1 -θ ⊤ u 2 + 2ϵ = arg min θ 1 + θ ⊤ u -1 2 + 1 -θ ⊤ u +1 2 .The gradient of the objective function with respect to θ is-2 1 -θ ⊤ u -1 u -1 + 2 1 + θ ⊤ u +1 u +1 .Therefore, the optimal cosine-similarity classifier θ ⋆ satisfies the following equation:1 -θ ⋆⊤ u -1 u -1 = 1 + θ ⋆⊤ u +1 u +1 . Then, 1 -θ ⋆⊤ u -1 = 1 + θ ⋆⊤ u +1 or 1 -θ ⋆⊤ u -1 = -1 -θ ⋆⊤ u +1 .The second equation is not true for any u -1 and u +1 . Hence,θ ⋆⊤ = u +1 -u -1 ∥u +1 -u -1 ∥ .Corollary 1. The classifier θ ⋆ , with respect to V -1 and V +1 , satisfies the double inequalities ofE v∈V-1 θ ⋆⊤ v < 0 < E v∈V+1 θ ⋆⊤ v . Proof. Based on the inequalities u ⊤ +1 v +1 > u ⊤ +1 v -1 and u ⊤ -1 v +1 < u ⊤ -1 v -1 , E v∈V-1 θ ⋆⊤ v = u ⊤ +1 v -1 -u ⊤ -1 v -1 ∥u +1 -u -1 ∥ < 0 < E v∈V+1 θ ⋆⊤ v = u ⊤ +1 v +1 -u ⊤ -1 v +1 ∥u +1 -u -1 ∥ .Low-quality image detection. We additionally demonstrate the applicability of our method, HFTT, in detecting low-quality images, which are commonly unwanted visual data beyond OOD and hate images. Specifically, we assume the task of detecting corrupted images lurking within a raw visual dataset consisting of 1000 ImageNet classes. For this experiment, we employ ImageNet and ImageNet-C as in-distribution and out-distribution data, respectively. As shown in Table 3 , HFTT consistently surpasses existing methods in the detection of corrupted images. Medical image domain. We compare the performance of HFTT and MCM in the medical image domain. Specifically, we treat the ISIC-18 skin lesion diagnosis dataset [6] as in-distribution and the PathVQA [16] and PatchCamelyon [48] datasets as out-distribution. The ISIC-18 skin lesion diagnosis dataset is an image classification benchmark for seven skin disease categories. We apply MCM and HFTT to CLIP on the seven disease categories. Table 4 reveals a significantly low detection performance of MCM, attributed to the limited medical domain knowledge of CLIP. Even appending descriptions (generated by GPT-4) to the disease names does not yield favorable results for MCM (+ description). In contrast, our proposed method achieves significantly better results by leveraging the model knowledge and the medical-related information in the corpus. The utilization of medicalrelated information within the corpus by HFTT is evidenced by the fact that further improvements can be achieved by modifying the corpus to align with the medical domain (+ corpus engineering).The experimental results on other pre-trained models. As discussed in Section 3.1 of our paper, our method assumes that text and image are well-aligned through contrastive learning, similar to CLIP. Therefore, if CLIP is used as the vision encoder, the text encoder must also be CLIP. If text and image are well-aligned, our method can be applied to models other than CLIP. Here, we provide the results for CLIP-L/14, BLIP-B/16, and BLIP-L/16 in addition to the CLIP-B/16 used in our study. Table 5 demonstrates that our method is effective across various vision-language models.HFTT vs. CLIPN and NegLabel. NegLabel [24] constructs an OOD corpus by selecting texts distant from the in-distribution texts from a predefined corpus, then compares the distances between the input image and those texts in the CLIP embedding space to detect OOD. While NegLabel shows high OOD detection performance on ImageNet (see Table 7 ), it has the following limitations compared to our method: 1) Although NegLabel does not require training additional parameters, it must compute the embeddings of all texts in the corpus and measure their similarity to the indistribution texts to find the optimal OOD corpus for a given in-distribution. Our training method also requires nearly the same cost as obtaining the embeddings of all texts within a predefined corpus and calculating the similarities between those embeddings and the task+trainable embeddings, as discussed in Section 4.1. Thus, NegLabel and our method require the same level of optimization cost; 2) Since NegLabel uses the embeddings of the determined OOD corpus as they are, it falls behind our method, which has trainable parameters, in terms of generalization. To demonstrate this, we further compare our method and NegLabel in the medical image domain. Specifically, we treat the ISIC-18 skin lesion diagnosis dataset [1] as in-distribution and the PathVQA [16] and PatchCamelyon [48] datasets as out-of-distribution. The ISIC-18 skin lesion diagnosis dataset is an image classification benchmark for seven skin disease categories. Table 6 illustrates the limitations of NegLabel in terms of generalization. While NegLabel fails to construct an effective OOD corpus for the medical image dataset, our method achieves significantly higher performance by learning optimal embeddings for detection.CLIPN utilizes an additional "no" text encoder alongside CLIP. This additional text encoder predicts the probability that a given object is not present in an image. Thus, CLIPN predicts whether a given image is in-distribution or out-distribution by using the original CLIP text encoder and the "no" text encoder to estimate the probabilities, respectively. Images with a low probability of being in-distribution and a high probability of being out-distribution are identified as OOD.Although CLIPN achieves high OOD detection performance on ImageNet (see Table 7 ), it has the following limitations compared to our method:1) CLIPN requires significantly higher inference costs due to the use of an additional text encoder; 2) While our method requires lightweight training that does not involve images, CLIPN demands extensive and expensive training of the "no" text encoder on large vision-language datasets; 3) CLIPN can only be applied to tasks where the distinction between in-distribution and out-distribution is clear and straightforward, such as classification datasets. This is because all training images must be classified as either "yes" or "no" images. Therefore, it is unsuitable for tasks dealing with abstract concepts, such as hateful image detection, as discussed in Section 4.3 of our paper; 4) Our method can be easily applied to any detection task defined in natural language, whereas CLIPN shows significantly degraded performance for in-distribution tasks that fall outside the training distribution of the "no" text encoder. In terms of applicability, our proposed method surpasses CLIPN. To demonstrate this, we further compare our method with CLIPN in the medical image domain. Table A illustrates the limitations of CLIPN in terms of generalization. While CLIPN effectively detects PCAM, it exhibits very low detection performance on PVQA. In contrast, our method achieves high performance on both OOD tasks.In this section, we analyze how different components affect the performance of HFTT.Textual data synthesis method. While we propose a textual data synthesis method that incurs no additional cost, alternative approaches beyond this can also be explored. We apply the following methods in conjunction with HFTT and list their results in Table 8: 1. WTO, CTO, DTO: Recently, Park et al. proposed a method that utilizes textual outliers instead of visual outliers in outlier exposure [19] . For our experiments, we use word-level textual outliers (WTO) generated using in-distribution images along with CLIP and BERT [8] , caption-level textual outliers (CTO) generated by an image captioning model [30] , and description-level textual outliers (DTO) created using a large language model. 2. Caption: We can consider the extensive use of image captions from LAION-400M [42] to substitute for the entire visual data distribution. 3. Deduplication: To experimentally compare Eq. 2 and 3, we applied HFTT after removing words from our word set that have meanings identical to ImageNet classes as much as possible.Table 8 reveals that textual outliers generated using additional models and in-distribution images show comparable or inferior performance to our textual data synthesis method. Furthermore, the captions results suggest that heavily relying on image captions does not effectively enhance the average detection performance for various OOD data. Lastly, there appears to be no discernible performance difference between the application of Eq. 1 and 4. This indicates that our proposed loss minimizes the need for human labor by eliminating the process of selecting out-of-distribution data without sacrificing performance.The focal loss hyper-parameter. HFTT incorporates the concept of focal loss to shape the decision boundary of detectors near the in-distribution. We observe its effect by incrementally increasing the focal loss hyper-parameter γ from zero. Table 9 demonstrates that using the focal loss (γ > 0) generally leads to better performance compared to not using it (γ = 0). Temperature. In HFTT, a temperature parameter is utilized for computing p (x). CLIP learns a temperature parameter during their pre-training phase, and we employ these same learned temperature values in all of our experiments. Table 10 illustrates that modifying the temperature value may reduce the efficacy of HFTT.The number of trainable embeddings. If the dimensionality of the data manifold in the joint embedding space of VLMs is low, HFTT can be effective even with a small number of trainable embeddings. To validate this, we present the OOD detection performance of HFTT in Table 11 , illustrating how it varies with the number of trainable embeddings (N ). Remarkably, HFTT can improve OOD detection performance on ImageNet, which has 1,000 classes, even with a very limited number of trainable embeddings. This is possible due to the inherently low dimension of data in the actual model output space [1] . • The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced.https://github.com/dwyl/english-words?tab=readme-ov-filehttps://github.com/openai/CLIP 


Decision: The paper originally received borderline scores leaning towards acceptance. The authors provided a detailed rebuttal and the paper has been discussed by the reviewers. The reviewers raised some concerns about missing experiments/comparisons (e.g. with NegLabel and CLIPN) that have been mostly addressed in the rebuttal. The authors included in their responses several new result and, overall, the reviewers where satisfied by the rebuttal and they all manteined or increased their scores to 6:WeakAccept. Overall this is a solid submission and the authors are encouraged to carefully follow and implement all the reviewers' suggestion in the final revision of the paper.

Reviews: review: paper_summary: paper_summary: This paper proposes an objective function for CLIP-based architecture to enhance out-of-distribution (OOD) detection. Instead of relying on OOD image data, the approach extracts OOD words from various sources and updates some trainable embeddings using predefined text embedding. Results show that the proposed approach outperforms previous methods that do not require in-distribution images and has comparable performance with methods that require in-distribution images.
strength_weakness: strength_weakness: \- The related work section lacks comprehensiveness. While the authors mention some CLIP-based methods, they overlook approaches closely related to their own. Notably, NegLabel (ICLR2024) and CLIPN (ICCV2023) also use textual features for OOD detection and are very similar to the proposed method. The authors should include these works in their related work section, provide benchmarks, and critically analyze the advantages of their approach compared to these existing methods.

\- In the experimental results, authors compare results with CLIPN and NegLabel only in Tab. 2, while they are not mentioned in Tab.1. Also, those two methods are not even cited nor explained in related work. I believe it might have been a last minute effort.

\- Some methods in the literature do not require any training (e.g. NegLabel). Given that the proposed approach involves additional training (even though without images), it is crucial to highlight the differences with these training-free methods.
questions: questions: Authors should explain why the comparison with NegLabel and CLIPN is partial and the method are not even cited by the paper.

In addition authors should highlight advantages and disadvantages of their method which is in-domain image free versus approaches that are training free, such as NegLabel.
limitations: limitations: Authors addressed the limitations.
paper_summary: paper_summary: This paper focuses on textual training methods to remove undesirable (such as biased or offensive) visual content and proposes a method for detecting unwanted visual content using only synthetic textual data to partition visual data. The classifier trained on textual content can be successfully transferred to visual content. The method consists of an objective function and a textual data synthesis method. The design of the loss does not require data annotation and the textual data synthesis method can emulate unknown visual data distribution into the training process with no extra cost. The proposed method was proven to be effective for out-of-distribution detection and hateful image detection.
strength_weakness: strength_weakness: I do not see a significant weakness in the paper since it’s well-organized. Several clarification questions are listed below.
questions: questions: 1. This idea of training on textual content for transferring to visual content is interesting, could the authors provide more insights on more examples of applications of this method except for the OOD detection and hateful image detection? Is it possible to apply it to more binary classification tasks?
2. What encoders are used to obtain the task embedding? The authors mentioned using CLIP as the vision backbone, and is the text encoder during training also from CLIP?
3. How do you decide the number of task embeddings?
4. The paper mentions using the proposed method to detect unwanted content and uses two tasks (OOD detection and hateful image detection. If I understand correctly, my key concern is that the focus of this paper is closer to a novel method for binary classification with an application that classifies unwanted content. Could the authors share more thoughts regarding this? If so, I would suggest slightly paraphrasing the introduction to better reflect the contribution and indicate more potential applications.
5. Can this method be extended to multi-class classifications?
limitations: limitations: The authors adequately addressed the limitations.
paper_summary: paper_summary: This paper introduces an efficient and effective text-only training method for detecting undesired visual content. Its key contributions include a theoretical demonstration of how text data can substitute visual data, a new loss function, and a method for synthesizing textual data. These efforts aim to segregate data from one mode using only the dataset from the other mode. The method is evaluated through experiments on OOD detection tasks and hateful image detection, demonstrating comparable performance.
strength_weakness: strength_weakness: 1. The trainable embedding learning process is unclear. According to the description, trainable embeddings are learned for N out-distribution data instances, which are then frozen during the test phase. It appears that the image embeddings obtained by the image encoder align with the frozen embeddings. If this is the case, the alignment process should be clearly elucidated. If not, the method for using the learned embeddings for images needs to be explained. The parameter N, representing the number of out-distribution data used during training, intuitively suggests that a larger N would enhance accurate distribution learning. However, the results in Table 8 present a contradictory viewpoint. More explanation should be given.
2. The proposed HFTT method can be applied to different pretrained models and OOD tasks. In addition to the image CLIP model, it is recommended that the authors conduct experiments using pretrained video-text models to assess performance changes in related video detection tasks. This additional experiment can shed light on how pretrained models influence performance when only one mode is utilized for learning, considering the crucial role of pre-learned cross-modal alignment knowledge in enabling text-only learning.
questions: questions: My main questions are (1) making the trainable embedding learning clearer and (2) the test on other pre-trained models to view how the  pre-learned cross-modal alignment knowledge impacts the text-only learning.
limitations: limitations: The main limitations of this paper also lie in the two concerns: unclear description of the trainable embedding learning and no examination of other pre-trained vision/video-text models.
paper_summary: paper_summary: This paper proposes a purely textual-based training method for detecting out-of-distribution or hateful images. The authors train an additional embedding layer over frozen CLIP encoders using text data. The authors propose the use of a novel loss function for this training. The method improves performance over baselines for most of the datasets that they evaluate on.
strength_weakness: strength_weakness: The writing structure could be improved. The textual synthesis aspect is not clear to me and should be discussed in more detail in the main paper. The method could also be used for generic image classification tasks which brings me to question the choice of why such classification results have not been shown. Some more discussion over the baselines would be good too.
questions: questions: Clarifications to my questions in weaknesses section is sufficient
limitations: limitations: The authors have described limitations.

Comments: title: I keep my score
replying to the response
comment: Although authors did not answer to my question about the reasons for having only a partial comparison with NegLabel and CLIPN in the main paper, the authors provided a complete and clear comparison with the mentioned methods. Thus I keep my positive score.
Thanks for the response. It well addressed my concern-1. For my concern-2, they promise to include the results in their next version. So, I keep my rating Weak Accept.
I appreciate the detailed response from the authors and all my concerns are addressed. I will increase my rating from 5 to 6 and suggest that the authors include the information in the revised paper.
I thank the authors for their rebuttal. The rebuttal answers my concerns, thus I am raising my score. I would suggest the authors to include these discussions in the paper.