Out-of-Distribution Detection with Negative Prompts
Authors: Jun Nie
Yonggang Zhang
Zhen Fang
Tongliang Liu
Bo Han
Xinmei Tian
Venue: ICLR 2024 poster (2024)
Keywords: out-of-distribution dection
CLIP
prompt learning
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Out-of-distribution (OOD) detection is indispensable for open-world machine learning models. Inspired by recent success in large pre-trained language-vision models, e.g., CLIP, advanced works have achieved impressive OOD detection results by matching the *similarity* between image features and features of learned prompts, i.e., positive prompts. However, existing works typically struggle with OOD samples having similar features with those of known classes. One straightforward approach is to introduce negative prompts to achieve a *dissimilarity* matching, which further assesses the anomaly level of image features by introducing the absence of specific features. Unfortunately, our experimental observations show that either employing a prompt like "not a photo of a" or learning a prompt to represent "not containing" fails to capture the dissimilarity for identifying OOD samples. The failure may be contributed to the diversity of negative features, i.e., tons of features could indicate features not belonging to a known class. To this end, we propose to learn a set of negative prompts for each class. The learned positive prompt (for all classes) and negative prompts (for each class) are leveraged to measure the similarity and dissimilarity in the feature space simultaneously, enabling more accurate detection of OOD samples. Extensive experiments are conducted on diverse OOD detection benchmarks, showing the effectiveness of our proposed method.

Published as a conference paper at ICLR 2024
OUT-OF-DISTRIBUTION DETECTION WITH NEGATIVE PROMPTS
Jun Nie! Yonggang Zhang?* Zhen Fang" University of Science and Technology of China University of Technology Sydney Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
Tongliang Liu" Bo Han Xinmei Tian 15
TTMRR Group, Hong Kong Baptist University "Sydney Al Centre, The University of Sydney
ABSTRACT
(OOD) detection is indispensable for open-world machine learning models. Inspired by recent success in large pre-trained language-vision models, e.g., CLIP, advanced works have achieved impressive OOD detection results by matching the similarity between image features and features of learned prompts, i... positive prompts. However, existing works typically struggle with OOD samples having features similar to those of known classes. One straightfor- ward approach is introduce negative prompts achieve dissimilarity matching, which further assesses the anomaly level of image features by introducing the absence of specific features. Unfortunately, our experimental observations show that employing a prompt like "not photo of a" or learning shared prompt for all classes fails to capture the dissimilarity for identifying OOD samples. The failure may be attributed to the diversity of negative features, i.e., tons of features could indicate features not belonging to a known class. To this end, we propose to learn a set of negative prompts for each class. The learned positive prompt (for all classes) and negative prompts (for each class) are leveraged to measure the similarity and dissimilarity in the feature space simultaneously, enabling more accurate detection of OOD samples. Extensive experiments are conducted on diverse OOD detection benchmarks, showing the effectiveness of our proposed method.
INTRODUCTION
Deep neural networks have achieved remarkable success with large-scale labeled data available. Most deep learning methods are designed in closed-set environments, where models are trained under the in-distribution (ID) assumption that the label space at test time remains the same as that of the training samples (Huang et al. 2017). However, in reality, samples from new classes may emerge spontaneously, thereby breaking this assumption. To address this issue, distrr-iistiiuution (OOD) detection (Bendale & Boult, 2016) is gaining more attention. In OOD detection, the model is required not only to classify ID samples precisely but also to discern OOD samples accurately.
Many representative OOD detection methods are based on the post-hoe strategy (Hendrycks & Gimpel 2017 Liang et al. 2018. [iu et al.) 20200 (Sun et al., 202T, Sun & Li (2022), which identifies OOD samples by analyzing the properties of predictions made by a well-trained model on ID samples. However, the performance of post-hoe methods heavily relies on the quality of the extracted features from the well-traince model. In general, identifiable features for ID classification do not necessarily aid in the detection of OOD samples. For example, in the ID classification tasks for cats and horses, it is clear that the cat ears and horseshoes however, these features are not always applicable for OOD detection, especially when other animals, e.g., tigers, with ears similar cat ears involved as OOD, see Figure T The gap between classification and OOD detection is that in classification, we only need to consider the difference between a cat and a horse, where we can say the object with a pair of pointed ears is a cat. But in OOD detection, we need to consider the difference between a cat and all the remaining classes, where we should say the object with pair of pointed ears, pudgy face, a long beard and other corresponding features is cat, and the object without a pair of pointed ears or without a pair of
the identifiable semantic features for ID classification,
"Correspondence Yonggang Zhang Sonygrhan@@com.hkenie www.hkww............. hk); Xinmei Tian (xinmei Sinmei@uste.ccuun wwwttuuteeeea edu...

---

Published as a conference paper at ICLR 2024
Traditional OOD detection methods:
curs
tiger are similar
Perform classification and get following rules:
horseshoes
data
OOD data
with f - calss, not class, - without fi
In classification:
and detect given sample as ID) or OOD based whether fa is detected.
21 (horseshoes) - horse
Vi (pointed ears) - cat
However, these rules only hold for specific classification task.
An ood sample such as tiger sharing similar cars with cat is misclassified as ID class cat.
V'
ood detection we need stricter rules:
OOD detection, need more complete description cat:
with and fiz and fa calss, not class, without fa fiz or f'a...
F23
and V3
We try uncover more features by exploring what not class,
So that tiger without Vs can be correctly judged not
P4
The features learned by ID samples for ID classification are insufficient for OOD detection. Cat ears and horseshoes are enough to distinguish between cats and horses, but they cannot distinguish between the ID class cat and the OOD class tiger, tigers have cars similar to those of cats.
Figure I:
pointed ears or without long beard or other corresponding features is not cat. Inspired by the causal understanding in (Zhang et al., 2022), we argue that simply using features learned from classification tasks under limited classes for OOD detection is not reasonable.
An effective solution involves utilizing large models like CLIP (Radford et al. 2021 trained on extensive data. Unlike models tailored for specific classification tasks on ID samples, CLIP can generate distinctive image features for all classes, both ID and OOD. This raises a pivotal question: how can we leverage CLIP's extracted featuure features for OOD detection? Some previous works (Ming et al., [022 Fort et al. 2021) propose an approach by matching the similarity between image features and text features of ID classes. Specifically, hand-crafted or learned prompts like "a photo of [class]" are fed to the text encoder to compute text features. The cosine similarity between these text features and the image feature determines the sample's likelihood of belonging to "[class]."
One of the primary limitations of this approach lies in its exclusive reliance on the positive features of ID classes. For example, When the given image input tiger, and we provide "a photo of a cat" as prompt, CLIP will assign high score because the tiger shares similar image features, like ears, with a cat. It completely neglects the distinguishing features that differentiate a tiger from a cat. To address this challenge, we propose to construct negative prompts such "not photo of to achieve a dissimilarity matching. Our intuition to leverage negative prompts like "not a photo of a" to identify objects absent in image. For instance, consider examples like "not photo of a cat". With this prompt, our objective is to steer CLIPP's attention toward features in the image unrelated cats (i.e., negative features). It is worth noting that the description "not a photo of a cat" is an incorrect prompt for cats, but an accurate one for tigers. However, crafting effective negative prompts is not straightforward task. Our empirical results show that simply using "not a photo of cat" as negative prompt leads to a higher matching score with photo of a cat than with a photo of a tiger. This aligns with the findings of recent study (Yüksekgönül et al. 2022).
Previous works have focused on prompt representation learning, such as CoOp (Zhou et al. 2022by and CoCoOp (Zhou al., 2022a), where they aim to improve image classification accuracy based on pre-trained vision-language models. In contrast to these works, our work targets learning negative prompts that inform the network of what is "not." To mitigate this challenge, in this paper, we propose novel method named Learn to Say No (LSN), which can learn suitable negative prompts to tell the network what is "not." Unlike positive prompt learning, where the learned positive prompts are usually shared across different classes most positive information about class is contained in the class name, the negative information about a class can not be carried by class name alone. The negative features of class usually diverse. To this end, we learn set of negative prompts for
2

---

Published as a conference paper at ICLR 2024
"a photo of a [class]"
"not photo of [class]"
Figure Qualitative comparison of the learned positive prompts and negative prompts.
each class. To further improve performance, we also learn positive prompts to replace hand-designed prompts such as "a photo of a". The simultaneous incorporation of positive and negative prompts contributes to a more accurate OOD uncertainty estimation of the pre-trained CLIP model. As shown in Figure 2 we visualize heat maps depicting the learned positive and negative prompts. These visualizations reveal distinct regions of interest for positive and negative prompts. Using both positive and negative prompts to detect samples allows focusing on more diverse features of the samples, thus achieving better OOD detection performance.
The work most similar to ours is RPL (Chen et al. 2020) and CLIPN (Wang et al. 2023a). RPL learns reciprocal points negative representations corresponding to each known class. However, RPL constructs the information of "not belong to a class" using features learned from classification tasks ID classes. This approach has been demonstrated be less rational. CLIPN leverages large-scale dataset CC3M (Sharma et al., 2018) to fine-tune the text encoder, endowing CLIP with the ability to say "no". In contrast, in this paper, we focus on learning prompts uncover the encoded knowledg of negative features, which benefits from the powerful visual representation capabilities of CLIP and can be done very short time. We summarize our contributions as follows:
We propose to use negative prompts to answer positively what is "not" to complement the feature representations, thus improving the ability of the model handle OOD samples.
We reveal that CLIP cannot understand the meaning of "not" very well. For this reason, we learn a set of negative prompts rather than using hand-designed prompt.
Extensive experiments show that learned prompts greatly improve the OOD uncertainty estimation of CLIP, and an ablation study is conducted to understand the efficacy of LSN.
2 PROBLEM SETUP
Let X' and y = ..,,,,,,. lc} be the image space and ID label space, respectively. Note that ) is a set containing words, e.g., y = (cat, dog, ... horse], and C' is the number of ID classes. Given the ID feature random variable Xin E X' and OOD feature random variable X E X' we use the Pxt and F xou to represent the ID marginal distribution and OOD marginal distribution, respectively.
Image and text encoders with CLIP-like model. Given any input image x and any label E V, we extract features of x and le using the CIke--ike model f, which consists of image encoder ff and text encoder g. Then, the extracted features are fxx and ((..), Vc 1, ..., C, where (.)) represents the positive prompt template for ID labels and means the function composition.
OOD detection. In the classical classification task, we assume that each image will definitely belong to a certain ID class. OOD detection, the test samples may contain some unknown images P X and the categories of these images are beyond ID label space, i.c., ( cL y.
Given ID training samples T = {(11, y1), ..., (Xn, Mn)} } drawn from IP Xia, OOD detection aim to learn an OOD predictor G using T,, such that 1) the predictor G can classify ID samples into correct ID classes and 2) the predictor G can detect OOD samples as OOD.
Score functions. Following many representative OOD detection methods (Hendrycks & Gimpel 2017; Liang et al.] 2018 Liu et al.] 2020), given a threshold y and a score function S, then x is detected III sample if and only if 5xx 2 7:
(x) ID, if Sxx) 7; otherwise, Gx( OOD.
(1)
3

---

Published as a conference paper at ICLR 2024
The performance of OOD detection depends on how design score function S to make OOD samples obtain lower scores while ID samples have higher scores.
PROPOSED ALGORITHM
3.1 CAN CLIP UNDERSTAND THE MEANING "NOT"?
While CLIP has demonstrated impressive performance in various zero-shot and few-shot tasks, recent research Yüükskköönül et al. 2022) reveals that state-of-the-arl VLMs operate like bags-of-words, lacking relational understanding leading errors in linking objects to their attributes, and showing severe lack of order sensitivity. Our study further underscores CLIP's challenge in comprehending the concept of "not." As depicted in Figure when presented with an image an elephant, CLIP surprisingly ranks "not photo of an elephant" as a more favorable match than "not photo of dog." contrary to reality. Additionally, Figure when neither of the two provided text inputs accurately describes the picture, CLIP tends to assign higher probability the input containing the word "not." From these observations, it is inferred that CLIP does not completely comprehend the meaning of "not." " This may be attributed to the training method utilized by CLIP.
photo of an elephant
1.00
not photo of an elephant 1.00
a photo of a dog 0.00
not photo of a dog
0.00
Figure 3: Each time, we feed CLIP with two text inputs and ask it to make a choice based on the given image. When the given image an elephant and the given text inputs are "not photo of an elephant" and "not a photo of a dog," CLIP makes the wrong choice.
photo of dog
0.71
photo of dog
0.34
photo of cat
0.29
not a photo of a cat
0.66
Figure 4: In the two experiments before (left) and after (right), just adding "not" to the low-scoring text input, CLIP makes a different judgment, indicating that CLIP is sensitive to "not."
3.2 ALGORITHM DESIGN
Since CLIP lacks the ability comprehend the concept of "not," using the original "this is not photo of" as negative prompts do not have the desired effect. Next, we introduce our strategy on how learn effective negative prompts to help model understand what "not".
Preliminaries: visual prompt learning, For pre-trained vision-language models, the text input, known prompt, plays key role in downstream tasks. However, identifying the right prompt is moontriivial task, which often takes a significant amount of time for word tuning. To bypass it, in CoOp (Zhou et al., 2022b), the authors propose to learn prompt representations automatically. The learnable prompt (l.) for a given class 4, is designed with the following form:
UL, ((.) [1i, U2 Uz, W.,
where U1, U2... are learned vectors and can be analog to the context of the hand-designed prompt such as "a photo of a," each v; has the same dimension as word embeddings (i.c., 512 for CLIP), and is the class token of base class l,, The leamned prompt context [e,,.. vt.] can be shared across classes, or it can be class-specific. among classes, and negative prompts are designed be class-specific. To improve generalization performance, CoCoOp (Zhou et al.] (022a) further learns Meta-Net built with a two-layer bottleneck structure (Linear-RRed Reln-RRlU.i Renerr generate for each image inputconnittionll token m(x). Then
our experiments, positive prompts are designed to shared
4

---

Published as a conference paper at ICLR 2024
the token is added to each context token (x 0; mx)) for i 1,2,, L. To learn prompts, the prediction probability to any ID training sample (x, y4) T is computed as follows:
exp exp (fx)s(((())))))))) (U)))/T)
Pi Ey exp (cos (f (x:). V ((+))/ /T)
where f and g represent the image and text encoders, respectively. cos(-.-) denotes the cosine similarity. The cross- ross-entropy loss is applied to optimize learnable prompt V while keeping the CLIP image and text encoders frozen:
temperature parameter, and
L
E (xT log
Negative classifier. Many OOD detection methods identify OOD samples by analyzing the properties of predictions of the neural networks trained on ID samples (such neural networks trained in conventional way are called positive classifiers). Thus, the performance of such methods heavily relies on the quality of the features extracted from the well-trained model. However, due to the inertia of neural networks, the features extracted by the model, which is trained only on samples, are often inadequate. To alleviate this problem, we propose to learn negative classifier for each ID) class to mine the negative features. To be more specific, for e-th negative classifier, needs to mine the general negative features that samples from class don't 't have but samples from all other classes have. Thus, the e-th negative classifier will produce low activation to samples from class le and produce high activation to other classes. By learning additional negative classifiers, allow models make decisions from both sides based on different features. A toy example is described below:
Consider triple classification task with two ID samples per class (details are shown in Figure 5):
0OD sample:
sample
class
sample
only consider positive feature:
0
clasal
sumple sample
sumple sample 6
consider positive and negative features:
class 2
class
clasal
Figure 5: A toy example of a negative classifier. Each geometric figure represents a feature.
When performing a triple classification task, it is clear that the model can easily use as the repre- sentative feature of "class 1", as the representative feature of "class 2", and as the representative feature of "class 3". When dealing with an OOD sample with descriptive features in Figure (5] the model trained by the triple classification task can easily classify it as "class 1" with high confidence because the model focuses on the learned feature and ignores the other anomalous features. How- ever, when performing a negative classification task, will be used to represent "not class 1", will be used to represent "not class 2", will be used to represent "not class 3". When we add the additional negative classifier, the negative classifier focuses on another feature and thus judges it as "not class 1". The positive classifier judges it "class 1." while the negative classifier judges it as "not class 1", thus reducing the confidence of the model's prediction the OOD sample.
Negative prompt learning. As described above, want to learn negative prompts to capture negative features. These negative prompts construct negative classifiers, thus the powerful visual features extracted by clip can be fully utilized. According to the example in Figure such negative prompts V should satisfy the following two properties:
The negative prompt representation (la) = [w1 02 UL, w.] for given class Le should produce a low match with images whose label is l.. it represents negation.
The negative prompt representation (l)) [V1, ..... Uz we] for given class 1. should produce a high match with all images whose labels are not l. as "this is not a photo of a [CLASS]" is the correct description for all images whose labels not "[CLASS]]
5

---

Published as a conference paper at ICLR 2024
dog
rabit
learnable negative
prompt
Nis
N12
Nuw
class
encoder
encader
N2s
N22
Naxi
Jeurmalle positive prompt
enender
Mez
Naz
Nex
rabit
R
positive teat Reatures
nogative text Reauures
Positive Loss
Negative Loss
image features
Meta-Net
Meta-Net
Figure 6: An overview of LSN. The CLIP text encoder and image encoder are fixed. We optimize positive prompts and negative prompts by Eq. equation 4 and Eq. equation 8 respectively. When use CoOp to learn prompts, we don't 't use the Meta-Net.
Based on the above two properties, we design a novel loss L to optimize the negative prompts:
f = - -B log
(5)
where
exp -1/r-1-ooe co V (U:))/+) Pj ΣLEey exp (1/T - cos (f (f(x)). V(.)) /T)
(6)
In positive prompt learning, most the positive features of a class are carried by the class names, and the learned positive prompt only serves as calibration, so learning a unified positive prompt for all classes sufficient. However, in negative prompt learning, the situation is quite different. The negative features of class cannot be carried by the class name but should be included in the learned negative prompts, and the negative features of a class are often diverse. Therefore, in negative prompt learning, learn class-specific prompts, and for each class, learn of negative prompts. We encourage the learned negative prompts to be different by the following semantic orthogonality loss:
C
K
Creg
(co 12)
j=
where C: -g go Vi(l) is the text feature of i-th negative prompts for class le, For each class, learn K different negative prompts to attend to different negative features. The overall loss function combines the empirical classification loss and regularisation term
-- = L + XLreg:
(8)
where À is a hyper-parameter. An overview of our method is shown in Figure And for testing, we the difference of MCM score (Ming al. 2022) between positive prompts and negative prompts
as the score function: given sample x. the score is
exp (cos (f(x), g V())) S(x) E exp (cos (f(x) go V(()) /T)
exp (cos (f(x), V /T) Σ Σ exp (cos (f(x), V V(G)) /T)
min
Leveraging the designed score 5(x), we can endow OOD detection with the ability say "no".
EXPERIMENTS
4.1 DATASETS
In order to fully validate the effectiveness of our proposed method, we conduct experiments on both large-scale and small-scale datasets. For the large-scale dataset, categorics from ImageNet-11 1k (Deng et al.) 2009) as ID dataset, following the MCM (Ming et al. 2022), and the selected classes kept consistent with MCM. For the OOD test datasets,
100 randomly selected

---

Published as a conference paper at ICLR 2024
Table OOD detection performance on ImageNet-100 as ID. Values are percentages. Bold numbers are superior results. 1 indicates larger values are better, and indicates smaller values are better. denotes the results obtained from the relevant paper, and * denotes the results reproduced by ourself.
OOD Dataset
Average
ID ACC
Methods
iNaturalist
SUN
Places
Textures
FPR951
AUROCT
FPR95I,
AUROC1
FPROS
AUROCT
FPR95L
AUROC1
FPRO5]
AUROC1
MCM
18.13
96.77
36.45
94.54
34.52
94.36
41.22
92.25
32.58
94,48
87.88
MSP"
23.55
95.92
37.02
92.45
40.76
91.23
24.40
94.90
31.43
93.63
91.93
Vim
20.11
96.22
38.56
93.12
44,01
87.33
33.12
93.24
33.95
92.48
91.93
VOS*
12.55
97.53
39.65
92.78
38.84
92.89
15.27
97.19
26.58
95.10
91.87
NPOS*
9.56
97.94
14.93
97.04
17.51
96.33
7.59
98.07
12.40
97.35
91.89
CaOp'
9.30
97.95
11.64
97.61
17.45
96.53
15.94
96.90
13.58
97.25
92.88
CoCoOp
11.76
97.84
14.28
97.13
15.16
96.73
18.27
96.54
14.86
97.06
92.24
CoOp 1.SN (ours)
5.74
98.32
12.42
97.53
14.62
96.89
9.17
97.64
92.88
CoCoDp + LSN (ours)
4.93
98.92
8,23
97.98
12,82
97.19
8.26
98.11
8,56
98.05
92.24
use iNaturalist (Horn et al.) 2018), SUN (Xiao et al.) 2010), PLACES (Zhou et al. 2018), and TEXTURE (Cimpoi et al. (014) following MCM. For completeness, we also conduct experiments on the full ImageNet- Ik dataset. For the small-scale dataset, following ZOC (Esmacilpour et al.
2022), we evaluate the performance of our method on splits of CIFARIO 10 (Krizhevsky et al. (2009),
CIFAR 100 (Krizhevsky et al. 2009), CIFAR+10, CIFAR+50 and Tiny (Le & Yang 2015).
For CIFAR+IO, non-animal classes in CIFAR10 are selected as ID classes, and 10 animal classes
in CIFAR100 are selected OOD classes. For CIFAR+50,, 4 non-animal classes in CIFAR10
selected as ID classes, and 50 animal classes in CIFARIO0 100 are selected as OOD classes
4.2 MODEL ARCHITECTURE AND IMPLEMENTATION DETAILS
Experimental setup. In experiments, we mainly used CLIP as the pre-trained model. For ImageNet- 100 and ImageNet-1k Ik benchmarks, following the same setup in MCM (Ming et al. 2022), we use CLIP-B/16 the backbone model. For CIFARIO benchmark, in agreement with ZOC, we use CLIP-B/S3 B/32. All the images are resized to size 224 * 224. For some baselines that do not require text encoder, we use CLIP's image encoder as their backbone. For positive prompt learning, we closely follow the original implementations in CoOp and CoCoOp. And for negative prompt learning, we use Adam as optimizer (Kingma Ba 2015) for faster convergence. The initial learning rate is set to 1e-3. For each class, learn three sets of negative prompts, and À is set to 0.1.
Evaluation Metrics. For evaluation, we use the following metrics: (1) the false positive rate (FPR95) of OOD samples when the true positive rate of in-distribution samples is at 95%, (2) the area under the receiver operating characteristic curve (AUROC), and (3) ID classification accuracy (ID ACC). When determining the classes of ID samples, we only positive prompts.
4.3
BASELINE METHODS
We compare LSN with several baseline methods as follows, MCM (Ming et al., 2022). MSP (Fort
et al.
2021), ODIN (Liang al. 2018), Energy (Liu et al. 2020), Gram (Huang et al. 2021
Vim
Wang et al. 2022a), KNN (Sun et al. (0226). VOS (Du et al. (022). NPOS (Too et al.
2023
ZOC
Esmaeilpou lpour al. 2022) and CLIPN Wang et al. 2023a). We also compared LSN with vanilla
prompt learning methods such as CoOp (Zhou et al. 2022b) and CoCoOp (Zhou et al. 2022a).
4.4
MAIN RESULTS
Table I Table
2 and Table eehhibit evaluations of our method on several datasets. Results show that LSN achieves the highest average performance compared to other baselines. Compared to the OOD detection baseline MCM for the vision-languue language model. our method LSN reduces the FPR95 from 32.58% to 8.56% on the ImageNet- 100 dataset, a great improvement of 24.02%. Even compared to recent very strong baseline work NPOS, our method still achieves higher performance. Please note that NPOS needs to fine-tune CLIP using all the training samples and needs to use synthetic outliers, while our method only needs to use part of data to train prompts. Even so, our approach still beats NPOS. On the large-scale dataset ImagtNet- Ik, compared to MCM, LSN also reduces the FPR95 from 43.55% to 30.22%, a significant improvement of 13.33%. Compared to CLIPN, LSN achieves consistent performance at smaller computational cost. On the small-scale datasets CIFARIO, 10, CIFARIO0 100, CIFAR+10, CIFAR+ 100, and Tinylmagenet, LSN also outperforms previous baseline works. The superior performance shows that the learned positive prompts and negative
7

---

Published as conference paper at ICLR 2024
Table 2: OOD detection performance on ImageNet-lly as ID.
Methods
iNaturalist
SUN
Places
Textures
Average
ID ACC
FPR95]
AUROC1
FPR95]
AUROCT
FPR95
AUROCT
FPR95L
AUROCI
FPR95]
AUROCT
MCM
58.05
43.55
90.62
68.53
MSP
54.05
87.43
73.37
78.03
72.98
78.03
68.85
79.06
67.31
80.64
79.64
ODIN
54.04
87.17
85.54
51.67
47.75
88,
79.64
Energy"
29.75
94.68
53. 18
87.33
56.40
85.60
51.35
88.00
47.67
88.
79.64
GradNorm
81.50
72.56
82.00
72.86
73.70
79.36
70.26
80.82
72.35
79.64
Vim
32.19
93.16
54.01
87.19
60.67
83.75
53.94
87.18
50,20
87.82
79.64
KNN'
29.17
94.52
35.62
92.67
39.61
91.02
64.35
85.67
42.19
90.97
79.64
VOS'
31,65
94.53
43.03
91.92
41.62
90.23
56.67
86.74
43.24
90.86
79.64
VOS+*
28.99
94.62
36.88
92.57
38,39
91.23
61.02
86,33
41.32
91.19
79.58
NPOS
43.77
90.44
89.44
46.12
91.22
79.42
CoOp'
29.47
94.89
31.34
93.36
40.28
90.07
$4.25
87.58
38.83
91.47
72.93
CoCoOp
30.74
53.84
38.63
CLIPN
23.94
95.27
26.17
93.93
33.45
92.28
40.83
90.93
31.10
93.10
68.53
CoOp LSN (ours)
23.48
95.47
29.84
93.45
36.43
90.87
38.16
89.52
31.97
92.33
72.93
CoCoOp I.SN (purs)
21.56
95.83
26.32
94.35
34.48
91.25
38.54
90.42
30.22
92.96
71.89
Table OOD detection performance in AUROC on CIFARIO, CIFARI00, CIFAR+10, CIFAR+50
and Iinylmagenet. Each result in the table the average of 5 splits of each dataset. We do not show the FPR95 score for ZOC it reported in the original paper.
Dataset
CIFARIO
CIFARI00
CIFAR+10
CIFAR+50
Tinylmagenet
Methods
FPR95L
AUROCT
FPR95
AUROCT
FPR95,
AUROC
FPR951
AUROCT
FPR95L
AUROC
MCM*
51.87
90.06
79.18
82.68
46.68
94.03
46.66
94.20
74.36
84.60
ZOC*
93.00
82.10
97.80
97.60
84,60
MSP
50.31
91.63
72.39
82.59
12.82
97.32
15.60
96.54
68.00
84.15
CoOp
35.21
94.44
72.24
84.67
14,26
97.36
26.37
95.86
63.24
87.84
CoCoOp
38.16
69.38
85.03
11.69
20.57
66.56
87.42
CoOp LSN (ours)
25.16
95.23
53.28
86.14
6.27
98.75
8.25
98.21
53.27
88.15
CoCoOp ISNN (ours)
27.63
95.17
51.27
86.32
5.92
98.93
7.65
98.32
56.83
88,32
prompts greatly improve the out-of-distribution detection capability of the model, demonstrating the effectiveness of the proposed method.
4.5 EMPIRICAL ANALYSIS
In this section, conduct experiments ablation study. Unless otherwise noted, experiments conducted on the ImageNet- 100 dataset with the model of CLIP-B/16 and the method of CoCoOp.
The effect of positive prompts and negative prompts. We conduct ablation studies to examine the effectiveness of learned positive prompts and negative prompts, respectively. Experimental results show that both learned positive prompts and negative prompts improve the out-of-distribution detection ability of the model, as shown in Table The proposed semantic orthogonality loss also contributes to performance improvement. In addition this, we conduct comparison experiment of learning multiple (three) positive prompts. However, experimental results show that learning multiple positive prompts does not further improve performance, while learning negative prompts simultaneously does, further demonstrating the necessity of learning negative prompts.
The gap between positive prompt learning and negative prompt learning. Although our approach to learning negative prompts looks basically the same as CoOp and CoCoOp, there is a fundamental difference between positive prompt learning and negative prompt learning. In positive prompt learning, using a shared prompt across classes is enough to achieve good performance. This is because in positive prompt learning, the positive features each class are carried by the class names, and the positive prompt is only used to calibrate these features to the downstream dataset. In negative prompt learning, on the other hand, the situation is completely different. The negative features need to be contained in the learned negative prompts, and the class names appear be less important. As shown in Table [5] for positive prompt learning, there is little performance difference between using class-shared prompts and class-specific prompts, while when class names are used, the performance degradation is more clear. In negative prompt learning, using class-specific prompts can lead to large performance gains, while the class names are not important.
8

---

Published as a conference paper at ICLR 2024
Table 4: Both positive and negative prompts can improve the OOD detection ability.
OOD Dataset
Methods
iNaturalist
SUN
Places
Textures
Average
FPRO5,,
AUROC1
FPR95I
AUROCT
FPR95]
AUROCT
FPR95]
AUROCT
FPR95,
AUROCT
MCM full model
18.13
96.77
36.45
94,54
34,52
94.36
41,22
92.25
32.58
94.48
4.93
98.92
8.23
97.98
12.82
97.19
8.26
98.11
8.56
98.05
w/o positive prompts
5.16
98.31
20.48
95.53
20.92
95.58
11.77
97.52
14.58
96.73
w/o negative prompts
11.76
97.84
14.28
97.13
15.16
18.27
96.54
14.86
97.06
w/o semantic orthogonality loss
5.01
98.48
10.13
97.68
13.31
96.85
14.47
97.89
10.73
97.73
muki positive prompts
11.37
97.90
15.24
97.09
16.45
96.58
17.79
96.62
15.21
97.04
Table 5: The gap between positive prompt learning and negative prompt learning.
OOD Dataset
Methods
iNaturalist
SUN
Places
Textures
Average
FPR95
AUROCT
FPR955
AUROCT
FPR95]
AUROCT
FPR95L
AUROCT
FPR95,,
AUROC
pos-specifie-w name
11.65
97.66
16.27
96.84
17.92
96.62
19.63
96.42
16.61
96.89
pos-shared-ye w name
11.76
97.84
14.28
97.13
15.16
96.73
18.27
96.54
14.86
97.06
pos-specific-wou name
21.83
96.62
26.72
95.49
22.49
96.15
21.82
96.24
23.22
96.12
neg-specific-w name
13.46
97.14
28.99
94.01
29.44
93.94
15.88
96.79
21.94
95.47
neg-shured-w
81.78
83.62
86.21
77.95
85,45
78.69
71.64
88.83
81.27
82.27
neg-specific-woo name
12.86
97.25
30.01
93.39
28.79
94.44
18.29
96.65
22.48
95.43
5 RELATED WORK
Duttof-distr detection. The goal of OOD detection is to enable the model to discriminate between ID samples and OOD samples while maintaining classification accuracy on ID samples. The representative OOD detection methods mainly include: Post-hoc Detection (Hendrycks & Cimpell 2017) (Liu et al. 2020) (Park et al. 2023). Confidence Enhancement Methods (Hein
al... 2019) (Bitterwolr al. 2020) (Wang al. 20226) and Outlier Exposure (Hendrycks et al.
2019) (Wang et al.) 2023b). Among them, post post-hoc methods have the advantage of being easy to use without modifying the training procedure and objective. Recently, inspired by the success of pre-trained dision-languag models, some works have enriched OOD detection from a single-modal to a multi-modal regime. Ming et al. (Ming et al. 2022) proposes maximum concept matching to align visual features with textual concepts. (Ming & Li, 2023) further investigates the effect of fine-tuning on OOD detection in large vision-language models, and the MCM score is highlighted as effective. These CLid-based OOD detection methods deliver superior performance in a simple way.
Prompt learning. recent years, large vision-language models (VLMs) such as CLIP (Rafford et al., 2021) have demonstrated surprising results in zero-shot and few-shot learning tasks. When applied to downstream tasks, the performance of VLMs can be greatly affected by the prompts. Task-specific prompts can significantly improve performance, but require laborious prompt engineering. To this end, inspired by prompt learning in language tasks, a series of prompt learning methods (Zhou et al., 2022b Lu al. 2022) in computer vision are proposed. Unlike the above methods, we aim to improve the out-of -iitt--dittiitution detection capability of the model through learning both positive and negative prompts, thus fully leveraging the capabilities of CLIP for OOD detection.
Complementary label learning, Complementary Label Learning (CLL) is a new problem in weakly supervised learning that allows models to learn from complementary labels. (Ishida et al. 2017) proposes an unbiased risk estimator (URE) within several specific multi-class loss functions to the classification risk and theoretically establishes an estimation error bound. (Yu et al., 2018) consideres the biased CLs. (Feng et al.) (020)) derives an URE of the ordinary risk for multiple complementary labels, and improves it by minimizing properly chosen upper bounds. LSN takes all ID labels except the correct label as complementary labels.
CONCLUSION
In this paper, we propose negative classifiers to accurately identify when an image does not belong to a particular category. We discuss that the features learned by the negative classifier can be an effective complement to the features learned by the positive classifier. Further, instead of learning such negative classifiers from scratch, construct them top of CLIP. With the help of CLIP, we build such negative classifiers by learning negative prompts. Extensive experiments on standard benchmarks indicate the effectiveness of the proposed method LSN.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This work was supported in part by NSFC No. 62222117, the Fundamental Research Funds for the Central Universities under contract WK3490000005, and KY2100000117. YGZ and BH were supported by the NSFC General Program No. 62376235, Guangdong Basic and Applied Basic Research Foundation No. 2022A151501 1652, HKBU Faculty Niche Research Areas No. RC-FNRA- IG/22-23/SCI/04, and HKBU CSD Departmental Incentive Scheme. TL is partially supported by the following Australian Research Council projects: FT220100318, DP220102121, LP2000000) L.P20200999, and (C190100033
LP220100527,
ETHIC STATEMENT
This paper does not raise any ethical concerns. This study does not involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conflicts of interest and sponsorship, discriminaaion/bias/aaaaee concerns, privacy and security issues, legal compliance, and research integrity issues.
REPRODUCIBILITY STATEMENT
We summarize our efforts below to facilitate reproducible results:
Methodology. Our method is fully documented Section 3.2 with the pseudo algorithm
detailed in Algorithm [ Hyperparameters specified in Section 4.2 with thorough ablation study provided in Section [4.5
Datasets. We use publicly available datasets, which are descrihed in detail Section 4.2 and Appendix
Open Source. Code is available https//gtthbb.oom/jjuzzzdbbbuuuu
REFERENCES
Yong Hyun Ahn, Gycong-Moon Park, and Scong Tae Kim. Line: Out-of-distribution detection by leveraging important neurons. In The IEEE / CVF Computer Vision and Pattern Recognition Conference, CVPR, 2023.
Abhijit Bendale and Terrance E. Boult. Towards open set deep networks. In The IEEE / CVF Computer Vision and Pattern Recognition Conference, CVPR, 2016.
Julian Bitterwolf, Alexander Meinke, and Matthias Hein. Certifiably adversarially robust detection of out-of duft----distiittioon data. In Advances in Neural Information P Processing System, NeurlPS, 2020.
Guangyao Chen, Limeng Qiao, Yemin Shi, Peixi Peng, Jia Li, Tiejun Huang, Shiliang Pu, and Yonghong Tian. Learning open set network with discriminative reciprocal points. In Proceedings of the European Conference on Computer ECCi,,,ECCV, 2020.
Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describ- ing textures in the wild. In The IEEE/ / CVF Computer Vision and Pattern Recognition Conference, CVPR, 2014.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. The IEEE / CVF Computer Vision and Pattern Recognition Conference, CVPR, 2009.
Andrija Djurisic. Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for ICIRR 2023.
detection. In International Conference on Learning Representations,
Xuefeng Du Zhaoning Wang, Mu Cai, and Yixuan Li. VOS: learning what you don't 't know by virtual outlier synthesis. In International Conference on Learning Representations, ICLR, 2022.
10

---

Published as a conference paper at ICLR 2024
Sepideh Esmaeilpour, Bing Liu, Eric Robertson, and Lei Shu. Zero-shot distr--iistriuution detection based on the pre-trained model CLIP. In The AAAI Conference on Artificial Intelligence, AAAI, 2022.
Lei Feng, Takuo Kaneko, Bo Han, Gang Niu, Bo An, and Masashi Sugiyama. Learning with multiple complementary labels. In International Conference Machine Learning, ICML, 2020.
Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-of-distribution detection. In Advances in Neural Information Processing Systems, NeurIPS, 2021.
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high- confidence predictions far away from the training data and how to mitigate the problem. In The IEEE / CVF Computer Vision and Pattern Recognition Conference, CVPR. 2019.
Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and out-of distrriibtt examples in neural networks. In International Conference on Learning Representations, ICLR, 2017.
Dan Hendrycks, Mantas Mazeika, and Thomas G. Dietterich. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations, ICLR, 2019.
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alexander Shepard, Hartwig Adam, Pietro Perona, and Serge J. Belongie. The inaturalist species classification and detection dataset. In The IEEE / CVF Computer Vision and Puttern Recognition Conference, CVPR, 2018 8.
Ping Hu, Ximeng Sun, Stan Sclaroff, and Kate Saenko. Dualcoop++: Fast and effective adaptation to multi-label recognition with limited annotations. arXiv preprint (eXiv:2308.01890 2023.
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In The IEEE/ / CVF Computer Vision and Pattern Recognition Conference, CVPR, 2017.
Rui Huang, Andrew Geng, and Yixuan Li. On the importance gradients for detecting distributional shifts in the wild. In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing System, NeurIPS, 2021.
Takashi Ishida, Gang Niu, Weihua Hu, and Masashi Sugiyama. Learning from complementary labels. In Advances in Neural Information Processing System, NeurIPS, 2017.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), International Conference on Learning Representations, ICLR, 2015.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 23/N, 7(7):3, 2015.
Junnan Li, Dongxu Li. Caiming Xiong, and Steven C. H. Hoi. BLIP: bootstrapping language-iman pre-training for unified vision-language understanding and generation. International Conference Machine Learning, ICML, 2022.
Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference Learning Representations, ICLR, 2018.
Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based out-of-distribution detection. In Advances Neural Information Processing Systems, NeurlPS, 2020.
Yuning Lu. Jianzhuang Liu, Yonggang Zhang, Yajing Liu, and Xinmei Tian. Prompt distribution learning. In The IEEE/ CVF Computer Vision and Pattern Recognition Conference, CVPR, 2022.
Yifei Ming and Yixuan Li. How does fine- fnn-ttning impact language models? Int. J. Comput. Vis, 2023.
detection for vision-
11

---

Published as a conference paper at ICLR 2024
Yifei Ming. Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. Delving into out-of- distribution detection with -anguuge representations. In Advances in Neural Information Processing Systems, NeurlPS, 2022.
Jaewoo Park, Yoon Gyo Jung, and Andrew Beng Jin Teoh. Nearest neighbor guidance for out- of-distribution detection. In IEEE/CVF International Conference on Computer Vision, ICCV, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, ICML, 2021.
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of ACI,, 201 8.
Ximeng Sun, Ping Hu, and Kate Saenko. Dualcoop: Fast adaptation to multi-label recognition with limited annotations. In Advances in Neural Information Processing System, NeurIPS, 2022a.
Yiyou Sun and Yixuan Li. DICE: leveraging sparsification for nutto--distribution detection. In Proceedings of European Conference on Computer Vision, 2022.
Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of disto--dittiibutionn detection with rectified activations. In Advances in Neural Information Processing Systems, NeurlPS, 2021.
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International Conference on Machine Learming, ICML, 2022b.
Leitian Tao, Xuefeng Du, Xiaojin Zhu, and Yixuan Li. Non-parametric outlier synthesis. In International Conference on Learning Representations, ICLR, 2023.
Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: -iittr--itttittionn with virtual- logit matching. The IEEE/ CVF Computer Vision and Pattern Recognition Conference, CVPR. 2022a.
Hualiang Wang, Yi Li, Huifeng Yao, and Xiaomeng Li. CLIPN for zero-shot OOD detection: Teaching CLIP to say no. In IEEE/CVF International Conference on Computer Vision, ICCV, 2023a.
Qizhou Wang, Feng Liu, Yonggang Zhang, Jing Zhang, Chen Gong, Tongliang Liu, and Bo Han. Watermarking for dustof-distribution detection. In Advances in Neural Information Processing System, NewrIPS, 2022b.
Qizhou Wang, Zhen Fang, Yonggang Zhang, Feng Liu, Yixuan Li, and Bo Han. Learning to augment distributions for out-o--distibbttion detection. In Advances in Neural Information Processing System, NeurlPS, 2023b.
Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. SUN database: Large-scale scene recognition from abbey to zoo. In The IEEE / CVF Computer Vision and Pattern Recognition Conference, CVPR, 2010.
Xiyu Yu, Tongliang Liu, Mingming Gong, and Dacheng Tao. Learning with biased complementary labels. In Proceedings of the European Conference on Computer Vision,ECCV, 2018.
Mert Yüksekgönül, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why languaee models behave like bags-of-words, and what to do about it? International Conference on Learning Representations, ICLR, 2022.
Yonggang Zhang, Mingming Gong, Tongliang Liu, Gang Niu, Xinmei Tian, Bo Han, Bernhard Schölkopf. and Kun Zhang. Adversarial robustness through the lens of causality. In International Conference on Learning Representations, ICLR, 2022.
Bolei Zhou, Ágata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell., 40(6): 1452-1464, 2018.
12

---

Published as a conference paper at ICLR 2024
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In The IEEE/ CVF Computer Vision and Pattern Recognition Conference, CVPR, 2022a.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning prompt for vision- language models. Int. J. Comput. Vis., 2022b.
APPENDIX
A LIMITATIONS
LSN relies heavily the features learned by CLIP. If the features extracted by CLIP itself for some categories images are not strongly discriminative, the effect of learning the prompts based on these features may be poor.
Compared CoOp, LSN doubles the training and inference time.
LSN uses only small number of samples to learn the prompts to improve the model's OOD detection. However, limited by the small number of learnable parameters, the model's detection ability is not improved more when increasing the training samples further, which results LSN not benefiting from more training samples.
B ALGORITHM OF LSN
We summarize our algorithm in implementation, as shown in Algorithm| !
Algorithm Learn to Say No (LSN)
Input: Pre-trained VLM encoders for image f and text ID training samples T; ID labels y
Output: Score function S and OOD predictor G
1: Randomly initialize the positive prompts V and negative prompts V
2: for Epoch=1 do
3: Sample mini-batch {(x1, 71),... ...,, (Xx, [6)] from ID training samples [
4: Compute image features f (xx) I.....b
5:
Compute positive text embeddings g V(l,,, c= L...C
6:
Compute negative text embeddings
((c,,, c=
7: Compute positive loss L using Eq.equation 4 and update V by gradient descent
8: Compute negative loss - using Eq.equation and update V by gradient descent
9: Compute the score function by Eq.equation 9 and obtain OOD predictor by Eq. equation]
C DETAILS OF DATASETS
In this section, we provide dataset details.
C.O.1 IMAGENET BENCHMARK
Following MCM (Ming 2022), we choose 100 classes from ImageNet- 1k (Deng et al. 2009 to create ImageNet- 100. The chosen classes are the same as MCM:
n03877845, n03000684, n031 nO3710721, n02825657, n02113186, n01817953, n04239074,
n02002556, n04356056, n03187595, n03355925, n03125729, n02058221, n03016997,
n02843684, n04371430, n01944390, n03887697, n04037443, n02493793, n01518878, n03844681..
n04179913, n01871265 n03866082, no3 n01910747, n03388549, n03908714, n01855032,
n02134084, n03400231, n04483307. m04488333,
n03721384, m02033041, n01775062, n02808304, n13052670,
n01601694, n04136333, m03272562, n03895866, n03995372, n06785654, n02111889, n03447721.
n03666591, m04376876, m03929855, n02128757, n02326432, n07614500, n01695060, n02484975.
n02105412, n04090263, n03127925, m04550184, n04606251, n02488702, n03404251, n03633091.
13

---

Published as a conference paper at ICLR 2024
n02091635, n03457902, n n02483333, n04466996, n02871552 0 n01490041,
n02107312, n01632458, n03394916, n04147183, n04418357, n03218198, n01917289, n02102318, n02088364, m09835506, n02095570, m03982430, n04041544, m04562935, n03933933, n01843065, n02128925, n02480495, m03425413, m03935335, n02971356, n02124075, n07714571, n03133878, n02097130, n021 13799, n09399592, n03594945.
Similarly, we use subsets from iNaturalist (Horn et al. 2018), SUN (Xiao et al., 2010), Places (Zhou
al. 2018) and Texture (Cimpoi al. 2014) as OOD datasets, which are created by Huang al. (?). The classes from OOD datasets do not overlap with Ikk. A brief description of them is as follows:
iNaturalist contains images from the natural world. It has 13 super-categories and 5089 sub- categories covering plants, insects, birds, mammals, and so on. The subset containing 110 plant classes not showing in ImageNet-1k k is chosen as the OOD test set.
SUN contains 899 categories that cover more than indoor, urban, and natural places. We use the subset that contains 50 natural objects that do not overlap with ImageNet- Ik.
Places contains photos labeled with scene semantic categories from three macro-classes: Indoor, Nature, and Urban. We use subset sampled from 50 categories that are not present in ImageNet- k.
Texture contains images of textures and abstracted patterns. As no categories overlap with ImageNet- 1k, we use the entire dataset.
C.0.2 CIFARIO BENCHMARK
Following ZOC (Esmaeilpour et al. 2022). we also evaluate our methods LSN on CI- FARIO (Krizhevsky al. 2009), CIFAR 100 (Krizhevsky al. 2009), CIFAR+10 (Krizhevsky et al.) 2009). CIFAR+50 (Krizhevsky et al. 2009) and Tinylmagenet datasets (Krizhevsky et al. 2009). For CIFARIO, 6 classes are selected ID classes, and the 4 remaining classes are selected as OOD classes. Experiments are conducted on 5 different splits. For CIFAR+10, 4 non-animal classes from CIFARIO are selected as ID) classes, and 10 animal classes from CIFARIOS are selected as OOD classes. For 550 4 non-animal classes from CIFARIO are selected ID classes, and all 50 animal classes from CIFAR 100 are selected as OOD classes. For CIFAR 100, 20 consecutive classes are selected as ID classes. The remaining 80 classes are selected as OOD classes. For Tiny Imagenet,,, 20 classes are used as ID classes, and the remaining 180 classes are used as OOD classes. All the split is the same ZOC. A more detailed split information is shown in Table 6
D EXPERIMENTAL DETAILS
In this section, we provide the implementation details.
D.O.1 SOFTWARE AND HARDWARE
We use Python 3.8.16 and Pytorch 1.12.1 and several NVIDIA GeForce RTX 2080, 3090, and 4090 GPUs.
D.0.2 TRAINING DETAILS
As mentioned in the formal paper, we learn both positive prompts and negative prompts. For positive prompts, we use unified context, and for negative prompts, we use class-specific context. As the CLIP image encoder and text encoder are frozen in experiments, we extract images' features in advance to speed up the training process rather than using original images. For positive prompts, training done with SGD and an initial learning rate of 0.002. The scaling factor 7 is set to 0.01. When we use CoOp. the prompt length is set to 16, the max epoch is set to 50, and the batch size is set to When we use CoCoOp. the prompt length is set to 4, the max epoch is set to 10, and the batch size is set to 1. For negative prompts, training is done with Adam, and an initial learning rate of 0.001. The prompt length is set to 16. The scaling factor is set to 0.05. When 100, and batch size is set to 8. When we use CoCoOp. the max epoch is set to 10, and batch size is set to 1. As the learned negative prompts benefit from large number of training samples, balance
CoOp, the max epoch is set to
14

---

Published as a conference paper at ICLR 2024
Table Data splits for CIFARIO benchmark. The numbers in the table represent the class indices for ID classes except CIFAR+ cases. For CIFAR+ experiments, we provide OOD classes since non-animal classes are utilized for ID classes. For CIFAR+50, there is only one split, and for the sake of the table's completeness, we have repeated it here 5 times.
split
split 2
split 3
split 4
split 5
CIFARIO
0, 2,
0, 7, 8.
0, 5, 6, 7, 8
3,4, 5, 7, 8,
0, 1, 2, 3, 7, 8
0,1,2,3,
20,21,22,23,
40,41,42,43,
60,61,62,63,
80,81,82,83,
4,5,6,7,
24,25,26.27.
44,45,46,47,
64,65,66,67,
84,85,86,87,
CIFAR100
8,9,10,11,
28,29,30.31,
48,49,50,51,
68,69,70,71,
88,89,90.91,
12,13,14,15,
32.33,34.35,
52,53,54.55,
72,73,74,75,
92,93,94,95
16,17,18.19
36,37,38,39
56,57,58,59
76,77,78,79
96,97,98,99
31,65,45,98,77,
11,97,98,75,66,
15,24,, 11,1,64,42,45,
19,42,21,78,98,
CIFAR+10
93,34,63,26.44
77,95,99,9,,7
34,32,88,63,2
38,66,67,63,44
72,15,46,66,3
19,29,11,1,31,
19,29,11,1,11, 19,29,11,1,31,
19,29,11,1,1,, 19,29,11,1,11,
97,80,74.64,42,
97,80,74,64,42, 97,80.74.64442,
97,80,74,64442, 97,80,74,64,42,
65,21,24.78,45.
65,21,24.78.45.
97.80.74,64,42, 97,80.74,64.42, 97,80.74,64,42, 97,80,74,64,42, 97,80,74,64,17, 97,80,444444 97.80.74,64,42, 97,80.74.64,42, 97,80,74,64,442.
73,14,6,98,36,
73,14,6,98,36,
55,72,43.35,27.
55.72.43.35,27,
CIFAR+50
50.15,18.46,75.
50.15,18,46,75.
97,80.74,64,42, 97,80,74,64,42, 97,80,74,64,42, 97.80,74,64,42, 97,80,74,64,42, 97,80,4,,64444,
38,66,77,95,99,
38,66,77,95,99,
93,4.34.32,88.
93,4334.2,,88,
97.80,74,64,42, 97,80.74.64,42, 94,8044,4422
67,30,63,26,79.
97,80,74,64,42, 97,80,74,64,42 97.80.74,64,42, 67,30,63,26,79,
44,7,2.91,3
44,7,2,91,3
97,80.74,64,42, 97,80,74,64,42, 97,80.74,64,42,
192.112,145.107,
156,157,167,1755
128.132,123,72, 10.13,, 102,79,47,106,
91,180,144.193,
153,11,147,0,
154,35,86,10. 90,167,61,13,
59,93.145,10,
Tinylmagenet
124,159,49,12, 199,171,132,60. 10.125.116,28,
188,28,85,89, 54,78,82,107,
62,175,76,183,
72,124.54,77, 157.169,104.166
87.190,101.111,
91,82,116,65.
48,130,38.186.
193,71.131192 80,25,140,46
96,41,134.25
44,8,29.26
time cost and performance, on ImageNet-100, CIFAR1O, CIFAR100, CIFAR+10, CIFAR+50, and Tinylmagenet, 128 samples per class. And ImageNet- 1k, use 64 samples per class.
D.0.3 TEST DETAILS
For testing, we use the difference of similarities between positive prompts and negative prompts as a score function. In formal paper, give abbreviated form. Here, describe it in more detail.
Once we finish the training process, we get the positive prompts V and negative prompts V. Then get the positive prompt representation Vl,.) and negative prompt representation (l,.) for each class l, E Dn (2..c.. For any given test sample x, we calculate the positive cosine similarity between the image feature fx) and the positive text feature g V(..):
s, = cos (f(x).g go V(l.) = f(x) g V(.. [f(x)] [|g o V
(10)
Following MCM, we calculate the maximum concept matching (MCM) score as follows:
(xx(x
SMCM (x; Vin) = max
(11)
E (²)(x)/T'
Similarly, caleulate the negative cosine similarity between the image feature f(x) and the negative text feature o (la):
s. = cos (f(x). V(+)) f(x) g V(L.)
(12)
[[(x)]] g V(1.
15

---

Published as a conference paper at ICLR 2024
Table OOD detection performance on ImageNet- 100 as ID.
OOD Dataset
Average
INaturalist
SUN
Places
Textures
Methods
FPR95.
AUROCT
FPR95.
AUROCT
FPR951
AUROCT
FPR95
AUROC
FPR95,
AUROCT
MCM
18.02
96.98
47.99
92.67
53.63
91.12
45.19
92.02
41.21
93.19
DualCoOp
15.98
97.02
24.67
95.71
36.05
93.11
17.18
96.72
23.47
95.64
LSN
13.59
97.67
17.85
96.32
26.66
94.26
20.72
96.65
19.70
96.22
Then calculate the minimum concept matching (MinCM) score follows:
SMMMCM (x; = min Dj=1 e
(13)
where т and f are the temperature factors. We set 5 and 7 to be 1 and respectively, as we use different temperature factors when training.
Finally, the score function for OOD detection is the difference between MCM and MinCM:
S(x) = SMCM (x; Vin) - SMICCM (x; Via)
(14)
E COMPARISON WITH DUALCOOP.
similar concept to LSN appears in DualCoOp (Sun et al. 2022a). DualCoOp focuses on multi- label recognition (MLR), where each input image includes multiple objects, and the model needs to identify the classes of all the objects correctly. Similar to our work, DualCoOp learns a pair of prompts to provide positive and negative contexts for the target class. Instead of using hand-crafted thresholding to determine positive labels, the learned prompts naturally result in positive and negative classifier, so the existence of the target class the image can be easily decided by comparing their scores. However, although both DualCoOp and LSN propose negative prompts, their roles are different. In DualCoOp. negative prompts act as threshold for positive prompts of the corresponding category, whereas in LSN, negative prompts focus on learning generic negative features across all other categories except the corresponding category. In DualCoOp, positive and negative prompts optimized simultaneously with Asymmetric Loss, which makes the learning of negative logits intractable, resulting in DualCoOp tending to generate false negative responses, which is consistent with the observation in (Hu et al. 2023). In contrast, in LSN, the learning of positive and negative prompts is separated, and the learned negative prompts successfully focus on different regions than the positive prompts, as demonstrated in Figure 2 Besides, DualCoOp mainly focuses multi-label recognition (MLR), which is a closed-set problem. This needs the class names of the unseen classes for testing. In contrast, LSN focuses on OOD detection, which is an open-set problem. This does not need unseen classes. We compare OOD detection performance of DualCoOp and LSN on ImageNet- 100 with CLIP ResNet-50. As shown in Table 7 DualCoOp.
the result shows that LSN outperforms
F COMPARISON WITH OTHER POS-HOC METHODS.
To enrich our analysis, we further conduct comparative experiments with some recent pos-hoc methods (Par et al.) 2023 Djurisic et al., 2023) [Snn et al. 2022b Ahn al. 2023). Note that here simply replace the backbone used in these works with the CLIP ViT-b/l6 model without further fine-tuning. The results on ImageNet- is shown Table
Our experimental results show that these post-hoc methods don't perform well on the raw CLIP model. We believe a major reason for this is the difference in training data. Many pos-hoc methods designed on ImageNet pre-trained networks, which corresponds the fact that only ID data used during training. In contrast, when training CLIP, both ID data and OOD data are used. The difference training data leads to different activations of OOD data.
16

---

Published as a conference paper at ICLR 2024
Table 8: OOD detection performance ImageNet-Ile IK as ID.
OOD Dataset
Average
Methods
iNaturalist
SUN
Places
Textures
FPR95L
AUROC
FPR95
AUROCT
FPR951
AUROCT
FPR95
AUROC
FPR951
AUROCT
MCM
32.08
94.41
39.21
92.28
44.88
89.83
58.05
85.96
43.55
90.62
NNGuide
97.83
78.32
93.35
81.36
86.93
82.48
93.98
78.59
93.02
80.18
ASH
58.48
87.61
56.35
86,06
58.77
84.34
86.87
61.41
65.11
79.85
KNN
99.36
68.34
97.06
68.05
93.40
72.01
98.38
68.67
97.05
69.27
LINe
73.00
87.34
58.83
89.27
58.92
86.68
60.33
86.01
62.77
87.32
LSN
21.56
95.83
26.32
94.35
34.48
91.25
38.54
90.42
30.22
92.96
Table 9: On the effect of model architectures.
OOD Dataset
Average
Methods
iNaturalist
SUN
Places
Textures
FPR95
AUROC1
FPR951
AUROC1
FPR95_
AUROCT
FPR951
AUROC1
FPR95,,
AUROC
MCM/RN50X4
14.56
97.28
36.09
94.64
36.17
93.99
45.05
92.68
32.97
94.65
LSN/RN50x4
5.37
98.64
14.36
96.84
18.47
97.73
97.31
MCM/CL.IP -B3322
10.01
97.80
94.33
35.67
94.11
40.78
92.96
30.66
94.80
LSN/CLIP- BB32
3.87
99.11
10.69
97.23
15.37
98.24
97.86
MCM/CL.IP-/IIC
18.13
96.77
36.45
94.54
34.52
94.36
41.22
92.25
32.58
94.48
LSN/CLIP-B/IL
4.93
98.92
8.23
97.98
12.82
97.19
98.11
98.05
G LSN WITH DIFFERENT NETWORK ARCHITECTURES.
To show the effectiveness of our method on different model architectures, conducted experiments with CLIP-B/16, CLIP-B/32, and different model structures, our method goes well beyond the baseline MCM.
models, respectively. As shown in Table 9 with
H LSN WITH OTHER VLMs.
We further conduct experiments other VLMs, such as BLIP (L et al. 2022 As shown in Table 10 LSN also works well on BLIP.
I LSN UNDER DIFFERENT NUMBER OF ID CLASSES.
By varying numbers of ID classes, we performed experiments on ImageNet-10, ImageNet-20, ImageNet- 100, and ImageNet-IK lk. The results are as shown in Table indicating that LSN is robust to the number classes.
J THE EFFECT OF THE NUMBER OF LABELED TRAINING SAMPLES PER CLASS.
To study the effect of different numbers of training set samples on the learned positive prompts and negative prompts, vary the number of training set samples per class. As shown in Figure 7 the performance of both positive prompts and negative prompts increases gradually as the number of training samples increases, and the increase of negative prompts is higher than that of positive prompts. Since negative prompts do not perform well with small training sample sizes, we use a relatively large number of training samples in our experiments.
K MORE EXAMPLES ABOUT CLIP's ABILITY ON UNDERSTANDING "NOT"
We exhibit examples show the CLIP's ability to understand "not": Figure g The results of the experiment are consistent with the conclusion in the formal paper. To test the model on more data, the OOD datasets iNaturalist, SUN, Places, and Texture on ImageNet benchmark to
17

---

Published as a conference paper at ICLR 2024
Table 10: LSN with BLIP.
OOD Dutaset
Average
iNaturalist
SUN
Places
Textures
Methods
FPR95
AUROCT
FPR95
AUROC1
FPR95,,
AUROCT
FPR95]
AUROCT
FPR95.
AUROCT
MCM
46.45
93.37
81.76
77.14
81.05
48.10
90.92
62.54
86.77
CoOp
27.83
95.26
45.85
88.26
47.38
36,28
92.93
39.33
90.96
CoOp L.SN
16.33
96.73
39.48
90.94
41.76
90.42
31.59
94.27
32.29
93.09
Table 11: LSN under different number of ID classes.
OOD Dataset
Average
Methods
INaturalist
SUN
Places
Textures
FPR95,
AUROCT
FPR95,
AUROCT
FPR95,
AUROC1
FPR95,
AUROCT
FPR95,
AUROC
ImageNct-10/MCN
0.12
99.80
0.29
99.79
0.88
99.62
0,04
99.90
0.33
99.78
ImageNet-
0.05
99.90
0.09
99.91
0.15
99.89
0.01
99.99
0.08
99.92
20MM
1.02
99.66
2.55
99.50
4.40
99.11
2.43
99.03
2.60
99.32
ImageNet-20/LSN
0.34
99.88
0.76
99.74
0.95
99.75
0.43
99.86
0.62
99.81
ImageNet100/MMM
18.13
96.77
36.45
94.54
34.52
94.36
41,22
92.25
32.58
94.48
mageNNc=1000LSN
4.93
98.92
8.23
97.98
12.82
97.19
8.26
98.11
8.56
98.05
ImageNer- 1KMMM
32.08
94.41
39.21
92.28
44.88
89.83
58.05
85.96
43.55
90.62
ImageNet-1K/LSSN
21.56
95.83
26.32
94.35
34.48
91.25
38.54
90.42
30.22
92.96
evaluate CLIP's ability to understand the meaning of "not." For every sample, we ask CLIP to pick class name of
between "a photo of [class]" and "not a photo of a [class]." where "[class]" ImageNet-lk. lk. For each class, we count the number of samples categorized as "a photo of a [class]" and the number of samples categorized as "not a photo of [class]." As shown in Figure 9 10 it is clear that CLIP tends to choose "not photo of [class]" as the correct description rather than "a photo of [class]".
18

---

Published as a conference paper at ICLR 2024
100
On the effect of num of shot
98
96
94
92
90
88
86
84
shots
16 shots negative prompts
64 shots positive and negatives prompts
128 shots
positive prompts
Figure 7: The effect on the number of labeled training samples per class.
photo of
1.00
photo of house
1.00
photo girl
0.99
a photo an elephant
photo an car
0.00
photo dog
0.01
not photo of cat
0.99
not photo of a house 0.99
not photo of a girl 0.94
not photo of an elephant
0.01
photo of 0.01
photo of dog 0.06
photo of dog a photo an elephant
0.98
photo of photo a planc
0.81
dog photo cat
0.69
0.19
a photo of a dog not a photo of an elephant 0.56
0.44
photo of a car
0.38
photo of a dog not photo of a cat 0.76
0.24
not photo of a plane 0.62
Figure 8: We explore the ability of the model to understand the meaning of nnot." When the correct class name of sample is given, CLIP tends to choose the one that has the correct class name as the sample's description, matter whether "not" exists or But when the correct class name isn given, CLIP tends to choose the one that has "not" as the sample's correct description.
19

---

Published as a conference paper at ICLR 2024
10000
32000
4000
A.prven of Tiiee nota photo of (ilect
envent on Cruned
400
520
$20
2000
260
400
6200
1163
Figure 9: Statistical result iNaturalist.
Figure 11: Statistical result SUN.
5000
2000
a2o
200
Clas
Figure 10: Statistical result on Places.
Figure 12: Statistical result on Texture.
20


Reviews:
[{"review_id": "NE4OI9xVNH", "reviewer": "Reviewer_Dyvx", "summary": "The paper proposes how to use CLIP for OOD detection with negated prompts that include the negation word 'not'. Using the learnable prompts embeddings, the method trains the model by freezing the CLIP encoders based on a contrastive loss. The proposed method shows improvement over the baseline CLIP for OOD detection.", "questions": "I suggest the authors to properly address the above weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is clearly written with descriptive visual figures.\n2. Experiments have been extensively conducted across a set of diverse datasets including small ones and large ones.", "weaknesses": "1. Comparison and related works to state-of-the-arts are missing (e.g., NNGuide [1], ASH [2], CLIPN [3], [4])\n2. The performance is too behind the state-of-the-art\n3. The main concept of the paper is too similar to CLIPN\n4. The performance improvement is very marginal\n\n[1]\n[2] \n[3] Wang, Hualiang, et al. \"Clipn for zero-shot ood detection: Teaching clip to say no.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n[4] Ming, Yifei, et al. \"Delving into out-of-distribution detection with vision-language representations.\" Advances in Neural Information Processing Systems 35 (2022): 35087-35102.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "k8iGBv0uZJ", "reviewer": "Reviewer_r91y", "summary": "In this paper, the authors perform OOD detection based on generic features learned from a large pre-trained language-vision model by matching the similarity between image features and features of learned positive prompts and negative prompts.\nThe core innovation of this paper is the proposed LSN module to learn a set of negative prompts for each ID category to help the network to comprehend the concept of \"not.\" They mine general negative features that are not present in a category but are present in all other categories by proposing a new loss in prompt learning. In the test, the MCM scores of the cosine similarity of positive prompts and negative prompts to image features are used as OOD detection metrics. Extensive experiments on various ood detection benchmarks have been conducted to demonstrate the effectiveness of the method proposed in this work.", "questions": "1, This method is very dependent on the features learned by CLIP. If the features extracted by CLIP itself for some categories of images are not strongly discriminative, the effect of learning the prompts based on these features may be poor.\n2, The way to learn negative prompts is to mine the general negative features that each class of samples does not have but all other classes have, i.e., the negative classifier produces low activation values for that class and high activation values for other classes. Does this result in learning what are actually generic background features rather than general features that all other classes have?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1、This work utilizes the generic feature extraction capability of CLIP and does not need to finetune the image encoder and text encoder. It only needs to learn the appropriate positive and negative prompts by LSN for OOD detection. Therefore this method has high generality and low complexity.\n2、SOTA performance is achieved in different benchmark experiments.", "weaknesses": "The overall prompt learning approach is still based on CoOp without a lot of innovation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MjGr3pVibz", "reviewer": "Reviewer_ZZhi", "summary": "In this paper, the authors present a negative prompt tuning method with CLIP to improve the OOD performance. Specifically, the authors learn class-specific prompts for each category. A semantic orthogonality loss is also applied to encourage diverse negative prompts. The negative prompts are also considered in the evaluation.", "questions": "1. What's the ID dataset in Table 3?\n2. Why do CoOp/CoCoOp and CoOp/CoCoOp + LSN achieve the same ID results in Table 1 and Table 2?\n3. The ID results of CoOp/CoCoOp appear to be significantly lower than other baselines such as NPOS in Table 2. Can the authors explain some reasons?\n\nSome minor suggestions that do not affect my final rating:\n1. It is suggested to use `\\citep' rather than `\\cite' in the latex code\n2. Typo: 'we use we use' at the bottom of page 6", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed LSN method achieves good performance over baselines on various OOD benchmarks.\n2. The authors provide sufficient ablation studies to show the effectiveness of each proposed component.\n3. The proposed idea is simple and easy to understand.", "weaknesses": "1. This paper is related to negative learning or learning with complementary labels. The authors may consider adding some related discussion in the related work section.\n2. The proposed method may double the training and inference time with the negative prompts.\n3. I found a related work that the authors may add discussion in the related work section:\n\" How Does Fine-Tuning Impact Out-of-Distribution Detection for Vision-Language Models? , IJCV 2023.\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PI1XxpT0f2", "reviewer": "Reviewer_ayMs", "summary": "This paper focuses on the out-of-distribution detection problem, which aims to precisely classify samples of known categories, and accurately discern samples of unknown categroies. To facilitate the recognition the out-of-distribution examples, a CLIP-based method is proposed, where a set of learnable negative prompts for each class are introduced. Promising results are obtained compared to existing out-of-distribution detection methods.", "questions": "In equation(9), the positive score and the negative score are obtained independently. If the obtained positive category is different from the negative category, could this lead to mistakenly recognizing a correct positive prediction as out-of-distribution?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. This paper is clearly written and easy to follow. The weakness of the hand-crafted prompts is clearly interpreted and the motivation is reasonable.\n2. The proposed approach is simple and intuitive.\n3. Promising experimental results are achieved compared to existing OOD detection and prompt-based methods.", "weaknesses": "1. Meta-Net in Figure 6 is not introduced in the paper.\n2. The motivation and model design of this paper are similar to DualCoOp. The authors claim that the proposed method could learn negative prompts to capture negative features compared to DualCoOp, but there is no visual or quantitative evidence to verify this statement. Besides, the OOD detection performance of the DualCoOp design is not experimentally verified.\n3. Some existing OOD detection methods proposed in 2022 and 2023 are not compared or discussed[2][3][4].\n4. It seems that the authors have submitted this paper with an ICLR 2023 template.\n\n[1] Dualcoop: Fast adaptation to multi-label recognition with limited annotations.\n\n[2] Out-of-Distribution Detection with Deep Nearest Neighbors.\n\n[3] LINe: Out-of-Distribution Detection by Leveraging Important Neurons.\n\n[4] Decoupling MaxLogit for Out-of-Distribution Detection", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)