Exploring Large Language Models for Multi-Modal Out-of-Distribution Detection
Authors: Yi Dai
Hao Lang
Kaisheng Zeng
Fei Huang
Yongbin Li
Venue: EMNLP 2023 Findings (2023)
Keywords: OOD Detection
TL;DR: LLMs for Multi-Modal OOD Detection

Abstract: Out-of-distribution (OOD) detection is essential for reliable and trustworthy machine learning.
Recent multi-modal OOD detection leverages textual information from in-distribution (ID) class names for visual OOD detection, yet it currently neglects the rich contextual information of ID classes.
Large language models (LLMs) encode a wealth of world knowledge and can be prompted to generate descriptive features for each class.
Indiscriminately using such knowledge causes catastrophic damage to OOD detection due to LLMs' hallucinations, as is observed by our analysis.
In this paper, we propose to apply world knowledge to enhance OOD detection performance through selective generation from LLMs.
Specifically, we introduce a consistency-based uncertainty calibration method to estimate the confidence score of each generation.
We further extract visual objects from each image to fully capitalize on the aforementioned world knowledge.
Extensive experiments demonstrate that our method consistently outperforms the state-of-the-art.

Exploring Large Language Models for Multi-Modal Out-of-Distriuution Detection
Yi Dai Hao Lang Kaisheng Zeng', Fei Huang,, Yongbin Li
Department of Computer Science and Technology, Tsinghua University' Alibaba Group lang, f. F.huag,, shuide. 1yb)8al waaalliiabbbanncce (dai-y21, zks19)@mails. tsinghua. edu.
Abstract
webbed feet
Block Seathers
cut-of-distribution (OOD) detection is essen- tial for reliable and trustworthy machine leamn- ing. Recent multi-modal OOD detection lever- ages textual information from in-distribution (ID) class names for visual OOD detection, yet it currently neglects the rich contextual infor- mation of ID classes. Large language models (LLMs) encode a wealth of world ledgl and can be prompted to generate descriptive features for each class. Indiscriminately using such knowledge causes catastrophic damage to OOD detection due LLMs' hallucinations, as is observed by our analysis. In this paper, we propose to apply world knowledge to enhance OOD detection performance through selective generation from LLMs. Specifically, we intro- uncertainty calibra- tion method to estimate the confidence score of each generation. We further extract visual objects from each image fully capitalize the aforementioned world knowledge. Exten- sive experiments demonstrate that our method consistently outperforms the state-of-the-art.
long. slander
bill
long nesk
World knowledge from LLMs:
descriptivn frettures of swrm
black
Figure 1: World knowledge from large language models facilitate the detection of visual objects.
OOD detection problem that explores the seman- tic information conveyed in class labels for visual OOD detection, relying on large-scale pre-trained lasion-language models such as CLIP (Radford et al., 2021).
duce consistency-beee
In this paper, apply world knowledge from large language models (LLMs) (Petroni al., 2019a) multi-modal OOD detection by gener- ating descriptive features for class names (Menon and Vondrick, 2023). As illustrated Figure 1, find black swan, look for its long neck, webbed feet, and black feathers. These descriptors provide rich additional semantic information for ID classes, which can lead to more robust estimation of OOD uncertainty (Ming et al., 2022a), i.e., measuring the distance from the visual features of input to the closest textual features of ID classes.
1 Introduction
Machine learning models deployed in the wild of- ten encounter out-of-distribution (OOD) samples that not seen in the training phase (Bendale and Boult, 2015; Fei and Liu, 2016). A reliable model should not only obtain high performance on samples from seen distributions, i.e., in-distribution (ID) samples, but also accurately detect OOD sam- ples for caution (Amodei et al., 2016; Boult et al., 2019; Dai et al., 2023b). Most existing OOD de- tection methods are built upon single-modal in- puts, e.g., visual inputs (Hsu et al., 2020; Liu et al., 2020) textual inputs (Zhou et al., 2021; Zhan al., 2021). Recently, Esmaeilpour al. (2022); Ming al. (2022a) attempt to tackle multi-modal
However, the knowledge encoding of LLMs such as GPT-3 (Brown et al., 2020) lossy (Peng et al., 2023) and tends to hallucinate (Ji et al., 2023), which can cause damage when applied for OOD detection tasks. As shown in Figure 2, LLMs gener- ate unfaithful descriptors for class "hen", assuming featherless head appearing in a hen. Indiscrimi- nately employing generated descriptive features model ID classes brings noise to the inference pro- cess due to LLMS' hallucinations. Moreover, this issue becomes more severe as OOD detection deals with samples in an unbounded feature space (Shen
Work done while the author was interning at Alibaba.
Equal contribution. Corresponding author.

---

al., 2021). Collisions between OOD samples and ID classes with augmented descriptors would be common.
detection for the first time by generating de- scriptive features for ID class names.
We analyse LLMs' hallucinations which can cause damage to OOD detection. A selective generation framework is introduced and an uncertainty calibration method in LLMs is developed to tackle the hallucination issuc.
To address the challenge mentioned above, we propose an approach for selective generation of
high-quality descriptive features from LLMs, while abstaining from low-quality unfaithful ones (Ren et al., 2022). Recent studies show LLMs can pre- dict the quality of their outputs, i.c., providing cal- ibrated confidence scores for each prediction that accurately reflects the likelihood of the predicted answer being correct (Kadavath et al., 2022; Si et al., 2022a). Unfortunately, descriptors of class name generated by LLMs are long-fomm and struc- tured intermediate results for the ultimate OOD detection task, and calibration of LLMs for gener- ating such long open-ended text (Lee et al., 2022) is still in its infancy.
We detect objects in an image and represent them with their predicted class names to fur- ther explore world knowledge from LLMs. Our extensive experimentation on various datasets shows that our method consistently outperforms the state-of-the-art.
2 Related Work
OOD Detection is widely investigated vi- sion classification problems (Yang et al., 2021), and also in text classification problems (Lang et al., 2023). Existing approaches try to improve the OOD detection performance by logits-based scores (Hendrycks and Gimpel, 2017a; Liu et al., 2020), distance-based OOD detectors (Lee et al., 2018; Sun et al., 2022), robust representation learn- ing (Winkens et al., 2020; Zhou al., 2021), and generated pscudo OOD samples (Shu et al., 2021; Lang et al., 2022).
We perform uncertainty calibration in LLMs by exploring a consistency- -bassd approach (Wang et al., 2022). We assume if the same correct predic- tion is consistent throughout multiple generations, then it could serve as a strong sign that LLMs are confident about the prediction (Si et al., 2022b). Instead of computing literal similarity, we define consistency between multiple outputs from LLMs for a given input based on whether they can re- trieve similar items from a fixed set of unlabeled images. Specifically, for each descriptor, we first retrieve a subset of images, leveraging the joint lasiun-language representations. Then, we measure generation consistency by calculating the overlap between these image subsets.
Multi-modal OOD detection is recently studied by Fort et al. (2021a); Esmaeilpour et al. (2022); Ming et al. (2022a), which leverages textual infor- mation for visual OOD detection. These works do not explore world knowledge from LLMs.
To further capitalize on the world knowledge expressed in descriptors from LLMs, we employ a general object detector to detect all the candi- date objects (concepts) in an image (Cai et al., 2022) and represent them with their predicted class names (Chen et al., 2023b) such as "mirror", "chair", and "sink" (see Figure 5). These visual concepts provide valuable contextual information about an image in the textual space and can po- tentially match descriptive features of an ID class if the image belongs to that class. Accordingly, we improve our distance metric of input samples from ID classes by considering the similarity be- tween image visual concepts and ID class descrip- tive features in language representations. Our key contributions are summarized as follows:
Large language models like GPT3 (Brown et al., 2020) can serve as a knowledge base and help various tasks (Petroni et al., 2019b; Dai et al.. 2023a). While some works demonstrate that world knowledge from LLMs can provide substantial aid to vision tasks (Yang al., 2022) such as vision classification (Menon and Vondrick, 2023), its ef- ficacy in multi-modal OOD detection is currently underexplored. Moreover, as L.MSs tend halluci- nate and generate unfaithful facts (Ji et al., 2023). additional effects are needed to explore LLMs ef- fectively.
Uncertainty Calibration provides confidence scores for predictions to safely explore LLMs, helping users decide when trust LLMS outputs. Recent studies examine calibration of LLMs in multiple-choice and generation QA tasks (Kada- vath et al., 2022; Si et al., 2022a; Kuhn et al., 2023). In multi-modal OOD detection task, open-ended
We apply world knowledge from large lan- guage models (LLMs) to multi-modal OOD

---

Classitication(t) CIJP+Desp.
OOD Detection())
goldfish
ID dataset
CLIPP
CLIP
CLJP+Desp.
longg flowing tail
ImageNet-Ik
64.05
68.03
42.74
48.99
.small, black eyes
CUB-200
56.35
$7.75
7.09
4,72
small mouth
Stanford-Can
61.56
-dorall fins
101
85.61
88.50
1.86
3.86
Oxford-Pee Pet 81.88
86,92
1.70
3.52
hen
featherless head
Table 1: Effect of class name descriptors from LLMs in classification and OOD detection tasks. CLIP and CLIP+Desp. VLMs based methods without and with descriptors. Classification is evaluated by accuracy and OOD detection is evaluated by FPR95 (averaged on iNaturalist, SUN, Places, Texture OOD datasets).
round body shape
yellow beak
webbed red red-orange legs
Figure 2: Cases of descriptors generated by LLMs. Un- faithful descriptors may appear due to hallucinations.
in previous approach fail to improve the OOD de- tection performance in a few datasets. As shown in Table 1, although descriptors can improve the classification performance in all five datasets, they degenerate the OOD detection performance in four ID datasets. We hypothesize this because LLMs generate unfaithful descriptors due to hallucina- tions (see cases in Figure 2), which bring noise to the OOD detection process.
text (Lee et al., 2022) are generated to provide de- scriptive features for ID classes (Menon and Von- drick, 2023), and calibration in this task is yet un- derexplored.
3 Backkround
3.1 Problem Setup
We start by formulating the multi-modal OOD de- tection problem, following Ming et al. (2022a). We denote the input and label space by X' and V, re- spectively. y is set of class labels/names refer- ring to the known ID classes. The goal of OOD detection to detect samples that do not belong to any of the known classes or assign a test sam- ple to one the known classes. We formulate the OOD detection as a binary classification problem: G(x; X + {0, 1}. where E X' de- notes an input image, I and 'T are image encoder and text encoder from pre-trained vision-language models (VLMs), respectively. The joint vision- language embeddings of VLMMs associate objects in visual and textual modalities well. Note that there no training data samples provided to train the OOD detector.
To verify hypothesis, visualize ID sam- ples from ImageNet- 1k dataset and OOD samples from iNaturalist dataset, together with their origi- nal class names, based on aligned vision-language features (Radford et al., 2021). As illustrated Figure 3(b), class names of ID samples may coin- cide with these of OOD samples, when augmented with descriptors from LLMs. Thus, it is improper to indiscriminately adopt these descriptors.
The above assumptions are also evidenced in Fig- ure 4. ID samples obtain higher similarity scores to their classes when augmented with descriptors (Figure 4(a)), which show the effect of descriptors for the classification task. Meanwhile, OOD sam- ples gain larger maximum similarity scores with ID classes with descriptors (Figure 4(b)), which makes OOD detection more difficult.
3.2 Analyzing Class Name Descriptors from LLMs
Method
Recent work has demonstrated that class name de- scriptors, ie., descriptive features for distinguish- ing a known object category in a photograph gen- erated by prompting LLMs (see Section 4.2 for more details), can improve zero-shot visual classi- fication performance (Menon and Vondrick, 2023) in close-word world setting (Vapnik, 1991). A natural extension of this work is to leverage the descriptors for OOD detection in an open world (Fei and Liu, 2016), which largely unexplored. Unfortunately, we find that the descriptors used
4.1 Overview
In this study, we build the multi-modal OOD de- tector following four steps: 1. Generate set of descriptors d(c) for each class name E y by prompting LLMs; 2. Estimate confidence score for descriptors d(c) with uncertainty calibration: 3. Detect visual objects for each test image x; 4. Build an OOD detector with selective generation of descriptors and image visual objects. Figure 5 shows an overview of our approach.

---

ID Text
ID Text Class Descriptor
Text with Class Nama Selective Descriptor
1D Text
ID Text
D Text
GOD Image
DO0 Image
DOD Im
(a)
(b)
Figure 3: t-SNE visualization and OOD samples, which are from ImageNet- Ik iNaturalist datasets, respectively. ID/OOD Image represent visual representations of images, and ID/OOD Text are textual representations their class names, based on CLIP. Each line denotes pair of vision-language representations for one sample.
Similanty to Class Name (ID Samples)
Maximum similarity (0OD Samples]
consistency- based (Wang al., 2022) uncertainty calibration method to estimate a confidence score for each generation, which helps decide when trust the LLMs outputs. We assume if the same correct prediction is consistent throughout multiple generations, then it shows that LLMs are confident about the prediction (Si et al., 2022b), thus the generation results are more trustworthy.
Name
(9)
It is non-trivial to directly extend previous consistency-baad methods to settings. Specifi- cally, we leverage LLMs generate long-form and structured descriptor lists without fixed candidate answer set. Any permutation of the descriptor list can convey the same meaning, despite the incon- sistency in surface form. Meanwhile, the outputs of LLMs are intermediate results for our OOD de- tection task. It is challenging to measure quality of these intermediate results while maintaining a tangible connection to the ultimate detection task.
Figure 4: Left: Similarities between ID samples and their class names; Right: Maximum similarities be- tween OOD samples and ID classes. ID samples from mageNet-1k KK and OOD samples from Naturalist.
4.2 Descriptor Generation
To apply world knowledge from L.MSs for OOD detection, we generate set of descriptors d(c) for each known class name è by prompting LLMs (see Figure 6), following (Menon and Vondrick, 2023). We randomly select visual category and manually compose descriptors to use as 1-shot in- context example. We prompt LLMss to describe the visual features for distinguishing category in photograph. The generated list composes the d(c). Figure 2 shows cases of generated descrip- tors, which include shape, size, color, and object parts in natural language.
We take inspiration from prior code generation works (Li al., 2022: Chen al., 2023a), which prompt LLMs generate structured codes aiming at solving programming tasks. Multiple codes are sampled from LLMs for one input and then execu- tion results are obtained by executing these codes. The code with the most frequent execution result is selected as the final prediction. In similar manner, quantify the characteristics of descriptor sets through their retrieval feedbacks from fixed set of unlabeled images, and define their consistency according to consensus among retrieval results. Specifically, we propose a three-stage consistency- based uncertainty calibration method.
4.3 Uncertainty Calibration
As Figure 2 illustrates. LLMs may generate unfaith- ful descriptors due to hallucinations, which would hurt the performance OOD detection if applied indiscriminately. address this issue, we design a

---

ID classes y
1. Descriptor Generation
2. Uncertainty Calibration
OOD Detection
3. Object Detection
Test images
barber shop
ID Image
(barber shop)
basketball
barber shop
aa barber chair
barber shop which has
sink
barber mirrors sink
1.Mirror
Prompt festures for distinguishing a (class name) photo?
M
2.Chair 3.Sink
round
brown leather
basketball
20 ounces
basketball
OOD image
p(cE=D.4
1.8icyele
2.0ag
Large Language Model
Vision-Laguuage Model
General Object
Figure 5: Our multi modal OOD detection framework. For each image x and ID classes y 1. Generate descriptor set for each class e y by prompting LLMs; Estimate confidence score p(c) for each descriptor set: 3. Detect objects and represent them with object names; 4. Compute the maximum class matching score Smax(
Q: What are useful visual features for distinguishing a lemur a photo?
17. Note that text similarity between descriptor sets can also be used in consistency computation.
Q
prompt
A: There are several useful visual features
Stage III. We compute the confidence score p(c) for descriptor set d(c) as is* where 8" is the largest group in S(c).
to tell there a lemur photo:
furry bodies
Aprmpt
long tail
large eyes
4.4
Visual Object Detection
Q: What are useful visual features for distinguishing a goldfish in a photo?
Prest
To further capitalize on the world knowledge con- veyed in generated descriptors, we introduce a gen- eral object detector with a vocabulary of 600 object categories to detect visual objects uxx) for each testing image (Cai et al., 2022). Specifically, v(x) consists of detected objects' class names, such "mirror", "chair", and "sink" in pho- tograph of a barber shop (see Figure 5).
A: There several useful visual features
to tell there is goldfish in photo:
long, flowing tail small, black eyes
Asest
a small mouth dorsal fins
Figure 6: Example prompt for generating descriptors of the category goldfish. The trailing '-' guides LLMs to generate text the form of bulleted list.
4.5 OOD Detection
For each ID class name c, descriptor set d(c) is used to augment the representation of e its con- fidence score p(c) above threshold 7, otherwise is used to represent that class only. Thus, the textual features for class name are:
Stage We sample n sets of descriptors D(c) {di (c), .. ,.,)...,,(())) from LLMs for each ID class name e C y.
Stage II. We cluster descriptor sets D(c) into groups S(c). where each group S(c) is com- prised of descriptor sets that consist with each other. We define descriptor consistency, C(..),, which re- tains any two descriptor sets that share the same characteristics through retrieval feedback. Con- cretely, we retrieve top k images from an unla- beled image set M via retriever R(.) for each set d E Dcc). The resulting image subset for descrip- tor set is denoted R(d) E 1). where m is the size of M and entry of the vector is 1 if the j-th image of M the retrieved subset. Finally, we assume descriptor consistency C(d, d') holds cosine similarity between R(d) and R(d') is above
t(c) = { {o(d]]d E d(c)), d(c)), if p(c) 2n
{ch, otherwise,
where d is one descriptor in the set d(c) and g(-) transforms d into the form {c) which has (d).
For input image x, calculate the class wise matching score for each ID class name
sx(x) (I(x), T())
E (T((v)
vEt(e)
(E ret(c)
where (-s-) denotes the cosine similarity func- tion, the left term computes the similarity between image visual representations and class name tex- tual representations, and the right term measures

---

the similarity between detected image objects and class names in the text space.
vocabulary of 600 objects. See more details in Appendix B.
Lastly, we
define the maximum class
5.3 Baselines
matching score as: Smax (
maxe Edey (x)() similar to Ming al.
ex)(sx(x))
We compared our method with competitive base- lines: MOS (Huang and Li, 2021) divides ID classes into small groups with similar concepts to improve OOD detection; 2. Fort et al. (Fort et al., 2021b) finetunes a full ViT model pre-trained on the ID dataset; 3. Energy (Liu al., 2020) pro- poses logit-based score to detect OOD samples; 4. MSP (Hendrycks and Gimpel. 2017b) employs the maximum classification probability of samples estimate OOD uncertainty; 5. MCM (Ming et al., 2022a) estimates OOD uncertainty with the maxi- mum similarity between the embeddings of sam- ple and ID class names; 6. Menon et al. (Menon and Vondrick, 2023) prompts LLMs to generate descriptors of each class cues for image classifi- cation. We extend it to OOD detection and use the maximum classification probability as measure of OOD uncertainty (Hendrycks and Gimpel, 201 7b)..).
(2022a). Our OOD detection function can be defined as:
-(x,), Smax
2 Smar (x:
(2)
where I represents ID class and O indicates OOD conventionally. À is chosen threshold.
5 Experiments
5.1 Datasets and Metries
Following recent works (Ming al., 2022a), we use large-scale datasets that are more realistic and complex. We consider the following ID datascts: variants of ImageNet (Deng et al., 2009), CUB-200 (Wah et al., 201 Stanford- Cars (Krause et al., 2013), Food-101 (Bossard et al., 2014), Oxford-Pet (Parkhi et al., 2012). For OOD datasets, we use iNaturalist (Van Horn et al., 2018), SUN (Xiao et al., 2010), Places (Zhou et al., 2017), and Texture (Cimpoi al., 2014).
Datasets
5.4 Main Results
To evaluate the scalability of our method in real- world scenarios, we compare it with recent OOD detection baselines on the ImageNet- k dataset (ID) in Table 2. It can be seen that our method out- performs all competitive zero-shot methods. Com- pared with the best rest-performing zero-shot baseline MCM, it reduces FPR95 by 5.03%. We can also observe that: Indiscriminately employing knowl- edge from L.MSs (i.e., Menon et al.) degenerates the OOD detection performance. This indicates the adverse impact of LLMs' hallucinations and under- lines the importance of selective generation from LLMs. 2. Despite being training-free, our method favorably matches or even outperforms some strong task-specific baselines that require training (e.g., MOS). It shows the advantage of incorporating world knowledge from LLMs for OOD detection.
Metries For evaluation, we use these metries (1) the false positive rate (FPR95) of OOD samples
when the true positive rate of ID samples is at 95%,
(2) the area under the receiver operating character- istic curve (AUROC).
5.2 Implementation Details
In our experiments, we adopt CLIP (Radford et al., 2021) as the pre-trained vision-language model. Specifically, we mainly use CLIP-B/16 (CLIP-B), which consists of ViT-B/16 Transformer as the image encoder and masked self-attention Trans- former ( (Vaswani et al., 2017) as the text encoder. We also use CLIP-L/14 (CLIP-L) as represen- tative of large models. To generate descriptors, we query text-davinci-Of) (Ouyang et al., 2022) with sampling temperature T = 0.7 and maxi- mum token length of 100. We construct the un- labeled image set M through the random selection of = 50000 images from the training set of ImageNet. The retriever R(-) retrieves k 50 im- ages from M. We set the threshold 1 0.9 and 7 = 0.5. In visual object detection, we employ the object detection model CBNerV2-Swin-BBl (Cai et al., 2022) as a general object detector with a
We further evaluate the effectiveness of our method on hard OOD inputs. Specifically, two kinds of hard OOD are considered, i.e., semanti- cally hard OOD (Winkens et al., 2020) and spurious OOD (Ming et al., 2022b). As shown in Table 3, our method exhibits robust OOD detection capa- bility and outperforms all competitive baselines, e.g., improvement of 1.93% in FPR95 compared the pert-peeronmin baseline MCM. We can also observe that zero-shot methods generally obtain higher performance than task-specific baselines.

---

OOD Dataset
Average
Method
iNaturalist
SUN
Places
Texture
FPR95, AUROCT FPR9S, AUROCT FPRO5 AUROCT FPR95, AUROCI
FPR95, AUROCT
spec-fe (training required)
MOS (BIT)
9.28
98.15
40.63
92.01
49.54
89.06
60,43
81.23
39.97
90.11
Fort et (VTVB)
15.07
96.64
$4.12
86.37
57.99
85.24
53.32
84.77
45.12
88.25
(VT--L.
96.51
52.34
Energy (CLIP-B)
21.59
95.99
34.28
93.15
36.64
91.82
51.18
88.09
35.92
92.26
Energy (CIIP-1.)
10.62
97.52
30.46
93.83
32.25
93.01
44.35
89.64
29.42
93,50
MSP (CIIP-B)
40.89
88.63
65.81
81.24
67.90
80.14
64.96
78.16 18.16
$9.89
82.04
MSP (CLIP-L)
34.54
92.62
61.18
83.68
84.10
59.27
$3.71
85.68
Zero-shot
(no training required)
MCM (CLIP-B)
30.91
94.61
37.59
92.57
44.69
89,77
57.77
86.11
42.74
90.77
MCM (CLIP-L)
28.38
94.95
29.00
94.14
35.42
92.00
59,88
84.8%
38.17
91.49
Menon et al. (CLIP-B)
41.23
92.09
44.12
91.72
49.74
88.91
60. 89
85 07
48.99
89.45
Menon et (CLIP-L)
33.26
93.92
30,29
94.18
37.30
91.78
84,40
40.17
91.07
Obj. (CLIP-B)
23.67
95.40
37.19
92.57
43.97
89.77
56.97
86.33
40.45
91.02
w/o Obj. (CLIP-L)
28.20
95.22
27.81
94.44
33.22
91.56
56.37
86.05
36.41
91.82
Calib. (CLIP-B)
48.30
90.53
41.17
92.11
45.08
56,91
87.01
47.87
89.81
Calib. (CLIP-L)
40.41
92.09
29.90
52.93
87.17
39.81
91.09
w/o Know. (CLIP-B)
30.19
94.81
37.39
91.56
43.63
89.76
57.30
86.17
42.13
90.58
w/o Know (CLIP-L)
28.66
94.88
33.25
93.70
40.00
91.31
59.09
85.41
40.26
91.33
Ours (CLIP-B)
22.88
95.54
34,29
92.60
41.63
89.87
52.02
87.71
37.71
91.43
Ours (CLIP-L)
26.47
95.10
26.35
94.56
33.13
91.77
51.77
87.45
34.43
92.22
Table OOD detection performance for ImageNet-IK 1 ID. The performances of all task-specific baselines from Ming et al. (2022a)..
ID
ImageNet-100 ImageNet-20
ImageNet-20 ImageNet-10 10
Waterbirds Spurious OOD
Average
Method
OOD
FPR95 AUROCT FPR951 AUROCT FPR95, AUROCT FPR951 AUROCI
Task-specific (training required)
MOS (BiT)
24.60
95.30
41.80
92.63
78.21
78.60
48.20
88.84
Fort (VIT-B)
8.14
98.07
11.71
98 09.08
4.63
98.57
8.16
98.24
Energy (CLIP-B)
15.23
96.87
15.20
96.90
41.51
89.30
23.98
MSP (CLIP-B)
9.38
98.31
12.51
97.70
39.57
90.99
20.49
95.67
Zero-shot (no training required)
MCM (CLIP-B)
5.00
98.31
12.91
98 09
5.87
98.36
7.93
98.25
Menon et al. (CLIP-B)
5.80
98.65
13.09
98.08
5.57
98.45
8.15
98.39
w/o Obj. (CLIP-B)
4.70
98.71
10.78
98.25
4.88
98.46
6.79
98.47
w/o Calib, (CLIP-B)
6.00
98.50
11.14
98.04
5.17
98.49
7.44
98.34
w/o Know. (CLIP-B)
5.00
98.73
11.38
98.22
4.86
98.39
7.08
98.45
Ours (CLIP-B)
4.20
98.77
9.24
98.26
4.56
98.62
6.00
98.55
Table Performance comparison on hard OOD detection tasks.
This indicates that exposing model to training set may suffer from bias and spurious correlations. We also make comparisons on larger number of ID and OOD datasets in Appendix A.
descriptors from LLMs. Results in Table 2 and Table show that method outperforms all the above variants. Specifically, Incorporating knowledge from LLMs (see w/o Know.) improves the OOD detection performance by 4.42%. This verifies that world knowledge is important multi modill OOD detection. 2. Both uncertainty calibration (see w/o Calib.) and visual object detection (see w/o Obj.) help to improve the OOD detection performance.
observe that:
5.5 Ablation Studies
Model Components Ablation studies are carried out to validate the effectiveness of each main com- ponent in our model. Specifically, the following variants are investigated: 1. w/o Obj. removes the visual object detection step, i.e.. only the left term in Eq. is adopted. 2. w/o Calib. removes the uncertainty calibration step and indiscriminately uses descriptors from LLMs. w/o Know, only uses class names to represent each class without
Uncertainty Calibration To evaluate our pro- posed uncertainty calibration method, we perform ablation on alternatives: Confidence (Si et al., 2022a) leverages language model probabilities

---

Categories
Variants
PPR95,
AUROCT
Set #1
Set
Textual Similarity
Uncertainty
Confidence
40.81
91.16
barber chair
barber ber pole
barber chair
barber pole
1.00
Self-consistency
41.64
90.93
Calibration
Self-evaluation
40.83
90.82
sinks
sinks
Retrieval Feedback
mirrors
mirrors
Visual Object Class Sim.
38.55
91.05
shelves
Derection
Simple Det.
39.40
90.97
products
Ours
37.71
91.43
Set #1
Set #2
Textual Similarity
wet
Table Ablation variants of uncertainty calibration and visual object detection. We use the average performance on four OOD datasets with ImageNet- Ik as ID.
hairless nose
white
0.94
X
white or light
the head and chest
grey head
Retrieval Feedback
furry bodies
furry bodies
0.98
clawed hands
prominent lips
within top-5 retrieved
images for
generated descriptors the confidence score. 2. Self-consistency (Wang et al., 2022) makes multi- ple predictions for one input and makes use of the frequency of the majority prediction as the confi- dence score. 3. Self-evaluation (Kadavath et al., 2022) asks LLMs to first propose answers and then estimate the probability that these answers are cor- rect. Results in Table show that our uncertainty calibration method performs better than other vari- ants. This further indicates that dedicated uncer- tainty calibration approaches should be explored to safely explore generations from LLMs.
Figure 7: Case study on descriptor sets and correspond- ing retrieval feedbacks.
41.0
Random Diversity
40.5
40.0
39.5
FPR95(
39.0
38.0
Visual Object Detection We evaluate the visual object detection module by implementing the fol- lowing variants: 1. Class Sim. uses class name c instead of the descriptive features t(c) in the right term of Eq. 1. 2. Simple Det. adopts simple object detection model with smaller vocabulary of 80 objects (Li al., 2023). As shown in Table 4, our method outperforms the above variants. Specif- ically, we can observe that: 1. Calculating the similarity between detected image concept names and ID class names without descriptors degenerates the OOD detection performance. 2. Using a gen- eral object detection model with large vocabulary of objects helps to improve the performance.
37.5
37.0
100
Propartion of Images (%)
Figure 8: Analysis of unlabeled image
down-sampling from the original unlabeled image set (" ""Rando")), or removing images from certain categories ("Diversity""). We can observe that: 1. Our method achieves superior OOD detection per- formance along with the increase of unlabeled im- age data. 2. Unlabeled image sets that lack diver- sity achieve limited detection performance, espe- cially in small sizes.
5.6 Further Analysis
Conclusion
Cases of Retrieval Feedback We provide a case study where descriptor sets for the same class are similar/dissimilar in textual form. Figure 7 illus- trates that with low textual similarity and vari- ations in textual form, two deseriptor sets can have consistent retrieval feedback if they accurately cap- ture the descriptive features of the same object.
In this paper, we introduce a novel framework for multi-modal out-of-distributtin detection. It em- ploys world knowledge from large language mod- els (LLMs) to characterize ID classes. An uncer- tainty calibration method is introduced to tackle the issue of LLMs' ballucinations, and visual object detection proposed fully capitalize the gen- erated world knowledge. Experiments on variety of OOD detection tasks show the effectiveness of our method, demonstrating its exciting property where world knowledge can be reliably exploited
Analysis of Unlabeled Image Set Figure shows the effect of unlabeled image set with vary- ing sizes on the OOD detection performance. We compose image subsets either through random

---

via LLMs by evaluating their uncertainty,
Likun Cai, Zhi Zhang, Yi Zhu, Li Zhang, Mu Li, and Xiangyang Xue. 2022. Bigdetection: A large-scale benchmark for improved object detector pre-training. In Proceedings the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 4777- 4787.
Limitations
We identify one major limitation of this work is its input modality. Specifically, our method is lim- (OOD)
ited to detecting visual inputs and ignores inputs in other modalities such as textual, audio, electroencephalogram (EEG) and robotic features. These modalities provide valuable information that can be used to construct better OOD detectors. Fortunately, through multi-modal pre-training models (Xu et al., 2021; Huo et al., 2021), we can obtain robust representations in vari- ous modalities.
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2023a. Teaching large language models to self-debug. arXiv preprint 2304, 051000128.
Zhenfang Chen, Qinhong Zhou, Yikang Shen, Yining Hong, Hao Zhang, and Chuang Gan. 2023b. See, think, confirm: Interactive prompting between vision and language models for knowledge-based visual reasoning. arXiv preprint arXiv:2301.055 005226..........
Mircea Cimpoi, Subhransu Maji. lasonas Kokkinos. Sammy Mohamed, and Andrea Vedaldi. 2014. De- scribing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606-3613.
Ethics Statement
This work does not raise any direct ethical issues. In the proposed work, we seek to develop zero- shot multi- OOD detection model equipped with world knowledge from LLMs, and we believe this work can benefit the field of OOD detection, with the potential to benefit other fields requiring trustworthy models. All experiments are conducted on open datasets.
Yi Dai, Hao Lang. Yinhe Zheng, Fei Huang, and Yong- bin Li. 2023a. Long-tailed open world. arXiv preprint arXiv:2305.0660 066576
tailed question answering in an
Yi Dai, Hao Lang, Yinhe Zheng, Bowen Yu, Fei Huang. and Yongbin Li. 2023b. Domain incremental life- long learning in open world. arXiv preprint arXiv:2305.0655.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hier- archical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. leee.
References
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. Concrete problems in ai safety. arXiv preprint arXiv:/606.06565.
Sepideh Esmaeilpour, Bing Liu, Eric Robertson, and Lei Shu. 2022. Zero-shot out-of-distribution detection based on the pre-trained model clip. In Proceedings of the AAAI conference on artificial intelligence, vol- ume 36, pages
Abbijit Bendale and Terrance Boult. 2015. Towards open world recognition. Proceedings of the IEEE conference on computer vision and pattern recogni- rion, pages 1893-1902.
Geli Fei and Bing Liu. 2016. Breaking the closed world assumption in text classification. Proceedings of the 2016 Conference of the North American Chap- of the Association for Computational Linguistics: Human Language Technologies, pages 506-514.
Lukas Bossard, Mathieu Guillaumin, and Luc Van Gool. 2014.
discriminative components with random forests. In Computer Vision-ECCT 2014: 13th European Conference, Zurich, Switzer- land, September 6-12, 2014, Proceedings, Part VI 13, pages 446 461. Springer.
Stanislav Fort, Jie Ren, and Balaji Lakshminarayaman. 2021a. Exploring the limits of tection. In Advances in Neural Information Process- ing Systems, volume 34, pages 7068-7081. Curran Associates, Inc.
de-
Terrance E Boult, Steve Cruz, Akshay Raj Dhamija, Manuel Gunther, James Henrydoss, and Walter J Scheirer. 2019. Learning and the unknown: Survey- ing steps toward open world recognition. In Proceed- ings of the AAAI conference on artificial intelligence, volume 33, pages 9801-9807.
Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. 202 Exploring the limits of out-of-distributtion detection. In Conference on Neural Information Pro- cessing Systems (NeurIPS).
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 1877-1919
Dan Hendrycks and Kevin Gimpel. 2017a. A baseline for detecting misclassified and out-of-distribution ex- amples in neural networks. Proceedings of the Inter- national Conference on Learning Representations.

---

Dan Hendrycks and Kevin Gimpel. 2017b. baseline for detecting misclassified and out diitrii-btiiittion ex- amples in neural networks. In International Confer- ence on Learning Representations (ICLR).
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. Ad- vances in neural information processing systems, 31.
Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. 2020. Generalized odin: Detecting out-of- distribution image without learning from out-of- distribution data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pages 10951-10960.
Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pas- cale N Fung. Mohammad Shoeybi, and Bryan Catan- zaro. 2022. Factuality enhanced language models for open-ended text generation. Advances in Neural Information Processing Systems, 35:34586-3459
Chuyi Li, Lulu Li, Yifei Geng, Hongliang Jiang, Meng Cheng, Bo Zhang, Zaidan Ke, Xiaoming Xu, and Xiangxiang Chu. 2023. Yolov6 v3. 0: scll-scale reloading. arXiv preprint arXiv:2301.058
Rui Huang and Yixuan Li. 2021. Mos: Towards scaling out-of-distribution detection for large semantic space. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8710-8719.
Yujia Li, David Choi, Junyoung Chung. Nate Kushman, Julian Schrittwesser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level code generation with alphacode. Science, 37866624)
Yuqi Huo, Manli Zhang, Guangzhen Liu, Haoyu Lu, Yizhao Gao, Guoxing Yang. Jingyuan Wen, Heng Zhang, Baogui Xu, Weihao Zheng, Zongzheng Xi, Yueqian Yang, Anwen Hu, Jinming Zhao, Ruichen Li, Yida Zhao, Liang Zhang. Yuqing Song, Xin Hong, Wanqing Cui, Dan Yang Hou, Yingyan Li, Junyi Li, Peiyu Liu. Zheng Gong, Chuhao Jin, Yuchong Sun, Shizhe Chen. Zhiwu Lu, Zhicheng Dou. Qin Jin, Yanyan Lan, Wayne Xin Zhao, Ruihua Song, and Ji-Ro Ro-R Wen. 2021. Wenlan: Bridging vision and language by large-scale multi-mode pre-training. CoRR, abs/2103.0656/
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan 2020. Energy-based based distrrbdiitiiittio detection. Proceedings of the Advances in Neural Information Processing Systems.
Sachit Menon and Carl Vondrick. 2023. Visual classifi- cation via description from large language models. In The Eleventh International Conference on Learning Representations.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang. Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12):1 1-38.
Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. 2022a. Delving into out-of- distribution detection with vision-language represen- tations. arXiv preprint arXiv:2211. 221.2245..44
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, al. 2022. Language models (mostly) know what they know. arXiv preprint arXiv.2207.05221.
Yifei Ming. Hang Yin, and Yixuan Li. 2022b. On the impact of spurious correlation for out-of-distribution detection. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pages 10051- 10059.
Jonathan Krause, Michael Stark. Jia Deng, and Li Fei- Fei. 2013. 3d object representations for fine-gri ane-grained categorization. In Proceedings of the IEFE inter- national conference on computer vision workshops, pages 554-561.
Long Ouyang. Jeffrey Wu, Xu Jiang, Diogo Almeida. Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744,
Lorenz Kuhn, Yarin Gal, and Sehastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for un- certainty estimation in natural language generation. arXiv preprint arv: 230::2302.00000 09664.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. 2012. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, pages 3498 1288-505.11 IEEE
Hao Lang, Yinhe Zheng. Yixuan Li, Jian Sun, Fei Huang, and Yongbin Li. 2023. A survey on out- of-distribution detection in nlp. arXiv preprint arXiv:2305.03236.
Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng.
Yujia Xie. Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint a3Xiv:2302.1 12813.
Hao Lang, Yinhe Zheng, Jian Sun. Fei Huang, Luo Si. and Yongbin Li. 2022. Estimating soft labels for oft-of-domain intent detection. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 261-276, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019a. Language models as knowl- edge bases? In Proceedings of the 2019 Conference

---

on Empirical Methods Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNIPP), pages 2473..
species classification and detection dataset. In Pro- ceedings of the IEEÉ conference on computer vision and pattern recognition, pages 8769-8778.
Vladimir Vapnik. 1991. Principles of risk minimization for learning theory. Advances in neural information processing systems, 4.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019b. Language models knowl- edge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463-2473, Hong Kong, China. Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, & ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems, volume 30. Curran Associates, Inc.
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Be- longie. 2011. The caltech-ucsd birds-200-2011 dataset. Technical Report SNNS-TR-201--00 1-001, Cali- fornia Institute of Technology.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- tr, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learn- ing transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8748 8768. PMLR.
Wenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang, Zhiqi Li, Xizhou Zhu, Xiaowei Hu, Tong Lu, Lewei Lu, Hongsheng Li, Xiaogang Wang, and Yu Qiao. 2023. Internimage: Exploring large-scale vision foundation models with deformable convolutions. In Proceedings of the IEEF/CVF Conference on Com- puter Vision and Puttern Recognition (CVPR), pages
Jie Ren, Jiaming Luo. Yao Zhao, Kundan Krishna, Mo- hammad Saleh, Balaji Lakshminarayanan, and Pe- ter J Liu. 2022. Out-of-distribution detection and selective generation for conditional language models. arXiv preprint vrXiv2209..55556
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022. Self-consisteney im- proves chain of thought reasoning in language mod- els. arXiv preprint arXiv:2203. 11171.
Yilin Shen, Yen- Ven-Chang Hsu, Avik Ray, and Hongxia Jin. 2021. Enhancing the generalization for intent classification and out-of-domain detection slu. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2443- 2453.
Jim Winkens, Rudy Bunel, Abhijit Guha Roy, Robert Stanforth, Vivek Natarajan, Joseph R Ledsam, Patri- cia MacWilliams, Pushmeet Kohli, Alan Karthike- salingam, Simon Kohl, et al. 2020. Contrastive train- ing for improved ou disdiitiibttion detection. arXiv preprint arXiv:2007.0520 0055666.
Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. 2010. Sun database: Large-scale scene recognition from abbey to 200. In 2010 IEEE computer society conference on computer vision and pattern recognition, pages -392.. IEEE
Lei Shu, Yassine Benajiba, Saab Mansour, and Yi 7hang. 2021. Odist: Open world classification via distribu- tionally shifted instances. In Findings of the Associ- ation for Computational Linguistics: EMNLP 2021, pages 3751-3756.
Chenglei Si. Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan GradeGra and Li- juan Wang. 2022a. Prompting gpt-3 to be reliable. arXiv preprint 20.00022
Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang. Yijuan Lu, Dinci Florencio, Cha Zhang, Wanxiang Che, Min Zhang, and Lidong Zhou. 2021. Layo Multi modal pre-training for visually-rich document understanding. In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pages Association for Computational Linguisties.
Chenglei Si, Chen Zhao, Sewon Min, and Jordan Boyd- Graber. 2022b. Re-examining calibration: The case of question answering. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2814-2829.
Online.
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. 2022. Out-of- detection with deep nearest neighbors. International Conference on Machine Learning, pages 20827-20840. PMLR.
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. 2021. Generalized out-tt-distiibuttinn detection: A survey. arXiv preprint arXiv:21 10, 11334.
Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Yumao Lu, Zicheng Liu, and Lijuan Wang 2022. An empirical study of gpt-3 for few-shot knowledge- based vqa. In Proceedings of the AAAI Conference
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. 2018. The inaturalist

---

on Artificial Intelligence, volume 36, pages 3081- 3089.
To obtain retrieval feedback for each descrip- tor set, We construct fixed set of unlabeled im- ages, denoted as M, through the random selection of m = 50000 images spanning 1000 categories. These images are extracted from the training set of ImageNet- without corresponding labels. For descriptor set d, the retriever R(()) retrieves top k similar images from M:
Li-Ming Zhan, Haowen Liang, Bo Liu, Lu Fan, Xiao- Ming Wu, and Albert YS Lam. 2021. Out-of-scope intent detection with supersuue and discrimi- native training. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 1ith International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 3521-3532.
R'(d) = argmax E ([[(x),T(d))
MCM.
XEM ded
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Places: A 10 million image database for scene recognition. IEEE transactions pattern analysis and machine intelli- gence, 40(6); 1452-1464.
From the retrieved image subset R'(d) we derive binary vector R(d), with entry j equal 1 if the j-th image of M is in R'(d). In order to de- termine whether two descriptor sets, d and d', are consistent with each other, denoted as C(d, d'), we incorporate the following two constraints:
Wenxuan Zhou, Fangyu Liu, and Muhao Chen. 2021. Contrastive out-of-distne diitt--iiittitttion detection for pretrained transformers. arXiv preprint arXiv:2104.08812
C(d, d') -1 [o((((d) R(d')) 2 n)
Zhuofan Zong. Guanglu Song, and Yu Liu. 2023. Detrs with collaborative hybrid assignments training. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision (ICCV), pages 6748-6758.
1 I[o( Edea dead T(d) Eves T(d') d'
(3)
m/
d
where the first constraint measures the cosine sim- ilarity between R(d) and R(d'), the second con- straint computes the cosine similarity between the averaged textual embeddings of descriptors in d and d'. Note that the second constraint that com- putes textual similarity optional in our method. We evaluate its impact by constructing an ablation variant relying solely on the first constraint. Its average performance ImageNet- 1k ID dataset is 38.59 in FPR95 and 91.37 in AUROC, which outperforms the other shot baselines as well. We set & 50 for image retrieval and 17 = 0.9, n/ = 0.99 for consis- tency computation. We set y = 0.5 as the confi- dence threshold for p(c).
A More Results
We use an extra collection of ID datasets to showcase the versatility of our method: CUB-200 (Wah al., 2011), STANFORD- CARS (Krause et al., 2013), FOOD-101 (Bossard et al., 2014), OXFORD-PET (Parkhi et al., 2012), and three variants of ImageNet con- structed by Ming et al. (2022a), i.e., ImageNet- 10, ImageNet-20, ImageNet-100. The results are shown in Table 5, demonstrating that our method offers superior performance various multi-modal OOD detection tasks without training.
four OOD datasets with
In visual object detection, we use CBNerV2- Swin-Base from Cai et al. (2022) with vocabulary of 600 objects as our general object detector. To construct the ablation variant "Simple Det.", we employ YOL0v6-L6 from Li et al. (2023) with a smaller vocabulary of 80 categories.
B More Implementation Details
In our experiments, we adopt CLIP (Radford et al., 2021) as the pre-trained vision-language model. Specifically, we mainly use CLJP-B/16 (CLIP-B), which consists of a ViT-B/16 Transformer as the image encoder and a masked self-attention Trans- former (Vaswani et al., 2017) as the text encoder. representa- We also use CL14-L/14 (CLIP-L) tive of large models. To obtain world knowledge corresponding to each class, we query texi-davinci- 003 (Ouyang al., 2022) with a sampling tem- perature of 0.7 and a maximum token length of 100.
C Robustness to Sampling Temperature T.
We vary sampling temperature T for LLM gen- eration among (0.3, 0.5, 0.7, 0.9, 1.1). It can be seen Figure 9 that regardless of the temperature. our method consistently outperforms the ablation variant "w/o Know." which does not incorporate additional world knowledge from LLMs. We can

---

ours
43
w/o Know.
FPR951
40
37
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
Temperature
Figure 9: Effect of the sampling temperature T for LLM generation.
also observe that an intermediate temperature of 0.7 can lead to the best performance.
D Reliability under Different LLMs, Image Detectors and OOD Detectors
To further verify the reliability of our method, we perform OOD detection using our method un- der different LLMs (GPT-4, ChatGPT, Claude-1, Claude-2, Bard and text-davinci-003), image detec- (YOLOv6 (Li al., 2023), Internlmage (Wang al., 2023), Bigdetection (Cai al., 2022), Co- DETR (Zong et al., 2023)), and OOD detectors (CLIP-based OOD detector without softmax scal- ing (Ming et al., 2022a)). We use ImageNet- I as ID dataset, and NNaturaissSSUN/Plccss/TTttt OOD datasets. As shown in Table 6, our method is reliable when using different LLMs, image detec- tors and OOD detectors.
as

---

OOD Dataset
Average
ID Dutaset
Method
iNaturalist
SUN
Places
Texture
FPR95,
AUROC
FPR95,
AUROCT
FPR95] AUROCT
FPR95],
AUROCT
FPR95,,
AUROCT
MCM
9.83
98.24
4.93
99.10
6.65
98.57
6.97
98.75
7.09
98,
Menon et al.
8.72
98.36
3.16
99.36
3.89
99.00 0..08
3.58
99.28
4.72
99 .0.
CUB-200
w/o Obj-
7.62
98.51
2.36
99,49
3,24
99.18
3.46
99.24
4.17
99.11
w/o Calib.
8.85
98.09
3.60
99.29
4.64
98.92
3.09
99.38
5.04
98. 92
Know.
Ours
2.33
99.50
3.20
97
7.55
98.45
2.43
99.44
3.88
99.15
MCM
0.05
99.77
0.02
99.95
0.24
99.89
0.02
99.96
0.08
99 89
Menon et
0.07
99.82
99.96
99.90
99.96
99,91
Stanford-Cars
w/o Obj.
0.04
99.77
0.02
99.95
0.24
99.90
0.02
99.96
0.08
99.90
wlo Calib.
0.05
99.81
0.05
99.96
0,30
99.90
0.02
99.96
0.10
99.90 90
w/o Know:
0.06
99.76
99.95
0.26
99.88 89
0,02
99.96
0.09
Ours
0.05
99.75
0.02
99.96
0.24
99.90
0.02
99.96
0.08
99.91
MCM
0.72
99.76
0.90
99.75
1.86
99.58
4. 3.04
98.65 62
1.36
99.45 /33
Menon
99.73
98.28
99.08
Food-101
wlo Obj.
0.72
99.77
1,02
99.74
1,93
99.55
4.17
98.63
1.96
99 .2.42
w/o Calib.
98.87
99.65
3.77
99.22
w/o Know.
0.76
99.76
1.06
99.73
2.12
99.54
4,20
98.59
2.04
99.40
Ours
0.64
99.78
0.86
99.75
1.86
99.57
3.87
98.65
1.81
99.44
MCM
2.85
99.36
1.06
99.73
2.11
99.56
99.81
1.70
99.61 99.61
Menon
8.14
98.69
99 66
3.26
99.37
1,28
99.73
3.52
36
Oxtord-Pet
wlo Obj. 2.81
99.32
1,02
99.72
2.01
99.55
0.83
99.80
1.67
99 .066
Calib.
8.41
99.33
w/o Know:
2.82
99.32
1,00
99.71
2.04
99.99 55
0.87
99 80
1.68
99 .9.9
Ours
99.37
99.74
99.58
99.62
MCM
0.12
99.80
0.29
99.79
0,88
99.62
99.90
0,33
99.78
Menon et
0.13
99.81
0.28
99.82
0.90
99.66
0.04
99.91
0.34
99.80
ImageNet-10
wlo Ohj.
0.10
99.89
0.24
99.89
0.88
72
0.04
99.97
0.31
99.87
w/o Calib.
0.18
99.84
0.29
99.89
1.04
99.96
0.38
99.88 83
w/o Know.
0.13
99.81
0,28
99.79
0.92
99 62
0.04
99.91
0,34
99.78
99.97
MCM
99,50
Menon et al.
1.01
99.60
1.72
99.51
3.71
99.29
3.10
98.85
2.37
99.32
ImageNet-20
Obj.
0.97
99.59
1.72
99.51
3.61
99.29
98.87
99.32
w/o Calib.
0.33
99.78
2.07
99.50
3.37
99.26
2.11
98.93
1.97
99.37
w/o Know.
0.75
99.71
2.08
99.52
3.67
99.20
2,43
98.85 83
2,23
99.32
Ours 0.65
99.63
1,58
99,59
3.19
99.30
2,06
98.94
1.87
99.37
MCM 18.13
96.77
36.45
94,54
34.52
94,36
41.22
92.25
32.58 58
94,48
Menon et al.
22.83
96.24
27.00
95.41
31.24
94.55
42.55
92.08
30.91
94.57
ImageNet-
Obj.
wlo Calib.
21.73
96.38
24.35
95.75
28.07
94.93
39.06
92.70
28.30 30
94.94
Know, 18.48
96.61
31.09
95.29
29.71
39.07
92.83
29.59
94.97
Ours 18.13
96.76
22.02
96.14
26.52
95.21
38.65
93.01
26.33
95.28
Table 5: Zero-shot OOD detection performance based on CLIPP-B166 with various ID datasets.
Method
INaturalist
SUN
Places
Texture
Averaage
FPR95J AUROCI FPR95]
AUROCA FPR95 AUROCT
AUROCO
EPROS AUROC?
MCM
30.91
94.61
37.59
92.57
44.69
89.77
57.77
86.11
42.74
90.77
Opri(GPT-4)
22. 22.56
95.52
33.49
92.81
40.87
90.53
51.89
87.66
37.20
91.63
Ours(ChaGPP)
21.97
95.67
33.68
92.58
41.50
90.14
50.96
88.09
37.03
91.70
Different LLMs
Ours(Claude-1)
24.92
95.63
34.97
92.
42.72
54.13
39.19
91,22
OlasCCaudde22)
25.83
95.08
34.71
92.65
41.65
90.86
53.44
87.02
38.91
91.40
Qurs(Rard)
25.77 74
95.07
92.72
42.12
$3.48
87.02
38.88
Ourstext-davinci-ouss
22.30 88
34.29
92.60
41.63
52,02
87.71
37.71
91,43
Ous((inemtinge))
23.41
95.45
92.63
42.13
$3.65
87.30
38.57
Image
OonSCO-DEER)
22.10
95.69
33.48
92.78
42.92
51.77
87.78
37.57
91.51
Detectors
23.72
95.42
34.29
92.65
43.26
89.84
56.32
86.82
39.40
91.18
Bigdetection)
Different
C1 IP has-hadddiioo softmax)
61.66
89.31
64.39
87.43
63.67
85.95
86.61
71.68
69.08
83.59
OOD Detectors
Ours(w/o soltmax)
60.20
65.13
Table 6: Zero-shot OOD detection performance using different LLMs, image detectors and OOD detectors.


Reviews:
[{"review_id": "p2U1jPRWcv", "reviewer": "Reviewer_QiKW", "summary": "", "questions": "1. Your method highly relies on the LLM and image detector. Although you have designed a consistency-based confidence estimation method, it is well-known that LLM still suffers from hallucination issues, which will significantly affect the soundness of your method.\n2. Can you split your Table 2 into multiple tables or subtables, e.g. one for comparing on CLIP-B, one for comparing on CLIP-L, and one for ablation study? \n3. You have adopted the text-davinci-003 to generate the description. Why don't you try the ChatGPT and GPT-4?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive feedback and useful suggestions! We are glad you find it to be \"well-written\", \"the first work to adopt LLM for multi-modal OOD detection,\" and noted the empirical benefits. We address your thoughts point by point below.\n\n--\"The reliability of LLM and image detector is an issue\"\n\n1. We thought the reliability is an important point to evaluate our proposed method. To measure the reliability, we conduct experiments on multiple datasets with different distributions, and the experimental results are shown in Table 2 (large-scale datasets), Table 3 (semantically and spurious hard OOD datasets), and Table 5 (larger number of ID and OOD datasets). All these experiments demonstrate that our method consistently outperforms the state-of-the-art.\n\n2. To further verify the reliability of our method, we perform OOD detection using our method under different LLMs, image detectors, and OOD detectors.We use ImageNet-1K as ID dataset, and iNaturalist/SUN/Places/Texture as OOD datasets. (Metrics are FPR95/AUROC)  \n    * Our method is reliable under diferent LLMs: We generate descriptions from various LLMs (GPT-4, ChatGPT, Claude-1, Claude-2, Bard and text-davinci-003), and our method consistently outperforms the strongest baseline MCM (Ming et al., 2022).\n    * Our method is reliable under diferent image detectors: We detect objects from images using various image detectors (YOLOv6 [1], InternImage [2], Bigdetection, Co-DETR [3]), and our method still consistently outperforms MCM.\n    * Our method is reliable under different OOD detectors: We also apply our method on CLIP-based OOD detector without softmax scaling (Ming et al., 2022), which can enhance its OOD detection performance using descriptors from LLMs.\n\n[1].Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., ... & Wei, X. (2022). YOLOv6: A single-stage object detection framework for industrial applications. arXiv preprint arXiv:2209.02976.\n\n[2].Wang, W., Dai, J., Chen, Z., Huang, Z., Li, Z., Zhu, X., ... & Qiao, Y. (2023). Internimage: Exploring large-scale vision foundation models with deformable convolutions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 14408-14419).\n\n[3].Zong, Z., Song, G., & Liu, Y. (2022). Detrs with collaborative hybrid assignments training. arXiv preprint arXiv:2211.12860.\n\n|Group| Method | iNaturalist | SUN       | Places    | Texture   | Average   |\n|---| ---------------------------- | ----------- | --------- | --------- | --------- | --------- |\n|| MCM                          | 30.91/94.61       | 37.59/92.57     | 44.69/89.77     | 57.77/86.11     | 42.74/90.77     |\n|**Different LLMs**| Ours(GPT-4)                  | 22.56/95.52       | **33.49**/92.81 | **40.87**/**90.53** | 51.89/87.66     | 37.20/91.63     |\n|| Ours(ChatGPT)                | **21.97**/**95.67**   | 33.68/**92.88**     | 41.50/90.14     | **50.96**/**88.09** | **37.03**/**91.70** |\n|| Ours(Claude-1)                  | 24.92/95.63       | 34.97/92.60 | 42.72/89.72 | 54.13/86.94     | 39.19/91.22     |\n|| Ours(Claude-2)                  | 25.83/95.08       | 34.71/92.65 | 41.65/90.86 | 53.44/87.02     | 38.91/91.40     |\n|| Ours(Bard)                  | 25.74/95.07       | 34.17/92.72 | 42.12/89.85 | 53.48/87.02     | 38.88/91.17     |\n|| Ours(text-davinci-003)|22.88/95.54|34.29/92.60|41.63/89.87|52.02/87.71|37.71/91.43|\n|**Different image detectors**|Ours(InternImage)|23.41/95.45|35.10/92.63|42.13/**89.90**|53.65/87.30|38.57/91.32|\n||Ours(Co-DETR)|**22.10**/**95.69**|**33.48**/**92.78**|42.92/89.80|**51.77**/**87.78**|**37.57**/**91.51**|\n||Ours(YOLOv6)|23.72/95.42|34.29/92.65|43.26/89.84|56.32/86.82|39.40/91.18|\n||Ours(Bigdetection)|22.88/95.54|34.29/92.60|**41.63**/89.87|52.02/87.71|37.71/91.43|\n|**Different OOD detectors**|CLIP-based(w/o softmax)|61.66/89.31|64.39/87.43|63.67/85.95|86.61/71.68|69.08/83.59|\n||Ours(w/o softmax)|**59.87**/**89.65**|**61.79**/**87.90**|**60.20**/**86.27**|**78.67**/**72.84**|**65.13**/**84.17**|\n\n--\"Table 2 has too much information to show the significance of this paper.\"\n\n1. Thank you for this constructive suggestion. We will show main results and ablation studies separately in our revised paper.\n\n--\"Although you have designed a consistency-based confidence estimation method, it is well-known that LLM still suffers from hallucination issues, which will significantly affect the soundness of your method.\"\n\n1. You make a good point that our method would be affected by hallucination issues of LLMs, which is also evidenced in Section 3.2 of our paper. Hence, we propose a consistency-based uncertainty calibration method for selective generation of descriptors (Section 4).\n\n2. To evaluate the effectiveness and reliability of our method, we perform experiments under different distributional datasets, LLMs, image detectors, and OOD detectors (See the first responce). All these results show that our method can outperform the state-of-the-art. \n\n--\"Can you split your Table 2 into multiple tables or subtables, e.g. one for comparing on CLIP-B, one for comparing on CLIP-L, and one for ablation study?\"\n\n1. Thank you for your constructive feedback. We will refine this for better presentation.\n\n--\"You have adopted the text-davinci-003 to generate the description. Why don't you try the ChatGPT and GPT-4?\"\n\n1. Thank you for your valuable suggestion. We evaluate the performance of our method with text-davinci-003, ChatGPT, GPT-4, Claude-1, Claude-2 and Bard to verify the reliability (See the first responce).\n\n--\"The proposed method can outperform but can not significantly and consistently outperform other methods.\"\n\n1. We compare our method with the strongest zero-shot baseline MCM on multiple datasets. It consistently outperforms MCM by a large margin, e.g., it reduces FPR95 by 5.03% in Table 2.\n2. Outperforming all task-specific methods (training required) does pose a challenge. Nonetheless, our method outperforms all task-specific baselines except Energy (37.71 vs. 35.92 in FPR95) in Table 2. Our method outperforms all task-specific baselines in Table 3.\n3. We'd like to kindly point out that reviewer ReNH also endorses our method \"consistently outperforms the state-of-the-art.\"\n\nGiven these encouraging new results, we hope you might be convinced to increase your score. We appreciate your feedback strengthening our paper!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper is well-written and easy to follow.\n2. The method that adopts the knowledge from LLM for downstream tasks is a promising direction.\n3. The evaluation is comprehensive.", "reasons_to_reject": "1. The reliability of LLM and image detector is an issue for the soundness of this paper.\n2. Table 2 has too much information to show the significance of this paper. It will be much better if the ablation study can be shown in a new table.", "questions_to_authors": "1. Your method highly relies on the LLM and image detector. Although you have designed a consistency-based confidence estimation method, it is well-known that LLM still suffers from hallucination issues, which will significantly affect the soundness of your method.\n2. Can you split your Table 2 into multiple tables or subtables, e.g. one for comparing on CLIP-B, one for comparing on CLIP-L, and one for ablation study? \n3. You have adopted the text-davinci-003 to generate the description. Why don't you try the ChatGPT and GPT-4?", "paper_topic_and_main_contributions": "Paper summary\n\nOut-of-distribution is an important topic in reliable AI. Multi-modal OOD, which leverages both text and vision information to detect OOD, has not been fully studied yet. In this paper, the authors adopt the Large Language Model on multi-modal OOD by generating descriptive features for class names. To mitigate the hallucination issue, the authors also adopt a consistency-based confidence estimation method to select high-quality descriptions. A general object detector detects all the candidate objects in the image and detects the OOD by comparing the detected object and generated description from LLM. \n\nTo evaluate the effectiveness of the proposed method, the authors conduct the experiment on several datasets and comparing with 6 baseline methods. \n\nPresentation: Good\nThis paper is well-written and easy to follow.\n\nNovelty: Fair\nThis paper is the first work to adopt LLM to generate descriptions for multi-modal OOD detection. \n\nSoundness: Poor\nThe biggest issue is the reliability of  LLM and image detector, which may affect the soundness of the method.\n\nSignificant: Fair\nThe proposed method can outperform but can not significantly and consistently outperform other methods.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "No", "Typos_Grammar_Style_and_Presentation_Improvements": "No", "Justification_for_Ethical_Concerns": "No"}, "extra_text": {}}, {"review_id": "krSy5I8eQG", "reviewer": "Reviewer_ReNH", "summary": "", "questions": "1. How to select an object detector that has the right label list to achieve good performance in a new domain (not necessarily covered by common COCO 80 classes)\n\n2. what is the inference speed of the entire system?\n\n3. Visual cues of an object type might be ambiguous (there are numerous types of \"birds\"). Using LLM to generate key visual look may lead to ambiguous features?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your attentive comments! We are glad you thought \"the paper addresses a novel and important problem\", and noted the effectiveness of our method. We address your feedback point by point below.\n\n--\"using LLM to detect OOD means the LLM has strong knowledge of the given class name. This may be limiting to common object types that can be understood by LLM.\"\n\n1. LLMs contain broad world knowledge of massive object types, which have been shown to enhance vision classification performance consistently and significantly (Menon et al.,ICLR2023).\n\n2. We also evaluate our OOD detection method on mutiple realistic and complex datasets with different distributions, and the experimental results are shown in Table 2 (large-scale datasets), Table 3 (semantically and spurious hard OOD datasets), and Table 5 (larger number of ID and OOD datasets). All these experiments demonstrate that our method consistently outperforms the state-of-the-art.\n\n3. Meanwhile, we propose a consistency-based uncertainty calibration method in case LLM does not understand the object type, which is a key contribution of our work.\n\n--\"How to select an object detector that has the right label list to achieve good performance in a new domain (not necessarily covered by common COCO 80 classes)\"\n\n1. We use a general object detector with a large vocabulary to detect image objects aiming at fully capitalizing on the world knowledge from LLMs, rather than ensuring all the objects in an image are detected.\n2. We evaluate the reliability of our method with different detectors (YOLOv6 trained with COCO, 80 classes; InternImage trained with OpenImage, 600 classes; Bigdetection, 600 classes; Co-DETR trained with LVIS, 1203 classes). Metrics are FPR95/AUROC. It can be seen that using a general object detection model helps to improve the performance consistently. Even though most objects in samples from Texture are not covered by COCO (only 26 classes are detected by YOLOv6), YOLOv6 still improves the FPR95 score in Texture by 0.65.\n\n|Method|iNaturalist|SUN|Places|Texture|Average|\n|--|--|--|--|--|--|\n|MCM| 30.91/94.61       | 37.59/92.57     | 44.69/89.77     | 57.77/86.11     | 42.74/90.77     |\n|Ours(InternImage)|23.41/95.45|35.10/92.63|42.13/**89.90**|53.65/87.30|38.57/91.32|\n|Ours(Co-DETR)|**22.10**/**95.69**|**33.48**/**92.78**|42.92/89.80|**51.77**/**87.78**|**37.57**/**91.51**|\n|Ours(YOLOv6)|23.72/95.42|34.29/92.65|43.26/89.84|56.32/86.82|39.40/91.18|\n|Ours(Bigdetection)|22.88/95.54|34.29/92.60|**41.63**/89.87|52.02/87.71|37.71/91.43|\n\n3. We will further explore object detectors with open vocabulary, which enables the model to detect unseen object categories and provides more contextual information.\n\n\n--\"what is the inference speed of the entire system?\"\n1. We measured the inference speed with ImageNet-1K as ID dataset, and iNaturalist,SUN,Places,Texture as OOD datasets, which have 85640 samples totally. Our method takes 1194.83s(14ms/sample), and the baseline MCM takes 783.02s(9ms/sample). We will include these details in our revised paper.\n\n--\"Visual cues of an object type might be ambiguous (there are numerous types of \"birds\"). Using LLM to generate key visual look may lead to ambiguous features?\"\n1. You got that right! LLMs are required to generate visual features for **distinguishing** an object category. We also found LLMs may generate unfaithful features due to hallucinations (Figure 2), which can result in collisions between OOD samples and ID classes. In this light, we apply self-consistency to estimate the confidence of LLMs and selectively generate high-quality descriptive features.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper addresses a novel and important problem in multi-modal OOD detection by leveraging world knowledge from LLMs\n2. The proposed uncertainty calibration method effectively tackles the hallucination issue in LLMs, which is crucial for improving OOD detection performance\n3. The use of visual objects extracted from images provides valuable contextual information and enhances the overall approach\n4. The experimental results show consistent improvements over the state-of-the-art, demonstrating the effectiveness of the proposed method", "reasons_to_reject": "1. using LLM to detect OOD means the LLM has strong knowledge of the given class name. This may be limiting to common object types that can be understood by LLM.", "questions_to_authors": "1. How to select an object detector that has the right label list to achieve good performance in a new domain (not necessarily covered by common COCO 80 classes)\n\n2. what is the inference speed of the entire system?\n\n3. Visual cues of an object type might be ambiguous (there are numerous types of \"birds\"). Using LLM to generate key visual look may lead to ambiguous features?", "paper_topic_and_main_contributions": "This paper focuses on improving out-of-distribution (OOD) detection in multi-modal machine learning models by leveraging world knowledge from large language models (LLMs). The authors propose generating descriptive features for in-distribution (ID) class names using LLMs, which provide rich semantic information that can enhance OOD detection performance. However, LLMs can hallucinate and generate unfaithful facts, which can negatively impact OOD detection. To address this issue, the authors introduce a consistency-based uncertainty calibration method to estimate the confidence score of each generation. Additionally, they extract visual objects from images to capitalize on the world knowledge expressed in the generated descriptors.\n\nContribution are:\n1. It applies world knowledge from LLMs to multi-modal OOD detection for the first time by generating descriptive features for ID class names.\n2. The authors analyze LLMs' hallucinations, which can cause damage to OOD detection, and propose a selective generation framework and an uncertainty calibration method in LLMs to tackle this issue.\n3. The paper detects objects in an image and represents them with their predicted class names to further explore world knowledge from LLMs\n4. Extensive experimentation on various datasets demonstrates that the proposed method consistently outperforms the state-of-the-art.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kSYR4WCcdi", "reviewer": "Reviewer_qF8f", "summary": "", "questions": "- The validation of the hypotheses presented by the authors is not sufficiently concrete. In lines 207-210, the authors hypothesize that LLM generates unfaithful descriptors, which may negatively impact OOD detection. To verify this hypothesis they conduct experiments in lines 211-227. However, we can verify the difference when descriptors are used or not . In other words, those experiments do not directly confirm the correlation with faithfulness. Consequently, the relationship between descriptor use and OOD performance can be confirmed, but it is difficult to establish a clear connection with unfaithfulness as a variable. There could be many factors at play besides faithfulness.  For example, when the generated descriptors are too general, we can also explain the steep increase in the OOD input.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed, helpful feedback! We’re glad that you found it \"good performance without additional training\", \"novel and sensible\", and \"diligently compared through experiments\". We address your thoughts point by point below.\n\n--\"The explanation of specific experimental setups is somewhat lacking, often showing tables or figures without a clear description.\"\n   * \"in lines 203-206, the description of the experimental setup is missing, and only performance results are presented. There is no explanation of how the descriptors were extracted, or how they were utilized for classification or OOD detection\": Kindly note that we describe the experimental setup in the caption of Table 1, including ID and OOD datasets, CLIP based feature extractor, and the metrics. We describe the experimental results and conclusions of Table 1 in lines 203-206 without repeating the setup due to space limit. We detail the descriptor generation process in Section 4.2, and we reference this in lines 194-195.\n   * \" in lines 211-217, Fig3, there is no explanation of how the representations for each input were obtained, or how the descriptors were used in this process\": We describe the representation extraction process in the caption of Figure 3, including ID and OOD datasets, CLIP based feature extractor, and t-SNE visualization. We reference Figure 3 in lines 211-217 and describe what we can learn from it.\n   * \"In lines 220-227 (Fig4), it is unclear how the similarity was measured.\": We apologize for any confusions. We describe how we measured the similarity between samples and class names augmented with descriptors in Eq. 1 (Section 4.5). Your feedback matters to us and we will provide elaborated details in our revised paper.\n\n\n--\"Utilizing a third-party model to create descriptors seems to dilute the advantages of the model.\"\n\n1. We apply world knowledge from LLMs to multi-modal OOD detection for the first time.\n2. We analyse LLMs’ hallucinations which can cause damage to OOD detection.\n3. We develop a consistency-based uncertainty calibration method to selectively generate high-quality descriptors from LLMs to tackle the above issue.\n\n--\"The validation of the hypotheses presented by the authors is not sufficiently concrete. In lines 207-210, the authors hypothesize that LLM generates unfaithful descriptors, which may negatively impact OOD detection. To verify this hypothesis they conduct experiments in lines 211-227. However, we can verify the difference when descriptors are used or not . In other words, those experiments do not directly confirm the correlation with faithfulness. Consequently, the relationship between descriptor use and OOD performance can be confirmed, but it is difficult to establish a clear connection with unfaithfulness as a variable. There could be many factors at play besides faithfulness. For example, when the generated descriptors are too general, we can also explain the steep increase in the OOD input.\"\n\n1. We'd like to clarify that faithfulness reflects whether LLM adheres to the source input (Ji et al., ACM Computing Surveys 2023). We prompted the LLMs to generate \"useful visual features for distinguishing a category in a photo\". Hence, faithful descriptors are required to be precise and distinguishable for each class name. Analysis in lines 211-227 demonstrates that:\n   * First, descriptors provide useful features, enhancing the similarity between ID classes and their samples (Figure 4a).\n   * Second, some descriptors are not distinguishable and bring in noise features (Figure 3,4b), leading to collisions between ID and OOD samples.\n\n2. We will evaluate the faithfulness of descriptors by manual annotation, and report the results and their connections with detection performence degeneration in the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The methodology demonstrates good performance without additional training.\n- Particularly the method for calibrating long-form and structured descriptors, which is both novel and sensible.\n- Various methodologies were diligently compared through experiments, and the strengths of each component are well illustrated from ablations.", "reasons_to_reject": "- The explanation of specific experimental setups is somewhat lacking, often showing tables or figures without a clear description. This makes it difficult to evaluate the validity of the experiments conducted by the authors:\n1. Specifically, in lines 203-206, the description of the experimental setup is missing, and only performance results are presented. There is no explanation of how the descriptors were extracted, or how they were utilized for classification or OOD detection.\n2. Similarly, in lines 211-217, Fig3, there is no explanation of how the representations for each input were obtained, or how the descriptors were used in this process.\n3. In lines 220-227 (Fig4), it is unclear how the similarity was measured. According to Fig2, there are various descriptors for a single object, but it is not evident how a single similarity score was derived from these diverse descriptors.\n\n- Utilizing a third-party model to create descriptors seems to dilute the advantages of the model.", "questions_to_authors": "- The validation of the hypotheses presented by the authors is not sufficiently concrete. In lines 207-210, the authors hypothesize that LLM generates unfaithful descriptors, which may negatively impact OOD detection. To verify this hypothesis they conduct experiments in lines 211-227. However, we can verify the difference when descriptors are used or not . In other words, those experiments do not directly confirm the correlation with faithfulness. Consequently, the relationship between descriptor use and OOD performance can be confirmed, but it is difficult to establish a clear connection with unfaithfulness as a variable. There could be many factors at play besides faithfulness.  For example, when the generated descriptors are too general, we can also explain the steep increase in the OOD input.", "paper_topic_and_main_contributions": "This paper is a study on a multimodal out-of-distribution detection task, where it addresses the problem using a combination of text and image modalities with the help of a vision language model. The paper proposes a method to improve OOD detection performance by extracting high-quality descriptors related to each class label and utilizing them as additional information source to capture anomalous que. Specifically, it extracts multiple descriptors for each class label and calibrates them in multiple steps to obtain higher quality and faithful descriptors. Through experiments, the proposed method demonstrates better performance compared to existing approaches and shows that each component of the model contributes to performance improvement.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- The paper is difficult to read due to frequent back-and-forth references to figures and tables in the early parts (pg 1-3)."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper is a study on a multimodal out-of-distribution detection task, where it addresses the problem using a combination of text and image modalities with the help of a vision language model. With the help of large language models, we are able to imagine some of the image descriptions. These descriptions are able to provide more fine-grained information for the CLIP model to discern. The model also requires an object detection module to understand the objects inside the image. 

The experimental results show that the proposed method can achieve better scores for OOD detection for multi-modal aligning. However, as reviewer qf8F pointed out: Utilizing a third-party model to create descriptors seems to dilute the advantages of the model. The proposed method is indeed a pipeline, consisting of several modules together. This can greatly increase the complexity of the problem. Such a drawback makes the method a bit limited in terms of novelty.

Based on the current limitation, we would recommend maybe accepting to Findings.
Decision: Accept-Findings