Exploring Large Language Models for Multi-Modal Out-of-Distribution Detection (2023)

Abstract: Out-of-distribution (OOD) detection is essential for reliable and trustworthy machine learning. Recent multi-modal OOD detection leverages textual information from in-distribution (ID) class names for visual OOD detection, yet it currently neglects the rich contextual information of ID classes. Large language models (LLMs) encode a wealth of world knowledge and can be prompted to generate descriptive features for each class. Indiscriminately using such knowledge causes catastrophic damage to OOD detection due to LLMs’ hallucinations, as is observed by our analysis. In this paper, we propose to apply world knowledge to enhance OOD detection performance through selective generation from LLMs. Specifically, we introduce a consistency-based uncertainty calibration method to estimate the confidence score of each generation. We further extract visual objects from each image to fully capitalize on the aforementioned world knowledge. Extensive experiments demonstrate that our method consistently outperforms the state-of-the-art.

## introduction machine learning models deployed in the wild often encounter out-of-distribution (ood) samples that are not seen in the training phase @xcite . a reliable model should not only obtain high performance on samples from seen distributions, i.e., in-distribution (id) samples, but also accurately detect ood samples for caution @xcite @xcite . most existing ood detection methods are built upon single-modal inputs, e.g., visual inputs @xcite or textual inputs @xcite . recently, @xcite ; @xcite attempt to tackle multi-modal in this paper, we apply world knowledge from large language models (llms) @xcite to multi-modal ood detection by generating descriptive features for class names @xcite . as illustrated in figure 1 , to find a black swan, look for its long neck, webbed feet, and black feathers. these descriptors provide rich additional semantic information for id classes, which can lead to a more robust estimation of ood uncertainty @xcite , i.e., measuring the distance from the visual features of an input to the closest textual features of id classes. however, the knowledge encoding of llms such as gpt-3 @xcite is lossy @xcite and tends to hallucinate @xcite , which can cause damage when applied for ood detection tasks. as shown in figure 2 , llms generate unfaithful descriptors for class "hen", assuming a featherless head appearing in a hen. indiscriminately employing generated descriptive features to model id classes brings noise to the inference process due to llms' hallucinations. moreover, this issue becomes more severe as ood detection deals with samples in an unbounded feature space @xcite . collisions between ood samples and id classes with augmented descriptors would be common. to address the challenge mentioned above, we propose an approach for selective generation of high-quality descriptive features from llms, while abstaining from low-quality unfaithful ones @xcite . recent studies show llms can predict the quality of their outputs, i.e., providing calibrated confidence scores for each prediction that accurately reflects the likelihood of the predicted answer being correct @xcite . unfortunately, descriptors of a class name generated by llms are long-form and structured intermediate results for the ultimate ood detection task, and calibration of llms for generating such long open-ended text @xcite is still in its infancy. we perform uncertainty calibration in llms by exploring a consistency-based approach @xcite . we assume if the same correct prediction is consistent throughout multiple generations, then it could serve as a strong sign that llms are confident about the prediction @xcite . instead of computing literal similarity, we define consistency between multiple outputs from llms for a given input based on whether they can retrieve similar items from a fixed set of unlabeled images. specifically, for each descriptor, we first retrieve a subset of images, leveraging the joint vision-language representations. then, we measure generation consistency by calculating the overlap between these image subsets. to further capitalize on the world knowledge expressed in descriptors from llms, we employ a general object detector to detect all the candidate objects (concepts) in an image @xcite and represent them with their predicted class names @xcite such as "mirror", "chair", and "sink" (see figure 5 ). these visual concepts provide valuable contextual information about an image in the textual space and can potentially match descriptive features of an id class if the image belongs to that class. accordingly, we improve our distance metric of input samples from id classes by considering the similarity between image visual concepts and id class descriptive features in language representations. our key contributions are summarized as follows: our extensive experimentation on various datasets shows that our method consistently outperforms the state-of-the-art. large language models like gpt3 @xcite can serve as a knowledge base and help various tasks @xcite . while some works demonstrate that world knowledge from llms can provide substantial aid to vision tasks @xcite such as vision classification @xcite , its efficacy in multi-modal ood detection is currently underexplored. moreover, as llms tend to hallucinate and generate unfaithful facts @xcite , additional effects are needed to explore llms effectively. uncertainty calibration provides confidence scores for predictions to safely explore llms, helping users decide when to trust llms outputs. recent studies examine calibration of llms in multiple-choice and generation qa tasks @xcite @xcite text @xcite are generated to provide descriptive features for id classes @xcite , and calibration in this task is yet underexplored. ## conclusion in this paper, we introduce a novel framework for multi-modal out-of-distribution detection. it employs world knowledge from large language models (llms) to characterize id classes. an uncertainty calibration method is introduced to tackle the issue of llms' hallucinations, and visual object detection is proposed to fully capitalize on the generated world knowledge. experiments on a variety of ood detection tasks show the effectiveness of our method, demonstrating its exciting property where world knowledge can be reliably exploited via llms by evaluating their uncertainty.