Improving Open-World Classification with Disentangled Foreground and Background Features

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 20:18:20

Abstract: Detecting out-of-distribution (OOD) inputs is a principal task for ensuring the safety of deploying deep-neural-network classifiers in open-world scenarios. OOD samples can be drawn from arbitrary distributions and exhibit deviations from in-distribution (ID) data in various dimensions, such as foreground features (e.g., objects in CIFAR100 images vs. that in CIFAR10 images) and background features (e.g., textural images vs. objects in CIFAR10). Existing methods can confound foreground and background features in training, failing to utilize the background features for OOD detection. This paper considers the importance of feature disentanglement in open-world classification and proposes the simultaneous exploitation of both foreground and background features to support the detection of OOD inputs in open-world classification. To this end, we propose a novel framework that first disentangles foreground and background features from ID training samples via a dense prediction approach, and then learns a new classifier that can evaluate the OOD scores of test images from both foreground and background features. It is a generic framework that allows for a seamless combination with various existing OOD detection methods. Extensive experiments show that our approach 1) can substantially enhance the performance of four different state-of-the-art (SotA) OOD detection methods on multiple widely-used OOD datasets with diverse background features, and 2) achieves new SotA performance on these benchmarks.

Main paper: 
 related work: 
 Post-hoc Approaches. Modelling the uncertainty of pre-trained DNN directly without retraining the network is one popular approach for OOD detection [2, 8, 10, 12, 20, 29, 33, 38, 43, 48, 54] . Hendrycks et al. [16] propose the uncertainty of DNNs and establish a baseline for OOD detection by maximum softmax probability (MSP). ODIN [27] introduces input perturbation and temperature scaling to enhance MSP. Lee et al. [26] propose the deep Mahalanobis distance-based detectors, which compute the distance-based OOD scores from the pre-trained networks' features. Liu et al. [28] calculate the logsunexp on logit as the energy OOD score. ReAct [39] reduces the DNN's overconfidence in OOD samples by activation clipping, which further enhances the energy scores. MaSF [13] considers the empirical distribution of each layer and channel in the CNN and returns a p-value as the OOD score. ViM [42] attempts to utilize not only primitive semantic features but also their residuals to define more effective logit-based OOD scores. Recently, DML [51] reformulate the logit into cosine similarity and logit norm and propose to use flexibility balanced MaxCosine and MaxNorm. This type of approach relies on the foreground semantic features learned by the pre-trained networks, which neglects other relevant features, such as background features.Fine-tuning Approaches. Another dominant approach is to fine-tune the classification networks for adapting to the OOD detection tasks. In this line of research, Hsu et al. [19] further improve ODIN by decomposing confidence scoring. Zaeemzadeh et al. [49] project ID samples into a one-dimensional subspace during training. Some other studies [21, 41] group ID data and assume them as OOD samples for each other to guide the network training. Some studies [3, 37] have noticed the influence of background features on OOD detection, but they focus on solving the confusion between background and semantic features, and ignore the positive influence of background features on OOD detection itself. Outlier Exposure (OE) [17] introduces auxiliary outlier data to train the network and improve its OOD detection performance. Such approaches can use real outliers [4, 6, 28, 31, 34, 45, 47] or synthetic ones from generative models [25] . The performance of this approach often depends on the quality of the outlier data. Fine-tuning the networks may also lead to the loss of semantic information and consequently degraded ID classification accuracy. 
proposed approach: 
 Problem Statement. Given a set of training samples X = {x 𝑖 , y 𝑖 } 𝑁 𝑖=1 drawn from an in-distribution P X with label space Y = {𝑦 𝑗 } 𝐾 𝑗=1 , and let 𝑓 : X → R Y be a classifier trained on the in-distribution samples X, then the goal of OOD detection is to obtain a new decision function 𝑔 to discriminate whether x come from P X or out-of-distribution data P 𝑜𝑢𝑡 :𝑔(x, 𝑓 ) = 1 if x ∈ P 𝑜𝑢𝑡 , 0 if x ∈ P X .The difference between P X and P 𝑜𝑢𝑡 determines the difficulty of detecting the OOD samples. Existing OOD detection approaches focus on the difference between P X and P 𝑜𝑢𝑡 based on the semantic information of the class label space Y, neglecting other relevant dimensions such as the background feature space. This work aim to learn the background features and leverage them to complement these foreground-based OOD detection approaches.Using semantic of foreground objects only to detect OOD samples can often be successful when the OOD samples have some dominant semantics that are different from the ID images. However, this type of approach would fail to work effectively when the OOD samples do not have clear object semantics and/or exhibit some similar semantic appearance to the ID samples, e.g., the images illustrated in Fig. 1 . Motivated by this, we introduce a generic framework DFB, in which the model disentangle the foreground and background features of the in-distribution data and learns ID background features, upon which different existing OOD detection methods can be applied with the learned background representations to detect OOD samples from both of the foreground and background dimensions.A high-level overview of our proposed framework is provided in Fig. 2 .( DFB aims to learn distinct representations of foreground and background information in images, while also considering them as indistribution features. The key challenge here is how to locate these EQUATIONwhere W 𝑦 x is the classification weight of the trained classifier 𝜙 corresponding to the groundtruth class 𝑦 x of x, and 𝜙 cnn (x) (𝑖,𝑗 ) obtains the feature vector at the unit (𝑖, 𝑗) in the feature map extracted by the feature extractor in 𝜙 from image x. M 𝑦 x ∈ R 𝐻 ×𝑊 is an attention map indicating a pixel-wise semantic score of x relative to its groundtruth class 𝑦 x . We then define a foreground decision threshold 𝜃 to generate the fine-grained pseudo labels of background pixels and foreground pixels by:EQUATIONwhere the attention scores are normalized into the range [0, 1] and 𝜃 = 0.5 is used. We then leverage these pseudo labels of the foreground and background pixels, Ŷ, to train a (𝐾 + 1)-class dense prediction network 𝑓 Θ 𝑑 : X → {0, 1} 𝐾 ×𝐻 ×𝑊 via a pixel-level cross entropy loss:EQUATIONwhere 𝑓 (x, Θ 𝑑 ) (𝑖,𝑗 ) outputs a prediction vector consisting of prediction probabilities of the 𝐾 + 1 classes at the image pixel (𝑖, 𝑗), and ŷ(𝑖,𝑗 ) denotes the corresponding pseudo labels at the same pixel. In doing so, 𝑓 Θ 𝑑 learns both in-distribution foreground and background features.The pixel-level foreground and background features learned in the dense prediction network cannot be applied directly to the image classification task. We show below that the (𝐾 + 1)-class dense prediction network can be transformed to a (𝐾 + 1)-class image classification network in a lossless fashion: the dense prediction and the classification networks share the same weight parameters, and the classification network can be applied to image classification without re-training. Particularly, the dense prediction network 𝑓 Θ 𝑑 : X → {0, 1} (𝐾+1) ×𝐻 ×𝑊 can be decomposed into three main modules: 1) a feature extraction network 𝑓 Θ CNN : X → G consisting of a convolutional neural network that extracts the input image x ∈ R 3×𝐻 ×𝑊 into a smaller scale but larger dimensional feature map G ∈ R 𝐶 ×ℎ×𝑤 , 2) an upsampling module up(•) that upsamples the feature map G to original input size 𝐻 ×𝑊 , typically implemented using bilinear interpolation, and 3) a 1x1 convolution classifier 𝑓 Θ cls : G → L that computes the logit for each pixel in the feature map and outputs a logit map L ∈ R (𝐾+1) ×𝐻 ×𝑊 . The size of weights of the convolutional classifier is 𝐶 × (𝐾 + 1). Thus, the dense prediction network 𝑓 Θ 𝑑 can be denoted as:EQUATIONwhereΘ 𝑑 = {Θ CNN , Θ cls }.For a classification network 𝑓 Θ 𝑐 : X → R Y , it can be similarly decomposed into three main modules: 1) a feature extraction network 𝑓 Θ CNN : X → G with the same function as the dense prediction network, 2) a global average pooling GAP(•), which compresses the feature map of 𝐶 × 𝐻 ×𝑊 into a feature vector of size 𝐶 × 1 × 1, integrating the features of the full image, and 3) a linear classifier 𝑓 Θ cls : G → L, which computes the logit of the full image based on the feature vector with 𝐶 × (𝐾 + 1) weights. The classification network 𝑓 Θ 𝑐 can be denoted as:EQUATIONwhere Θ 𝑐 = {Θ CNN , Θ cls }. It is clear from Eqs. ( 4) and ( 5) that the only difference between 𝑓 Θ 𝑑 and 𝑓 Θ 𝑐 is the upsampling module and the GAP module, sharing the same feature extraction network and the classifier. Further, the upsampling and the GAP modules are weight-free operations, which can be easily replaced with each other, as shown in Fig. 3 . In this way, we directly transfer the indistribution foreground and background features learned in the dense prediction network 𝑓 Θ 𝑑 to the image classification network 𝑓 Θ 𝑐 without any loss of the parameters learned in 𝑓 Θ 𝑑 . For a given test image, the classifier 𝑓 Θ 𝑐 yields a (𝐾 + 1)-dimensional logit vector, where the (𝐾 + 1)-th logit is focused on the in-distribution background features and can be used directly to detect OOD samples from the background dimension:EQUATIONwhere L x = 𝑓 (GAP(𝑓 (x, Θ CNN )), Θ cls ) is a (K+1)-dimensional prediction logit vector yielded by 𝑓 Θ 𝑐 .Although the background-based OOD score 𝑆 𝑏 can be used to detect OOD samples directly, it can miss the OOD samples whose detection relies heavily on the foreground features. Thus, we propose to utilize this background OOD score to complement existing SotA foreground-based OOD detectors. Particularly, since the first 𝐾 classification logits in 𝑓 Θ 𝑐 capture similar class semantics as the original 𝐾-class classifier, off-the-shelf post-hoc OOD detection methods that derive an OOD score from these 𝐾 classification logits can be plugged into DFB to obtain an OOD score from the foreground feature aspect. These foreground and background-based OOD scores are synthesized to achieve a joint foreground and background OOD detection.There are generally two types of post-hoc OOD detection approaches, including raw logit-based and softmax probability-based methods. Our background-based OOD score is based on an unbounded logit value, which can dominant the overall OOD score when combining with the foreground-based OOD score using the softmax output (its value is within [0, 1]). To avoid this situation, we take a different approach to combine the foreground and background-based OOD scores, depending on the type of the foregroundbased OOD detector used:EQUATIONwhere 𝑆 (x) is the final OOD score used to perform OOD detection in DFB, 𝑆 ℎ (x) = ℎ(x) denotes the OOD score obtained from using an existing foreground-based OOD scoring function ℎ, and 𝑇 is a temperature coefficient hyperparameter. In Eq. ( 7), to obtain faithful foreground-and-background-combined OOD score, the log function is used to constrain the value and the variance of the background scores 𝑆 𝑏 , while 𝑇 is used to adjust the distribution of the background scores to match that of the foreground scores. 
experiments: 
 Datasets. Following [13, 16, [26] [27] [28] 51] , we choose two widely used classification datasets: CIFAR10 and CIFAR100 [23] , as the indistribution datasets. As OOD samples are unknown during training, their respective training and test data are used as ID data, with samples from a different dataset added into the test set as the OOD data. To evaluate the effectiveness of our approach, four commonlyused OOD datasets consisting of natural image datasets with diverse background features are used, including SVHN [32] , Places365 [53] , Textures [7] , and CIFAR100/CIFAR10 [23] (CIFAR100 is used as OOD data when CIFAR10 is used as ID data, and vise versa [11, 36, 38] ) SVHN is a digit classification dataset cropped from pictures of house number plates, Places365 is a large-scale scene classification dataset,while Textures contains 5,640 texture images in the wild that do not contain specific objects and backgrounds. Images in all these three datasets exhibit largely different foreground and background distributions, so the three datasets contain strong out-of-distribution semantic and background features. On the other hand, both CIFAR10 and CIFAR100 are sampled from Tiny Images [40] and they share similar background features, so when they are used as OOD data for each other, the background features are weak. As a result, this pair of mutual OOD/ID combination is considered as hard OOD detection benchmarks [44] Implementation Details. We use BiT-M [22] , a variant of ResNetv2 architecture [14] , as the default network backbone throughout the experiments. The official release checkpoint of BiT-M-R50x1 trained on ImageNet-21K is used as our initial 𝐾-class in-distribution classification model. The model is further fine-tuned on the in-distribution dataset (CIFAR10/CIFAR100) with 20,000 steps using a batch size of 128. SGD is used as the optimizer with an initial learning rate of 0.003 and a momentum of 0.9. We decay the learning rate by a factor of 10 at 30%, 60%, and 90% of the training steps. All images were resized to 160x160 and randomly cropped to 128x128. The Mixup [50] with 𝛼 = 0.1 is also used to synthesize new image samples during training. We subsequently use CAM (Class Activation Mapping) [52] to generate the pseudo mask labels for each in-distribution image based on a multi-scale masking method used in [1] (see Appendix A. After that, the dense prediction model is converted to (𝐾 + 1)-class image classification model using Eq. ( 5). Four post-hoc OOD detection methods, including MSP [16] , ODIN [27] , Energy [28] , and ViM [42] , are used as the plug-in base models. They are respectively employed to combine with DFB to detect OOD samples in both of foreground and background features. To have fair and straightforward comparison, these four plug-in models are built upon the same 𝐾-class classification model as DFB. The temperature 𝑇 = 2.5 is used in Eq. ( 7) by default.We will release our code upon paper acceptance. Evaluation Metrics. We use three widely-used evaluation metrics for OOD detection, including: 1) FPR95 that evaluates the false positive rate of the OOD samples when the true positive rate of the in-distribution samples is 95%, 2) AUROC denotes the Area Under the Receiver Operating Characteristic curve, and 3) AUPR is the Area under the Precision-Recall curve. The ID images are the positive samples in calculating AUROC and AUPR to measure the OOD detection performance. In addition, we also report the Top-1 accuracy of classifying the in-distribution samples.The OOD detection results of DFB and its competing methods with CIFAR10 and CIFAR100 as in-distribution data are reported in Tabs. 1 and 2, respectively. Overall, DFB substantially improves four different SotA detection methods in all three evaluation metrics on both datasets, and obtains new SotA performance. We discuss the results in detail as follows. Enhancing Different OOD Detection Methods. Four different SotA methods -MSP, ODIN, Energy, and ViM -are used as foreground-based OOD detection baseline models and plugged into DFB to perform joint foreground and background OOD detection. Their results are shown at the bottom of Tabs. 1 and 2.Compared to all the four plug-in base models, DFB can significantly improve the performance of all evaluation metrics in terms Table 1 : OOD detection results with CIFAR10 as in-distribution data. All methods are based on ID training data without using any external outlier data. † indicates that the results are taken from the original paper, and other methods are reproduced using the same network architecture. Four post-hoc foreground OOD detection methods are respectively plugged into our method 'X'-DFB, with improved results highlighted in red and in blue otherwise. The best result per dataset is boldfaced.Average CIFAR100 SVHN Places365 Textures FPR95↓ /AUROC↑ /AUPR↑ MaxLogit [15] [ICML '22] 39.11/85.07/78. 13 of the average results over the four OOD datasets on both of the CIFAR10 and CIFAR100 datasets. In particular, for the averaged improvement across the four base models, DFB boosts the FPR95 by 10.38%, the AUROC by 3.92% and the AUPR by 6.96% AUPR in Tab. 1; and similarly, it boosts the FPR95 by 11.78% , the AU-ROC by 4.8% and the AUPR by 5.41% in Tab. 2. Note that even for the base model ViM, the most recent SotA method, DFB can still considerably enhance its performance, especially on some datasets where ViM does not work well, such as the SVHN, Places365, and Textures datasets, resulting in over 6% reduction in FPR95 and 2.5% increase in both AUROC and AUPR on the CIFAR100 data. DFB shows slight performance drops on some metrics of CIFAR and SVHN OOD datasets, which are mainly caused by suboptimal combinations of the foreground and background OOD scores with the default temperature setting. The explanation would be discussed in detail using Fig. 6 in Sec. 4.2.Comparison to SotA Methods. DFB is also compared with five very recent SotA methods, including MaxLogit [15] , KL-Matching [15] , ReAct [39] , MaSF [13] and DML+ [51] , with their results reported at the top of Tabs. 1 and 2. Among all our four DFB methods and the SotA methods, ViM-DFB is consistently the best performer except the CIFAR10 data in Tab. Background Score AUROC=99.880%In-distribution Out-of-distribution CIFAR10 vs. Textures (Vanilla) CIFAR10 vs. Textures (DFB) is impressive that although the base models MSP, ODIN and Energy that largely underperform the SotA competing methods, DFB can significantly boost their performance and outperform these SotA competing methods on nearly all cases in Tabs. 1 and 2.The Reasons behind the Effectiveness of DFB. We aim to understand the effectiveness of DFB from two perspectives, including the foreground and background OOD scoring, and the latent features learned in DFB, with the results on the Textures dataset reported in Figs. 4 and 5 respectively. We can see in Fig. 4 that the background OOD scores in DFB enable a significantly better ID and OOD separation than the foreground OOD scores, indicating that the ID and OOD samples can be easier to be separated by looking from the background features than the semantic features since there can be more background differences than the foreground ones in each ID/OOD image. From the feature representation perspective, compared to the features learned in the vanilla 𝐾-class classifier in We can observe that the performance of all methods in CIFAR100 vs. CIFAR10 gradually improves as the temperature increases. This is because the increase of 𝑇 narrows down the distribution of background scores, thus making the final OOD scores emphasizing more on the foreground OOD scores, which are more effective in the OOD datasets like CIFAR100 (ID) vs. CIFAR10 (OOD) where the background difference is very small. In contrast, the performance of all methods in CIFAR100 (ID) vs. Places365 (OOD) gradually decreases as the temperature increases. This is because the background distribution difference dominates over the foreground difference in such cases, on which enlarging the distribution of background OOD scores is more effective. 𝑇 = 2.5 is generally a good trade-off of the foreground and background OOD scores, and it is thus used by default in DFB. Note that adjusting 𝑇 generally does not bring the overall performance down below the baseline performance, showing the effectiveness of DFB using different 𝑇 values.In-distribution Classification Accuracy. A potential risk of modifying the primitive classification network for OOD detection is the large degradation of the in-distribution classification accuracy. As shown in Tab. 4, our proposed DFB does not have this issue, as DFB has only 0.12% top-1 accuracy drop on the CIFAR10 dataset and improves the classification performance by 0.23% on the CIFAR100 dataset. This result indicates that the dense prediction training in DFB ensures effective learning of foreground features, while learning the background features. The 0.23% accuracy increase on CIFAR100 also indicates that the dense prediction task can also improve the foreground feature learning for in-distribution classification.Extending to Large-scale Semantic Space. A further challenge for OOD detection is on datasets with a large number of ID classes and high-resolution images, e.g., ImageNet-1k [9] . Fig. 7 presents the detection performance of DFB using ImageNet-1k as in-distribution dataset and on four OOD datasets, including two new high resolution datasets, ImageNet-O [18] and SUN [46] . To examine the impact of the number of classes, we show the results using C ∈ {200, 300, 500, 1000} randomly selected ID classes from ImageNet-1k (𝐶 = 1000 is the full ImageNet-1k data; see Appendix B.2 for more details). The results show that DFB can consistently and significantly outperform its base model Energy with increasing number of ID classes on four diverse OOD datasets, indicating the effectiveness of DFB working in large-scale semantic space. On the other hand, as expected, both Energy and DFB are challlenged by the large semantic space, and thus, their performance decreases with more ID classes. Extending to large-scale semantic space is a general challenge for existing OOD detectors. We leave it for future work. 
conclusions: 
 In this paper, we reveal the importance of disentangling foreground and background features in open-world classification and introduce background features for OOD detection that are neglected in current approaches. We further propose a novel OOD detection framework DFB that utilizes dense prediction networks to segment the foreground and background from in-distribution training data, and jointly learn foreground and background features. It then leverages these background features to define background OOD scores and seamlessly combines them with existing foregroundbased OOD methods to detect OOD samples from both foreground and background aspects. Comprehensive results on popular OOD benchmarks with diverse background features show that DFB can significantly improve the detection performance of four different existing methods. Through this work, we promote the design of OOD detection algorithms to achieve more holistic OOD detection in real-world applications.  
: 
 ) and OOD datasets (CIFAR100 [23] , SVHN [32] , Places365 [53] , Textures [7] ) with their attention maps from a vanilla classifier and our proposed DFB classifier. Vanilla classifiers tend to focus on objects unrelated to the ID class, e.g., the person on a horse (ID class), due to spurious correlation. In OOD data, vanilla classifiers struggle to localize objects within the image and treat the background features as foreground for ID classification. By disentangling foreground and background features, DFB effectively addresses these issue.especially in safety-critical applications like autonomous driving or medical systems [5] . In open-world applications, encountering unexpected OOD inputs is a common occurrence. Consequently, detecting and rejecting these OOD inputs has emerged as a significant challenge in the secure deployment of deep neural networks. OOD detection approaches are designed to address this problem, which aim to detect and reject these OOD samples while guaranteeing the classification of in-distribution data [16] . There are generally two groups of OOD detection approaches. One of them are post-hoc approaches that work with a trained classification network to derive OOD scores without re-training or fine-tuning of the network, e.g., by using maximum softmax probability of the network outputs [16] , maximum logits [15] , or the Mahalanobis distance between the input and the class centroids of ID data [26] . Another group of approaches fine-tunes the classifiers with different methods, such as the use of pseudo OOD samples [4, 17, 28, 31, 34, 45, 47] . Most of these methods, especially the post-hoc methods, are primarily based on the "foreground features" to detect OOD samples. These are the features that exhibit the semantics of the in-distribution classes, such as the appearance features of the 'horse' class images in the CIFAR10 image classification, as shown in Fig. 1 . This introduces two problems: 1) Owing to dataset bias, the foreground features learned by vanilla classifiers may develop spurious correlations [30] with other objects in the image. For instance, a correlation may be incorrectly drawn between horses and riders. These spurious correlations introduce irrelevant semantics, which can lead to the misclassification of not only ID images but also OOD images. 2) Focusing on foreground features overlooks other dimensions that could also be important for OOD detection, as OOD samples can be drawn from arbitrary distributions and can exhibit deviations from in-distribution (ID) data in various dimensions. One such dimension is the set of "background features" that exhibit no class semantics. We observe that the regions of interest for the classifiers on OOD samples often tend to be the background, as shown in the CIFAR100 example in Fig. 1 . This phenomenon indicates that existing classifiers have a foreground-background confusion issue, and they misclassify OOD samples to the ID classes with similar semantic features to these background features.This paper considers the importance of disentangling foreground and background features in OOD detection and proposes to leverage background features to enhance the OOD detection methods that are based on foreground features. To this end, we introduce a novel generic framework, called DFB, that can Disentangle the Foreground and Background features from ID training samples by a dense prediction approach, with which different existing foreground-based OOD detection methods can be seamlessly combined to learn the in-distribution features from both the foreground and background dimensions. Specifically, given a trained 𝐾-class classification network where 𝐾 is the number of in-distribution classes, DFB first generates pseudo semantic segmentation masks by a weakly-supervised segmentation approach that uses image-level labels to locate discriminative regions in the images. These pseudo segmentation masks are then utilized to train a (𝐾 + 1)-class dense prediction network, with the first 𝐾 classes being the original 𝐾 ID classes and the (𝐾 + 1)-th class corresponding to the ID background features. The dense prediction network is further converted into a (𝐾 + 1)-class classification network by adding a global pooling layer. The conversion is lossless and requires no re-training. In doing so, the (𝐾 + 1)-class classifier learns both foreground and background ID features. Different existing foreground-based methods, such as the post-hoc methods, can be applied to the first 𝐾 prediction outputs to obtain semantic OOD scores, while the (𝐾 + 1)-th prediction can be directly used to define background OOD scores. Combining these semantic and background OOD scores enable OOD detection from both foreground and background features.As depicted in Fig. 1 , the proposed DFB effectively disentangles foreground and background features. In the ID data CIFAR10, DFB can more accurately locate the ID objects than the vanilla classifier. In the OOD data CIFAR100, which contains significant foreground objects, DFB successfully disentangle between the foreground and background objects, both of which are important for detecting the OOD. In the other three OOD datasets without prominent foreground objects, the foreground branch of DFB focuses on fewer areas compared to the vanilla classifier while its background branch recognizes most of the background areas, in which the background OOD scores would exert greater influence on the OOD detection. Note that the influence of semantic and background features is determined by the area of these features, so a hyperparameter with a fixed value can well control these two OOD scores across diverse OOD datasets; no careful tuning of this hyperparameter is required per OOD dataset.In summary, we make the following main contributions:• This work studies the importance of disentangling foreground and background features and proposes to synthesize both foreground and background features for more effective OOD detection in diverse real-world applications. This provides a new insight into the OOD detection problem.  
 
 Appendix:  


Decision: The paper investigates Out-of-Distribution (OOD) detection by separating foreground and background features, achieving promising performance. However, a primary concern is that the study mainly focuses on small-scale datasets in the main paper. The authors do provide supplementary results on the ImageNet dataset in the rebuttal. With two reviewers recommending acceptance and one suggesting rejection but not opposing acceptance, the AC recommends accepting the paper. The authors are advised to address the reviewers' comments in their rebuttal.

Reviews: review: paper_summary: paper_summary: This work proposes a new framework that separately extracts foreground and background features for open-world classification.
strength_weakness: strength_weakness: - The idea of the paper is easy to follow. The presentation is good.
- The improved performances are achieved compared to baselines.
limitations: limitations: - Is this work doing OOD detection or open-world recognition? As I learn, open-world learning involves incrementally learning or discovering new classes of data. 
- Are “horse” and “person” unknown classes in the first column of Figure 1? There are not many differences between Vanilla and DFB-FG.
- The details of “Segment and Learn” in Figure 2 are missing.
- The results of more challenging datasets should be provided. The results of baselines are saturated.
paper_summary: paper_summary: This paper ocuses on improving open-world classification by introducing a new framework that disentangles foreground and background features in ID training samples. This work leverages both foreground and background features to enhance the detection of OOD inputs in the open-world classification setting. The method is evaluated on CIFAR10 and CIFAR100 in-distribution datasets.
strength_weakness: strength_weakness: 1. The paper proposes a method that enhances OOD detection by separating foreground features and background features, and provides a analysis of the motivation behind this approach.
2. The experimental results show the effectiveness of the method.
limitations: limitations: 1. The paper constructs an OOD detection method from the perspective of foreground and background features, which is somewhat similar to work on visual interpretability. Can the authors analyze this from the perspectives of saliency analysis and visual attribution methods, and compare with some similar OOD detection methods using the similar methods?
2. The paper only evaluate the method on CIFAR-10 and CIFAR-100 datasets, and do not compare it with more recent works based on features. Additionally, can the method discussed in the paper also perform well in scenarios with higher resolution images, such as ImageNet-1K?
paper_summary: paper_summary: This paper focuses on the problem of open-world classification and propose an new approach named Disentangle the Foreground and Background features (DFB) that is able to achieve out-of-distribution samples detection. The proposed DFB utilizes dense prediction networks to segment the foreground and background from in-distribution training data, and jointly learn foreground and background features. Then, the 
background features is utilized to calculate the background OOD scores and then combined with the OOD scores from foreground features for OOD detection.The experimental results on some OOD benchmarks show the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. The topic of OOD detection is important in community of computer vision and has some relations with the community of multi-media.
2. The idea of improving the accuracy of outlier detection by utilizing the background features generated by a module like CAM has some novelty.
3. The experimental results show the effectiveness of the proposed method in some extent.
4. The paper is generally written well and organized clearly. I do not find obvious typo or grammatical errors.
limitations: limitations: 1. The concept of open-world in the title fails to exactly match the propose method. Indeed, this paper focuses on the problem of out-of-distribution detection or novelty detection. As explained in the following reference paper, the concept of open-world recognition usually includes not only outlier detection, but also incremental learning. However, I do not find any content about incremental learning for update the model in this paper. I suggest the authors change open-world into open-set.
Abhijit Bendale, Terrance Boult. Towards Open World Recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2015.
2. The motivation of utilizing the background features to improve the performance of OOD detection has not been explained clearly. Please explain why the (K+1)-th dimension of the prediction logit vector yielded by $f_{\Theta_c}$ can be directly used as the OOD score. According to the training procedure, this score should reflect the similarity between the test sample and the background of the in-distribution samples. I do not quite understand the relationship between it and the OOD samples.
3. Considering that the capability of semantic segmentation for each in-distribution class is learned in a weakly-supervised manner, why DFB can solve the problem of spurious correlations with other objects caused by dataset bias? If the horse always appears with a person on it the training data, I do not expect the model can distinguish the horse and the person in the image without more supervised information. Please explain it.
4. In the third line of the first paragraph of section 3.1, "this type of approach" should be "approaches of this type".