WeiPer: OOD Detection using Weight Perturbations of Class Projections

Authors: maximilian  granz
manuel  heurich
tim  landgraf
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 17:16:44

Abstract: Recent advances in out-of-distribution (OOD) detection on image data show that pre-trained neural network classifiers can separate in-distribution (ID) from OOD data well, leveraging the class-discriminative ability of the model itself. Methods have been proposed that either use logit information directly or that process the model's penultimate layer activations. With "WeiPer", we introduce perturbations of the class projections in the final fully connected layer which creates a richer representation of the input. We show that this simple trick can improve the OOD detection performance of a variety of methods and additionally propose a distance-based method that leverages the properties of the augmented WeiPer space. We achieve state-of-the-art OOD detection results across multiple benchmarks of the OpenOOD framework, especially pronounced in difficult settings in which OOD samples are positioned close to the training set distribution. We support our findings with theoretical motivations and empirical observations, and run extensive ablations to provide insights into why WeiPer works. Our code is available at: https://github.com/mgranz/weiper.

Main paper: 
 introduction: 
 Out-of-Distribution (OOD) detection has emerged as a pivotal area of machine learning research. It addresses the challenge of recognizing input data that deviates significantly from the distribution seen during training. This capability is critical because machine learning models, particularly deep neural networks, are known to make overconfident and incorrect predictions on such unseen data Hendrycks & Gimpel (2016) . The need for OOD detection is driven by practical considerations. In real-world applications, a model frequently encounters data that is not represented in its training set. For instance, in autonomous driving, a system trained in one geographic location might face drastically different road conditions in another. Without robust OOD detection, these models risk making unsafe decisions Amodei et al. (2016) .Over the last few years, the field has made significant steps towards setting up benchmarks and open baseline implementations. Thanks to the efforts of the OpenOOD team Zhang et al. (2023b) ; Yang We visualize densities of CIFAR10 (ID, blue) and CIFAR100 (OOD, red) as contour plots along the two logit dimensions spanned by w 0 and w 1 , zoomed in on the positive cluster of class zero. The blue axis denotes the vector associated with that class, and one of its perturbations is depicted by the turquoise line. Right: When projecting the data onto both vectors, we obtain the densities shown in the top and bottom panel, respectively. The vertical blue lines mark the 5-percentile (highest 5%) of the true ID data (CIFAR10, blue). At this decision boundary, the classifier would produce false positives in the marked dashed red tail area.A single perturbation of the class-associated vector yields already a reduction of the false positive rate (FPR) from 1.34% to 0.79%. Visually, we confirm that OOD data mostly resides close to 0, extending into the positive cluster in a particular conical shape, which is exploited by the cone of WeiPer vectors. et al. (2022) , we can evaluate new methods across CIFAR10, CIFAR100 and ImageNet, and compare them against a variety of methods, on the same network checkpoints. To this date, however, there is no single method outperforming the competition on all datasets Tajwar et al. (2021) , which indicates a variety of ways in which OOD data differs from the training set. Here, we introduce WeiPer, a method that can be applied to any pretrained model, any training loss used, with no limitation on the data modality to separate ID and OOD datapoints. WeiPer creates a representation of the data by projecting the latent representation of the penultimate layer onto a cone of vectors around the class-projections of the final layer's weight matrix. This allows extracting additional structural information on the training distribution compared to using the class projections alone and specifically exploits the fact that the OOD data often extends into the cluster of positive samples of the respective class in a conical shape (see Figure 1 ). In addition to WeiPer, our KL-divergence-based method WeiPer+KLD represents a novel OOD detection score that is based on the following observation:When ignoring the individual dimensions and examining the activation distribution across all dimensions, we observe that ID samples exhibit a similar "fingerprint" distribution. The more feature dimensions there are, the better our estimate of this source distribution becomes. We demonstrate that measuring the discrepancy between the per-sample distribution and the training set's mean distribution in the augmented WeiPer space leads to improved OOD detection accuracy. We evaluate WeiPer on OpenOOD using our proposed KL-divergence-based scoring function (KLD), MSP Hendrycks & Gimpel (2016) , and ReAct Sun et al. (2021) . Additionally, we conduct an ablation study to understand the influence of each component of WeiPer and analyze WeiPer's performance.Our results confirm that the weight perturbations allow WeiPer to outperform the competition on two out of eight benchmarks, demonstrating consistently better performance on near OOD tasks. WeiPer represents a versatile, off-the-shelf method for state-of-the-art post-hoc OOD detection. However, the performance of WeiPer comes at a cost: The larger the WeiPer space, the more memory is required.In summary, we present the following contributions:• We discover that OOD detection can be improved by considering linear projections of the penultimate layer that correlate with the final output, i.e., the class representations. We construct these projections by perturbing the weights of the final layer.• We uncover a fingerprint-like nature of the ID samples in both the penultimate space and our newly found perturbed space, proposing a novel post-hoc detection method that leverages this structure. The activation distributions of the penultimate space and our WeiPer space over the dimensions of each sample are similar for each ID input, yielding distributions in both spaces that we compare to the mean ID distribution using KL divergence.• We evaluate our findings by testing the proposed methods and two other MSP-based methods on the perturbed class projections using the OpenOOD benchmark, achieving state-of-the-art performance on near OOD tasks.2 Related work OOD detection. Generally, we can distinguish two types of OOD detection methods -one that requires retraining of the model, including novel loss variants, data augmentations, or even outlier exposure settings. Here, we focus on post-hoc methods that can be added with little effort to any existing pipeline. They can be applied to any pretrained model, irrespective of its architecture, loss objective or data modality. Post-hoc methods can be distinguished further in: 1) Confidence-based methods Guo et al. (2017) ; Hendrycks et al. (2022a,b) ; Liu et al. (2023a Liu et al. ( , 2020)) ; Wang et al. (2022) process samples in the model's logit space, i.e. using the network directly to detect ID/OOD data points. A prominent example is the Maximum Softmax Probability (MSP) (Hendrycks & Gimpel, 2016) which simply uses the maximum logit as the main OOD decision metric. Some methods Ahn et al. (2023) ; Djurisic et al. (2022) ; Sun et al. (2021) additionally introduce transformations such as cutoffs of the features in the penultimate layer or masks on the weight matrix Sun & Li (2021) Random projections from the latent space of the neural network have been described in the context of generative modeling Bonneel et al. (2014) ; Jerome H. Friedman & Schroeder (1984) ; Kolouri et al. (2016) ; Liutkus et al. (2019) ; Nguyen et al. (2021) ; Paty & Cuturi (2019) , e.g. to improve the Wasserstein distance calculation or for robustness. A previous work described random projections from the penultimate layer to detect out-of-distribution samples with a normalizing flow Kuan & Mueller (2022) .3 Method 
limitations: 
 Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: In the Limitations section, we address two points of critique that we feel are most relevant: the number of hyperparameters and the memory consumption (also analyzed in Table 5 ). In our methods section, we clearly state our assumptions which are be justified by observations (e.g. in the figures) and by the empirical results. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 5. Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We provide our implementation in the supplementary material. It is integrated in the OpenOOD framework, which makes it easy to setup. All required packages will be installed when setting up OpenOOD. The benchmark datasets are publicly available. When published, we will release a public GitHub repository with our implementation. Guidelines:• The answer NA means that paper does not include experiments requiring code. Justification: We describe the training details of the used baseline models and their checkpoints in the Experiments section. With our method functioning in a post-hoc fashion, these training checkpoints remain unchanged. The optimization details for our postprocessor are also located in section 4. For the specific hyperparameter ranges and found configurations see Table 7 and Table 8 . Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
preliminaries: 
 We consider a pretrained neural network classifier f : X → R C that maps samples x from an input space X ∈ R D to a logit vector f (x) ∈ R C , by applying a linear projection W fc to the feature representation in the penultimate layer(z 1 , ..., z K ) T = z = h(x) = (h 1 (x), ..., h K (x)) T (1)such that f (x) = W T fc z, with D, K and C representing the dimensionality of the input, the penultimate layer and the output layer, respectively. We define the rows of the final weight layer to be w 1 , ..., w C .In the following it is useful to introduce Z as random vector from which we draw our latent samples. We denote the densities of the latent activations of the training data with p Ztrain , of the test data with p Ztest and those of OOD samples with p Zood admitted by the random vectors Z train , Z test and Z ood , respectively. To ease notation, we will treat Z and all its subsets, both as sets, e.g. Z train is the set of all training activations in the penultimate layer.An OOD detector is a binary classifier O that decides if samples are drawn from an ID or OOD distribution by usually only considering samples drawn from p Ztest or p Zood . Commonly, this is achieved by thresholding a scalar score function S.O(x) = ID if S(x) > λ OOD otherwise (2)For MSP, the score function is simply the maximum softmax probabilityEQUATIONNote, that for clarity, we define MSP also as a function of the logits. Other methods propose metrics on the penultimate layer, e.g. by incorporating distance measures between a given latent activation z of a new sample and the distribution of activations p Ztrain of the training set.A neural network classifier maps the data distribution to the distribution of the logits W fc Z. The training objective of the network ensures an optimal separation of classes and lets the model learn to exploit features in Z specific to the training distribution. OOD samples, hence, often yield lower logit scores. Confidence methods leverage this property, but could potentially be improved by capturing more of the underlying distribution of the penultimate layer. A confidence score measures properties of the logit distribution W fc Z. Is there additional information in the penultimate layer of the network, and if so, how can we utilize it?Applying the weight matrix W fc to the penultimate space can be understood as C projections of Z onto the row vectors w. According the Cramer-Wold theorem (Cramér & Wold (1936) ), we can reconstruct the source density p Z from all one-dimensional linear projections, and Cuesta-Albertos et al. (2007) has shown that a K-dimensional subset of projections suffices (for more details see Appendix A.1.1). The question remains which projections extract the most relevant information?Drawing vectors w ∈ W = N (0, I) from a standard normal and projecting onto them often results in similar densities for ID and OOD data, i.e. w T Z train ≈ w T Z ood , deteriorating detection performance (see Table 1 , RP). This aligns with Papyan et al. (2020) , suggesting limited information in the penultimate layer compared to the logits. We hypothesize that the latent distribution shows relevant structure only along certain dimensions. We applied PCA to the latent activations Z and inspected the resulting projections. This analysis supports the notion that the informative dimensions lie in the directions of the class projections w 1 , ..., w C (see Appendix A.1.3). Hence, we construct projections that correlate with these vectors but at the same time deviate enough to obtain new information.W fc W (δ, r) =      W 1 W 2 . . . W r      δ KLD MSP rC(K) samples E z∈Z (q z ) p W z DKL ←→ T ks (p W z ) T ks MSP WeiPer (x) := 1 r r i=1 MSP W i z W z z p z T ks (p z ) T ksFigure 2 : WeiPer perturbs the weight vectors of W fc by an angle controlled by δ. For each weight, we construct r perturbations resulting in r weight matrices W 1 , ..., W r . KLD: For WeiPer+KLD, we treat z 1 , ..., z k ∼ p z and w T 1,1 z, ..., w T r,C z ∼ p W z as samples of the same distribution induced by z and W z, respectively. We approximate the densities with histograms and smooth the result with uniform kernel T ks . Afterwards, we compare the densities T ks (q z ) with the mean distribution over the training samples E z∈Ztrain (q z ) for q z = p z and q z = p W z , respectively. MSP: For a score function S on the logit space R C , we define the perturbed score S WeiPer as the mean over all the perturbed logit spaces W z. We choose S = MSP and call the resulting detector MSP WeiPer .We define perturbations η, drawn from a standard normal and add them to W fc . To ensure that all perturbed vectors have the same angular deviation from the original weight vector, we normalize the perturbations to be the same length as their corresponding row vector and multiply them by a factor δ:EQUATIONfor i = 1, ..., r, where δ represents the length ratio between w j and the perturbation η i . For large K = dim(Z), w j and ηi are almost orthogonal and thus δ actually adjusts the angle α ≈ arctan(δ) of the perturbed vector bundle. We set δ to be constant across all j = 1, ..., K and treat both δ and r as hyperparameters. This proceedure is related to the Distributional Sliced Wasserstein distance Nguyen et al. (2021) as they sample projections from a distribution such that the mean angle between the projections is greater than arccos(C) for a constant C.The whole set of vectors we define isEQUATIONWe can think of the resulting weight matrix W as r repetitions of the weight matrix W fc on which we add perturbation matrices Hi . The j-th row Hi,j corresponds to a perturbation vector ηj , normalized to match the respective row w j .EQUATIONSince W i Z = W fc Z + H i Z, we call W Z the perturbed logit space. Our weight perturbations method, we call WeiPer, essentially increases the output dimension of a model. Hence, it can be combined with many scoring functions. We demonstrate this with the two following postprocessors.If the perturbations do not deviate too much from the class projections w j , i.e. the row vectors of the final layer, the class cluster will still be separated from the other classes in the new projections and we can apply MSP on the perturbed logit space. In fact, we find that class clusters on the perturbed projections can be better distinguished from the OOD cluster than on the original class projection Figure 3 : Histogram of all 512 activations in the penultimate layer (left pair) and the activations in WeiPer space (right pair) of a ResNet18 trained on CIFAR10. We perturb the weight matrix 100 times to produce a 10 • 100 = 1000-dimensional perturbed logit space. For each pair, the left panel shows the mean distribution over all samples (ID = CIFAR10, OOD = CIFAR100). The right panels show the distribution p z and p W z , respectively, for two randomly chosen samples with smoothing applied (s1 = s 2 = 2)defined by W fc (see improvements of WeiPer+MSP(x) over MSP in Table 2 ). Figure 1 illustrates a visual example. We calculate the MSP on the perturbed logit space asEQUATIONthe mean over all the maximum softmax predictions of the perturbed logits. We analyze why MSP WeiPer could be capable of capturing more of the penultimate layer distribution than MSP in Appendix A.1.2.Following our line of argument motivated by Theorem A.1, it seems natural to choose a density-based score function. When pooling all activations of the penultimate layer, an ID sample's activation distribution exhibits remarkable differences to that of an OOD sample. We observe the following properties:• The majority of samples exhibit a bimodal distribution of their penultimate activations. An activation either belongs to the mode close to zero, or to the second mode (and rarely takes values in between). • ID samples share a similar activation distribution. The mean activation distribution can serve as a prototype -see the upper left panel in Figure 3 . • The activation distribution is specific to the ID samples, i.e. the activation distribution of OOD samples differs from its distribution of ID samples and thus from the ID prototype.Concluding on all three points, we make the assumption that all features z = h(x) of an ID input x can be thought of as samplesEQUATIONof the same underlying activation distribution p z . Furthermore, the density of p z matches the mean distribution over all ID samplesp z ≈ E z ′ ∈Ztrain [p z ′ ].(9) We assume, the same is true for the logits. They naturally separate into a non-class cluster and a class cluster with the ratio 1 : C -1. Here, we could apply the same procedure, but especially for datasets with a small number of classes we would only get C samples. This is where the cone of WeiPer vectors creates an advantage: They sit at a fixed angle to a class projection and thus preserve the class structure similarly across each projection onto one vector of the cone (e.g., like Figure 1 right -bottom panel). Analogous to Equation (9), we treat each projectionw T 1,1 z, ..., w T r,C z ∼ p W z (10)as a sample of the same underlying distribution and observe thatEQUATIONWe demonstrate both behaviors in Figure 3 .In by convolving p with a uniform kernel k s of size s and prevent densities from being zero by adding ε > 0 which we set to the fixed values ε := 0.01 for the penultimate layer and ε := 0.025 for the WeiPer space. Note, that tuning both epsilons might increase performance as we observed in early stages of our experiments, but will add two additional hyperparameters. We normalize the density to sum up to one again, here defined by normalize. Afterwards, we compare each of the densities with the KL divergence, respectively:EQUATIONwhere q z is either p z or p W z . We discuss why our method does not suffer from the curse-ofdimensionality in contrast to other methods as investigated by Ghosal et al. (2023) in Appendix A.1.5WeiPer+KLD combines the KL divergence on the penultimate space, the KL divergence and MSP on the perturbed logit space into one final score:EQUATIONThe full list of hyperparameters is r and δ for the WeiPer application and n bins , λ 1 , λ 2 , s 1 , s 2 for the KL divergence score function. Figure 2 provides a visual explanation and a quick overview of WeiPer and both its postprocessors. 
experiments: 
 Setup. We evaluate WeiPer using the OpenOOD Zhang et al. (2023b) framework that includes three vision benchmarks: CIFAR10 Krizhevsky ( 2009), CIFAR100 Krizhevsky (2009), and ImageNet Deng et al. (2009) . Each of them contains a respective ID dataset D in and several OOD datasets, subdivided into near datasets D near and far datasets D far (see Table 1 ). The terms near and far indicate their similarity to D in and, therefore, the difficulty of separating their samples.OpenOOD also provides three model checkpoints trained on each CIFAR dataset whereas for ImageNet the methods are evaluated on a single official torchvision Marcel & Rodriguez (2010) checkpoint of ResNet50 He et al. (2016) and ViT-B/16 Dosovitskiy et al. (2020) respectively. We report our scores together with the results of Zhang et al. (2023b) in Table 2 .Due to resource constraints, we only evaluate our methods on the models trained with the standard preprocessor, that includes random cropping, horizontal flipping and normalizing, on the cross entropy objective. Additionally to the KL divergence score function and MSP, we evaluate WeiPer on ReAct. But instead of combining ReAct with the energy-based score function Liu et al. (2020) as in OpenOOD, we apply MSP WeiPer and call it WeiPer+ReAct. The hyperparameters of our methods were tuned by finding the best combination over a predefined and discrete range of values on the OpenOOD validation sets to assure a fair comparison to the competition (see Table 8 ). Results. Table 2 reports the performance of WeiPer in comparison to the state-of-the-art OOD detectors on each benchmark. We compare our approach based on the D near and D far detection performances and report the mean over all datasets in each category. Table 3 portrays the mean relative performance on D near and D far of every postprocessor. The score is calculated as follows:S rel (P ) := 1 3 A CIFAR10 (P ) + A CIFAR100 (P ) + 1 2 (A ImageNet(ResNet50) (P ) + A ImageNet(ViT)) (P )) (15)where A D (P ) := AUROC Dnear / far (P ) max P ∈P AUROC Dnear / far (P )It is designed such that each result on each dataset D is equally weighted and scoring 1.0 means that the postprocessor P is top performing across all datasets.WeiPer+KLD achieves three out of eight top AUROC scores and the best performance on all near benchmarks, establishing a new state of the art performance by a significant margin (see Table 3 ).Especially for the most challenging benchmark, separating D near on ImageNet with a ResNet50, we outperform our strongest competitor, ASH Djurisic et al. (2022) , by 1.88% AUROC (we even achieve an AUROC score of 80.29 when using a 1M training samples instead of 300k, see Table 6 ).Additionally, WeiPer+KLD performs well on many far benchmarks, being the best method for ResNet50 on ImageNet, reaching into the top three positions on CIFAR10 far and into the top three on the CIFAR100 far benchmark. With its relative performance in than classes and therefore compresses the class clusters. Some dimensions may thus compress two classes while others represent a feature specific to only one class. This introduces more noise into p z which could impair the detection performance. Future experiments will reveal whether WeiPer benefits from higher dimensionalities of the latent space.WeiPer on existing methods. Additionally, WeiPer enhances the MSP performance by 1-4.1% AUROC across all benchmarks and WeiPer+ReAct consistently outperforms ReAct with an energybased score, although in their evaluation, this variant was better than ReAct+MSP (see Table 3 ).Ablation study. We determine the effect of each hyperparameter in Figure 4 by freezing single hyperparameters and optimizing only the one in question. As expected, increasing the number of random perturbations r leads to a better median performance, while the standard deviation decreases for larger r. Note, that it is possible to have better performance for lower r by rerolling the weights a few times and choosing the best performing ones. All methods show a significant performance boost compared to using no perturbations δ = 0 and seem to be best at δ = 2 for WeiPer+KLD, which corresponds to an angle of α ≈ 63 • and δ = 4 (α ≈ 76 • ) for MSP and ReAct.On CIFAR10, WeiPer+KLD only improves marginally by applying MSP WeiPer , which is not the case for the other benchmarks (see Table 7 ), where λ 2 > 0. Furthermore, we study the performance of random projections that are independent from the weights W fc . We show that using only random projections (RP, see Table 2 ) without adding MSP WeiPer , we are hardly able to detect any OOD samples. This supports the claim that utilizing the class directions is necessary. The supplementary material presents all the other KLD-specific hyperparameters and we also investigate their influences to the performance in Figure 6 . We outline the selected parameters for each benchmark in Table 7 . The left pair shows results on CIFAR10, the right pair corresponds to ImageNet (using ResNet18 for both). Models were tested using their respective near OOD datasets. The panels corresponding to δ depict AUROC performance minus the initial AUROC performance at δ = 0. The graphs show the mean over 25 runs and the shaded area around them represents the value range (min to max) over those runs. All other parameters of the methods were fixed to the optimal setting. 
limitations: 
 WeiPer+KLD has more hyperparameters than other competitors: 6 in total. As discussed in the previous section, r can be seen as a memory / performance trade-off (see Figure 4 ). In the supplementary material (see Figure 6 ) we investigate the other parameters and find that they all have only one local maximum in the range we were searching and should thus be easy to optimize. We tried to choose the same smoothing size s 1 = s 2 for both densities, but the ablations show that both are optimal at different sizes. While MSP WeiPer is not really used for CIFAR10 (λ 2 ≈ 0) it is beneficial for CIFAR100 and ImageNet. As WeiPer blows up the dimension we also conduct a memory and time comparison to other methods in Table 4 and Table 5 . We demonstrate that with a combination of a confidence and a distance based metric it is possible to achieve competitive near results across the board where all other methods seem to deteriorate in at least one benchmark. 
conclusion: 
 We show that multiple random perturbations of the class projections in the final layer of the network can provide additional information that we can exploit for detecting out-of-distribution samples.WeiPer creates a representation of the data by projecting the latent activation of a sample onto vector bundles around the class-specific weight vectors of the final layer. We then employ a new approach to construct a score allowing the subsequent separation of ID and OOD data. It relies on the fingerprintlike nature of features of the penultimate and the WeiPer-representations by assuming they were sampled by the same underlying distribution. In a thorough evaluation, we first show that WeiPer enhances MSP and ReAct+MSP performance significantly and show that WeiPer+KLD achieves top scores in most benchmarks, representing the new state-of-the-art solution in post-hoc OOD methods on near benchmarks. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: Due to WeiPers random nature our results exhibit noise dependent on the hyperparameter r and we display the minimum to maximum range in Figure 4 and Figure 6 . We chose minimum to maximum results instead of standard deviation as the maximal performing perturbations could be strategically sampled. Guidelines:• The answer NA means that the paper does not include experiments. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: Our work builds on data, the evaluation framework OpenOOD and previous OOD detectors that are introduced and cited in the Experiments section and Related Work section, respectively. Our contribution is licensed by CC BY 4.0. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: The implementation of our contribution is well annotated and we provide optimization details as well as a thorough description of our method, as it is the key contribution of our work. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: A ∞ k=1 (m k ) -1/k = ∞. If the set W XY = {w ∈ R K : w T X d = w T Y } has positive Lebesgue measure, then X d = Y .We provide a simple proof for the case that W XY = R K . For the complete proof, we refer to Cuesta-Albertos et al. (2007) .Proof. The characteristic function of a random vector X is defined asEQUATIONBy the uniqueness theorem, every random vector X has a unique characteristic funtion ϕ X . If the assumption in the theorem holds, then for all realizations x = X(ω) and y = Y (ω), we havew T x = w T y (18)and thus ϕ X = ϕ Y . Therefore we have X = Y by the uniqueness.Note, that is enough cover all the directions in R K instead of covering the whole space with the set of projectionsW . Since if tw ̸ ∈ W for t ∈ R, but w ∈ W ∩ W XY then tw ∈ W XY .For δ > 0 our set of perturbed class projections indeed covers the all directions if r → ∞.To generally apply the theorem, Z must be defined on a bounded set with finite measure (Hausdorff moment problem), which is true for virtually all practical problems. More importantly, W needs to be a K-dimensional subset of R K . Note that the theorem also applies for a set of weight matrices{W ∈ R K×K : W X d = W Y } (19)when their row vectors form a K dimensional set as their marginal distributions w T i X d = w T i Y would be equal for i = 1, ..., K. We are using this theorem solely as motivation since it is not possible to draw direct implications. However, with a score function S we are measuring properties of the logit distribution of the training data W fc Z train = (w T 1 Z train , ..., w T C Z train ) and check if they match the properties of some unknown logit distribution W fc Z that might be test data or OOD data. In the ideal case the logits match in distributionEQUATIONe.g. if D is a distance and S = -D, S(W fc Z test ) = 0 and S(W fc Z ood ) < 0. Now the theorem says that if we chose a K-dimensional set W of projections and we had a score function S WeiPer that fulfills the property of Equation ( 20) on the infinite dimensional space spanned by the projections of W , not just the distributions on the projection would be equal when S WeiPer is minimized but also the penultimate distributions Z train We argue that instead of taking random projections, we can utilize the class projections. Since we have a trained classifier at hand, it is likely that the informative dimensions are: span(w 1 , ..., w C ), the span of the row vectors of W fc . Hence, a better choice than Gaussian vectors for the set of projections W are vectors w that correlate with these basis vectors in W fc but at the same time deviate enough such that we obtain new information from projections onto them.We gave a brief description of our construction of the densities in the main paper. The formal definition is:EQUATIONHere l b is the bin length, b(t) is the bin range in which t falls, and [.] is the Iverson bracket which evaluates to one if true or zero if the statement is false. Note that we are dividing by l b such that the density integrates to one. This is the usual definition for descretizing a density into equal sized bins.In contrast to other distance-based methods, our KL divergence score does not suffer from the curse-of-dimensionality, which deteriorates the performance of methods like KNN Sun et al. (2022) as investigated by Ghosal et al. (2023) . We disregard the dimension and only consider the activations in the penultimate space or in the perturbed logit space. In our case, more dimensions means more samples to approximate the activation distribution p z . We believe that our method thrives when applied to networks with higher dimensional penultimate space, but this still needs to be evaluated in future experiments. However, in the perturbed logit space, we can control the size of the space with r (see Figure 4 ). Our ablation results show that increasing r and thus blowing up the dimensionality only increases performance.Table 4 : Time taken in seconds to setup the method or for inference. More precisely, we measure the time of postprocessor.setup() and postprocessor.inference() in OpenOODs evaluator.py . We take the mean over 20 iterations and denote the standard deviation after the ± sign. We mark the maximal time in bold and underline the second longest time taken. We compare WeiPer+KLD (r = 100 as in the paper) to its closest competitors and show that it is competitive with other methods in terms of computation time. While WeiPer+KLD is on the higher end of the spectrum in terms of computation time, it remains comparable to other methods. It's worth noting that computation time can be adjusted by trading off performance with lower r values. OpenOODs evaluator.py before and after its execution with psutil . We take the mean over 20 iterations of the data between the 20% and 80% quantile to diminish the effect of outliers (e.g., caused by interfering processes) and denote the standard deviation after the ± sign. We mark the maximal memory in bold and underline the second highest demand. WeiPer+KLDs (r = 100) memory consumption for its setup is among the lower demanding methods while it has a comparably high demand for inference. Note that for optimal results we choose r = 100, but smaller values of r also provide competitive results (see Figure 4 ). Thus memory can be traded against performance where resources are constraint. We fixed the optimal hyperparameters and varied the one parameter in question by conducting 10 runs over the same fixed parameter setting on CIFAR10 and ImageNet as ID against their near OOD datasets. We report the mean and the minimum to maximum range (transparent). We set r = 5 instead of r = 100 for ImageNet to save resources. Thus the noise on the results is stronger for the ImageNet ablations. All of the parameters except the kernel sizes only have a single local maximum which indicates that they should be easy to optimize. The most important parameters seem to be the kernel sizes s 1 and s 2 that we use for smoothing followed by n bins . Note that s 1 and s 2 have a different optimum, which means it is not possible to simply choose s 1 = s 2 and reduce the count of hyperparameters. λ 1 = 0 is the score function without the KLD specific WeiPer application. λ 2 is the application of MSP WeiPer which is not beneficial for CIFAR10, but shows to be effective on ImageNet.Table 7 : The hyperparameter sets for each experiment. The number of repeats r was predefined since we found increasing it always boosts the performance at the cost of time and memory consumption. Figure 7 : Density plots for CIFAR10 and CIFAR100 of a ResNet18 trained for 300 epochs. We present the densities as in Figure 3 , but this time we show it for more datasets and for two different samples z 1 , z 2 in the penultimate and the perturbed logit space, respectively. The OOD set for the ID set CIFAR10 is CIFAR100 and vice versa for CIFAR100. The range of the y-axis is adjusted such that the differences between ID and OOD become visible. We therefore report the mean over the maximum density max p of the penultimate dimensions to show, up to which value the maximum would go. We apply smoothing over k s neighbors in each plot and construct the histograms with n bins bins. We report the parameters in Table 9 . The class clusters and the activation clusters are clearly visible for CIFAR10 and merge into the bigger cluster for CIFAR100, probably because of the lower class to non-class ratio. It is harder to see for CIFAR100, but for both datasets, it seems harder for the OOD data to sample in the class cluster or the activated feature cluster.Hyperparameter CIFAR10 CIFAR100 ImageNet-1K ImageNet-1K (ResNet18) (ResNet18) (ResNet50) (ViT-B/ViT-B/16 ). We chose SSB-hard as OOD set and apply the same plotting procedure as defined for CIFAR10/CIFAR100. For the respective plotting parameters, see All experiments are conducted on a local machine with the following key specifications: AMD EPYC 7543 (32-Core Processor) with 256GB RAM and 1x NVIDIA RTX A5000 (24GB VRAM). To streamline the experimental process, we pre-compute the penultimate output of each backbone model and dataset combination. This is possible as we do not alter the training objective for our evaluation, achieving a reduction of disk usage to <2.8GB when stored as FP16 tensors compared to sum of the original dataset sizes of CIFAR10, CIFAR100 and ImageNet-1K.For all benchmarks, the 24GB VRAM suffices for both, the model inference and postprocessor optimization. Depending on the chosen batch size, this offers a runtime / VRAM trade-off and is therefore well achievable on smaller GPUs.Excluding the inference step for the penultimate output, we report an inference time for the postprocessor optimization of 18 seconds for CIFAR10 and <10 minutes for ImageNet-1K per iteration. An iteration refers to processing the full dataset starting from the penultimate layer output. For the full duration without pre-processing, one would add the inference time of the respective ResNet[18/50] or ViT/16-B model.1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The abstract introduces the key contribution WeiPer, the KL-Divergence-based scoring function and the setting in which it outperforms its competition. Furthermore, our introduction briefly presents our methods mechanism and lists our contributions and mentions its limitations. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 


Decision: The paper introduces a novel approach to enhance out-of-distribution detection by perturbing the weight vectors in class projections, expanding the output dimension, and enabling richer representation extraction from neural networks. The authors have presented a significant amount of empirical evidence demonstrating improved performance across challenging benchmarks, particularly in near-OOD scenarios.

The primary concerns raised in the reviews centered around the additional computational and memory requirements of the WeiPer method, its stability across various datasets, and the need for tuning multiple hyperparameters. The authors' rebuttals have addressed these concerns satisfactorily. Given the unanimous support, the decision is to accept this paper. The authors are encouraged to incorporate the suggested enhancements in their final version.

Reviews: review: paper_summary: paper_summary: The paper introduces "WeiPer," a method that improves existing out-of-distribution (OOD) detection techniques by perturbing class projection weights. This method leverages the class-discriminative ability of pre-trained neural network classifiers by introducing weight perturbations in the final fully connected layer, creating richer representations of the input. The authors demonstrate that this technique significantly enhances OOD detection performance across multiple benchmarks, especially in challenging near-OOD scenarios. Additionally, a KL-divergence-based scoring method is proposed to utilize the properties of the augmented WeiPer space, supported by theoretical motivations and empirical observations.
strength_weakness: strength_weakness: 1. WeiPer introduces additional computational complexity and memory requirements, which might limit its applicability in resource-constrained environments.

2. The proposed method involves several hyperparameters that need tuning, potentially increasing the difficulty of practical implementation.

3. Compared with other methods, the superior performance is not very stable. While the method performs well across several benchmarks, additional validation on a wider variety of datasets would further demonstrate its generalizability.
questions: questions: Can you provide more discussion on the motivation of the approach and its contribution to making the OOD field more robust?
limitations: limitations: The authors have addressed the limitations.
paper_summary: paper_summary: The manuscript proposes a post-hoc OOD detection method, which is broadly applicable and can improve existing OOD detection methods.
strength_weakness: strength_weakness: Since memory usage is a limitation of the proposed methodology, the manuscript should present a comparison of computational cost (time and memory) between the multiple OOD detection methodologies that are considered.
questions: questions: How does your methodology compare to the state-of-the-art in terms of required memory and time?
limitations: limitations: The authors clearly stated the method's limitations.
paper_summary: paper_summary: This work proposes a component, WeiPer, that can benefit OOD detection. WeiPer adds random perturbations (sampled from standard normal) to the class projection weight vectors and essentially expands the output dimension (compared to the original logit space). WeiPer can be combined with multiple existing OOD detection scoring functions (e.g., MSP, ReAct). The authors further propose a KL-divergence-based scoring mechanism that works particularly well with WeiPer. Experiments show that WeiPer+KLD yields state-of-the-art results on the challenging OpenOOD benchmarks, including the near-OOD one on ImageNet-1K.
strength_weakness: strength_weakness: A few weaknesses have been discussed by the authors in Sec. 4 and 5, e.g., WeiPer is less powerful on ViT and could induce higher memory consumption.
questions: questions: Since WeiPer essentially expands the output logit space and is expected to encode richer information, is it possible to leverage WeiPer for other tasks such as OOD generalization, beyond OOD detection?
limitations: limitations: The authors adequately addressed the limitations.
paper_summary: paper_summary: The paper introduces WeiPer, an post-hoc method for out-of-distribution (OOD) detection that leverages weight perturbations in the final layer of neural networks to enhance detection performance. 

Contributions:

1. Introducing linear projections of the penultimate layer by perturbing the final layer's weights to improve OOD detection.

2. Discovering a fingerprint-like nature of in-distribution (ID) samples in both penultimate and newly perturbed spaces, leveraging this structure for a novel detection method.

3. Proposing a KL-divergence-based scoring function and evaluating it alongside MSP and ReAct methods, showing state-of-the-art performance on near OOD tasks using the OpenOOD benchmark.
strength_weakness: strength_weakness: **Related Work Missing Strong Baselines: Data Depths and Information Projections**

Data depths and information projections are gaining significant interest in the OOD detection community. However, these approaches are notably absent in the related work section. It is essential to address these gaps to provide a comprehensive overview of the field.

Relevant works include:

M. Darrin. "Unsupervised Layer-wise Score Aggregation for Textual OOD Detection."

M. Darrin. "Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data."

M. Picot. "Adversarial Attack Detection Under Realistic Constraints."

M. Picot. "A Simple Unsupervised Data Depth-based Method to Detect Adversarial Images."

M. Picot. "A Halfspace-Mass Depth-Based Method for Adversarial Attack Detection." TMLR 2023.

P. Colombo. "Beyond Mahalanobis Distance for Textual OOD Detection." NeurIPS 2022.

P. Colombo. "Toward Stronger Textual Attack Detectors."

**Lack of Strong Baselines for Empirical Comparison**

The current work lacks strong baseline comparisons, particularly those involving data depth methods. This omission undermines the robustness of empirical evaluations. Incorporating these baselines is crucial for a fair and thorough comparison with existing methods.

Key references that should be considered for baseline comparisons include:

E. Gomes. "A Functional Perspective on Multi-Layer Out-of-Distribution Detection."
M. Picot. "A Simple Unsupervised Data Depth-based Method to Detect Adversarial Images."
M. Picot. "A Halfspace-Mass Depth-Based Method for Adversarial Attack Detection." TMLR 2023.
questions: questions: 1. Why did you not mention the data depths and only rely on Mahanalobis?
2. Why not use information projections?
3. Can you compare against the baselines previously introduced?

**I thank the authors for their rebuttal, i have increased my grade.**
limitations: limitations: NA

Comments: title: Thanks for the rebuttal
comment: I thank the authors for the new experiments. For the camera-ready, I suggest including comprehensive comparisons on time and memory consumption between the proposed method and diverse alternative ones.

Moreover, I believe that the experiments using subsets of the training database, which the authors mentioned in their rebuttal, will be valuable for showing the practicality of their methodology. 

Overall, I am happy to increase my score to accept.
The author has basically addressed my doubts. Regarding the more discussion about motivation I mentioned, I think a broader description of motivation and contribution can be given in the context of the task. I am happy to improve my score to Borderline Accept.
I have read the rebuttal and have no further questions. I maintain my score.