Prototype-based Optimal Transport for Out-of-Distribution Detection

## Abstract

Abstract Detecting Out-of-Distribution (OOD) inputs is crucial for improving the reliability of deep neural networks in the real-world deployment. In this paper, inspired by the inherent distribution shift between ID and OOD data, we propose a novel method that leverages optimal transport to measure the distribution discrepancy between test inputs and ID prototypes. The resulting transport costs are used to quantify the individual contribution of each test input to the overall discrepancy, serving as a desirable measure for OOD detection. To address the issue that solely relying on the transport costs to ID prototypes is inadequate for identifying OOD inputs closer to ID data, we generate virtual outliers to approximate the OOD region via linear extrapolation. By combining the transport costs to ID prototypes with the costs to virtual outliers, the detection of OOD data near ID data is emphasized, thereby enhancing the distinction between ID and OOD inputs. Experiments demonstrate the superiority of our method over state-of-the-art methods.

## 1 Introduction

Deep neural networks (DNNs) deployed in real-world scenarios often encounter out-of-distribution (OOD) inputs, such as inputs not belonging to one of the DNN’s known classes.
Ideally, reliable DNNs should be aware of what they do not know. However, they typically make overconfident predictions on OOD data (Nalisnick et al., ).
This notorious behavior undermines the credibility of DNNs and could pose risks to involved users, particularly in safety-critical applications like autonomous driving (Filos et al., ) and biometric authentication (Wang & Deng, ).
This gives rise to the importance of OOD detection, which identifies whether an input is OOD and enables conservative rejection or transferring decision-making to humans for better handling.

The representations of ID data within the same class tend to be gathered together after the training process, as shown in Figure 1(a). In contrast, the representations of OOD data are relatively far away from ID data, as they are not involved in the training process. In other words, the distributions of ID and OOD representations in the latent space exhibit a distinct separation.
Therefore, we can expect that the distribution discrepancy between the representations of test inputs (i.e., a mixture of ID and OOD data) and pure ID data is primarily caused by the presence of OOD data. Such a distribution discrepancy motivates us to differentiate OOD data from test inputs by quantifying the individual contribution of each test input to the overall distribution discrepancy.

In this way, a critical question arises: how to measure the distribution discrepancy between test inputs and ID data, while quantifying the contribution of each test input? To this end, we utilize *optimal transport* (OT), a principled approach with rich geometric awareness for measuring the discrepancy between distributions.
Concretely, OT aims to minimize the total transport cost between two distributions to measure the distribution discrepancy based on a predefined cost function (typically the geometric distance between samples).
The smaller the total cost is, the closer the two distributions are.
Since the total cost comprises the sum of transport costs between sample pairs, OT facilitates a fine-grained assessment of individual sample contributions to the overall discrepancy, making it particularly well-suited for OOD detection. Furthermore, the transport cost captures the geometric differences between ID and OOD representations in the latent space, providing a geometrically meaningful interpretation.

Based on the above intuition, in this paper, we propose a novel OOD detection method called POT that utilizes the Prototype-based Optimal Transport.
Concretely, we first construct ID prototypes with the class-wise average of training sample representations to represent the distribution of ID data.
We then apply OT between the representations of test inputs and the ID prototypes, obtaining a transport cost matrix where each entry indicates the transport cost between the corresponding pair of test input and prototype.
As illustrated in Figure 1 (b), the total transport cost, calculated by summing all matrix entries, reflects the overall distribution discrepancy.
The transport cost from each test sample to all ID prototypes (i.e., the row sum), serves as a measure of individual contribution to the overall discrepancy, indicating the likelihood of being OOD.
However, the task of OOD detection remains inadequately addressed due to the presence of OOD data with smaller distribution shifts.
These OOD data lie closer to ID data in the latent space, rendering the transport costs to ID prototypes insufficient for detecting them.
To tackle this issue, we propose generating virtual outliers to approximate the OOD region, particularly the areas near ID prototypes, using linear extrapolation between ID prototypes and the average representation of test inputs.
By integrating the transport costs from test inputs to ID prototypes with the cost to virtual outliers, the detection of OOD samples with smaller distribution shifts could be emphasized, thereby enhancing the overall distinction between ID and OOD data.

Our key contributions are as follows: (1) We present a novel perspective for OOD detection by measuring distribution discrepancy and propose an effective detection method using prototype-based optimal transport. (2) Extensive experiments on various benchmark datasets demonstrate that our proposed method achieves state-of-the-art (SOTA) performance, outperforming 21 previous OOD detection methods.
Moreover, in the scenarios where training data is unavailable, our method consistently beats the robust competitors by a margin of 22.5% in FPR95 on the CIFAR-100 benchmark.

Figure: Figure 1: Illustration of our method for OOD detection. In (a), the representation distribution of OOD inputs is distinctly separated from ID inputs, visualized via t-SNE. The model is ResNet18 (He et al., ). The ID/OOD data is CIFAR-10 (Krizhevsky, ) and SVHN (Netzer et al., ). (b) shows a slice of the transport cost matrix, which is derived from the optimal transport between test inputs and ID prototypes (depicted as triangles). The row sum of a test input (labelled from A to L) represents the transport cost from it to all ID prototypes. Darker colors indicate higher transport costs. It is evident that the ID inputs (depicted as orange circles) generally incur lower transport costs compared to the OOD inputs (depicted as blue circles).
Refer to caption: extracted/5915266/ft/intro/illustration.png

## 2 Related Work

### 2.1 OOD Detection

OOD detection has attracted growing research attention in
recent years. Existing approaches can generally be categorized into two major lines:

(1) One line of work utilizes the outputs from pretrained models to design scoring functions for differentiating OOD samples. These post-hoc methods can be further divided into three subcategories.
1) The confidence-based methods (Hendrycks & Gimpel, ; Sun et al., ; Song et al., ; Hendrycks et al., ; Wang et al., ; Liu et al., ) adjusts model outputs to obtain the desired confidence, including maximum softmax probability (Hendrycks & Gimpel, ), energy (Liu et al., ), and generalized entropy (Liu et al., ).
2) The density-based methods (Hendrycks et al., ; Sun & Li, ; Zhang et al., ; Liu et al., ) identifies certain properties or patterns of ID data, such as neuron coverage (Liu et al., ), by learning the corresponding density functions, and the OOD samples that deviate from these properties or patterns tend to reside in low-density regions.
3) The distance-based methods (Lee et al., ; Ren et al., ; Sehwag et al., ; Sun et al., ) adopts distance metrics, e.g., Mahalanobis distance, between test input and ID samples or centroids differentiate OOD samples, under the assumption that OOD data lie relatively far away from ID data.
Different from these works, we introduce a novel perspective for OOD detection on measuring distribution discrepancy.
The most closely related work is dual divergence estimation (DDE) (Garg et al., ), which estimates the dual KL-Divergence between the test samples and ID samples.
However, while DDE estimates divergence in a dual space optimized by DNNs and relies heavily on the quality of transformed sample representations, POT enables direct measurement of distribution discrepancy in the latent space.

(2) Another line of work focuses on altering models with training-time regularization to amplify the differences between OOD and ID samples (Hendrycks et al., ; Chen et al., ; Ming et al., ; Zhang et al., ; Wang et al., ; Lu et al., ). For example, by incorporating a suitable loss, models are encouraged to produce predictions with uniform distributions (Hendrycks et al., ) or higher energy (Liu et al., ) for outlier data.
Building on this, some approaches investigate refining or synthesizing outliers to improve the performance of models.
For instance, Ming et al. () utilizes a posterior sampling-based technique to select the most informative OOD samples from the large outlier set, while Wang et al. () implicitly expands the auxiliary outliers by perturbing model parameters.
However, the computational overhead of retraining can be prohibitive, especially when the parameter scale is large.
Additionally, modifying models may also have side effects of degrading model performance on the original task.
In contrast, this paper focus on post-hoc methods, which are easy to implement and generally applicable across different models.
Such properties are highly practical for adopting OOD detection methods in real-world applications.

### 2.2 Optimal Transport

As a mathematical tool for comparing distributions, optimal transport (OT) has been successfully employed in diverse machine learning tasks, including domain adaptation (Courty et al., ; Turrisi et al., ), generative adversarial training (Arjovsky et al., ), object detection (Ge et al., ), and partial-label learning (Wang et al., ).
The most related one to our work is (Lu et al., ), which also applies OT for the OOD detection problem.
By assuming both an unlabeled and a labeled training set,  Lu et al. () uses OT to guide the clustering of samples for label assignment to the unlabeled samples, thereby augmenting the training data and facilitating the model retraining for OOD detection.
In contrast, our work operates in a post-hoc manner without the requirement of extra training data or model retraining.

## 3 The Proposed Method

### 3.1 Problem Setting

In the context of supervised multi-class classification, we denote the data space as $\mathcal{X}$ and the corresponding label space as $\mathcal{Y}=\{1,2,\cdots,C\}$.
The training dataset with in-distribution (ID) samples $\mathcal{D}_{tr}=\{(\bm{x}_{i},{y}_{i})\}^{n}_{i=1}$ is sampled from the joint distribution $P_{\mathcal{X}\mathcal{Y}}$.
The marginal distribution on $\mathcal{X}$ is denoted as $P_{\mathcal{X}}^{in}$.
The model trained on training data typically consists of a feature encoder $g$ : $\mathcal{X}\rightarrow\mathcal{R}^{d}$, mapping the input $\bm{x}\in\mathcal{X}$ to a $d$-dimensional representation, and a linear classification layer $f$ : $\mathcal{R}^{d}\rightarrow\mathcal{R}^{C}$, producing a logit vector containing classification confidence for each class.
Given the test inputs $\mathcal{D}_{te}=\{\bm{x}_{j}\}^{m}_{j=1}$, the goal of OOD detection is to identify whether $\bm{x}_{j}$ is out-of-distribution w.r.t $P_{\mathcal{X}}^{in}$.

### 3.2 Prototype-based Optimal Transport for OOD Detection

Constructing Class Prototypes.
The key idea of our method is to use OT to measure the distribution discrepancy between test inputs and ID data while quantifying the individual contribution of each test input.
A straightforward approach involves applying OT between test inputs and the training data.
However, the standard OT is essentially a linear programming problem, suffering from cubic time complexity and incurring prohibitive computational cost when adopted to large-scale training sets (Peyré et al., ).
An alternative is to sample a smaller subset of the training set for efficiency, but this can result in missing classes, leading to mismatches in ID inputs.
Such mismatches can exaggerate the contribution of ID inputs to the distribution discrepancy, thus incorrectly identifying them as OOD.
To overcome this, we propose to characterize each class with a prototype and align test inputs to these prototypes.
Specifically, given the training dataset $\mathcal{D}_{tr}=\{(\bm{x}_{i},y_{i})\}^{n}_{i=1}$, we construct each class prototype as the average representation for that class extracted from the feature encoder $g$:

$$ ${\bm{\eta}}_{c}=\frac{1}{N_{c}}\sum\nolimits_{i=1}^{n}g(\bm{x}_{i})\bm{1}{\{y_ {i}=c\}},$ (1) $$

where $N_{c}$ is the number of samples in class $c$.
The utilization of prototypes offers two key benefits: it reduces computational overhead by reducing the data scale and ensures all classes are represented, mitigating the risk of mismatches caused by missing classes.

OOD Detection Using Prototype-based Optimal Transport.
By formalizing an optimal transport problem between the representations of test inputs $\{\bm{z}_{j}=g(\bm{x}_{j})\}_{j=1}^{m}$ and ID prototypes $\{{\bm{\eta}}_{i}\}_{i=1}^{C}$,
we search for the minimal transport cost that represents the distribution discrepancy, while subject to the mass conservation constraint:

$$ $\begin{split}\min\limits_{\bm{\gamma}\in\Pi(\bm{\mu},\bm{\nu})}\langle\bm{E}&, \bm{\gamma}\rangle_{F}=\min\limits_{\bm{\gamma}\in\Pi(\bm{\mu},\bm{\nu})}\sum \nolimits_{i=1}^{C}\sum\nolimits_{j=1}^{m}E_{ij}\gamma_{ij}\\ \text{s.t. }\Pi(\bm{\mu},\bm{\nu})&=\{\bm{\gamma}\in\mathcal{R}_{+}^{C\times m }|\bm{\gamma}\bm{1}_{m}=\bm{\mu},\bm{\gamma}^{T}\bm{1}_{C}=\bm{\nu}\},\end{split}$ (2) $$

where $\langle\cdot,\cdot\rangle_{F}$ stands for the Frobenius dot-product. $\bm{\mu}$ and $\bm{\nu}$ are the probability simplexes of ID prototypes and test samples:

$$ $\begin{split}\bm{\mu}=\sum\nolimits_{i=1}^{C}p_{i}\delta_{{\bm{\eta}}_{i}} \qquad\text{and}\qquad\bm{\nu}=\sum\nolimits_{j=1}^{m}q_{j}\delta_{\bm{z}_{j}} ,\end{split}$ (3) $$

where $p_{i}=\frac{N_{i}}{\sum_{i=1}^{n}N_{i}}$, $q_{j}=\frac{1}{m}$, and $\delta_{{\bm{\eta}}_{i}}$ is the Dirac at position ${\bm{\eta}}_{i}$.
$\bm{\gamma}$ is a transport plan, essentially a joint probability matrix, with an entry $\gamma_{ij}$ describing the amount of probability mass transported from prototype ${\bm{\eta}}_{i}$ to test input $\bm{z}_{j}$.
All feasible transport plans constitute the transportation polytope $\Pi(\bm{\mu},\bm{\nu})$ (Cuturi, ).
$\bm{E}$ is the ground cost matrix, where the entry $E_{ij}$ denotes the point-to-point moving cost between ${\bm{\eta}}_{i}$ and $\bm{z}_{j}$, which is defined with the Euclidean distance as $E_{ij}=||{\bm{\eta}}_{i}-\bm{z}_{j}||_{2}$.

To efficiently resolve Equation 2, we introduce the entropic regularization term $H(\bm{\gamma})$ and express the optimization problem as:

$$ $\min\limits_{\bm{\gamma}\in\Pi(\bm{\mu},\bm{\nu})}\langle\bm{E},\bm{\gamma} \rangle_{F}-\lambda H(\bm{\gamma}),$ (4) $$

where $\lambda>0$ and $H(\bm{\gamma})=\sum_{i,j}\gamma_{ij}(\log\gamma_{ij}-1)$ (Peyré et al., ).
In this way, the optimal transport plan $\bm{\gamma}$ can be written as:

$$ $\bm{\gamma}=\text{Diag}(\bm{a})\bm{K}\text{Diag}(\bm{b}),\quad\text{where} \quad\bm{K}=\text{exp}(-\bm{E}/\lambda).$ (5) $$

Here, $\bm{a}\in\mathcal{R}^{C}$ and $\bm{b}\in\mathcal{R}^{m}$ are known as scaling variables.
This formulation can be solved much faster using the Sinkhorn-Knopp algorithm (Cuturi, ):

$$ $\bm{a}\leftarrow\bm{\mu}\oslash(\bm{K}\bm{b}),\quad\bm{b}\leftarrow\bm{\nu} \oslash(\bm{K}^{\top}\bm{a}),$ (6) $$

where $\oslash$ denotes element-wise division.
Detailed derivations are provided in Appendix A.
With the $\lambda$-strong convexity (Peyré et al., ), the entropic regularized OT could be solved in quadratic time complexity $O(nm)$, where $n$ and $m$ denote the number of data points in the two distributions, respectively.
Since the ID distribution is represented with a fixed number of prototypes, the prototype-based OT has linear time complexity $O(Cm)$.

To evaluate whether an input sample $\bm{x}_{j}\in\mathcal{D}_{te}$ is OOD or not, we derive the cost of moving it to all ID prototypes $\mathcal{T}_{j}$ by decomposing the total transport cost:

$$ $\langle\bm{E},\bm{\gamma}\rangle_{F}=\sum\nolimits_{j=1}^{m}\sum\nolimits_{i=1 }^{C}E_{ij}\gamma_{ij}:=\sum\nolimits_{j=1}^{m}\mathcal{T}_{j}.$ (7) $$

As the total transport cost serves as a measure of distributional discrepancy between ID and test data, a higher transport cost $\mathcal{T}$ for a test sample
indicates a greater deviation from the ID data, suggesting that the sample is more likely to be OOD data.
We emphasize that, while the solution of OT is the transport plan $\bm{\gamma}$, which is often the focus in many applications (Caron et al., ; Wang et al., ), our concentration lies on the transport costs between sample pairs.
Specifically, this is the product of the transport plan and the ground cost, which serves as a desirable and interpretable measure for OOD detection.

Data Augmentation via Linear Extrapolation.
As described above, the transport cost can serve as a measure for OOD detection. However, relying solely on the cost to ID prototypes is insufficient for discerning OOD data with smaller distribution shifts from ID data.
This is because such OOD data are located closer to ID data in the latent space and tend to incur lower transport costs to the ID prototypes.
Meanwhile, the mass conservation constraint inherent in OT may exacerbate this issue by enforcing transportation between OOD data and prototypes when ID inputs are sparse.
To address this issue, we propose to generate virtual outliers to approximate the OOD region using representation linear representation extrapolation.
By integrating the transport costs from test inputs to both virtual outliers and ID prototypes, we introduce a contrastive transport cost, which enhances the detection performance, particularly for the OOD inputs with smaller distribution shifts.

Given two representations $z_{i}$ and $z_{j}$, the linear representation extrapolation is defined as:

$$ $z^{*}=z_{i}+\omega(z_{j}-z_{i}),\quad s.t.\quad\omega>1\vee\omega<0.$ (8) $$

Instead of generating outliers aimlessly by enumerating available sample pairs, we construct virtual outliers $\mathcal{P}^{*}$ by combining the prototypes $\mathcal{P}$ and the average representation of test inputs $\mathcal{M}$:

$$ $\mathcal{P}^{*}=\{{\bm{\eta}}^{*}_{i}={\bm{\eta}}_{i}+\omega(\mathcal{M}-{\bm{ \eta}}_{i})+{\bm{\eta}}_{i}\in\mathcal{P}\},$ (9) $$

where $\omega>1$ ensures that the generated points $\mathcal{P}^{*}$ lies beyond $\mathcal{M}$.
The underlying intuition is that $\mathcal{M}$ can be expressed as a linear interpolation of the average representations of test ID samples $\mathcal{M}_{\text{in}}$ and test OOD samples $\mathcal{M}_{\text{out}}$:

$$ $\mathcal{M}:=\frac{N_{\text{in}}}{N}\mathcal{M}_{\text{in}}+\frac{N-N_{\text{ in}}}{N}\mathcal{M}_{\text{out}},$ (10) $$

where $N_{\text{in}}$ and $N_{\text{out}}$ denote the number of test ID samples and total test samples, respectively.
As illustrated in Figure 2, the point $\mathcal{M}$ resides between ID and OOD data, guiding the direction for outlier generation in response to distribution shifts.
As parameter $\omega$ increases, the generated virtual outliers progressively move away from the ID prototypes towards the OOD region in the latent space.
By choosing an appropriate parameter, we can control the location of generated virtual outliers to emphasise the detection of OOD inputs with smaller distributions shifts.
In contrast to the method (Zhu et al., ) that conducts informative extrapolation to synthesize numerous outliers during training with assumed auxiliary outliers, the linear representation extrapolation is a lightweight operation that does not require training or auxiliary outliers.

Figure: Figure 2: Illustration of virtual outlier generation. The average representation of test inputs $\mathcal{M}$ lies between the average of ID inputs $\mathcal{M}_{\text{in}}$ and average of OOD inputs $\mathcal{M}_{\text{out}}$. We generate virtual outliers to approximate the OOD region using linear extrapolation between $\mathcal{M}$ and ID prototypes.
Refer to caption: x1.png

Likewise, after generating the virtual outliers $\mathcal{P}^{*}$, we apply the entropic regularized OT between $\mathcal{P}^{*}$ and the test inputs to obtain the corresponding transport cost $\mathcal{T^{*}}$.
By taking the difference of transport costs from test inputs to the ID prototypes and to the virtual outliers, we derive the contrastive transport cost $\mathcal{T}-\mathcal{T^{*}}$ as the final OOD score.
Given the opposite trends of $\mathcal{T}$ and $\mathcal{T^{*}}$ in indicating whether a sample is ID or OOD, a higher contrastive transport cost denotes a higher likelihood of the test input being OOD.

## 4 Experiments

In this section, we preform extensive experiments over OOD detection benchmarks.All the experimental setup adheres to the latest version of OpenOOD, an open repository for benchmarking generalized OOD detection(^1^11https://github.com/Jingkang50/OpenOOD.) (Yang et al., ; Zhang et al., ).

### 4.1 Common Setup

Datasets.
We assess the performance of our proposed POT on the widely used CIFAR-100 benchmark and ImageNet-1k benchmark, which regard CIFAR-100 and ImageNet-1k as ID datasets, respectively.
For CIFAR-100, we use the standard split, with 50,000 training images and 10,000 test images. For ImageNet-1k, we utilize 50,000 images from the validation set as ID test set.
For each benchmark, OpenOOD splits OOD data to Far-OOD and Near-OOD based on their degrees of the semantic similarity with ID data.
Specifically, the CIFAR-100 benchmark utilizes four Far-OOD datasets: MNIST (Deng, ), SVHN (Netzer et al., ), Textures (Cimpoi et al., ), and Places365 (Zhou et al., ), along with two Near-OOD datasets: CIFAR-10 and Tiny Imagenet (Le & Yang, ).
For the large-scale ImageNet-1k benchmark, it considers iNaturalist (Horn et al., ), Textures (Cimpoi et al., ), and OpenImage-O (Wang et al., ) as Far-OOD datasets.
In terms of Near-OOD datasets, SSB-hard (Vaze et al., ) and NINCO (Bitterwolf et al., ) are included.

Models.
For the CIFAR-100 benchmark, we utilize ResNet18 (He et al., ) as the model backbone, which is trained on the ID training samples for 100 epochs.
We evaluate OOD detection methods over three checkpoints. For the ImageNet-1k benchmark, we employ ResNet50 and ViT-b16 (Dosovitskiy et al., ) models pretrained on ImageNet-1k and use the official checkpoints from PyTorch. For more training details, please refer to OpenOOD.

Baselines.
Since POT performs in a post-hoc manner, we primarily compare against 21 post-hoc OOD detection methods, including OpenMax (Bendale & Boult, ), MSP (Hendrycks & Gimpel, ), ODIN (Liang et al., ), MDS (Lee et al., ), MDSEns (Lee et al., ), RMDS (Ren et al., ), Gram (Sastry & Oore, ), EBO (Liu et al., ), GradNorm (Huang et al., ), ReAct (Sun et al., ), MLS (Hendrycks et al., ), KLM (Hendrycks et al., ), VIM (Wang et al., ), KNN (Sun et al., ), DICE (Sun & Li, ), RankFeat (Song et al., ), ASH (Djurisic et al., ), SHE (Zhang et al., ), GEN (Liu et al., ), DDE (Garg et al., ), NAC-UE (Liu et al., ). The results for DDE are reproduced using the official codebase (^2^22https://github.com/morganstanley/MSML/tree/main/papers/OOD_Detection_via_Dual_Divergence_Estimation) while the results for the remaining methods are sourced from the implementations in OpenOOD.

Hyperparameter tuning.
In line with OpenOOD, we use ID and OOD validation sets for hyperparameter selection.
Specifically, for the CIFAR-100 benchmark, 1,000 images are held out from the ID test set as the ID validation set, and 1,000 images spanning 20 categories from Tiny ImageNet (Le & Yang, ) are reserved as the OOD validation set. For the ImageNet-1k benchmark, 5,000 images from the ID test set and 1,763 images from OpenImage-O are held out for the ID and OOD validation sets, respectively.
Please note that all the validation samples are disjoint with the test samples.

Evaluation metrics. We report the following widely adopted metrics:
(1) area under the receiver operating characteristic curve (AUROC);
(2) false positive rate of OOD samples when the true positive rate of ID samples is equal to $95\%$ (FPR95).

Implementation details.
We consider the assumption of having access to the entire test set to be overly restrictive.
To this end, we relax the assumption by allowing test data arriving in batches.
Random batch division is applied to the test inputs, which are a mixture of ID and OOD test samples.
The default test batch size is set to $512$ and we also include ablation studies with varying batch sizes.

### 4.2 Empirical Results and Analysis

**Table 1: Far-OOD detection performance on CIFAR-100 benchmark. $\uparrow$ denotes the higher value is better, while $\downarrow$ indicates lower values are better. We format first and second results.**
| Method | MINIST | SVHN | Textures | Places365 | Average |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ |  |
| OpenMax | 53.97$\pm$4.71 | 75.89$\pm$1.40 | 52.81$\pm$1.89 | 82.05$\pm$1.55 | 56.16$\pm$1.86 | 80.46$\pm$0.10 | 54.99$\pm$1.42 | 79.22$\pm$0.41 | 54.48$\pm$0.63 | 79.40$\pm$0.41 |
| MSP | 57.24$\pm$4.67 | 76.08$\pm$1.86 | 58.43$\pm$2.61 | 78.68$\pm$0.95 | 61.79$\pm$1.31 | 77.32$\pm$0.71 | 56.64$\pm$0.87 | 79.22$\pm$0.29 | 58.52$\pm$1.12 | 77.83$\pm$0.45 |
| ODIN | 45.93$\pm$3.24 | 83.79$\pm$1.30 | 67.21$\pm$3.95 | 74.72$\pm$0.77 | 62.39$\pm$2.87 | 79.34$\pm$1.08 | 59.73$\pm$0.86 | 79.45$\pm$0.25 | 58.81$\pm$0.78 | 79.32$\pm$0.22 |
| MDS | 71.70$\pm$2.89 | 67.47$\pm$0.81 | 67.72$\pm$6.05 | 70.20$\pm$6.52 | 70.55$\pm$2.50 | 76.23$\pm$0.69 | 79.57$\pm$0.34 | 63.17$\pm$0.50 | 72.38$\pm$1.53 | 69.27$\pm$1.41 |
| MDSEns | 2.86$\pm$0.85 | 98.20$\pm$0.78 | 82.57$\pm$2.57 | 53.74$\pm$1.62 | 84.91$\pm$0.87 | 69.75$\pm$1.14 | 96.58$\pm$0.19 | 42.32$\pm$0.74 | 66.73$\pm$1.05 | 66.00$\pm$0.69 |
| RMDS | 51.99$\pm$6.34 | 79.78$\pm$2.50 | 51.10$\pm$3.62 | 85.09$\pm$1.09 | 54.06$\pm$1.02 | 83.61$\pm$0.52 | 53.58$\pm$0.33 | 83.39$\pm$0.47 | 52.68$\pm$0.65 | 82.97$\pm$0.42 |
| Gram | 53.35$\pm$7.51 | 80.78$\pm$4.14 | 20.40$\pm$1.69 | 95.47$\pm$0.58 | 89.84$\pm$2.87 | 70.61$\pm$1.44 | 95.03$\pm$0.63 | 46.09$\pm$1.28 | 64.66$\pm$2.30 | 73.24$\pm$1.05 |
| EBO | 52.62$\pm$3.83 | 79.18$\pm$1.36 | 53.19$\pm$3.25 | 82.28$\pm$1.78 | 62.38$\pm$2.08 | 78.35$\pm$0.84 | 57.70$\pm$0.87 | 79.50$\pm$0.23 | 56.47$\pm$1.41 | 79.83$\pm$0.62 |
| GradNorm | 86.96$\pm$1.45 | 65.35$\pm$1.12 | 69.38$\pm$8.40 | 77.23$\pm$4.88 | 92.37$\pm$0.58 | 64.58$\pm$0.13 | 85.41$\pm$0.39 | 69.66$\pm$0.17 | 83.53$\pm$2.01 | 69.20$\pm$1.08 |
| ReAct | 56.03$\pm$5.67 | 78.37$\pm$1.59 | 49.89$\pm$1.95 | 83.25$\pm$1.00 | 55.02$\pm$0.81 | 80.15$\pm$0.46 | 55.34$\pm$0.49 | 80.01$\pm$0.11 | 54.07$\pm$1.57 | 80.45$\pm$0.50 |
| MLS | 52.94$\pm$3.83 | 78.91$\pm$1.47 | 53.43$\pm$3.22 | 81.90$\pm$1.53 | 62.37$\pm$2.16 | 78.39$\pm$0.84 | 57.64$\pm$0.92 | 79.74$\pm$0.24 | 56.60$\pm$1.41 | 79.73$\pm$0.58 |
| KLM | 72.88$\pm$6.56 | 74.15$\pm$2.60 | 50.32$\pm$7.06 | 79.49$\pm$0.47 | 81.88$\pm$5.87 | 75.75$\pm$0.48 | 81.60$\pm$1.37 | 75.68$\pm$0.26 | 71.67$\pm$2.07 | 76.27$\pm$0.53 |
| VIM | 48.34$\pm$1.03 | 81.84$\pm$1.03 | 46.28$\pm$5.52 | 82.89$\pm$3.78 | 46.84$\pm$2.28 | 85.90$\pm$0.79 | 61.64$\pm$0.70 | 75.85$\pm$0.36 | 50.77$\pm$0.98 | 81.62$\pm$0.62 |
| KNN | 48.59$\pm$4.66 | 82.36$\pm$1.54 | 51.43$\pm$3.15 | 84.26$\pm$1.11 | 53.56$\pm$2.35 | 83.66$\pm$0.84 | 60.80$\pm$0.92 | 79.42$\pm$0.47 | 53.59$\pm$0.25 | 82.43$\pm$0.17 |
| DICE | 51.80$\pm$3.68 | 79.86$\pm$1.89 | 48.96$\pm$3.34 | 84.45$\pm$2.04 | 64.23$\pm$1.59 | 77.63$\pm$0.34 | 59.43$\pm$1.20 | 78.31$\pm$0.66 | 56.10$\pm$0.62 | 80.06$\pm$0.19 |
| RankFeat | 75.02$\pm$5.82 | 63.03$\pm$3.85 | 58.17$\pm$2.07 | 72.37$\pm$1.51 | 66.90$\pm$3.79 | 69.40$\pm$3.09 | 77.42$\pm$1.93 | 63.81$\pm$1.83 | 69.38$\pm$1.10 | 67.15$\pm$1.49 |
| ASH | 66.60$\pm$3.88 | 77.23$\pm$0.46 | 45.51$\pm$2.82 | 85.76$\pm$1.38 | 61.34$\pm$2.83 | 80.72$\pm$0.71 | 62.89$\pm$1.08 | 78.75$\pm$0.16 | 59.09$\pm$2.53 | 80.61$\pm$0.66 |
| SHE | 58.82$\pm$2.75 | 76.72$\pm$1.08 | 58.60$\pm$7.63 | 81.22$\pm$4.05 | 73.34$\pm$3.35 | 73.65$\pm$1.29 | 65.23$\pm$0.86 | 76.29$\pm$0.52 | 64.00$\pm$2.73 | 76.97$\pm$1.17 |
| GEN | 54.81$\pm$4.80 | 78.09$\pm$1.82 | 56.14$\pm$2.17 | 81.24$\pm$1.05 | 61.13$\pm$1.49 | 78.70$\pm$0.80 | 56.07$\pm$0.78 | 80.31$\pm$0.22 | 57.04$\pm$1.01 | 79.59$\pm$0.54 |
| DDE | 0.01$\pm$0.01 | 99.93$\pm$0.02 | 0.23$\pm$0.03 | 99.31$\pm$0.09 | 40.30$\pm$1.24 | 93.13$\pm$0.29 | 52.34$\pm$0.61 | 88.21$\pm$0.23 | 23.22$\pm$0.45 | 95.14$\pm$0.15 |
| NAC-UE | 21.44$\pm$5.22 | 93.24$\pm$1.33 | 24.23$\pm$3.88 | 92.43$\pm$1.03 | 40.19$\pm$1.97 | 89.34$\pm$0.56 | 73.93$\pm$1.52 | 72.92$\pm$0.78 | 39.95$\pm$1.36 | 86.98$\pm$0.26 |
| POT | 0.98$\pm$0.08 | 99.73$\pm$0.02 | 2.13$\pm$0.21 | 99.39$\pm$0.03 | 25.56$\pm$3.93 | 95.28$\pm$0.44 | 28.74$\pm$0.22 | 92.42$\pm$0.12 | 14.35$\pm$1.06 | 96.70$\pm$0.08 |

**Table 2: Near-OOD detection performance on CIFAR-100 benchmark. $\uparrow$ denotes the higher value is better, while $\downarrow$ indicates lower values are better.**
| Method | CIFAR-10 | Tiny ImageNet | Average |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ |  |
| OpenMax | 60.19$\pm$0.87 | 74.34$\pm$0.33 | 52.79$\pm$0.43 | 78.48$\pm$0.09 | 56.49$\pm$0.58 | 76.41$\pm$0.20 |
| MSP | 58.90$\pm$0.93 | 78.47$\pm$0.07 | 50.78$\pm$0.57 | 81.96$\pm$0.20 | 54.84$\pm$0.58 | 80.21$\pm$0.13 |
| ODIN | 60.61$\pm$0.52 | 78.18$\pm$0.14 | 55.28$\pm$0.45 | 81.53$\pm$0.10 | 57.95$\pm$0.45 | 79.86$\pm$0.11 |
| MDS | 88.01$\pm$0.51 | 55.89$\pm$0.22 | 78.68$\pm$1.48 | 61.83$\pm$0.19 | 83.35$\pm$0.76 | 58.86$\pm$0.09 |
| MDSEns | 95.94$\pm$0.15 | 43.85$\pm$0.31 | 95.76$\pm$0.13 | 49.14$\pm$0.22 | 95.85$\pm$0.05 | 46.49$\pm$0.25 |
| RMDS | 61.36$\pm$0.23 | 77.77$\pm$0.21 | 49.50$\pm$0.58 | 82.58$\pm$0.02 | 55.43$\pm$0.29 | 80.18$\pm$0.10 |
| Gram | 92.69$\pm$0.58 | 49.41$\pm$0.54 | 92.34$\pm$0.84 | 53.12$\pm$1.66 | 92.51$\pm$0.39 | 51.26$\pm$0.80 |
| EBO | 59.19$\pm$0.74 | 79.05$\pm$0.10 | 52.36$\pm$0.59 | 82.58$\pm$0.08 | 55.77$\pm$0.64 | 80.82$\pm$0.09 |
| GradNorm | 84.30$\pm$0.38 | 70.32$\pm$0.20 | 87.30$\pm$0.59 | 69.58$\pm$0.79 | 85.80$\pm$0.46 | 69.95$\pm$0.47 |
| ReAct | 61.29$\pm$0.43 | 78.65$\pm$0.05 | 51.64$\pm$0.41 | 82.72$\pm$0.08 | 56.47$\pm$0.42 | 80.69$\pm$0.06 |
| MLS | 59.10$\pm$0.63 | 79.21$\pm$0.10 | 52.19$\pm$0.42 | 82.74$\pm$0.08 | 55.64$\pm$0.52 | 80.97$\pm$0.09 |
| KLM | 84.77$\pm$2.99 | 73.92$\pm$0.23 | 71.59$\pm$0.79 | 79.16$\pm$0.30 | 78.18$\pm$1.30 | 76.54$\pm$0.24 |
| VIM | 70.63$\pm$0.44 | 72.21$\pm$0.42 | 54.54$\pm$0.31 | 77.87$\pm$0.13 | 62.59$\pm$0.26 | 75.04$\pm$0.14 |
| KNN | 72.82$\pm$0.50 | 77.01$\pm$0.26 | 49.63$\pm$0.61 | 83.31$\pm$0.16 | 61.22$\pm$0.15 | 80.16$\pm$0.15 |
| DICE | 60.98$\pm$1.10 | 78.04$\pm$0.32 | 55.36$\pm$0.59 | 80.50$\pm$0.25 | 58.17$\pm$0.50 | 79.27$\pm$0.22 |
| RankFeat | 82.78$\pm$1.56 | 58.04$\pm$2.36 | 78.37$\pm$1.09 | 65.63$\pm$0.24 | 80.57$\pm$1.11 | 61.84$\pm$1.29 |
| ASH | 68.06$\pm$0.41 | 76.47$\pm$0.30 | 63.47$\pm$1.10 | 79.79$\pm$0.24 | 65.77$\pm$0.49 | 78.13$\pm$0.17 |
| SHE | 60.47$\pm$0.58 | 78.13$\pm$0.02 | 58.42$\pm$0.76 | 79.52$\pm$0.33 | 59.45$\pm$0.34 | 78.83$\pm$0.17 |
| GEN | 58.65$\pm$0.92 | 79.40$\pm$0.06 | 49.82$\pm$0.29 | 83.15$\pm$0.15 | 54.23$\pm$0.54 | 81.27$\pm$0.10 |
| DDE | 62.35$\pm$2.12 | 81.32$\pm$0.28 | 61.20$\pm$2.11 | 80.34$\pm$0.95 | 61.78$\pm$2.11 | 80.83$\pm$0.60 |
| NAC-UE | 80.84$\pm$1.38 | 71.92$\pm$0.77 | 62.78$\pm$1.69 | 79.43$\pm$0.45 | 71.81$\pm$1.51 | 75.67$\pm$0.56 |
| POT | 41.63$\pm$2.28 | 87.51$\pm$0.44 | 46.94$\pm$0.43 | 85.50$\pm$0.04 | 44.28$\pm$1.26 | 86.51$\pm$0.20 |

**Table 3: OOD detection performance (AUROC$\uparrow$) on ImageNet-1k. See Table 7 and Table 8 for full results.**
| Backbone | Datasets | RMDS | ReAct | VIM | KNN | ASH | SHE | GEN | DDE | NAC-UE | POT |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ViT-b16 | iNaturalist | 96.09 | 86.09 | 95.71 | 91.45 | 50.47 | 93.56 | 93.53 | 97.10 | 93.69 | 99.38 |
| Openimage-O | 92.29 | 84.26 | 92.15 | 89.83 | 55.45 | 91.03 | 90.25 | 88.97 | 91.54 | 95.60 |  |
| Textures | 89.38 | 86.69 | 90.61 | 91.12 | 47.87 | 92.67 | 90.25 | 88.96 | 94.17 | 95.36 |  |
| Average (Far-OOD) | 92.59 | 85.68 | 92.82 | 90.80 | 51.26 | 92.42 | 91.34 | 91.68 | 93.13 | 96.78 |  |
| SSB-hard | 72.79 | 63.24 | 69.34 | 65.93 | 54.12 | 68.11 | 70.19 | 76.29 | 68.04 | 80.39 |  |
| NINCO | 87.28 | 75.45 | 84.63 | 82.22 | 53.07 | 84.16 | 82.47 | 81.32 | 82.45 | 87.51 |  |
| Average (Near-OOD) | 80.03 | 69.34 | 76.98 | 74.08 | 53.59 | 76.13 | 76.33 | 78.81 | 75.25 | 83.95 |  |
| ResNet-50 | iNaturalist | 87.27 | 96.32 | 89.54 | 86.31 | 97.04 | 92.58 | 92.44 | 83.64 | 96.43 | 99.45 |
| Openimage-O | 85.73 | 91.88 | 90.40 | 86.78 | 93.31 | 86.70 | 89.35 | 71.38 | 91.61 | 93.45 |  |
| Textures | 86.07 | 92.80 | 97.96 | 97.06 | 96.91 | 93.63 | 87.63 | 86.84 | 97.77 | 95.58 |  |
| Average (Far-OOD) | 86.36 | 93.67 | 92.63 | 90.05 | 95.76 | 90.97 | 89.81 | 80.62 | 95.27 | 96.16 |  |
| SSB-hard | 71.47 | 73.09 | 65.10 | 61.78 | 73.11 | 71.83 | 72.11 | 56.52 | 68.21 | 83.37 |  |
| NINCO | 82.22 | 81.71 | 78.54 | 79.41 | 83.37 | 76.42 | 81.71 | 62.93 | 81.16 | 78.13 |  |
| Average (Near-OOD) | 76.84 | 77.40 | 71.82 | 70.59 | 78.24 | 74.12 | 76.91 | 59.73 | 74.68 | 80.75 |  |

Main results. The results for Far-OOD and Near-OOD detection on the CIFAR-100 benchmark are presented in Table 1 and Table 2, respectively.
Our proposed POT consistently obtains either the best or second-best results across all datasets and OOD detection metrics. Specifically, on the Far-OOD track, POT achieves significant reductions in average FPR95, with decreases of 25.6% and 8.87% compared to the previous leading baselines NAC-UE and DDE, respectively. While Near-OOD samples are considered more intractable to detect due to their similarity in semantic and style with ID samples, POT demonstrates an even greater performance advantage in detecting them, as shown in Table 2. For instance, POT surpasses the next best method, GEN, by 9.96% in average FPR95 and 5.22% in average AUROC. The results of the large-scale ImageNet-1k benchmark are shown in Table 3, where only the AUROC values are reported due to space limitation. As can be seen, POT also consistently outperforms all baseline methods in terms of average performance across different backbones and OOD datasets.

**Table 4: Results on the ImageNet-1k with different training methods. We employ the ResNet-50 as the backbone. $\Delta$ represents the subtraction results between the default CE scheme and other training schemes.**
|  | Baseline | ASH | NAC-UE | POT |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Training | FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ |
| CE | - | - | 19.67 | 95.76 | 22.67 | 95.27 | 19.89 | 96.16 |
| ConfBranch | 51.17 | 83.97 | 78.77 | 72.64 | 32.21 | 93.63 | 20.34 | 95.90 |
| $\Delta$ | - | - | (+59.1) | (-23.12) | (+9.54) | (-1.64) | (+0.45) | (-0.26) |
| RotPred | 36.39 | 90.03 | 68.61 | 78.99 | 38.50 | 92.23 | 19.55 | 95.93 |
| $\Delta$ | - | - | (+48.94) | (-16.77) | (+15.83) | (-3.04) | (-0.34) | (-0.23) |
| GODIN | 51.03 | 85.50 | 57.06 | 88.07 | 29.44 | 93.99 | 22.74 | 95.25 |
| $\Delta$ | - | - | (+37.39) | (-7.69) | (+6.77) | (-1.28) | (+2.85) | (-0.91) |

Integration with training methods.
In the other line of work for OOD detection, training methods employ retraining strategies with training-time regularization to provide a modified model.
A important property of post-hoc methods is that they are applicable to different model architectures and training losses.
To this end, we examine the performance of post-hoc methods when integrated with established training methods.
We evaluate on the Far-OOD track of the ImageNet-1k benchmark using ResNet-50 as the backbone, comparing POT with ASH and NAC-UE, which have achieved top results on this benchmark (see Table 3).
Consistent with the experimental setup of NAC-UE, we utilize three training schemes: ConfBranch (DeVries & Taylor, ), RotPred (Hendrycks et al., ), and GODIN (Hsu et al., ), with softmax cross-entropy (CE) loss as the default training scheme for comparison.
We report the average results in Table 4, where Baseline refers to the detection method employed in the original paper and $\Delta$ denotes the subtraction results between the default CE scheme and other training schemes.
According to the results, POT has remarkable improvements on the baseline methods, while outperforming ASH and NAC-UE.
Importantly, POT maintains stable performance across three training schemes, whereas ASH and NAC-UE both exhibit notable performance degradation compared to the default CE training scheme.
Such results demonstrates that POT is generic to be seamlessly integrated with different training methods.

What if training data is unavailable.
Although existing post-hoc methods can be applied to pre-trained models without cumbersome retraining, many require access to at least a portion of training data (Liu et al., ). Considering some scenarios where training data is unavailable, such as commercial data involving privacy, some approaches further explore OOD detection without the requirement for training data such as ASH and GEN. To make POT applicable to such scenarios, we draw inspiration from the work (Tanwisuth et al., ) and construct the class prototypes with the neural network weights $\mathbf{W}\in\mathcal{R}^{d\times C}$ of the classification layer $f$. Each column of the weight matrix $\mathbf{W}$ corresponds to a $d$-dimensional class prototype. The underlying idea is that the process of learning class prototypes with learnable parameters in the latent space, is closely similar to the training process of the linear classification layer. In other words, obtaining prototypes in this way does not require any training data and additional parameters.

To verify the effectiveness of POT in such scenarios, we compare it to the baseline methods not requiring access to the training dataset. We report the average AUROC for both Far-OOD and Near-OOD datasets in Table 5. The results show that POT continues to outperform all competitors across all metrics. Notably, compared to GEN, POT significantly reduces the average FPR95 by 35.78% on the Far-OOD datasets and by 9.21% on the Near-OOD datasets of the CIFAR-100 benchmark.

**Table 5: OOD detection performance (AUROC$\uparrow$) of methods without the requirement for training data.**
|  | CIFAR-100 | ImageNet-1k |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Far-OOD | Near-OOD | Far-OOD | Near-OOD |  |  |  |  |
| Method | FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ |
| MSP | 58.53 | 77.83 | 54.84 | 80.21 | 51.77 | 86.03 | 81.56 | 73.56 |
| ODIN | 58.81 | 79.32 | 57.95 | 79.86 | 86.04 | 76.08 | 90.76 | 64.45 |
| EBO | 56.47 | 79.83 | 55.77 | 80.82 | 85.34 | 78.99 | 93.09 | 62.60 |
| GradNorm | 83.53 | 69.20 | 85.80 | 69.95 | 92.60 | 41.75 | 94.68 | 39.45 |
| ReAct | 54.07 | 80.45 | 56.47 | 80.69 | 53.96 | 85.68 | 84.42 | 69.34 |
| MLS | 56.60 | 79.73 | 55.64 | 80.97 | 78.91 | 83.55 | 92.06 | 68.42 |
| RankFeat | 69.38 | 67.15 | 80.57 | 61.84 | / | / | / | / |
| ASH | 59.09 | 80.61 | 65.77 | 78.13 | 96.69 | 51.26 | 95.07 | 53.60 |
| GEN | 57.04 | 79.59 | 54.23 | 81.27 | 32.16 | 91.34 | 70.66 | 76.33 |
| POT | 21.26 | 94.27 | 45.02 | 84.76 | 29.40 | 93.35 | 65.82 | 80.42 |

Figure: Figure 3: Ablation study on the effect of virtual outliers. We contrast the distribution for the transport cost score without virtual outliers (a & c) and the contrastive transport cost score with virtual outliers (b & d). The used models are ResNet-18 for CIFAR-100 and ViT-b16 for ImageNet-1k, respectively. The introduction of virtual outliers makes a more distinguishable score, leading to enhanced OOD detection performance.
Refer to caption: x2.png

### 4.3 Ablation study and analysis.

Ablation on virtual outliers.
In this ablation, we compare the OOD detection performance of POT with and without virtual outliers, which are generated to approximate the OOD region using linear representation extrapolation. Figure 3 displays the results, where we visualize the data distribution in the CIFAR-100 and ImageNet-1k benchmarks. Using the virtual outliers leads to clearer separability between ID and OOD samples, whereas the transport cost score without virtual outliers exhibits larger overlap. The results show that the introduction of virtual outliers enhances the distinguishability between ID and OOD, resulting in more effective OOD detection.

Parameter analysis. We ablate along individual parameters with ImageNet-1k as ID data in Figure 4, where the ViT-b16 is utilized as the backbone for analysis. In Figure 4 (a), we find that increasing the test batch size is beneficial for OOD detection. Although POT faces a performance degradation with the decrease of test batch size, its performance of lower batch sizes still hold superiority over the baseline methods. For instance, POT already achieves 93.18% average AUROC with the test batch size of 32, which outperforms the competitive rival NAC-UE (see Table 1). In Figure 4 (b), we can observe that as the regularization coefficient $\lambda$ increases, AUROC undergoes an ascending interval followed by a decrease interval, ultimately leading to convergence. In Figure 4 (c), we find that POT works better with a moderate $\omega$. This is intuitive as a lower $\omega$ may lead the generated virtual outliers close to ID samples, while a higher $\omega$ can generate virtual outliers far away both ID and OOD samples, leading the transport costs to virtual outliers indistinguishable.

Figure: Figure 4: Ablation across different parameters in POT including: (a) test batch size; (b) entropic regularization coefficient $\lambda$ of OT; (c) linear extrapolation parameter $\omega$ in generating virtual outlier.
Refer to caption: x3.png

**Table 6: OOD detection performance comparison from penultimate and logits layer.**
|  |  | Logits | Penultimate |  |  |
| --- | --- | --- | --- | --- | --- |
| Benchmarks | OOD | FPR95$\downarrow$ | AUROC$\uparrow$ | FPR95$\downarrow$ | AUROC$\uparrow$ |
|  | Far | 18.86 | 95.58 | 14.35 | 96.70 |
| CIFAR-100 | Near | 45.68 | 85.38 | 44.28 | 86.51 |
|  | Far | 20.26 | 95.49 | 15.59 | 96.78 |
| ImageNet-1k | Near | 67.97 | 80.45 | 60.15 | 83.95 |

Penultimate layer vs. logit layer.
In this paper, we follow the convention in feature-based methods by using the representations from the penultimate layer of neural network, as it is believed to preserve more information than the output from the top layer, also referred to as logits. To investigate the impact of layer selection, we provide evaluation on POT using representations from the penultimate layer or logits, with the average results presented in Table 6. From the results, using the representations from the penultimate layer achieves better performance than using the logits.

## 5 Conclusion

In this paper, we tackle the problem of OOD detection from a new perspective of measuring distribution discrepancy and quantifying the individual contribution of each test input.
To this end, we propose a novel method named POT, which utilizes the prototype-based OT to assess the discrepancy between test inputs and ID prototypes and use the obtaining transport costs for OOD detection.
By generating virtual outliers to approximate the OOD region, we combine the transport costs to ID prototypes with the costs to virtual outliers, resulting in a more effective contrastive transport cost for identifying OOD inputs.
Experimental results demonstrate that POT achieves better performance than 21 current methods.
Moreover, we show that POT is pluggable with existing training methods for OOD detection and is applicable to the scenarios where the training data is unavailable, highlighting its generic nature and high practicality.

## Appendix A Derivation of Sinkhorn-Knopp algorithm

Recall that the classical optimal transport (OT) problem, formulated as Equation 2, is a linear programming problem that incurs cubic time complexity, which is prohibitive in many applications.
To this end, we investigate a smoothed version with an entropic regularization term, formulated as Equation 4.
As a $\lambda$-strongly convex function, it has a unique optimal solution.
By introducing the associated Larangian, we can transform the primal problem with constraints into an unconstrained optimization problem, resulting in the following formulation:

$$ $\begin{split}\mathcal{L}(\bm{\gamma},\bm{\mu},\bm{\nu})&=\langle\bm{E},\bm{ \gamma}\rangle-\lambda H(\bm{\gamma})-\bm{u}^{\top}(\bm{\gamma}\bm{1}_{C}-\bm{ \mu})-\bm{v}^{\top}(\bm{Q}^{\top}\bm{1}_{m}-\bm{\nu}),\end{split}$ (11) $$

where $\bm{v}$ and $\bm{v}$ are Lagrange multipliers.
Taking partial derivation on transport plan $\gamma$ yields,

$$ $\displaystyle\frac{\partial\mathcal{L}(\bm{\gamma},\bm{u},\bm{v})}{\gamma_{ij}}$ $\displaystyle=E_{ij}+\lambda\log\gamma_{ij}-u_{i}-v_{j}=0$ (12) $\displaystyle\gamma_{ij}$ $\displaystyle=e^{(u_{i}-E_{ij}+v_{j})/\lambda}=\bm{a}_{i}K_{ij}\bm{b}_{j}$ $$

where $\bm{a}_{i}=e^{u_{i}/\lambda}$, $\bm{b}_{j}=e^{v_{j}/\lambda}$, and $\bm{K}=\text{exp}(-\bm{E}/\lambda)$.
Conveniently, the solution can be rewritten in matrix form as $\bm{\gamma}=\text{Diag}(\bm{a})\bm{K}\text{Diag}(\bm{b})$.
In this way, solving the primal problem in Equation 4 equals to finding the scaling variables $\bm{a}$ and $\bm{b}$, which must satisfy the following equations corresponding to the mass conservation constraints inherent to $\Pi(\bm{\mu},\bm{\nu})$:

$$ $\displaystyle\text{Diag}(\bm{a})\bm{K}\text{Diag}(\bm{b})\bm{1}_{m}$ $\displaystyle=\text{Diag}(\bm{a})(\bm{K}\bm{b})=\bm{\mu}$ (13) $\displaystyle\text{Diag}(\bm{b})\bm{K}^{\top}\text{Diag}(\bm{a})\bm{1}_{C}$ $\displaystyle=\text{Diag}(\bm{b})(\bm{K}^{\top}\bm{a})=\bm{\nu}$ $$

Known as a matrix scaling problem, the equation can be solved by modifying $\bm{a}$ and $\bm{b}$ iteratively:

$$ $\bm{a}\leftarrow\bm{\mu}\oslash(\bm{K}\bm{b}),\quad\bm{b}\leftarrow\bm{\nu} \oslash(\bm{K}^{\top}\bm{a}),$ (14) $$

where $\oslash$ denotes element-wise division.
The above iterative updates define the Sinkhorn-Knopp algorithm.

## Appendix B Full Experimental Results on ImageNet

**Table 7: Far-OOD detection performance on ImageNet-1k benchmark. We report the AUROC$\uparrow$ scores over two backbones, i.e., ResNet-50 and Vit-b16).**
| Method | Vit-b16 | ResNet-50 |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| iNaturalist | OpenImage-O | Textures | Average | iNaturalist | OpenImage-O | Textures | Average |  |
| OpenMax | 94.91 | 87.37 | 85.54 | 89.27 | 92.03 | 87.59 | 88.13 | 89.25 |
| MSP | 88.16 | 84.85 | 85.09 | 86.03 | 88.43 | 84.99 | 82.48 | 85.30 |
| ODIN | 79.53 | 71.46 | 77.24 | 76.08 | 91.13 | 88.27 | 89.01 | 89.47 |
| MDS | 96.00 | 92.35 | 89.39 | 92.58 | 63.67 | 68.74 | 89.72 | 74.04 |
| MDSEns | / | / | / | / | 61.95 | 61.14 | 80.02 | 67.71 |
| RMDS | 96.09 | 92.29 | 89.38 | 92.59 | 87.27 | 85.73 | 86.07 | 86.36 |
| EBO | 79.26 | 76.49 | 81.22 | 78.99 | 90.61 | 89.15 | 88.74 | 89.50 |
| GradNorm | 42.36 | 37.83 | 45.05 | 41.75 | 93.85 | 85.11 | 92.08 | 90.35 |
| ReAct | 86.09 | 84.26 | 86.69 | 85.68 | 96.32 | 91.88 | 92.80 | 93.67 |
| MLS | 85.25 | 81.61 | 83.79 | 83.55 | 91.15 | 89.26 | 88.42 | 89.61 |
| KLM | 89.54 | 86.96 | 86.51 | 87.67 | 90.77 | 87.35 | 84.69 | 87.60 |
| VIM | 95.71 | 92.15 | 90.61 | 92.82 | 89.54 | 90.40 | 97.96 | 92.63 |
| KNN | 91.45 | 89.83 | 91.12 | 90.80 | 86.31 | 86.78 | 97.06 | 90.05 |
| DICE | 82.55 | 82.33 | 82.26 | 82.38 | 92.50 | 88.46 | 92.07 | 91.01 |
| RankFeat | / | / | / | / | 40.10 | 50.94 | 70.96 | 54.00 |
| ASH | 50.47 | 55.45 | 47.87 | 51.26 | 97.04 | 93.31 | 96.91 | 95.76 |
| SHE | 93.56 | 91.03 | 92.67 | 92.42 | 92.58 | 86.70 | 93.63 | 90.97 |
| GEN | 93.53 | 90.25 | 90.25 | 91.34 | 92.44 | 89.35 | 87.63 | 89.81 |
| DDE | 97.10 | 88.97 | 88.96 | 91.68 | 83.64 | 71.38 | 86.84 | 80.62 |
| NAC-UE | 93.69 | 91.54 | 94.17 | 93.13 | 96.43 | 91.61 | 97.77 | 95.27 |
| POT | 99.38 | 95.60 | 95.36 | 96.78 | 99.45 | 93.45 | 95.58 | 96.16 |

**Table 8: Near-OOD detection performance on ImageNet-1k benchmark. We report the AUROC$\uparrow$ scores over two backbones, i.e., ResNet-50 and Vit-b16.**
| Method | Vit-b16 | ResNet-50 |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| SSB-hard | NINCO | Average | SSB-hard | NINCO | Average |  |
| OpenMax | 68.81 | 78.68 | 73.74 | 71.30 | 78.12 | 74.71 |
| MSP | 69.03 | 78.09 | 73.56 | 72.16 | 79.98 | 76.07 |
| ODIN | 63.69 | 65.20 | 64.45 | 71.84 | 77.73 | 74.78 |
| MDS | 71.45 | 86.49 | 78.97 | 47.15 | 62.19 | 54.67 |
| MDSEns | / | / | / | 44.89 | 55.64 | 50.27 |
| RMDS | 72.79 | 87.28 | 80.04 | 71.47 | 82.22 | 76.85 |
| EBO | 59.16 | 66.05 | 62.60 | 72.35 | 79.70 | 76.03 |
| GradNorm | 43.27 | 35.63 | 39.45 | 72.83 | 73.93 | 73.38 |
| ReAct | 63.24 | 75.45 | 69.34 | 73.09 | 81.71 | 77.40 |
| MLS | 64.45 | 72.39 | 68.42 | 72.75 | 80.41 | 76.58 |
| KLM | 68.11 | 80.64 | 74.37 | 71.19 | 81.87 | 76.53 |
| VIM | 69.34 | 84.63 | 76.99 | 65.10 | 78.54 | 71.82 |
| KNN | 65.93 | 82.22 | 74.07 | 61.78 | 79.41 | 70.60 |
| DICE | 60.01 | 71.91 | 65.96 | 70.84 | 75.98 | 73.41 |
| RankFeat | / | / | / | 55.87 | 46.10 | 50.98 |
| ASH | 54.12 | 53.07 | 53.60 | 73.11 | 83.37 | 78.24 |
| SHE | 68.11 | 84.16 | 76.13 | 71.83 | 76.42 | 74.13 |
| GEN | 70.19 | 82.47 | 76.33 | 72.11 | 81.71 | 76.91 |
| DDE | 76.29 | 81.32 | 78.81 | 56.52 | 62.93 | 59.72 |
| NAC | 68.04 | 82.45 | 75.25 | 68.21 | 81.16 | 74.68 |
| POT | 80.39 | 87.51 | 83.95 | 83.37 | 78.13 | 80.75 |

## Peer Reviews

confidence: 5
contribution: 3
initial_rating: 3
presentation: 2
questions: - Are you skeptical about the effectiveness of obtaining OOD data areas through simple linear extrapolation?
- Is it realistic for OOD detection to use test data for linear extrapolation?
soundness: 2
strengths: - This paper provides a new perspective on the OOD detection problem by combining the concepts of prototype and optimal transmission.
- The author conducted exhaustive experiments on multiple standard datasets, comparing the performance of POT with 21 existing technologies. The experiments are adequately designed and the results are clearly interpreted.
summary: Summary:This paper proposes a new prototype-based optimal transfer (POT) method to detect out-of-distribution (OOD) data by measuring the optimal total transfer cost between the test sample and the ID prototype and the virtual outlier. In addition, this paper also proposes to obtain the virtual outlier of OOD by simple linear extrapolation, and demonstrates its effectiveness by comparing with the existing technology. The paper has a clear structure and rigorous logic, covering relevant background, methods, experimental evaluation and results analysis.
weaknesses: - Formulas 2 and 3 in the paper mention that the calculation of the transmission plan and cost matrix comes from the ID prototype and test samples. The simple linear extrapolation to obtain the OOD area is also based on the test samples. This solution is not realistic under the task setting of OOD detection.
- The paper mentions that the OOD area is obtained by simple linear extrapolation, but based on the author's description and existing research, the difficulty of OOD samples lies in the approximate ID features in the feature space. The OOD area obtained by simple linear extrapolation may not be representative, and there is a lack of discussion on the interpretability of the process.
- The results of the paper method should also be compared with the results of some of the latest papers.

---

confidence: 4
contribution: 4
initial_rating: 6
presentation: 3
questions: 1.	Can the authors provide more discussions on the experimental results and ablation studies?
2.	Can the authors show the execute time of the proposed method compared with other methods?
soundness: 4
strengths: 1. This paper addresses the OOD detection problem by measuring distribution discrepancy and proposes a novel approach using prototype-based optimal transport.
2. To identify OOD data with smaller distribution shifts from ID data, this paper proposes to generate virtual outliers to approximate the OOD region using representation linear representation extrapolation.
3. Comprehensive experimental results on various benchmark datasets demonstrate the effectiveness of the proposed detection method.
summary: This paper focuses on detecting Out-of-Distribution (OOD) inputs, which is a crucial task for improving the reliability of deep neural networks. Since there exists an inherent distribution shift between ID and OOD data, the authors propose a novel method Prototype-based Optimal Transport (POT) which utilizes the prototype-based optimal transport to assess the discrepancy between test inputs and ID prototypes and use the obtaining transport costs for OOD detection. Since relying solely on the cost of ID prototypes is insufficient for discerning OOD data with smaller distribution shifts from ID data, they propose to generate virtual outliers to approximate the OOD region using representation linear representation extrapolation. Finally, experiments on several benchmark datasets are provided.
weaknesses: 1. The experimental results and ablation studies require more explanation and discussion, rather than just describing the phenomena.
2. Although this paper reduces computational complexity by introducing the entropy regularization term and the Sinkhorn-Knopp algorithm, the optimal transport problem is still computationally intensive on large-scale datasets. This may limit the application of the proposed method POT in resource-constrained environments.
3. Is the constraint w<0 in Eq. (8) necessary? Because there is no corresponding description in this paper, and this case was not considered in the parameter analysis.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: - The performance will depend on the feature encoder used. Is the method robust to the choice of the feature encoder? How large is the impact of the encoder? 
- What if we use more than one class prototypes for each class? 
- What are the \mathcal{P} and \mathcal{M} in Line 241? Better to formally specify them.
- Eqn (9): the second "+" should be ":"?
- Table 1, Textures FPR95, NAC-UE result: should be underlined rather than boldfaced.
- Maybe also underline the second best results in Table 2, so as to show the performance margin and be consistent with Table 1.
- Figure 3 shows the ablation result on virtual outliers on Far-OOD datasets. What about ablation study on Near-OOD datasets?
soundness: 3
strengths: - The work addressed an important topic of OOD detection, and designed a pratical method.
- The proposed method used two key techniques: optimal transport and virtual outliers via linear extrapolation. The work illustrate the insights behind these techniques quite clearly.
- The experiments are quite extensive and the results are quite convincing, showing the strong performance and investigating important aspects of the method like robustness to hyperparameters.
summary: The work considered the problem of OOD (Out-Of-Distribution) detection, and proposed a method based on optimal transport from test inputs to ID (in-distribution) prototypes. It first constructed the prototype of each class as the average representation for that class extracted from the feature encoder, then computed the optimal transport between the encoded feature of the test inputs and the class prototypes and used the sum of the transport costs from a test input to prototypes as an indication of the input's outlier-ness. It then noted that only using this is insufficient for detecting OOD data closer to ID, and proposed to generate virtual outliers by linear extrapolation between the prototypes and the mean of the test inputs, and finally used the difference between the transport cost to the ID prototypes and the cost to the virtual outliers as the final OOD score.

The work then performed extensive experiments. The results showed that the proposed method often outperforms the competitors and gets close-to-best performance otherwise. It can be used with different pretraining schemes without graceful performance degradation, and can use the network weights as the ID prototypes when training data are not available. Ablation study shows that the virtual outliers are important for the performance, and that the method is robust to hyperparameters: the extrapolation coefficient, the regularization coefficient, and the test batch size.
weaknesses: - See the questions below.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: Why do you claim that your method measures differences directly in the latent space? It is clear that you are also using features obtained from the feature extractor. When comparing with DDE, you state that DDE measures differences in the representation space, which is influenced by the transformations of the DNN features, while POT measures differences in the latent space.

I will adjust the score based on your response.
soundness: 3
strengths: 1.	Explore a new OOD score by measuring the discrepancy between test images and ID prototypes from the perspective of optimal transport, which is well-supported theoretically.
2.	Use linear extrapolation to regularize the decision boundary, offering a successful perspective on applying optimal transport in OOD detection.
summary: This paper addresses the inherent distributional differences between ID and OOD samples by proposing the use of optimal transport (OT) to measure the discrepancy between test inputs and ID prototypes, a method termed Prototype-based Optimal Transport (POT). The authors assess distances by calculating the transport cost from each test input to the ID prototypes. Additionally, they apply linear extrapolation to generate virtual outliers, effectively regularizing the decision boundary. By evaluating distributional differences between ID and OOD samples through optimal transport, this work introduces a novel perspective. Furthermore, the use of linear extrapolation enables optimal transport to function effectively within this framework.
weaknesses: 1.	In the use of OT, calculating ν for test samples in Equation (3) requires all test samples, which is impractical in real-world applications.
2.	When generating outliers via linear extrapolation, the method appears to rely on information from the test set, which is also impractical.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: (1) Can the author specify how the validation set is built in the OT formulation, especially the test batch size in Section 4.3? Specifically, my impression is that we only have 1 unique transport map from the validation set and we will use that for detecting OOD samples. As such, I'm not sure how varying the test batch size affect the result. Are we computing a distinct transport map for every test batch size? if so how do we decide the threshold for OOD samples for these test batch size? What if their statistic (portion between ID and OOD data) changes?

(2) How to apply this idea when the OOD set is small? How the validation set size affect the performance? 

(3) What is the fundamental idea that OT is better than other methods such as likelihood estimation?
soundness: 1
strengths: - The comparison with other baseline methods appears comprehensive and thorough.
summary: This paper introduces an approach to out-of-distribution (OOD) detection leveraging optimal transport (OT). The core idea is based on the observation that OOD inputs typically incur higher transport costs when mapped to the in-distribution (ID) in latent space. From this, they develop an OOD detection method by first compute the optimal transport plan from the test distribution to the ID distribution. OOD detection is then achieved by calculating the transport cost for each input and applying a threshold to determine whether the input is OOD. The author then benchmark their method with varied number of datasets.
weaknesses: One of my concern is that  the paper lacks a deeper understanding for why OT would be better than other OOD detection methods. Standard approaches have increasingly focused on understanding geometric representations in neural networks, either through uncertainty estimation [1] or, more rigorously, likelihood estimation [2], sometimes aided by reconstruction autoencoders [3]. Given the connection between geometric representation and OT through the transport map, I would expect interpretative insights along this line rather than a direct application of OT for OOD detection. Without this, the current proposal, in my opinion, is incremental compared to the existing distance-based approaches.

The method presentation is not clear. I have some confusion in terms of how the OT plan is constructed using both in-distribution (ID) inputs and out-of-distribution (OOD) datasets. See question (1).

Finally, similar ideas of using optimal transport to characterize OOD error have been explored with more concrete theoretically properties, as in [4].

[1] Cao, Senqi, and Zhongfei Zhang. "Deep hybrid models for out-of-distribution detection." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[2] Kamkari, Hamidreza, et al. "A Geometric Explanation of the Likelihood OOD Detection Paradox." Forty-first International Conference on Machine Learning.

[3] Zhou, Yibo. "Rethinking reconstruction autoencoder-based out-of-distribution detection." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[4] Lu, Yuzhe, et al. 'Characterizing out-of-distribution error via optimal transport.' Advances in Neural Information Processing Systems 36 (2023): 17602-17622."

Meta-review: This submission focuses on out-of-distribution (OOD) detection and proposes a novel method based on prototypes and optimal transport. Specifically, the method first calculates the optimal transport distance from each test batch to the in-distribution (ID) prototypes. To improve performance, the distance to virtual outliers, generated through linear extrapolation, is then incorporated into the OOD detection score.

One key weakness highlighted by reviewer #7P4j concerns the problem setting, where the proposed method relies on a minibatch of test samples to obtain the OOD detection scores. This approach is impractical for real-world applications and differs from existing OOD detection methods. Additionally, other reviewers, such as #zWXM, expressed similar concerns. The Area Chair concurs with these points and suggests that the authors address this issue in a future submission.

Decision: Reject