SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $b_{0}, b_{1}$  and  $b_{2}$  are uniform constants.
Algorithm 1 Distributional-Augmented OOD Learning (DAL)
Input: ID and OOD samples from  $D_{X_1Y_1}$  and  $D_{X_A}$ ;  
for st = 1 to num_step do  
Sample  $S_B$  and  $T_B$  from  $D_{X_1Y_1}$  and  $D_{X_A}$ ; Initialize  $\mathbf{p}^i \sim \mathcal{N}(\mathbf{0}, \sigma I)$ ,  $\forall i \in \{1, \dots, |T_B|\}$ ;  
for se = 1 to num_search do  
 $\psi^i = \nabla_{\mathbf{p}^i} \left[ \ell_{\mathrm{OE}} \left( \mathbf{h}(\mathbf{e}(\mathbf{x}_A^i + \mathbf{p}^i); \mathbf{e}(\mathbf{x}_A^i)) - \gamma \| \mathbf{p}^i \|_1 \right], \forall i \in \{1, \dots, |T_B|\} \right]$ ;  
 $\mathbf{p}^i \gets \mathbf{p}^i + \mathbf{ps} \psi^i$ ,  $\forall i \in \{1, \dots, |T_B|\}$   
end for  
 $\gamma \gets \min \left( \max \left( \gamma - \beta (\rho - \frac{1}{|T_B|} \sum_{i=1}^{|T_B|} \| \mathbf{p}^i \|, \gamma_{\max}), 0 \right) \right)$ ;  
 $\mathbf{w} \gets \mathbf{w} - 1r\nabla_{\mathbf{w}} \left[ \frac{1}{|T_B|} \sum_{i=1}^{|T_B|} \ell_{\mathrm{OE}}(\mathbf{h}(\mathbf{g}(\mathbf{x}_A^i) + \mathbf{p}^i)) + \alpha \frac{1}{|S_B|} \sum_{i=1}^{|S_B|} \ell(\mathbf{f}_{\mathbf{w}}; \mathbf{x}_I^i, y_I^i) \right]$ ;  
end for  
Output: model parameter w.
Furthermore, under proper conditions, one can show that the bound in Eq. (11) can attain  $\mathcal{O}(1 / \sqrt{n}) + \mathcal{O}(1 / \sqrt{m})$ , i.e.,  $R_{D}(\widehat{\mathbf{w}};\rho) - \min_{\mathbf{w}\in \mathcal{W}}R_{D}(\mathbf{w};\rho)\leq \mathcal{O}(1 / \sqrt{n}) + \mathcal{O}(1 / \sqrt{m})$ . Corollary 1 in Appendix C.5 gives an example to support the above claim. Next, we give a learning bound to estimate the expected detection risk in Eq. (2) w.r.t. the model  $\mathbf{f}_{\widehat{\mathbf{w}}}$  given by Eq. (8).
Theorem 3 (Risk Estimation). Given the same conditions in Theorem 2 and let  $\widehat{\mathbf{w}}$  be the solution of Eq. (8), which is given by  $\widehat{\mathbf{w}} \in \arg \min_{\mathbf{w} \in \mathcal{W}} \widehat{R}_D(\mathbf{w}; \rho)$ . If  $\ell_{\mathrm{OE}}(\mathbf{f}_{\mathbf{w}}; \mathbf{x})$  is  $L_c$ -Lipschitz w.r.t.  $c(\cdot, \cdot)$ , i.e.,  $|\ell_{\mathrm{OE}}(\mathbf{f}_{\mathbf{w}}; \mathbf{x}) - \ell_{\mathrm{OE}}(\mathbf{f}_{\mathbf{w}}; \mathbf{x}')| \leq L_c c(\mathbf{x}, \mathbf{x}')$ , then with the probability at least  $1 - 4e^{-t} > 0$ ,
$$
R _ {D} (\widehat {\mathbf {w}}) - \overbrace {\min  _ {\mathbf {w} \in \mathcal {W}} R _ {D} (\mathbf {w} ; \rho)} ^ {\text {a p p r o x i m a t e r i s k}} \leq \underbrace {\alpha L _ {c} \max  \left\{\mathrm {W} _ {c} \left(D _ {X _ {\mathrm {O}}}, D _ {X _ {\mathrm {A}}}\right) - \rho , 0 \right\} + \epsilon (n , m ; t)} _ {\text {e s t i m a t i o n e r r o r}},
$$
for any  $\rho > 0$ , where  $\epsilon(n, m; t)$  is defined in Theorem 2.
The bias term  $\alpha L_{c}\max \{\mathrm{W}_{c}(D_{X_{\mathrm{O}}},D_{X_{\mathrm{A}}}) - \rho ,0\} = 0$  when  $\rho$  is large enough. Hence, a large  $\rho$  implies a small estimation error. Although a larger  $\rho$  leads to better generalization ability, the approximate risk  $\min_{\mathbf{w}\in \mathcal{W}}R_D(\mathbf{w};\rho)$  may become larger. It implies that for practical effectiveness, i.e., small  $R_{D}(\widehat{\mathbf{w}})$ , there is a trade-off between the approximate risk  $\min_{\mathbf{w}\in \mathcal{W}}R_{D}(\mathbf{w};\rho)$  and the bias  $\alpha L_{c}\max \{\mathrm{W}_{c}(D_{X_{\mathrm{O}}},D_{X_{\mathrm{A}}}) - \rho ,0\}$  across different choices of  $\rho$ . Hence, we need to choose a proper  $\rho$  for open-world detection with unseen data (cf., Section 5.3).
4.2 Proposed Algorithm
In this section, we introduce the algorithm design for DAL, summarized in Algorithm 1. Due to the space limit, we provide further discussions in Appendix E.
Losses and Cost Function. Following Hendrycks et al. (2019), we adopt the cross entropy loss to realize  $\ell$  and the KL-divergence between model predictions and uniform distribution for  $\ell_{\mathrm{OE}}$ . We also define the cost function  $c$  by the  $l_{1}$  norm, namely,  $c(\mathbf{x},\mathbf{x}^{\prime}) = \| \mathbf{x} - \mathbf{x}^{\prime}\|_{1}$ .
Algorithm Design. By Theorem 1, we can address the primary problem in Eq. (8) by the dual problem in Eq. (9). Additionally, following Du et al. (2022a), we perturb for the worst OOD data in the embedding space. Denote the model  $\mathbf{f}_{\mathbf{w}} = \mathbf{h} \circ \mathbf{e}$  with  $\mathbf{h}$  the classifier and  $\mathbf{e}$  the feature extractor, we find the perturbation  $\mathbf{p}$  for the embedding features, i.e.,  $\mathbf{e}(\mathbf{x})$ , of the associated data  $\mathbf{x}$ . The perturbation  $\mathbf{p}$  should lead to the worst OOD case for the surrogate function in Theorem 1, namely,
$$
\phi_ {\gamma} (\mathbf {w}; \mathbf {e} (\mathbf {x})) = \sup  _ {\mathbf {p} \in \mathcal {E}} \left\{\ell_ {\mathrm {O E}} (\mathbf {h} (\mathbf {e} (\mathbf {x}) + \mathbf {p}); \mathbf {e} (\mathbf {x})) - \gamma \| \mathbf {p} \| _ {1} \right\},
$$
where  $\mathcal{E}$  denotes the space of embedding features. Note that we abuse the definition of  $\ell_{\mathrm{OE}}$ , emphasizing that we perturb the embedding features of  $\mathbf{e}(\mathbf{x})$  by  $\mathbf{p}$ .
Training and Inference. Our definition of  $\phi_{\gamma}(\mathbf{w};\mathbf{e}(\mathbf{x}))$  leads to a particular realization of Eq. (10), which is the learning objective of our DAL. It can be solved by stochastic gradient optimization for deep models, e.g., mini-batch stochastic gradient descent. After training, we use the MSP scoring function by default and discuss the possibility of other scoring functions in Appendix F.3.
Stochastic Realization. Algorithm 1 gives a stochastic realization of DAL, where ID and auxiliary OOD mini-batches are randomly sampled in each stochastic iteration, denoted by  $S_{\mathrm{B}}$  and  $T_{\mathrm{B}}$  respectively. Therein, we first find the perturbation  $\mathbf{p}$  that leads to the maximal  $\phi_{\gamma}(\mathbf{w}, \mathbf{e}(\mathbf{x}))$ . The value of  $\mathbf{p}$  is initialized by random Gaussian noise with the standard deviation  $\sigma$  and updated by gradient ascent for num_search steps with the perturbation strength  $\mathbf{ps}$ . Then we update  $\gamma$  by one step of gradient descent with the learning rate  $\beta$ , and further clipping between 0 and  $\gamma_{\max}$  to avoid extreme values. Finally, given the proper perturbations for the auxiliary OOD data in  $T_{\mathrm{B}}$ , we update the model parameter  $\mathbf{w}$  by one step of mini-batch gradient descent.
In this section, we mainly test DAL on the CIFAR (Krizhevsky and Hinton, 2009) benchmarks (as ID datasets). To begin with, we introduce the evaluation setups.
OD Datasets. We adopt the 80 Million Tiny Images (Torralba et al., 2008) as the auxiliary OOD dataset; Textures (Cimpoi et al., 2014), SVHN (Netzer et al., 2011), Places365 (Zhou et al., 2018), LSUN (Yu et al., 2015), and iSUN (Xu et al., 2015) as the (test-time) real OOD datasets. We eliminate those data whose labels coincide with ID cases.
Pre-training Setups. We employ Wide ResNet-40-2 (Zagoruyko and Komodakis, 2016) trained for 200 epochs via empirical risk minimization, with a batch size 64, momentum 0.9, and initial learning rate 0.1. The learning rate is divided by 10 after 100 and 150 epochs.
Hyper-parameters Tuning Strategy. The hyper-parameters are tuned based on the validation data, separated from the training ID and auxiliary OOD data, which is a common strategy in OOD detection with outlier exposure field (Hendrycks et al., 2019; Chen et al., 2021). Specifically, we fix  $\sigma = 0.001$ , num_search = 10, and adopt the grid search to choose  $\gamma_{\mathrm{max}}$  from  $\{0.1, 0.5, 1, 5, 10, 50\}$ ;  $\beta$  from  $\{1e^{-3}, 5e^{-3}, 1e^{-2}, 5e^{-2}, 1e^{-1}, 5e^{-1}, 1, 5\}$ ;  $\rho$  from  $\{1e^{-2}, 1e^{-1}, 1, 10, 100\}$ ; ps from  $\{1e^{-3}, 1e^{-2}, 1e^{-1}, 1, 10, 100\}$ ;  $\alpha$  from  $\{0.1, 0.5, 1.0, 1.5, 2.0\}$ .
Hyper-parameters Setups. For CIFAR-10, DAL is run for 50 epochs with the ID batch size 128, the OOD batch size 256, the initial learning rate 0.07,  $\gamma_{\mathrm{max}} = 10$ ,  $\beta = 0.01$ ,  $\rho = 10$ ,  $\mathsf{ps} = 1$ , and  $\alpha = 1$ . For CIFAR-100, DAL is run for 50 epochs with the ID batch size 128, the OOD batch size 256, the initial learning rate 0.07,  $\gamma_{\mathrm{max}} = 10$ ,  $\beta = 0.005$ ,  $\rho = 10$ , and  $\mathsf{ps} = 1$ , and  $\alpha = 1$ . For both cases, we employ cosine decay (Loshchilov and Hutter, 2017) for the model learning rate.
Baseline Methods. We compare DAL with representative methods, including MSP (Hendrycks and Gimpel, 2017), Free Energy (Liu et al., 2020), ASH (Djurisic et al., 2023), ReAct (Sun et al., 2021), Mahalanobis (Lee et al., 2018a), KNN (Sun et al., 2022), KNN+ (Sun et al., 2022), CSI (Tack et al., 2020), VOS (Du et al., 2022a), Outlier Exposure (OE) (Hendrycks et al., 2019), Energy-OE (Liu et al., 2020), ATOM (Chen et al., 2021), DOE (Wang et al., 2023), and POEM (Ming et al., 2022). We adopt their suggested setups but unify the backbones for fairness.
Evaluation Metrics. The detection performance is evaluated via two representative metrics, which are both threshold-independent: the false positive rate of OOD data when the true positive rate of ID data is at  $95\%$  (FPR95); and the area under the receiver operating characteristic curve (AUROC), which can be viewed as the probability of the ID case having greater score than that of the OOD case.
Due to the space limit, we test our DAL with more advanced scoring strategies in Appendix F.3 and conduct experiments on the more complex ImageNet (Deng et al., 2009) dataset in Appendix F.10.
5.1 Main Results
The main results are summarized in Table 1, where we report the detailed results across the considered real OOD datasets. First, we reveal that using auxiliary OOD data can generally lead to better results than using only ID information, indicating that outlier exposure remains a promising direction worth studying. However, as demonstrated in Section 3, the OOD distribution discrepancy can hurt its open-world detection power, while previous works typically oversee such an important issue. Therefore, our DAL, which can alleviate the OOD distribution discrepancy, reveals a large improvement over the original outlier exposure. Specifically, comparing with the conventional outlier exposure, our method reveals 1.99 and 0.13 average improvements w.r.t. FPR95 and AUROC on the CIFAR-10 dataset, and 13.46 and 3.65 of the average improvements on CIFAR-100 dataset. For advanced works that consider the OOD sampling strategies, e.g., ATOM and POEM, DAL can achieve much better results, especially for the CIFAR-100 case. The reason is that these methods mainly consider the situations
Table 1: Comparison between our method and advanced methods on the CIFAR benchmarks.  $\downarrow$  (or  $\uparrow$ ) indicates smaller (or larger) values are preferred, and a bold font indicates the best result in a column. Methods are grouped based on 1) using ID data only and 2) using additional information about auxiliary OOD data. Two groups are separated by the horizontal line for each ID case.
Method	SVHN	LSUN	iSUN	Textures	Places365	Average
FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑
CIFAR-10
Using ID data only
MSP	48.89	91.97	25.53	96.49	56.44	89.86	59.68	88.42	60.19	88.36	50.15	91.02
Free Energy	35.21	91.24	4.42	99.06	33.84	92.56	52.46	85.35	40.11	90.02	33.21	91.64
ASH	33.98	91.79	4.76	98.98	34.38	92.64	50.90	86.07	40.89	89.79	32.98	91.85
Mahalanobis	12.21	97.70	57.25	89.58	79.74	77.87	15.20	95.40	68.81	82.39	46.64	88.59
KNN	26.56	95.93	27.52	95.43	33.55	93.15	37.62	93.07	41.67	91.21	33.38	93.76
KNN+	3.28	99.33	2.24	98.90	17.85	95.65	10.87	97.72	30.63	94.98	12.97	97.32
CSI	17.37	97.69	6.75	98.46	12.58	97.95	25.65	94.70	40.00	92.05	20.47	96.17
VOS	36.55	93.30	9.98	98.03	28.93	94.25	52.83	85.74	39.56	89.71	33.57	92.21
Using ID data and auxiliary OOD data
OE	2.36	99.27	1.15	99.68	2.48	99.34	5.35	98.88	11.99	97.23	4.67	98.88
Energy-OE	0.97	99.54	1.00	99.15	2.32	99.27	3.42	99.18	9.57	97.44	3.46	98.91
ATOM	1.00	99.59	0.61	99.53	2.15	99.40	2.52	99.10	7.93	97.27	2.84	98.97
DOE	1.80	99.37	0.25	99.65	2.00	99.36	5.65	98.75	10.15	97.28	3.97	98.88
POEM	1.20	99.53	0.80	99.10	1.47	99.26	2.93	99.13	7.65	97.35	2.81	98.87
DAL	0.80	99.65	0.90	99.46	1.70	99.34	2.30	99.14	7.65	97.45	2.68	99.01
CIFAR-100
Using ID data only
MSP	84.39	71.18	60.36	85.59	82.63	75.69	83.32	73.59	82.37	73.69	78.61	75.95
Free Energy	85.24	73.71	23.05	95.89	81.11	79.02	79.63	76.35	80.18	75.65	69.84	80.12
ASH	70.09	83.56	13.20	97.71	69.87	82.56	63.69	83.59	79.70	74.87	59.31	84.46
Mahalanobis	51.00	88.70	91.60	69.69	38.48	91.86	47.07	89.09	82.70	74.18	72.37	82.70
KNN	52.10	88.83	68.82	79.00	42.17	90.59	42.79	89.07	92.21	61.08	59.62	81.71
KNN+	32.50	93.86	47.41	84.93	39.82	91.12	43.05	88.55	63.26	79.28	45.20	87.55
CSI	64.50	84.62	25.88	95.93	70.62	80.83	61.50	86.74	83.08	77.11	61.12	95.05
VOS	78.06	92.59	40.40	92.90	85.77	70.20	82.46	77.22	82.31	75.47	73.80	91.67
Using ID data and auxiliary OOD data
OE	46.73	90.54	16.30	96.98	47.97	88.43	50.39	88.27	54.30	87.11	43.14	90.27
Energy-OE	35.34	94.74	16.27	97.25	33.21	93.25	46.13	90.62	50.45	90.04	36.28	93.18
ATOM	24.80	95.15	17.83	96.76	47.83	91.06	44.86	91.80	53.92	88.88	37.84	92.73
DOE	43.10	91.83	13.95	97.56	47.25	87.88	49.40	88.62	51.05	88.08	40.95	90.79
POEM	22.27	96.28	13.66	97.52	42.46	91.97	45.94	90.42	49.50	90.21	34.77	93.28
DAL	19.35	96.21	16.05	96.78	26.05	94.23	37.60	91.57	49.35	90.81	29.68	93.92
where the model capacity is not enough to learn from all the auxiliary OOD data, deviating from our considered issue in OOD distribution discrepancy. Moreover, for the previous works that adopt similar concepts in the worst-case OOD learning, e.g., VOS and DOE, DAL also reveals better results, with 1.29 and 30.89 improvements on the CIFAR-10 dataset and 11.27 and 44.12 improvements on the CIFAR-100 dataset w.r.t. FPR95. It indicates that our theoretical-driven scheme can also guide the algorithm designs with practical effectiveness. Note that many previous works use advanced scoring strategies other than MSP, and thus our experiment above is not completely fair to us. Therefore, in Appendix F.3, we also combine DAL with many advanced scoring strategies other than MSP, which can further improve our performance.
5.2 Hard OOD Detection
We further consider hard OOD scenarios (Sun et al., 2022), of which the test OOD data are very similar to that of the ID cases. Following the common setup (Sun et al., 2022) with the CIFAR-10 dataset being the ID case, we evaluate our DAL on three hard OOD datasets, namely, LSUN-Fix (Yu et al., 2015), ImageNet-Resize (Deng et al., 2009), and CIFAR-100. Note that data in ImageNet-Resize (1000 classes) with the same semantic space as TinyImageNet (200 classes) are removed. We select a set of strong baselines that
are competent in hard OOD detection, summarizing the experiments in Table 2. As we can see, our method can beat these advanced methods across the considered datasets, even for the challenging
Table 2: Comparison between our method and advanced methods on hard OOD detection.  $\downarrow$  (or  $\uparrow$  ) indicates smaller (or larger) values are preferred, and a bold font indicates the best result in a column.
Methods	LSUN-Fix	ImageNet-Resize	CIFAR-100
FPR95 ↓	AUROC ↑	FPR95 ↓	AUROC ↑	FPR95 ↓	AUROC ↑
Using ID data only
Free Energy	6.42	98.85	46.46	89.02	50.47	87.08
ASH	4.00	98.20	46.18	88.85	54.31	83.71
KNN+	24.88	95.75	30.52	94.85	40.00	89.11
CSI	39.79	93.63	37.47	93.93	45.64	87.64
Using ID data and auxiliary OOD data
OE	1.75	99.47	6.76	98.58	29.40	94.20
DOE	1.97	98.71	5.98	98.75	29.75	94.24
POEM	1.24	98.93	6.56	98.37	35.11	91.80
DAL	1.39	99.47	5.60	98.80	25.45	94.34
Figure 2: Illustrations of embedding features for the ID, the auxiliary OOD (A-OOD), and the worst OOD (W-OOD) data. We adopt the t-SNE visualization on the CIFAR-10 dataset and illustrate the results before and after DAL training.
Figure 3: FPR95 curves across various  $\rho$  on the CIFAR-100. We report both the results for the average real OOD data (REAL) and the auxiliary OOD data (AUXI), where we consider two hyperparameter setups, i.e.,  $ps = 1$  and  $ps = 10$ .
CIFAR-10 versus CIFAR-100 setting. The reason is that our distributional augmentation directly learns from OOD data close to ID pattern, which can cover hard OOD cases.
We further conduct an ablation study to demonstrate two mechanisms that mainly contribute to our open-world effectiveness, namely, OOD data generation and Wasserstein ball constraint.
OD Data Generation. DAL learns from the worst OOD data to mitigate the OOD distribution discrepancy. To understand such an OOD generation scheme, we employ the t-SNE visualization (Van der Maaten and Hinton, 2008) for the ID, the auxiliary OOD, and the worst OOD data. Figure 2 summarizes the results before and after DAL training. Before training, the ID and auxiliary OOD data overlap largely, indicating that the original model is not effective at distinguishing between them. Then, DAL does not directly train the model on auxiliary OOD data but instead perturbs it to further confuse the model beyond the overlap region. After DAL training, the overlap region between ID and auxiliary OOD data shrinks. Additionally, perturbing the original OOD data becomes more difficult, indicating that the model has learned to handle various worst-case OOD scenarios.
Wasserstein Ball Constraint. The choice of  $\rho$  determines the radius of the Wasserstein ball. Larger values of  $\rho$  reduce estimation error and improve model generalization, as stated in Theorem 3. However, larger values of  $\rho$  also increase the approximate risk  $\min_{\mathbf{w} \in \mathcal{W}} R_D(\mathbf{w}; \rho)$  as it becomes more challenging to ensure uniform model performance with increased distributional perturbation. Figure 3 shows the FPR95 curves on the CIFAR-100 dataset for both the real and the surrogate OOD data, revealing the trade-off in selecting  $\rho$ . Here, we consider two setups of  $\rho$ , i.e.,  $\mathrm{ps} = 1$  (default) and  $\mathrm{ps} = 10$  (large perturbation strength). First, when the perturbation strength is very large (i.e.,  $\mathrm{ps} = 10$ ), the model can easily fail for training if the value of  $\rho$  is also large (e.g.,  $\rho = 100$ ), indicating that large value of  $\rho$  can lead to a large approximation error. However, such an issue can be overcome by selecting a relatively small value of  $\rho$  (e.g.,  $\rho = 0.5$ ).
Outlier exposure is one of the most powerful methods in OOD detection, but the discrepancy between the auxiliary and (unseen) real OOD data can hinder its practical effectiveness. To address such an issue, we have formalized it as the OOD distribution discrepancy and developed an effective learning framework to mitigate its negative impacts. Specifically, we consider a specific distribution set that contains all distributions in a Wasserstein ball centered on the auxiliary OOD distribution. Then, models trained over worst-case OOD data in the ball can ensure improved performance toward open-world OOD detection. Overall, as pioneers in critically analyzing the open-world setting with theoretical analysis, we are committed to raising attention to the OOD distribution discrepancy issue and encouraging further research in this direction.
QZW, YGZ, and BH were supported by the NSFC Young Scientists Fund No. 62006202, NSFC General Program No. 62376235, Guangdong Basic and Applied Basic Research Foundation No. 2022A1515011652, HKBU Faculty Niche Research Areas No. RC-FNRA-IG/22-23/SCI/04, and HKBU CSD Departmental Incentive Scheme. FL was supported by Australian Research Council (ARC) under Award No. DP230101540, and by NSF and CSIRO Responsible AI Program under Award No. 2303037. YXL was supported by the AFOSR Young Investigator Program under award number FA9550-23-1-0184, National Science Foundation (NSF) Award No. IIS-2237037 & IIS-2331669, and Office of Naval Research under grant number N00014-23-1-2643.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017.
Zhen Fang, Yixuan Li, Jie Lu, Jiahua Dong, Bo Han, and Feng Liu. Is out-of-distribution detection learnable? In NeurIPS, 2022.
Saikiran Bulusu, Bhavya Kailkhura, Bo Li, P Varshney, and Dawn Song. Anomalous instance detection in deep learning: A survey. Technical report, Lawrence Livermore National Lab., 2020.
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: a survey. arXiv preprint arXiv:2110.11334, 2021.
Tianshi Cao, Chin-Wei Huang, David Yu-Tung Hui, and Joseph Paul Cohen. A benchmark of medical out of distribution detection. arXiv preprint arXiv:2007.04250, 2020.
Zheyan Shen, Jiashuo Liu, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of-distribution generalization: A survey. arXiv preprint arXiv:2108.13624, 2021.
Dan Hendrycks, Mantas Mazeika, and Thomas G. Dietterich. Deep anomaly detection with outlier exposure. In ICLR, 2019.
Yifei Ming, Ying Fan, and Yixuan Li. POEM: Out-of-distribution detection with posterior sampling. In ICML, 2022.
Qizhou Wang, Junjie Ye, Feng Liu, Quanyu Dai, Marcus Kalander, Tongliang Liu, Jianye Hao, and Bo Han. Out-of-distribution detection with implicit outlier transformation. In ICLR, 2023.
Cédric Villani. Topics in Optimal Transportation. American Mathematical Society, 2021.
Cédric Villani. Optimal Transport: Old and New. Springer, 2008.
Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. VOS: Learning what you don't know by virtual outlier synthesis. In ICLR, 2022a.
Akshay Mehra, Bhavya Kailkhura, Pin-Yu Chen, and Jihun Hamm. On certifying and improving generalization to unseen domains. arXiv preprint arXiv:2206.12364, 2022.
Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. ATOM: Robustifying out-of-distribution detection using outlier mining. In ECML, 2021.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS, 2018a.
Weitang Liu, Xiaoyun Wang, John D Owens, and Yixuan Li. Energy-based out-of-distribution detection. In NeurIPS, 2020.
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In ICML, 2022.
Aman Sinha, Hongseok Namkoong, and John C. Duchi. Certifying some distributional robustness with principled adversarial training. In ICLR, 2018.