SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. In CS 231N, 2015.
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrated classifiers for detecting out-of-distribution samples. In ICLR, 2018a.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS, 2018b.
Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In ICLR, 2018.
Ziqian Lin, Sreya Dutta Roy, and Yixuan Li. Mood: Multi-level out-of-distribution detection. In CVPR, 2021.
Weitang Liu, Xiaoyun Wang, John D Owens, and Yixuan Li. Energy-based out-of-distribution detection. In NeurIPS, 2020.
Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In ICLR, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
Christopher D. Manning and Hinrich Schütze. Foundations of Statistical Natural Language Processing. The MIT Press, 1999.
Yifei Ming, Ying Fan, and Yixuan Li. Poem: Out-of-distribution detection with posterior sampling. In ICML, 2022.
Sina Mohseni, Mandar Pitale, J. B. S. Yadawa, and Zhangyang Wang. Self-supervised learning for generalizable out-of-distribution detection. In AAAI, 2020.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR, 2015.
Aristotelis-Angelos Papadopoulos, Mohammad Reza Rajati, Nazim Shaikh, and Jiamian Wang. Outlier exposure with confidence control for out-of-distribution detection. Neurocomputing, 2021.
Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, and Lihi Zelnik-Manor. Imagenet-21k pretraining for the masses. In NeurIPS Datasets and Benchmarks Track, 2021.
Mohammadreza Salehi, Hossein Mirzaei, Dan Hendrycks, Yixuan Li, Mohammad Hossein Rohban, and Mohammad Sabokrou. A unified survey on anomaly, novelty, open-set, and out-of-distribution detection: Solutions and future challenges. In arXiv, 2021.
Vikash Sehwag, Mung Chiang, and Prateek Mittal. SSD: A unified framework for self-supervised outlier detection. In ICLR, 2021.
Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribution detection. In ECCV, 2022.
Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of-distribution detection with rectified activations. In NeurIPS, 2021.
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In ICML, 2022.
Yiyou Sun, Zhenmei Shi, Yingyu Liang, and Yixuan Li. When and how does known class help discover unknown ones? provable understandings through spectral analysis. In ICML, 2023.
Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. CSI: Novelty detection via contrastive learning on distributionally shifted instances. In NeurIPS, 2020.
Leitian Tao, Xuefeng Du, Xiaojin Zhu, and Yixuan Li. Non-parametric outlier synthesis. In ICLR, 2023.
William R Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two sample. Biometrika, 1933.
Qizhou Wang, Junjie Ye, Feng Liu, Quanyu Dai, Marcus Kalander, Tongliang Liu, Jianye HAO, and Bo Han. Out-of-distribution detection with implicit outlier transformation. In ICLR, 2023.
Zitai Wang, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, and Qingming Huang. Opena: Towards aoc-oriented open-set recognition. In NeurIPS, 2022.
Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li. Mitigating neural network overconfidence with logit normalization. In ICML, 2022.
Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulkarni, and Jianxiong Xiao. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. In arXiv, 2015.
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. In arXiv, 2021.
Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, WENXUAN PENG, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, et al. Openood: Benchmarking generalized out-of-distribution detection. In NeurIPS Datasets and Benchmarks Track, 2022.
Zhiyong Yang, Qianqian Xu, Shilong Bao, Peisong Wen, Yuan He, Xiaochun Cao, and Qingming Huang. Auc-oriented domain adaptation: From theory to algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. In arXiv, 2015.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In CVPR, 2019.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019.
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. Mixup: Beyond empirical risk minimization. In ICLR, 2018.
Jingyang Zhang, Nathan Inkawich, Randolph Linderman, Yiran Chen, and Hai Li. Mixture outlier exposure: Towards out-of-distribution detection in fine-grained environments. In WACV, 2023.
Bolei Zhou, Agata Lapedriza, Antonio Torralba, and Aude Oliva. Places: An image database for deep scene understanding. Journal of Vision, 2017.
Jianing Zhu, Hengzhuang Li, Jiangchao Yao, Tongliang Liu, Jianliang Xu, and Bo Han. Unleashing mask: Explore the intrinsic out-of-distribution detection capability. In ICML, 2023.
Appendix for DivOE
The whole Appendix is organized as follows. In Appendix A, we present the detailed definitions and implementation of those post-hoc score functions and several OE-based methods that are considered in our exploration. In Appendix B, we formally present the theoretical analysis about DivOE. In Appendix C, we provide our extra experimental details and more comprehensive results with further discussion on the underlying implications. Finally, in Appendix D, we discuss the potential broader impact and limitations of our work.
Below we summarize several important aspects to facilitate reproducible results:
- Datasets. The datasets we used are all publicly accessible, which is introduced in Section 4.1. For methods involving auxiliary outliers, we strictly follow previous works [Liu et al., 2020, Sun et al., 2021] to avoid overlap between the auxiliary dataset (Tiny-ImageNet) and any other OOD datasets.
- Assumption. We set our experiments to a fine-tuning scenario where a well-trained model on the original classification task is available, and the training samples are also available for subsequent fine-tuning [Hendrycks et al., 2019].
- Open source. We open our source code in the repository: DivOE
- Environment. All experiments are conducted with multiple runs on NVIDIA GeForce RTX 3090 GPUs with Python 3.7 and PyTorch 1.12.
A Details about Considered Baselines and Metrics
In this section, we provide the details about the baselines for the scoring functions and fine-tuning with auxiliary outliers, as well as the corresponding hyper-parameters and other related metrics that are considered in our work. Additionally, we provide further discussion on the comparison with two related works, e.g., MixOE and DOE, and also present the critical differences between the diversifying with previous concept of informative.
Maximum Softmax Probability (MSP). [Hendrycks and Gimpel, 2017] proposes to use maximum softmax probability to discriminate ID and OOD samples. The score is defined as follows,
$$
S _ {\mathrm {M S P}} (x; f) = \max  _ {c} P (y = c | x; f) = \max  \operatorname {s o f t m a x} (f (x)), \tag {12}
$$
where  $f$  represents the given well-trained model and  $c$  is one of the classes  $\mathcal{Y} = \{1, \dots, C\}$ . The larger softmax score indicates the larger probability for a sample to be ID data, reflecting the model's confidence on the sample.
ODIN. [Liang et al., 2018] designed the ODIN score, leveraging the temperature scaling and tiny perturbations to widen the gap between the distributions of ID and OOD samples. The ODIN score is defined as follows,
$$
S _ {\mathrm {O D I N}} (x; f) = \max  _ {c} P (y = c | \tilde {x}; f) = \max  \operatorname {s o f t m a x} \left(\frac {f (\tilde {x})}{T}\right), \tag {13}
$$
where  $\tilde{x}$  represents the perturbed samples (controlled by  $\epsilon$ ),  $T$  represents the temperature. For fair comparison, we adopt the suggested hyperparameters [Liang et al., 2018]:  $\epsilon = 1.4 \times 10^{-3}$ ,  $T = 1.0 \times 10^{4}$ .
Mahalanobis. [Lee et al., 2018b] introduces a Mahalanobis distance-based confidence score, exploiting the feature space of the neural networks by inspecting the class conditional Gaussian distributions. The Mahalanobis distance score is defined as follows,
$$
S _ {\text {M a h a l a n o b i s}} (x; f) = \max  _ {c} - \left(f (x) - \hat {\mu} _ {c}\right) ^ {T} \hat {\Sigma} ^ {- 1} \left(f (x) - \hat {\mu} _ {c}\right), \tag {14}
$$
where  $\hat{\mu}_c$  represents the estimated mean of multivariate Gaussian distribution of class  $c$ ,  $\hat{\Sigma}$  represents the estimated tied covariance of the  $C$  class-conditional Gaussian distributions.
Energy. [Liu et al., 2020] proposes to use the Energy of the predicted logits to distinguish the ID and OOD samples. The Energy score is defined as follows,
$$
S _ {\text {E n e r g y}} (x; f) = - T \log \sum_ {c = 1} ^ {C} e ^ {f (x) _ {c} / T}, \tag {15}
$$
where  $T$  represents the temperature parameter. As theoretically illustrated in Liu et al. [2020], a lower Energy score indicates a higher probability for a sample to be ID. Following [Liu et al., 2020], we fix the  $T$  to 1.0 throughout all experiments.
ASH. [Djurisic et al., 2023] designs a extremely simple, post-hoc method called Activation SHaping for OOD detection. It removes a large portion of an input's activation at a late layer and adjusts the rest of the activation values by scaling them up or assigning them a constant value. The simplified representation is then passed throughout the rest of the network. The logit output is used to classify ID samples and calculate scores for OOD detection as usual. We adopt the energy score and apply the ASH-S version with the hyperparameter  $p = 95$  for CIFAR-10 and  $p = 85$  for CIFAR-100, as suggested by [Djurisic et al., 2023].
Outlier Exposure (OE). [Hendrycks et al., 2019] initiates a promising approach towards OOD detections by involving outliers to force apart the distributions of ID and OOD samples. In the experiments, we use the cross-entropy from  $f(x_{\mathrm{out}})$  to the uniform distribution as the  $\mathcal{L}_{\mathrm{OE}}$  [Lee et al., 2018a],
$$
\mathcal {L} _ {f} = \mathbb {E} _ {\mathcal {D} _ {\text {i n}}} \left[ \ell_ {\mathrm {C E}} (f (x), y) \right] + \lambda \mathbb {E} _ {\mathcal {D} _ {\text {o u t}} ^ {\mathrm {s}}} \left[ \log \sum_ {c = 1} ^ {C} e ^ {f (x) _ {c}} - \mathbb {E} _ {\mathcal {D} _ {\text {o u t}} ^ {\mathrm {s}}} (f (x)) \right]. \tag {16}
$$
Energy (w.  $\mathcal{D}_{\mathrm{aux}}$ ). In addition to using the Energy as a post-hoc score to distinguish ID and OOD samples, [Liu et al., 2020] proposes an Energy-bounded objective to further separate the two distributions. The OE objective is as follows,
$$
\mathcal {L} _ {\mathrm {O E}} = \mathbb {E} _ {\mathcal {D} _ {\text {i n}} ^ {\mathrm {s}}} \left(\max  \left(0, S _ {\text {E n e r g y}} (x, f) - m _ {\text {i n}}\right)\right) ^ {2} + \mathbb {E} _ {\mathcal {D} _ {\text {o u t}} ^ {\mathrm {s}}} \left(\max  \left(0, m _ {\text {o u t}} - S _ {\text {E n e r g y}} (x, f)\right)\right) ^ {2}. \tag {17}
$$
We keep the thresholds same to [Liu et al., 2020]:  $m_{\mathrm{in}} = -23.0$  for CIFAR-10 and  $m_{\mathrm{in}} = -25.0$  for CIFAR-100,  $m_{\mathrm{out}} = -5.0$  for both CIFAR-10 and CIFAR-100.
NTOM. [Chen et al., 2021] greedily exploits informative auxiliary data to tighten the decision boundary between ID and OOD samples. It samples hard outliers with lower OOD scores to regularize the model for OOD detection. For a fair comparison, we adopt the energy score for both outlier sampling and model training and choose the suggested sampling parameter  $p = 0$  [Chen et al., 2021].
POEM. [Ming et al., 2022] explores the Thompson sampling strategy [Thompson, 1933] to make the most use of outliers to learn a tight decision boundary. The details of Thompson sampling can refer to Ming et al. [2022].
MixOE. [Zhang et al., 2023] proposes to mix ID data and auxiliary outliers to cover the broad region where fine-grained OOD samples lie. The model is then trained such that the prediction confidence linearly declines as the input shifts from ID distribution to OOD distribution. Considering [Zhang et al., 2023] chooses different OOD test datasets from our work, we set the hyperparameters  $\alpha = 1$  and  $\beta = 0.1$  which are verified to be optimal by our tuning experiments.
DOE. [Wang et al., 2023] proposes to implicitly synthesize virtual outliers via model perturbation, which enhances the generalization of OE to better optimize the learning target [Foret et al., 2021].
A.1 A Closer Look at Comparison with MixOE and DOE
In this part, we discuss more about the difference of our DOE with previous MixOE and DOE.
Conceptually, DivOE has a different underlying motivation from MixOE and DOE. Our DivOE is proposed for diversifying outlier exposure by extrapolating auxiliary outliers. In comparison, MixOE focuses on enhancing the fine-grained OOD detection by interpolation between ID samples and OOD
samples; DOE focuses on improving the generalization of the original outlier exposure by exploring model-level perturbation.
Technically, DivOE has a different algorithmic design from the others. Specifically, DivOE was proposed as a general framework for outlier extrapolation, which bases on OOD samples. It is different from MixOE which directly mix-up ID and OOD samples; It is also different from DOE that constructs the model perturbation for better optimization. It is worthy noting our DivOE is from a different perspective (data-level) compared with DOE (model-level), and they are orthogonal and combinable. DOE regards regularizing training on the same outliers with different weight perturbations as implicitly synthesized but does not change the outliers. The optimization of DOE is constrained by the original training samples and is also sensitive to tuning the model perturbation.
Table 3: Comparison of DivOE and DOE across different ID classification tasks.
Dataset	Method	FPR95↓	AUROC↑	AUPR↑	ID-ACC↑
ImageNet	OE	61.91	81.57	95.49	75.90
DivOE (OE backbone)	60.01	81.96	95.59	75.74
DOE	65.90	79.32	90.61	75.38
DivOE (DOE backbone)	64.23	79.46	91.21	75.10
SVHN	OE	3.99	99.33	99.85	91.86
DivOE (OE backbone)	2.24	99.60	99.91	91.72
DOE	8.93	97.47	99.44	91.11
DivOE (DOE backbone)	4.24	98.68	99.62	91.50
CIFAR100	OE	27.67	91.89	97.93	75.41
DivOE (OE backbone)	24.80	92.91	98.22	75.24
DOE	30.69	93.13	98.39	71.41
DivOE (DOE backbone)	20.26	95.41	98.91	71.43
CIFAR10	OE	13.76	97.53	99.43	94.51
DivOE (OE backbone)	11.66	97.82	99.48	94.66
DOE	7.39	98.29	99.58	92.82
DivOE (DOE backbone)	6.51	98.60	99.65	93.45
Empirically, as shown in Table 1, we have directly compared MixOE with our DivOE. The results show its performance is worse than our DivOE or even the original OE. For DOE, since it is from a different perspective and orthogonal, we compare DOE mainly in Table 2 of the original submission, and here we present Table 3 for further discussion. It shows: (1) DOE is sensitive to the ID classification task and even performs worse than the original OE in CIFAR-100/SVHN/ImageNet; (2) DivOE improves both OE and DOE under the corresponding backbones; (3) Even under unfair comparison between DivOE (OE backbone) vs. DOE, DivOE (OE backbone) still achieves the comparable or better performance on ImageNet/SVHN/CIFAR100 datasets.
A.2 Conceptual Comparison of Diversified and Informative
In this part, we provide further discussion on the comparison of diversified and informative.
Conceptually, "diversified" is proposed to characterize one internal property of auxiliary outliers, while the "informative" in [Chen et al., 2021] is a concept for relative comparison. The intuition behind "diversified" is that we hope the outliers can span more informative areas beyond the original ones, unlike sampling the most "informative" outliers (i.e., near-boundary OOD samples that are close to ID data). Intuitively, Group 1 in Figure 2 indicates the auxiliary outliers which span more informative areas among those other groups. In other words, the most diversified Group 1 also covers the areas of other groups. Here we also utilize the Maximum Mean Discrepancy (MMD) [Borgwardt et al., 2006] to show the difference between "diversified" in our submission and "informative" in Table 4. We measure the discrepancy between the sampled/manipulated outliers with the original ones, termed as MMD[new, original], to indicate the "informative"; measure the discrepancy half by half within the sampled/manipulated outliers, termed as MMD[half new, half new], to indicate the "diversified". The results show that DivOE owns lower "informative" due to the partial outlier extrapolation but higher "diversified" on the inner dispersion of the outliers.
Technically, in our DivOE, "diversified" guides the training to expand the auxiliary outliers to the extrapolated distributions but also consider the original ones. We re-summarized the results
Table 4: "Diversified" and "Informative" measured by MMD scores.
Dataset	Method	MMD[ new, original] (informative)	MMD[ half new, half new] (diversified)
CIFAR100	OE	-	0.059 ± 0.001
NTOM*	0.032 ± 0.000	0.056 ± 0.000
ATOM	0.088 ± 0.001	0.056 ± 0.000
DivOE	0.053 ± 0.001	0.113 ± 0.002
CIFAR10	OE	-	0.052 ± 0.000
NTOM*	0.030 ± 0.000	0.049 ± 0.001
ATOM	0.285 ± 0.005	0.069 ± 0.001
DivOE	0.118 ± 0.001	0.304 ± 0.007
of Figure 4(a) to Table 5 and show that our DivOE with the pursuit of "diversified" (conduct partial extrapolation) performs better than those totally considered "informative" (conduct total extrapolation). The results demonstrate the effectiveness and rationality of DivOE considering partial extrapolation for diversification.
Table 5: Comparison of DivOE pursuing "diversified" and "informative"
Dataset	Method	FPR95↓	AUROC↑	AUPR↑	ID-ACC↑
CIFAR100	OE	27.67	91.89	97.93	75.41
DivOE (r=1.0 for "informative")	43.18	88.11	97.11	75.43
DivOE (r=0.5 for "diversified")	24.80	92.91	98.22	75.24
CIFAR10	OE	13.76	97.53	99.43	94.51
DivOE (r=1.0 for "informative")	22.05	96.24	99.14	94.81
DivOE (r=0.5 for "diversified")	11.66	97.82	99.48	94.66
B Theoretical Analysis and Discussion
In this section, we formally present the theoretical implications with detailed definitions and assumptions. The complete theoretical analysis is based on the perspectives of sample complexity and OOD distribution gap in the previous works [Ming et al., 2022, Fang et al., 2022]. Different from the existing methods, we focus on the informative extrapolation proposed in our DivOE for diversifying the given auxiliary outliers towards the ID decision area. Intuitively, we provide an illustration of the conceptual comparison in Figure 5 to explain the differences. Different from the previous sampling-based methods which assume that the sampling space of the surrogate OOD distribution can be broad enough to cover the OOD decision area, our DivOE is targeted to conduct informative extrapolation to extend the given outlier exposure towards the ID decision area. In the following parts, we first introduce some preliminary setups and notations for the analyses.
Preliminary setups and notations. Following the prior works [Lee et al., 2018a, Sehwag et al., 2021, Ming et al., 2022], we assume the extracted feature approximately follows a Gaussian mixture model (GMM) with the equal class priors as  $\frac{1}{2}\mathcal{N}(\mu, \sigma^2\mathcal{I}) + \frac{1}{2}\mathcal{N}(-\mu, \sigma^2\mathcal{I})$ . To be specific,  $\mathcal{D}_{\mathrm{in}} = \mathcal{N}(\mu, \sigma^2\mathcal{I})$  and  $\mathcal{D}_{\mathrm{out}}^s = \mathcal{N}(-\mu, \sigma^2\mathcal{I})$ . Considering the hypothesis class as  $\mathcal{H} = \mathrm{sign}(\theta^T x), \theta \in \mathbb{R}^d$ . The classifier outputs 1 if  $x \sim \mathcal{D}_{\mathrm{in}}$  and outputs -1 if  $x \sim \mathcal{D}_{\mathrm{out}}^s$ .
First, we introduce the assumption about informative extrapolation on  $\mathcal{D}_{\mathrm{out}}^s$ . The assumption is empirically supported to be valid by the previous exploration on the informative extrapolation in Figures 2 and 3, where the extrapolated outliers are more informative and close to the decision boundary towards ID samples.
Assumption B.1 (Informative Extrapolation on  $\mathcal{D}_{\mathrm{out}}^s$ ). Considering the extrapolated version  $\mathcal{D}_{\mathrm{out}}^e$  of the  $\mathcal{D}_{\mathrm{out}}^s$  in our proposed DivOE, we assume that the data points  $x \sim \mathcal{D}_{\mathrm{out}}^e$  satisfy the extended constraint based on the boundary scores  $-|f_{\mathrm{outlier}}(x)|$  defined in POEM Ming et al. [2022]:
$$
\sum_ {i = 1} ^ {n} f _ {\text {o u t l i e r}} \leq (\alpha - \tau (\epsilon)) n, \tag {18}
$$
Figure 5: Illustration about the conceptual comparison of the previous assumption on auxiliary outliers in OE and that studied in our work. Left panel: the benign assumption on the auxiliary outliers, in which the surrogate OOD distribution  $\mathcal{D}_{\mathrm{out}}^s$  can be broad enough to cover the informative boundary outliers close to the decision area of ID data (in  $\mathcal{D}_{\mathrm{in}}$ ). Right panel: a more general assumption on the auxiliary outliers, considering that they can not well represent the unseen OOD distribution  $\mathcal{D}_{\mathrm{out}}$  since it is impractical or even infeasible to accurately pre-define and collect those boundary outliers.
where the  $f_{\mathrm{outlier}}$  is a function parameterized by some unknown ground truth weights and maps the high-dimensional input  $\mathbf{x}$  into a scalar. Generally, it represents the discrepancy between the  $\mathcal{D}_{\mathrm{out}}^e$  and the true  $\mathcal{D}_{\mathrm{out}}$ , indicated with the constraint  $\alpha - \tau(\epsilon)$  results from the informative extrapolation. The  $\tau(\epsilon)$  represents the extended diversity, which is positively related to the extrapolated strength  $\epsilon$  in general.
Given the above assumption, we can derive the following extended lemma based on that adopted in POEM [Ming et al., 2022].
Lemma B.2 (Constraint of Extrapolated  $\mathcal{D}_{\mathrm{out}}^e$ ). Assume the data points  $x \sim$  extrapolated  $\mathcal{D}_{\mathrm{out}}^e$  satisfy the following constraint for resulting in the following boundary margin:  $\sum_{i=1}^{n} |2x_i^T \mu| \leq n\sigma^2(\alpha - \tau(\epsilon))$ .
Proof of Lemma B.2. Given the Gaussian mixture model described in the previous setup, we can obtain the following expression by Bayes' rule of  $\mathbb{P}(\text{outlier}|x)$ ,
$$
\mathbb {P} (\text {o u t l i e r} | x) = \frac {\mathbb {P} (x | \text {o u t l i e r}) \mathbb {P} (\text {o u t l i e r})}{\mathbb {P} (x)} = \frac {1}{1 + e ^ {- \frac {1}{2 \sigma^ {2}} \left(d _ {\text {o u t l i e r}} (x) - d _ {\text {i n}} (x)\right)}}, \tag {19}
$$
where  $d_{\mathrm{outlier}}(x)) = (x + \mu)^{\top}(x + \mu), d_{\mathrm{in}}(x) = (x - \mu)^{\top}(x - \mu)$ , and  $\mathbb{P}(\mathrm{outlier}|x) = \frac{1}{1 + e^{-f_{\mathrm{outlier}}(x)}}$  according to its definition. Then we have:
$$
- f _ {\text {o u t l i e r}} = - \frac {1}{2 \sigma^ {2}} \left(d _ {\text {o u t l i e r}} (x) - d _ {\text {i n}} (x)\right), \tag {20}
$$
$$
- \left| f _ {\text {o u t l i e r}} \right| = - \frac {1}{2 \sigma^ {2}} \left| (x - \mu) ^ {\top} (x - \mu) - (x + \mu) ^ {\top} (x + \mu) \right| = - \frac {2}{\sigma^ {2}} \left| x ^ {\top} \mu \right|. \tag {21}
$$
Therefore, we can get the constraint as:  $\sum_{i = 1}^{n}|2x_i^T\mu |\leq n\sigma^2 (\alpha -\tau (\epsilon))$
With the previous assumption and lemma, then we present the analysis as below.