SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 13: OOD detection performance on CIFAR-100 [Krizhevsky, 2009] as ID. All methods are trained on the same backbone. Values are percentages.  $\uparrow$  indicates larger values are better, and  $\downarrow$  indicates smaller values are better. (averaged by multiple trials)
Method	OOD dataset	ID ACC
Textures	Places365	iSUN	LSUN-r	LSUN-c
FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑
MSP	82.90	73.53	82.31	74.08	83.28	75.10	83.06	74.08	66.41	83.85	75.96
ODIN	76.50	74.83	81.17	74.20	60.64	85.08	62.15	84.38	36.20	93.31	75.96
Mahalanobis	38.99	90.64	88.58	67.92	25.33	94.57	20.89	96.05	91.26	69.73	75.96
Energy	79.70	76.34	80.48	75.78	82.44	77.87	81.01	77.67	35.69	93.43	75.96
GradNorm	88.31	60.52	97.22	53.15	99.20	43.52	99.26	38.83	41.12	91.96	75.96
OE	54.11	84.69	77.42	76.27	0.07	99.99	0.04	100.00	4.58	99.06	75.50
Energy (w. Daux)	56.93	81.22	80.53	72.76	0.00	100.00	0.00	100.00	0.52	99.88	75.14
VOS	78.50	74.02	82.21	73.40	85.24	75.86	83.59	75.58	25.65	94.88	74.94
NTOM	59.65	81.76	80.40	72.46	0.00	100.00	0.00	100.00	2.24	99.46	75.31
POEM	51.72	84.16	79.94	72.84	0.00	100.00	0.00	100.00	0.34	99.92	75.88
MixOE	61.78	82.13	72.95	78.20	2.96	99.32	1.05	99.75	33.56	92.88	75.77
DivOE (Ours)	34.16	90.61	74.42	77.53	0.18	99.94	0.03	99.99	12.97	97.10	75.35
performance improvement of DivOE on the Textures and Places365 (that OOD test sets that the original OE does not perform well) exactly verifies that the informatively extrapolated outliers better represent the unseen OOD inputs that are close to the ID decision area.
Experiments on Sample Comparison It may be hard to align the sample number as different methods have different synthesis motivations and but we can actually present the number of generated samples. To facilitate the comparison, we summarize the specific outlier numbers that were manipulated in the adopted algorithm with its performance in OOD detection in Table 14. The results show that, compared with previous methods, either need go through all the outliers for sampling or change the training on all the auxiliary outliers, our DivOE can conduct extrapolation on a small portion of original outliers but achieve generally better performance across different ID classification tasks.
Table 14: Numbers of outliers manipulated or sampled during training
Dataset	Method	Numbers	FPR95↓	AUROC↑	AUPR↑	ID-ACC↑
CIFAR100	OE	0	27.67	91.89	97.93	75.41
NTOM	50000	27.58	91.08	97.52	75.41
POEM	50000	26.50	91.33	97.58	75.74
MixOE	50000	35.26	90.31	97.58	75.74
DivOE	25000	24.80	92.91	98.22	75.24
CIFAR100	OE	0	13.76	97.53	99.43	94.51
NTOM	50000	17.76	91.08	96.40	95.04
POEM	50000	17.81	90.96	96.36	94.90
MixOE	50000	13.57	97.47	99.39	94.66
DivOE	25000	11.66	97.82	99.48	94.66
Experiments on Smaller Number of Auxiliary Samples. Here, we added the experiments under different numbers of auxiliary outliers used for all OE-based methods. The experimental results are summarized in Table 15, which indicates that the performance of OOD detection can be generally better if we use more outliers. Under smaller number of samples, i.e., from 50000 to 1000 (relatively smaller than 50000 ID data), DivOE can consistently perform better than OE.
Table 15: Performance comparison by changing the numbers of auxiliary outliers.
Dataset	Method	Number (ratio to ID samples)	FPR95	AUROC	AUPR	ID-ACC
CIFAR100	OE	26.72	91.12	97.51	75.58
NTOM	29.18	90.72	97.45	75.77
POEM	50000 (1.0)	27.20	90.79	97.41	75.77
MixOE	35.28	90.29	97.57	75.67
DivOE	26.27	92.71	98.18	75.34
OE	33.64	89.98	97.36	75.92
NTOM	33.23	88.92	96.86	75.87
POEM	5000 (0.1)	33.27	88.59	96.73	75.64
MixOE	37.32	89.39	97.31	76.09
DivOE	32.80	90.89	97.71	75.62
OE	38.40	88.93	97.17	76.14
NTOM	38.11	89.12	97.45	76.12
POEM	1000 (0.02)	37.69	89.47	97.56	75.98
MixOE	40.78	87.98	96.89	76.10
DivOE	36.74	89.63	97.36	76.08
CIFAR10	OE	12.64	97.76	99.47	94.85
NTOM	19.84	90.12	96.23	94.80
POEM	50000 (1.0)	20.21	90.15	96.10	94.97
MixOE	14.26	97.31	99.35	94.92
DivOE	12.04	97.76	99.47	94.80
OE	15.16	97.31	99.35	94.95
NTOM	28.91	84.26	93.40	94.63
POEM	5000 (0.1)	27.46	86.56	94.68	94.78
MixOE	15.66	97.06	99.29	94.98
DivOE	14.41	97.43	99.40	94.81
OE	16.74	97.11	99.31	95.03
NTOM	31.90	81.93	91.99	94.86
POEM	1000 (0.02)	30.67	82.56	92.33	94.78
MixOE	15.80	97.07	99.29	95.11
DivOE	15.38	97.25	99.37	95.03
Experiments on Comprehensive Explorations of the Extrapolated Data In Tables 20 and 21, we present a more comprehensive comparison of the extrapolated outliers under the learning framework of our DivOE. To be more specific, we control the specific extrapolation ratio  $r$  and the extrapolated strength  $\epsilon$  to characterize the algorithm. From one perspective, we gradually increase the extrapolated strength  $\epsilon$  with the fixed extrapolation ratio  $r = 1$ , which means we conduct the full-batch manipulation on the original surrogate OOD distribution. The results demonstrate the failure of such an overall distribution perturbation with the significant performance drop on several OOD test sets (e.g., LSUN-r and LSUN-c). Intuitively, the overall distribution manipulation for the auxiliary outliers can not perform well towards the diversification target, as it is also inconsistent with the conceptual idea of our DivOE. From the other perspective, we decrease the extrapolation ratio  $r$  with the enlarged strength on informative extrapolation. Fortunately, with the diversified guidance introduced by our proposed objective in DivOE, we can find that even using  $\epsilon = 0.125$  can also enhance the performance of outlier exposure with different OOD test sets, resulting in a better performance in average.
Experiments on Using Different Synthetic Mechanism Here, we consider some advanced techniques using Generative Adversarial Network (GAN) to synthesizing new outliers for outlier exposure. Those methods [Lee et al., 2018a, Kong and Ramanan, 2021] additionally train generative models for generating extra outliers for data augmentation, which depends on the sample quality and actually are orthogonal to our extrapolation. We conduct experiments comparing and incorporating the method [Lee et al., 2018a] for informative extrapolation in Table 12 in attached PDF. We find that the
Table 16: Fine-grained OOD detection performance of DivOE on CIFAR-10 [Krizhevsky, 2009] as ID.
Method	OOD dataset	Average FPR95
Textures	Places365	iSUN	LSUN-r	LSUN-c
FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑
OE	21.88	96.14	45.22	91.64	0.00	100.00	0.00	100.00	1.02	99.84	13.62
DivOE (ε = 0.01, r = 1)	15.32	97.03	41.27	91.87	0.00	100.00	0.00	100.00	1.31	99.72	11.58
DivOE (ε = 0.01, r = 0.5)	14.53	97.19	44.73	91.31	0.00	100.00	0.00	100.00	2.06	99.63	12.26
DivOE (ε = 0.01, r = 0.25)	16.72	96.89	46.08	91.44	0.00	100.00	0.00	100.00	1.31	99.76	12.82
DivOE (ε = 0.01, r = 0.1)	17.35	96.91	45.96	91.63	0.00	100.00	0.00	100.00	1.32	99.79	12.93
DivOE (ε = 0.005, r = 1)	16.96	96.89	43.12	92.06	0.00	100.00	0.00	100.00	0.61	99.88	12.14
DivOE (ε = 0.005, r = 0.5)	16.85	96.94	46.57	91.36	0.00	100.00	0.00	100.00	1.07	99.81	12.90
DivOE (ε = 0.005, r = 0.1)	19.42	96.64	46.64	91.64	0.00	100.00	0.00	100.00	0.95	99.84	13.40
DivOE (ε = 0.05, r = 1)	27.61	95.64	53.79	89.86	2.78	99.55	1.87	99.68	24.20	96.44	22.05
DivOE (ε = 0.05, r = 0.5)	11.85	97.85	43.34	91.65	0.02	99.98	0.02	99.98	3.21	99.42	11.69
DivOE (ε = 0.05, r = 0.25)	12.29	97.76	42.94	91.88	0.01	99.99	0.01	99.99	2.60	99.54	11.57
DivOE (ε = 0.05, r = 0.1)	13.45	97.61	43.27	91.73	0.00	99.99	0.00	99.99	2.09	99.64	11.76
DivOE (ε = 0.125, r = 1)	37.58	94.00	56.40	89.24	5.59	99.13	4.17	99.37	24.54	96.44	25.66
DivOE (ε = 0.125, r = 0.5)	14.18	97.50	44.19	91.79	0.01	99.99	0.01	100.00	1.85	99.69	12.05
DivOE (ε = 0.125, r = 0.25)	15.92	97.33	43.76	92.03	0.00	100.00	0.00	100.00	1.39	99.78	12.21
DivOE (ε = 0.125, r = 0.1)	15.81	97.39	44.88	91.82	0.00	100.00	0.00	100.00	1.44	99.77	12.42
DivOE (1/4ε= 0.05, 1/4ε= 0.125)	12.28	97.73	44.97	91.51	0.07	99.98	0.02	99.99	3.03	99.47	12.08
Table 17: Fine-grained OOD detection performance of DivOE on CIFAR-100 as ID.
Method	OOD dataset	Average FPR95
Textures	Places365	iSUN	LSUN-r	LSUN-c
FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑
OE	54.31	84.59	77.18	76.30	0.09	99.98	0.03	100.00	4.84	99.04	27.29
DivOE (ε = 0.01, r = 1)	36.11	89.20	77.73	76.60	0.16	99.94	0.02	99.99	38.55	90.11	30.52
DivOE (ε = 0.01, r = 0.5)	36.30	89.34	78.79	76.17	0.14	99.96	0.01	99.99	30.08	93.72	29.06
DivOE (ε = 0.01, r = 0.25)	37.70	88.65	78.04	76.51	0.07	99.98	0.00	100.00	15.26	96.79	26.21
DivOE (ε = 0.01, r = 0.1)	38.89	88.23	75.94	77.05	0.08	99.98	0.00	100.00	7.92	98.23	24.57
DivOE (ε = 0.005, r = 1)	41.63	87.64	77.51	76.64	0.02	99.99	0.00	100.00	5.72	98.59	24.98
DivOE (ε = 0.005, r = 0.5)	40.33	87.90	78.21	76.15	0.05	99.98	0.00	100.00	9.38	97.97	25.59
DivOE (ε = 0.005, r = 0.1)	44.05	86.83	75.66	76.89	0.06	99.98	0.00	100.00	4.93	98.89	24.94
DivOE (ε = 0.05, r = 1)	62.51	82.89	81.42	75.20	7.43	98.26	3.69	99.17	60.99	85.11	43.21
DivOE (ε = 0.05, r = 0.5)	33.56	90.78	75.14	77.23	0.20	99.94	0.03	99.99	12.50	97.17	24.29
DivOE (ε = 0.05, r = 0.25)	35.29	90.20	74.44	77.50	0.09	99.97	0.01	99.99	11.25	97.48	24.21
DivOE (ε = 0.05, r = 0.1)	34.33	90.37	73.18	77.92	0.10	99.97	0.00	100.00	10.08	97.71	23.54
DivOE (ε = 0.125, r = 1)	71.13	80.04	81.62	74.76	19.69	95.29	13.69	96.79	62.33	84.94	49.69
DivOE (ε = 0.125, r = 0.5)	41.54	88.72	73.64	77.43	0.18	99.96	0.03	100.00	9.31	97.97	24.94
DivOE (ε = 0.125, r = 0.25)	45.16	87.92	74.25	77.66	0.10	99.98	0.00	100.00	7.29	98.46	25.36
DivOE (ε = 0.125, r = 0.1)	43.73	88.11	73.94	77.78	0.09	99.98	0.00	100.00	6.07	98.70	24.77
DivOE (½ε = 0.01 ¼ε = 0.125)	31.67	91.12	76.19	77.03	0.17	99.95	0.02	99.99	24.67	94.73	26.54
DivOE (½ε = 0.05 ¼ε = 0.125)	34.16	90.61	74.42	77.53	0.18	99.94	0.03	99.99	12.97	97.10	24.35
GAN-synthesized method is hard to train under our setups. When we incorporate DivOE into the generated sample, the performance can better enhanced than the original one of GAN-based method.
Experiments on Computational Cost To better understand the computational budget, we conduct additional experiments and summarize the time and memory results in Table 18, which shows that DivOE can achieve better performance with comparable time and memory costs with other OE-based methods. In addition, since the extra time requirement compared with the original OE is mainly from the inner-maximization of the informative extrapolation, we also explore the effect of perturbation noise on the OOD detection performance in Table 19. The results show that we can further save extra time by reducing the outlier manipulation times with maintaining the improvement.
Table 18: Time and memory cost of different methods.
Dataset	Method	Time for one iteration (s)	GPU Memory (MiB)	FPR95	AUROC	AUPR	ID-ACC
CIFAR100	OE	0.22	3769	27.67	91.89	97.93	75.41
NTOM	0.39	3775	27.58	91.08	97.52	75.41
POEM	0.79	3845	26.50	91.33	97.58	75.74
MixOE	0.53	3769	35.26	90.31	97.58	75.74
DivOE	0.57	3781	24.80	92.91	98.22	75.24
CIFAR10	OE	0.21	3769	13.76	97.53	99.43	94.51
NTOM	0.34	3733	17.76	91.08	96.40	95.04
POEM	0.66	3835	17.81	90.96	96.36	94.90
MixOE	0.47	3731	13.57	97.47	99.39	94.66
DivOE	0.53	3779	11.66	97.82	99.48	94.66
Table 19: Perturb number and average time of the extrapolation in DivOE.
Dataset	Method	Perturb Number	Average Time for One Iteration (s)	FPR95	AUROC	AUPR	ID-ACC
CIFAR100	OE	-	0.22	27.67	91.89	97.93	75.41
DivOE	1	0.41	25.58	92.60	98.14	75.59
DivOE	2	0.47	24.94	92.74	98.17	75.18
DivOE	5	0.57	24.80	92.91	98.22	75.24
CIFAR10	OE	-	0.21	13.76	97.53	99.43	94.51
DivOE	1	0.38	12.71	97.70	99.45	94.78
DivOE	2	0.44	12.10	97.68	99.45	95.59
DivOE	5	0.53	11.66	97.82	99.48	94.66
Experiments on Compatibility with other OE-based Methods In Tables 20 and 21, we provide the fine-grained performance on OOD detection of those OE-based methods [Chen et al., 2021, Ming et al., 2022, Wang et al., 2023] with our DivOE. The overall results again confirm the effectiveness of the previously proposed OE-based methods incorporating the informative extrapolation introduced by our DivOE. Especially, DivOE via the informative extrapolation helps these methods better represent the unseen OOD inputs close to the ID decision area. The overall averaged results can refer to Table 2.
Table 20: Fine-grained results of compatibility experiments with other OE-based methods on CI-FAR10 [Krizhevsky, 2009] as ID.
Method	OOD dataset	Average FPR95
Textures	Places365	iSUN	LSUN-r	LSUN-c
FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑
OE	21.88	96.14	45.22	91.64	0.00	100.00	0.00	100.00	1.02	99.84	13.62
NTOM	32.78	86.30	59.03	67.50	0.00	100.00	0.00	100.00	1.17	99.38	18.59
NTOM+DivOE	14.49	94.03	53.88	72.31	0.00	100.00	0.00	100.00	5.29	97.53	14.73
POEM	34.73	85.28	57.82	67.93	0.00	100.00	0.00	100.00	0.43	99.74	18.59
POEM+DivOE	21.61	91.32	53.95	71.28	0.0	100.00	0.00	100.00	1.48	99.10	15.41
MixOE	22.23	95.74	39.45	92.86	0.26	99.92	0.27	99.92	4.40	99.21	13.32
MixOE+DivOE	18.63	96.34	42.55	91.68	0.54	99.86	0.40	99.86	9.77	98.34	14.38
DOE	21.97	94.87	15.43	96.74	0.30	99.89	0.22	99.91	1.27	99.67	7.84
DOE+DivOE	13.37	97.05	16.28	96.53	0.25	99.92	0.14	99.95	2.00	99.49	6.41
Table 21: Fine-grained results of compatibility experiments with other OE-based methods on CI-FAR100 [Krizhevsky, 2009] as ID.
Method	OOD dataset	Average FPR95
Textures	Places365	iSUN	LSUN-r	LSUN-c
FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑
OE	54.31	84.59	77.18	76.30	0.09	99.98	0.03	100.00	4.84	99.04	27.29
NTOM	59.65	81.76	80.40	72.46	0.00	100.00	0.00	100.00	2.24	99.46	28.46
NTOM+DivOE	41.65	88.21	79.45	72.80	0.00	100.00	0.00	100.00	5.04	98.54	25.23
POEM	51.72	84.16	79.94	72.84	0.00	100.00	0.00	100.00	0.34	99.92	26.40
POEM+DivOE	43.61	87.13	80.09	72.52	0.00	100.00	0.00	100.00	1.69	99.55	25.08
MixOE	61.78	82.13	72.95	78.20	2.96	99.32	1.05	99.75	33.56	92.88	34.46
MixOE+DivOE	56.91	83.88	74.26	77.66	4.00	99.04	2.02	99.56	46.42	89.54	36.72
DOE	59.92	83.93	40.50	91.53	26.37	95.11	18.35	96.59	12.90	97.72	31.61
DOE+DivOE	43.77	89.24	40.48	91.44	0.75	99.71	0.17	99.84	17.16	97.01	20.46
Experiments with Limited Informative Auxiliary Outliers In Table 22, we present the detailed performance results of the experiments with limited informative auxiliary outliers in CIFAR-10. The experiments are designed to demonstrate the consistent effectiveness of our proposed DivOE with informative extrapolation. Different from the previous sampling-based methods (e.g., NTOM [Chen et al., 2021] and POEM [Ming et al., 2022]), which assume that the potential surrogate OOD space can be arbitrarily large and cover the boundary decision area for ID and OOD inputs, our DivOE introduces the diversification target which is more general and robust given different auxiliary outliers. With such a general learning target, DivOE can perform learning and extrapolation given the surrogate OOD distribution iteratively under a single learning framework during the fine-tuning process. Empirically, it results in better performance improvement across the different setups on the auxiliary outliers in Table 20. Since the NTOM and POEM both adopt the sampling strategy to sample part of outliers (90%), we keep the same number in OE and our DivOE for the fair comparisons.
Table 22: Experiments on limited informative auxiliary outliers on CIFAR10.
Method	Number of auxiliary outliers	FPR95↓
OE	50000	19.87
40000	19.86
30000	21.90
10000	26.19
5000	31.28
NTOM	50000	19.90
40000	22.29
30000	24.12
10000	25.89
5000	29.03
POEM	50000	20.30
40000	21.39
30000	21.77
10000	28.10
5000	27.42
DivOE	50000	15.95
40000	15.75
30000	16.82
10000	17.52
5000	20.96
Experiments on Extrapolation via Data Augmentation In Tables 24, 25, 26 and 27, we replace the detailed realization of the inner-maximization of the informative extrapolation with the conventional data augmentation method, i.e., Mixup [Zhang et al., 2018], and also explore the effectiveness of combining the CutMix [Yun et al., 2019] in the learning framework. The results also confirm that the high-level idea of our DivOE is general to adopt different data manipulation or synthetic strategies to achieve the diversification target. The averaged and fine-grained performance results demonstrate the effectiveness of the proposed DivOE, which diversifies the given outlier for effective OOD detection. In addition, we also conduct experiments and combine them with the inner-maximization function. Compared with the pixel level manipulation, more coarse-grained data augmentation can also boost the performance if it can be conducted in an optimization way. If the original outliers are far away from the ID samples, data augmentation can serve as the basis for inner-maximization.
Table 23: DivOE with different inner extrapolation methods.
Dataset	Method	Extrapolation Method	FPR95	AUROC	AUPR	ID-ACC
CIFAR100	OE	-	27.67	91.89	97.93	75.41
GAN-synthesized [c]	-	87.09	66.70	90.40	31.73
DivOE	PGD	24.80	92.91	98.22	75.24
DivOE	Mixup	25.70	92.46	98.07	76.06
DivOE	GAN-synthesized [c]	69.69	81.82	95.16	29.11
CIFAR10	OE	-	13.76	97.53	99.43	94.51
GAN-synthesized [c]	-	78.75	72.97	91.88	50.67
DivOE	PGD	11.66	97.82	99.48	94.66
DivOE	Mixup	12.87	97.59	99.42	94.83
DivOE	GAN-synthesized [c]	61.34	80.14	93.23	61.23
OOD detection is an important aspect of deploying reliable deep learning systems in the real world [Nguyen et al., 2015, Hendrycks et al., 2022]. It is especially important for those safety-critical applications [Bommasani et al., 2021] like financial or medical intelligence, in which a reliable
Table 24: The inner-maximization combined with different augmentation methods.
Dataset	Method	Augmentation Method	FPR95	AUROC	AUPR	ID-ACC
CIFAR100	OE	-	27.67	91.89	97.93	75.41
DivOE	Random	26.16	92.39	98.06	75.56
DivOE	PGD	24.80	92.91	98.22	75.24
DivOE	Mixup	25.70	92.46	98.07	76.06
DivOE	CutMix	28.46	91.70	97.89	75.62
DivOE	AugMix	25.40	92.60	98.12	75.58
DivOE	Mixup+PGD	25.14	92.79	98.18	75.78
DivOE	CutMix+PGD	26.15	92.37	98.06	75.51
DivOE	AugMix+PGD	24.92	93.01	98.24	75.48
CIFAR10	OE	-	13.76	97.53	99.43	94.51
DivOE	Random	13.19	97.60	99.43	94.81
DivOE	PGD	11.66	97.82	99.48	94.66
DivOE	Mixup	12.87	97.59	99.42	94.83
DivOE	CutMix	13.95	97.54	99.42	94.78
DivOE	AugMix	12.41	97.82	99.49	94.71
DivOE	Mixup+PGD	12.71	97.70	99.47	94.68
DivOE	CutMix+PGD	13.12	97.72	99.47	94.74
DivOE	AugMix+PGD	11.49	97.87	99.49	94.71
model should have the capability to distinguish those samples having different label spaces (e.g. animals) instead of giving a prediction based on existing classes (e.g., finance products or disease). Our research work studies a general and practical research problem in outlier exposure for effective OOD detection, considering the limited informative auxiliary outliers that can not well represent the boundary outliers for the pre-trained model on the classification tasks with ID samples. It is important for effective OOD detection to gain better empirical performance through learning from the auxiliary outliers, and also conceptually figure out the critical problem in this general learning framework.
Although we take a step forward in fine-tuning with auxiliary outliers for effective OOD detection, it is not the end of this direction since there are still many practical problems to be addressed. For example, although learning with the surrogate OOD data can boost the performance of the given model in identifying the OOD inputs, fine-tuning requires extra training time and computational cost, which may bring extra resources for adjusting. Same to other OE-based methods, how to reduce the extra training cost for the model regularization is also a future direction that is worth exploring.
Figure 8: Verification on the effect of extrapolation via Energy Score. The density of the energy score distributions of the  $\epsilon = 0.5$  extrapolated outliers under different numbers of outlier samples, 100000 for left panel, 50000 for left-middle panel, 10000 for right-middle panel, 1000 for right panel.
Table 25: Experiments of DivOE via Mixup (%) (averaged by five OOD test sets)
Din	Method	FPR95↓	AUROC↑	AUPR↑	ID-ACC↑
CIFAR-10	Energy (w. Daux)	18.69	90.51	96.15	94.59
Energy+Mixup (w. Daux)	14.82	92.72	96.96	94.84
Energy+DivOE (Mixup) (w. Daux)	15.02	92.35	96.78	94.98
Energy+CutMix (w. Daux)	19.28	90.30	96.04	94.90
NTOM (with MSP)	12.96	97.73	99.46	95.11
NTOM+Mixup	12.25	97.85	99.50	95.04
NTOM+DivOE (Mixup)	12.20	97.81	99.48	94.98
POEM	18.32	90.81	96.27	94.70
POEM+Mixup	14.70	92.76	97.00	94.98
POEM+DivOE (Mixup)	15.39	91.89	96.62	94.77
POEM+CutMix	17.70	91.25	96.49	94.88
OE	13.62	97.52	99.41	94.57
OE+Mixup	13.08	97.68	99.46	94.89
OE+DivOE (Mixup)	12.87	97.59	99.42	94.83
OE+CutMix	14.74	97.45	99.41	94.93
CIFAR-100	Energy (w. Daux)	27.49	90.72	97.39	75.23
Energy+Mixup (w. Daux)	25.27	92.02	97.81	75.49
Energy+DivOE (Mixup) (w. Daux)	24.47	92.04	97.79	75.53
Energy+CutMix (w. Daux)	29.87	89.72	97.13	75.59
NTOM (with MSP)	26.13	92.32	98.07	75.54
NTOM+Mixup	25.41	92.75	98.18	75.60
NTOM+DivOE (Mixup)	24.44	92.73	98.15	75.84
POEM	25.57	91.73	97.69	76.01
POEM+Mixup	24.46	92.19	97.84	75.57
POEM+DivOE (Mixup)	24.14	92.16	97.81	75.57
POEM+CutMix	28.23	90.75	97.41	75.90
OE	27.29	91.98	97.96	75.51
OE+Mixup	27.20	92.22	98.03	75.87
OE+DivOE (Mixup)	25.70	92.46	98.07	76.06
OE+CutMix	31.14	90.98	97.70	75.66
Table 26: OOD detection performance on CIFAR-10 [Krizhevsky, 2009] as ID. All methods are trained on the same backbone. Values are percentages.  $\uparrow$  indicates larger values are better, and  $\downarrow$  indicates smaller values are better. (averaged by multiple trials)
Method	OOD dataset	ID ACC
Textures	Places365	iSUN	LSUN-r	LSUN-c
FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑
Energy (w. Daux)	34.83	84.76	58.34	67.95	0.00	100.00	0.00	100.00	0.28	99.84	94.59
Energy+Mixup (w. Daux)	20.03	90.84	48.51	75.38	0.00	100.00	0.00	100.00	5.55	97.38	94.84
Energy+DivOE (Mixup) (w. Daux)	23.25	89.57	51.50	72.39	0.00	100.00	0.00	100.00	0.38	99.80	94.98
NTOM(with MSP)	21.04	96.31	43.27	92.43	0.00	100.00	0.00	100.00	0.52	99.92	95.11
NTOM+Mixup	17.98	96.83	42.13	92.65	0.00	100.00	0.00	100.00	1.12	99.79	95.04
NTOM+DivOE (Mixup)	18.12	96.76	42.16	92.40	0.00	100.00	0.00	100.00	0.71	99.88	94.98
POEM	34.52	84.54	57.00	69.55	0.00	100.00	0.00	100.00	0.09	99.96	94.70
POEM+Mixup	21.19	89.99	48.59	75.45	0.00	100.00	0.00	100.00	3.70	98.35	94.98
POEM+DivOE (Mixup)	22.36	89.03	54.29	70.61	0.00	100.00	0.00	100.00	0.32	99.83	94.77
OE	21.88	96.14	45.22	91.64	0.00	100.00	0.00	100.00	1.02	99.84	94.57
OE+Mixup	19.01	96.68	44.70	91.97	0.00	100.00	0.03	100.00	1.67	99.74	94.89
OE+DivOE (Mixup)	19.92	96.40	43.59	91.69	0.00	100.00	0.00	100.00	0.82	99.86	94.83
Table 27: OOD detection performance on CIFAR-100 [Krizhevsky, 2009] as ID. All methods are trained on the same backbone. Values are percentages.  $\uparrow$  indicates larger values are better, and  $\downarrow$  indicates smaller values are better. (averaged by multiple trials)
Method	OOD dataset	ID ACC
Textures	Places365	iSUN	LSUN-r	LSUN-c
FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑
Energy (w. Daux)	57.45	80.77	79.60	72.95	0.00	100.00	0.00	100.00	0.41	99.90	75.23
Energy+Mixup (w. Daux)	42.00	87.24	74.82	75.16	0.00	100.00	0.00	100.00	9.53	97.69	75.49
Energy+DivOE (Mixup) (w. Daux)	44.64	86.13	77.01	74.28	0.00	100.00	0.00	100.00	0.72	99.80	75.53
NTOM(with MSP)	52.99	85.03	75.53	77.08	0.01	100.00	0.00	100.00	2.14	99.52	75.54
NTOM+Mixup	42.36	88.06	74.07	77.89	0.09	99.98	0.00	100.00	10.53	97.84	75.60
NTOM+DivOE (Mixup)	44.37	87.21	74.32	77.20	0.02	99.99	0.01	100.00	3.50	99.23	75.84
POEM	51.55	83.56	76.23	75.10	0.00	100.00	0.00	100.00	0.09	99.98	76.01
POEM+Mixup	39.70	88.09	77.06	74.39	0.00	100.00	0.00	100.00	5.53	98.47	75.57
POEM+DivOE (Mixup)	43.30	86.49	77.12	74.43	0.00	100.00	0.00	100.00	0.28	99.90	75.57
OE	54.31	84.59	77.18	76.30	0.09	99.98	0.03	100.00	4.84	99.04	75.51
OE+Mixup	47.11	86.53	74.27	77.66	0.22	99.93	0.09	99.99	14.29	97.00	75.87
OE+DivOE (Mixup)	48.35	86.07	75.04	77.33	0.13	99.97	0.02	100.00	4.97	98.94	76.06
Figure 9: OOD loss difference (i.e., the maximization part in Eq. (4)) between original outliers and the extrapolated outliers with different  $\epsilon$  in DivOE.
€	Loss difference
0.01	10.84
0.025	16.64
0.05	18.81
0.075	18.64
0.1	18.50
0.15	19.03
0.2	20.29
0.5	24.16
Table 28: OOD loss difference (i.e., the maximization part in Eq. (4)) between original outliers and the extrapolated outliers with different  $\epsilon$  in DivOE.