SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbf {z} _ {\mathrm {O O D}} \in \mathcal {Z} ^ {\mathrm {O O D}} \text {w i t h} \mathcal {Z} ^ {\mathrm {O O D}} = \left\{\mathbf {z} \sim \mathcal {U} _ {Z} \mid \mathcal {M} (\mathbf {z}) \leq \tau \right\}, \tag {7}
$$
with  $\mathcal{U}_Z$  a pre-defined uniform distribution. For now, we can ensure that  $\mathcal{Z}^{\mathrm{ID}}\cap \mathcal{Z}^{\mathrm{OD}} = \emptyset$
However,  $\mathcal{Z}^{\mathrm{ID}}\cap \mathcal{Z}^{\mathrm{OOD}} = \emptyset$  does not imply  $G(\mathcal{Z}^{\mathrm{ID}})\cap G(\mathcal{Z}^{\mathrm{OOD}}) = \emptyset$ , i.e., C1 is not satisfied given arbitrary  $G(\cdot)$ . To this end, we suggest further ensuring the generator to be a distance-preserving function [83], which is a sufficient condition to ensure that the disjoint property can be transformed from the latent space into the data space. For distance preservation, we suggest a regularization term for the generator regularizing, which is given by
$$
\ell_ {\text {r e g}} \left(\mathbf {z} _ {1}, \mathbf {z} _ {2}; G\right) = - \frac {\mathbb {E} _ {\mathcal {U} _ {Z} \times \mathcal {U} _ {Z}} \left[ d \left(\mathbf {z} _ {1} , \mathbf {z} _ {2}\right) \cdot d (G \left(\mathbf {z} _ {1}\right) , G \left(\mathbf {z} _ {2}\right)) \right]}{\sqrt {\mathbb {E} _ {\mathcal {U} _ {Z} \times \mathcal {U} _ {Z}} \left[ d \left(\mathbf {z} _ {1} , \mathbf {z} _ {2}\right) \right] \mathbb {E} _ {\mathcal {U} _ {Z} \times \mathcal {U} _ {Z}} \left[ d (G \left(\mathbf {z} _ {1}\right) , G \left(\mathbf {z} _ {2}\right)) \right]}}, \tag {8}
$$
where  $d(\mathbf{z}_1,\mathbf{z}_2)$  is the centralized distance given by  $\| \mathbf{z}_1 - \mathbf{z}_2\| _2 - \mathbb{E}\left\| \mathbf{z}_1 - \mathbf{z}_2\right\| _2$ . Overall, Eq. (8) is the correlation for the distances between data pairs measured in the latent and the data space, stating that the closer distances in the latent space should indicate the closer distances in the data space.
In summary, to fulfill C1, we need to specify the regularizing procedure for the generator, overall summarized by the following two steps. First, we regularize the generator with the regularization in Eq. (8) to ensure the distance-preserving property. Then, we specify two disjoint regions in the latent space following Eqs. (6)-(7), of which the data after passing through the generator are taken as auxiliary ID and auxiliary OOD data, respectively. We also summarize the algorithm details in crafting the auxiliary OOD detection task in Appendix D.
4.2 Applying the Auxiliary Task
We have discussed a general strategy to construct the auxiliary OOD detection task, which can be learned by the predictor following an outlier exposure-based learning objective, similar to Eq. (2). Now, we fulfill C2, transferring model capability from the auxiliary task to real OOD detection.
In general, C2 states that we should align distributions between the auxiliary and the real ID data such that the auxiliary OOD data can benefit real OOD detection, thus overcoming the negative effects of mistaken OOD generation. Following Tang et al. [59], we align the auxiliary and the real ID distribution via supervised contrastive learning [29], pulling together data belonging to the same class in the embedding space meanwhile pushing apart data from different classes. Accordingly, we suggest the alignment loss for the auxiliary ID data  $\hat{\mathbf{x}}_{\mathrm{ID}}$ , following,
$$
\ell_ {\text {a l i g n}} \left(\hat {\mathbf {x}} _ {\mathrm {I D}}, \hat {y} _ {\mathrm {I D}}; \phi\right) = - \log \frac {1}{\left| \mathcal {N} _ {l} ^ {\hat {y} _ {\mathrm {I D}}} \right|} \sum_ {\mathbf {x} _ {\mathrm {I D}} ^ {p} \in \mathcal {N} _ {l} ^ {\hat {y} _ {\mathrm {I D}}}} \frac {\exp \left[ \phi \left(\hat {\mathbf {x}} _ {\mathrm {I D}}\right) \cdot \phi \left(\mathbf {x} _ {\mathrm {I D}} ^ {p}\right) \right]}{\sum_ {\mathbf {x} _ {\mathrm {I D}} ^ {a} \in \mathcal {N} _ {l}} \exp \left[ \phi \left(\hat {\mathbf {x}} _ {\mathrm {I D}}\right) \cdot \phi \left(\mathbf {x} _ {\mathrm {I D}} ^ {a}\right) \right]}. \tag {9}
$$
$\mathcal{N}_l$  is a set of data of size  $l$  drawn from the real ID distribution, namely,  $\mathcal{N}_l = \{\mathbf{x}_{\mathrm{ID}}^i | (\mathbf{x}_{\mathrm{ID}}^i, y_{\mathrm{ID}}^i) \sim \mathcal{P}_{X,Y}^{\mathrm{ID}}\}$ , for  $i \in \{1, \dots, l\}$ . Furthermore,  $\mathcal{N}_l^y$  is a subset of  $\mathcal{N}_l$  with ID data that below to the label of
$y$ , namely,  $\mathcal{N}_l^y = \{\mathbf{x}_{\mathrm{ID}}^i | (\mathbf{x}_{\mathrm{ID}}^i, y_{\mathrm{ID}}^i) \sim \mathcal{P}_{X,Y}^{\mathrm{ID}}\}$ , and  $y_{\mathrm{ID}}^i = \hat{y}_{\mathrm{ID}}$ , for  $i \in \{1, \dots, l\}$ . Accordingly, based on Theorem 1, we can ensure that learning from the auxiliary OOD detection task, i.e., small Eq. (5), can benefit real OOD detection, freeing from the mistaken OOD generation.
4.3 Overall Algorithm
We further summarize the training and the inferring procedure for the predictor of ATOL, given the crafted auxiliary ID and the auxiliary OOD distributions that satisfy C1 via Eqs. (6)-(7). The pseudo codes are further summarized in Appendix D due to the space limit.
Training Procedure. The overall learning objective consists of the real task learning, the auxiliary task learning, and the ID distribution alignment, which is given by:
$$
\begin{array}{l} \min  _ {\mathbf {h} = \rho \circ \phi} \overbrace {\mathbb {E} _ {\mathcal {P} _ {X , Y} ^ {\mathrm {I D}}} \left[ \ell_ {\mathrm {C E}} \left(\mathbf {x} _ {\mathrm {I D}} , y _ {\mathrm {I D}} ; \mathbf {h}\right) \right]} ^ {\text {(a) r e a l t a k l e a r n i n g}} + \overbrace {\mathbb {E} _ {\mathcal {G} _ {X , Y} ^ {\mathrm {I D}}} \left[ \ell_ {\mathrm {C E}} \left(\hat {\mathbf {x}} _ {\mathrm {I D}}, \hat {y} _ {\mathrm {I D}} ; \mathbf {h}\right) \right] + \lambda \mathbb {E} _ {\mathcal {G} _ {X} ^ {\mathrm {O O D}}} \left[ \ell_ {\mathrm {O E}} \left(\hat {\mathbf {x}} _ {\mathrm {O O D}} ; \mathbf {h}\right) \right]} ^ {\text {(b) a u x i a l y t a k l e a r n i n g}} + \tag {10} \\ \underbrace {\alpha \mathbb {E} _ {\mathcal {G} _ {X , Y} ^ {\mathrm {I D}}} \left[ \ell_ {\text {a l i g n}} \left(\hat {\mathbf {x}} _ {\mathrm {I D}} , \hat {y} _ {\mathrm {I D}} ; \phi\right) \right]} _ {\text {(c) I D d i s t r i b u t i o n a l i g n m e n t}}, \\ \end{array}
$$
where  $\alpha, \lambda \geq 0$  are the trade-off parameters. By finding the predictor that leads to the minimum of the Eq. (10), our ATOL can ensure the improved performance of real OOD detection. Note that, Eq. (10) can be realized in a stochastic manner, suitable for deep model training.
Inferring Procedure. We adopt the MaxLogit scoring [17] in OOD detection. Given a test input  $\mathbf{x}$ , the MaxLogit score is given by:
$$
s _ {\mathrm {M L}} (\mathbf {x}; \mathbf {h}) = \max  _ {k} \mathbf {h} _ {k} (\mathbf {x}), \tag {11}
$$
where  $\mathbf{h}_k(\cdot)$  denotes the  $k$ -th logit output. In general, the MaxLogit scoring is better than other commonly-used scoring functions, such as maximum softmax prediction [15], when facing large semantic space. Therefore, we choose MaxLogit instead of MSP for OOD scoring in our realization.
This section conducts extensive experiments for ATOL in OOD detection. In Section 5.1, we describe the experiment setup. In Section 5.2, we demonstrate the main results of our method against the data generation-based counterparts on both the CIFAR [30] and the ImageNet [8] benchmarks. In Section 5.3, we further conduct ablation studies to comprehensively analyze our method.
Backbones setups. For the CIFAR benchmarks, we employ the WRN-40-2 [78] as the backbone model. Following [36], models have been trained for 200 epochs via empirical risk minimization, with a batch size 64, momentum 0.9, and initial learning rate 0.1. The learning rate is divided by 10 after 100 and 150 epochs. For the ImageNet, we employ pre-trained ResNet-50 [13] on ImageNet, downloaded from the PyTorch official repository.
Generators setups. For the CIFAR benchmarks, following [31], we adopt the generator from the Deep Convolutional (DC) GAN [51]). Following the vanilla generation objective, DCGAN has been trained on the ID data, where the batch size is 64 and the initial learning rate is 0.0002. For the ImageNet benchmark, we adopt the generator from the BigGAN [2] model, designed for scaling generation of high-resolution images, where the pre-trained model biggan-256 can be downloaded from the TensorFlow hub. Note that our method can work on various generators except those mentioned above, even with a random-parameterized one, later shown in Section 5.3.
Baseline methods. We compare our ATOL with advanced data generation-based methods in OOD detection, including (1) BoundaryGAN [31], (2) ConfGAN [55], (3) ManifoldGAN [62], (4) G2D [50] (5) CMG [67]. For a fair comparison, all the methods use the same generator and pre-trained backbone without regularizing with outlier data.
Auxiliary task setups. Hyper-parameters are chosen based on the OOD detection performance on validation datasets. In latent space, the auxiliary ID distribution is the high density region of the
Table 1: Comparison in OOD detection on the CIFAR and ImageNet benchmarks.  $\downarrow$  (or  $\uparrow$ ) indicates smaller (or larger) values are preferred; a bold font indicates the best results in a column.
Methods	CIFAR-10	CIFAR-100	ImageNet
FPR95 ↓	AUROC ↑	FPR95 ↓	AUROC ↑	FPR95 ↓	AUROC ↑
BoundaryGAN [31]	55.60	86.46	76.72	75.79	85.48	66.78
ConfGAN [55]	31.57	93.01	74.86	77.67	74.88	77.03
ManifoldGAN [62]	26.68	94.09	73.54	77.40	72.50	77.73
G2D [50]	31.83	91.74	70.73	79.03	74.93	77.16
CMG [67]	39.83	92.83	79.60	77.51	72.95	77.63
ATOL (ours)	14.66	97.05	55.22	87.24	51.80	85.82
Mixture of Gaussian (MoG). Each sub-Gaussian in the MoG has the mean  $\pmb{\mu} = (\mu_1, \mu_2, \dots, \mu_m)$  and the same covariance matrix  $\sigma \cdot \pmb{I}$ , where  $m$  is the dimension of latent space,  $\mu_i$  is randomly selected from the set  $\{-\mu, \mu\}$ , and  $\pmb{I}$  is the identity matrix. The auxiliary OOD distribution is the uniform distribution except for high MoG density region, where each dimension has the same space size  $u$ . For the CIFAR benchmarks, the value of  $\alpha$  is set to 1,  $\mu$  is 5,  $\sigma$  is 0.1, and  $u$  is 8. For the ImageNet benchmarks, the value of  $\alpha$  is set to 1,  $\mu$  is 5,  $\sigma$  is 0.8, and  $u$  is 8.
Training details. For the CIFAR benchmarks, ATOL is run for 10 epochs and uses SGD with an initial learning rate 0.01 and the cosine decay [37]. The batch size is 64 for real ID cases, 64 for auxiliary ID cases, and 256 for auxiliary OOD cases. For the ImageNet benchmarks, ATOL is run for 10 epochs using SGD with an initial learning rate 0.0003 and a momentum 0.9. The learning rate is decayed by a factor of 10 at  $30\%$ ,  $60\%$ , and  $90\%$  of the training steps. Furthermore, the batch size is fixed to 32 for real ID cases, 32 for auxiliary ID cases, and 128 for auxiliary OOD cases.
Evaluation metrics. The OOD detection performance of a detection model is evaluated via two representative metrics, which are both threshold-independent [7]: the false positive rate of OOD data when the true positive rate of ID data is at  $95\%$  (FPR95); and the area under the receiver operating characteristic curve (AUROC), which can be viewed as the probability of the ID case having greater score than that of the OOD case.
We begin with our main experiments on the CIFAR and ImageNet benchmarks. Model performance is tested on commonly-used OOD datasets. For the CIFAR cases, we employed Texture [6], SVHN [47], Places365 [84], LSUN-Crop [76], LSUN-Resize [76], and iSUN [74]. For the ImageNet case, we employed iNaturalist [18], SUN [74], Places365 [84], and Texture [6]. In Table 1, we report the average performance (i.e., FPR95 and AUROC) regarding the OOD datasets mentioned above. Please refer to Tables 17-18 and 19 in Appendix F.1 for the detailed reports.
CIFAR benchmarks. Overall, our ATOL can lead to effective OOD detection, which generally demonstrates better results than the rivals by a large margin. In particular, compared with the advanced data generation-based method, e.g., ManifoldGAN [62], ATOL significantly improves the performance in OOD detection, revealing  $12.02\%$  and  $2.96\%$  average improvements w.r.t. FPR95 and AUROC on the CIFAR-10 dataset and  $18.32\%$  and  $9.84\%$  of the average improvements on the CIFAR-100 dataset. This highlights the superiority of our novel data generation strategy, relieving mistaken OOD generation as in previous methods.
ImageNet benchmark. We evaluate a large-scale OOD detection task based on ImageNet dataset. Compared to the CIFAR benchmarks above, the ImageNet task is more challenging due to the large semantic space and large amount of training data. As we can see, previous data generation-based methods reveal poor performance due to the increased difficulty in searching for the proper OOD-like data in ImageNet, indicating the critical mistaken OOD generation remains (cf., Appendix F.9). In contrast, our ATOL can alleviate this issue even for large-scale datasets, e.g., ImageNet, thus leading to superior results. In particular, our ATOL outperforms the best baseline ManifoldGAN [62] by  $20.70\%$  and  $8.09\%$  average improvement w.r.t. FPR95 and AUROC, which demonstrates the advantages of our ATOL to relieve mistaken OOD generation and benefit to real OOD detection.
Table 2: Effectiveness of auxiliary task crafting and learning scheme of ATOL.  $\downarrow$  (or  $\uparrow$  ) indicates smaller (or larger) values are preferred; a bold font indicates the best results in a row.
Ablation	ATOL w/o τ	ATOL w/o ℓreg	ATOL w/o ℓalign	ATOL
CIFAR-10	FPR95 ↓	87.64	41.13	22.29	14.54
AUROC ↑	59.32	90.38	94.08	97.01
CIFAR-100	FPR95 ↓	92.90	82.80	74.20	55.06
AUROC ↑	51.34	73.95	80.56	87.26
Table 3: Performance comparisons with different setups of the generator on CIFAR benchmarks;  $\downarrow$  (or  $\uparrow$ ) indicates smaller (or larger) values are preferred; a bold font indicates the best results in a row.
Generators	DCGAN	Rand-DCGAN	StyleGAN	BigGAN
CIFAR-10	FPR95 ↓	14.66	20.78	13.65	8.61
AUROC ↑	97.05	95.57	96.63	97.90
CIFAR-100	FPR95 ↓	55.52	65.69	42.62	36.72
AUROC ↑	86.21	83.26	89.17	91.33
To further demonstrate the effectiveness of our ATOL, we conduct extensive ablation studies to verify the contributions of each proposed component.
Auxiliary task crafting schemes. In Section 4.1, we introduce the realization of the auxiliary OOD detection task in a tractable way. Here, we study two components of generation: the disjoint supports for ID and OOD distributions in the latent space and the distance-preserving generator. In particular, we compare ATOL with two variants: 1) ATOL  $w / o \tau$ , where the ID and OOD distributions may overlap in latent space without separation via the threshold  $\tau$ ; 2) ATOL  $w / o \ell_{\mathrm{reg}}$ , where we directly use the generator to generate the auxiliary data without further regularization. As we can see in Table 2, the overlap between the auxiliary ID and the auxiliary OOD distributions in latent space will lead to the failure of the auxiliary OOD detection task, demonstrating the catastrophic performance (e.g.,  $92.20\%$  FPR95 and  $51.34\%$  AUROC on CIFAR-100). Moreover, without the regularized generator as a distance-preserving function, the ambiguity of generated data will compromise the performance of ATOL, which verifies the effectiveness of our generation scheme.
Auxiliary task learning schemes. In Section 4.2, we propose ID distribution alignment to transfer model capability from auxiliary OOD detection task to real OOD detection. In Table 2, we conduct experiments on a variant, namely, ATOL  $w / o$ $\ell_{\mathrm{align}}$ , where the predictor directly learns from the auxiliary task without further alignment. Accordingly, ATOL  $w / o$ $\ell_{\mathrm{align}}$  can reveal better results than ATOL  $w / o$ $\ell_{\mathrm{reg}}$ , with  $18.84\%$  and  $3.70\%$  further improvement w.r.t. FPR95 and AUROC on CIFAR-10 and  $8.60\%$  and  $6.61\%$  on CIFAR-100. However, the predictor learned on the auxiliary task cannot fit into the real OOD detection task. Thus, we further align the distributions between the auxiliary and real ID data in the transformed space, significantly improving the OOD detection performance with  $7.75\%$  on CIFAR-10 and  $19.14\%$  on CIFAR-100 w.r.t. FPR95.
Generator Setups. On CIFAR benchmarks, we use the generator of the DCGAN model fitting on the ID data in our ATOL. We further investigate whether ATOL remains effective when using generators with other setups, summarized in Table 3. First, we find that even with a random-parameterized DCGAN generator, i.e., Rand-DCGAN, our ATOL can still yield reliable performance surpassing its advanced counterparts. Such an observation suggests that our method requires relatively low training costs, a property not shared by previous methods. Second, we adopt more advanced generators, i.e., BigGAN [2] and StyleGAN-v2 [27]. As we can see, ATOL can benefit from better generators, leading to large improvement  $18.80\%$  and  $5.12\%$  w.r.t. FPR95 and AUROC on CIFAR-100 over the DCGAN case. Furthermore, even with complicated generators, our method demonstrates promising performance improvement over other methods with acceptable computational resources. Please refer to the Appendix E.8 and Appendix F.7 for more details.
Quantitative analysis on computation. Data generation-based methods are relatively expensive in computation, so the consumed training time is also of our interest. In Table 4, we compare the time costs of training for the considered data generation-based methods. Since our method does not need a complex generator training procedure during the OOD learning step, we can largely alleviate the expensive cost of computation inherited from the data generation-based approaches. Overall, our method requires minimal training costs compared to the advanced counterparts, which further reveals the efficiency of our method. Therefore, our method can not only lead to improved performance over previous methods, but it also requires fewer computation resources.
Table 4: Computational comparison with other counterparts. We report the per-epoch training time (measured by seconds) on CIFAR benchmarks.
Methods	Training time (s)
BoundaryGAN	106.98 ± 7.82
ConfGAN	111.37 ± 5.72
ManifoldGAN	159.23 ± 9.23
G2D	73.29 ± 4.09
CMG	254.22 ± 2.42
ATOL	57.22 ± 2.28
Data generation-based methods in OOD detection are a promising way to make the predictor learn to discern ID and OOD data without real OOD data. However, mistaken OOD generation significantly challenges their performance. To this end, we propose a powerful data generation-based learning method by the proposed auxiliary OOD detection task, largely relieving mistaken OOD generation. Extensive experiments show that our method can notably improve detection performance compared to advanced counterparts. Overall, our method can benefit applications for data generation-based OOD detection, and we intend further research to extend our method beyond image classification. We also hope our work can draw more attention from the community toward data generation-based methods, and we anticipate our auxiliary task-based learning scheme can benefit more directions in OOD detection, such as wild OOD detection [28].
HTZ, QZW and BH were supported by the NSFC Young Scientists Fund No. 62006202, NSFC General Program No. 62376235, Guangdong Basic and Applied Basic Research Foundation No. 2022A1515011652, HKBU Faculty Niche Research Areas No. RC-FNRA-IG/22-23/SCI/04, and HKBU CSD Departmental Incentive Scheme. XBX and TLL were partially supported by the following Australian Research Council projects: FT220100318, DP220102121, LP220100527, LP220200949, and IC190100031. FL was supported by Australian Research Council (ARC) under Award No. DP230101540, and by NSF and CSIRO Responsible AI Program under Award No. 2303037.
[1] Abhijit Bendale and Terrance E. Boult. Towards open set deep networks. In CVPR, 2016.
[2] Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In ICLR, 2019.
[3] Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha. ATOM: robustifying out-of-distribution detection using outlier mining. In ECML, 2021.
[4] Ling-Hao Chen, He Li, Wanyuan Zhang, Jianbin Huang, Xiaoke Ma, Jiangtao Cui, Ning Li, and Jaesoo Yoo. Anomman: Detect anomaly on multi-view attributed networks. arXiv preprint arXiv:2201.02822, 2022.
[5] Ling-Hao Chen, Jiawei Zhang, Yewen Li, Yiren Pang, Xiaobo Xia, and Tongliang Liu. Humanmac: Masked motion completion for human motion prediction. In ICCV, 2023.
[6] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In CVPR, 2014.
[7] Jesse Davis and Mark Goadrich. The relationship between precision-recall and ROC curves. In ICML, 2006.
[8] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
[9] Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out-of-distribution detection. In ICLR, 2023.
[10] Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. VOS: learning what you don't know by virtual outlier synthesis. In ICLR, 2022.
[11] Zhen Fang, Yixuan Li, Jie Lu, Jiahua Dong, Bo Han, and Feng Liu. Is out-of-distribution detection learnable? In NeurIPS, 2022.
[12] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS, 2014.
[13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
[14] Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In CVPR, 2019.
[15] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017.
[16] Dan Hendrycks, Mantas Mazeika, and Thomas G. Dietterich. Deep anomaly detection with outlier exposure. In ICLR, 2019.
[17] Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joseph Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real-world settings. In ICML, 2022.
[18] Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alexander Shepard, Hartwig Adam, Pietro Perona, and Serge J. Belongie. The inaturalist species classification and detection dataset. In CVPR, 2018.
[19] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, 2017.
[20] Rui Huang and Yixuan Li. MOS: towards scaling out-of-distribution detection for large semantic space. In CVPR, 2021.
[21] Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. In NeurIPS, 2021.
[22] Zhuo Huang, Chao Xue, Bo Han, Jian Yang, and Chen Gong. Universal semi-supervised learning. In NeurIPS, 2021.
[23] Zhuo Huang, Xiaobo Xia, Li Shen, Bo Han, Mingming Gong, Chen Gong, and Tongliang Liu. Harnessing out-of-distribution examples via augmenting content and style. In ICLR, 2023.
[24] Zhuo Huang, Miaoxi Zhu, Xiaobo Xia, Li Shen, Jun Yu, Chen Gong, Bo Han, Bo Du, and Tongliang Liu. Robust generalization against photon-limited corruptions via worst-case sharpness minimization. In CVPR, 2023.
[25] Conor Igoe, Youngseog Chung, Ian Char, and Jeff Schneider. How useful are gradients for ood detection really? arXiv preprint arXiv:2205.10439, 2022.
[26] Taewon Jeong and Heeyoung Kim. OOD-MAML: meta-learning for few-shot out-of-distribution detection and classification. In NeurIPS, 2020.