SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Xuefeng Du, Yiyou Sun, Xiaojin Zhu, Yixuan Li
{xfdu,sunyiyou,jerryzhu,sharonli}@cs.wisc.edu
Utilizing auxiliary outlier datasets to regularize the machine learning model has demonstrated promise for out-of-distribution (OOD) detection and safe prediction. Due to the labor intensity in data collection and cleaning, automating outlier data generation has been a long-desired alternative. Despite the appeal, generating photo-realistic outliers in the high dimensional pixel space has been an open challenge for the field. To tackle the problem, this paper proposes a new framework DREAM-OOD, which enables imagining photo-realistic outliers by way of diffusion models, provided with only the in-distribution (ID) data and classes. Specifically, DREAM-OOD learns a text-conditioned latent space based on ID data, and then samples outliers in the low-likelihood region via the latent, which can be decoded into images by the diffusion model. Different from prior works [1, 2], DREAM-OOD enables visualizing and understanding the imagined outliers, directly in the pixel space. We conduct comprehensive quantitative and qualitative studies to understand the efficacy of DREAM-OOD, and show that training with the samples generated by DREAM-OOD can benefit OOD detection performance. Code is publicly available at https://github.com/deeplearning-wisc/dream-ood.
Out-of-distribution (OOD) detection is critical for deploying machine learning models in the wild, where samples from novel classes can naturally emerge and should be flagged for caution. Concerningly, modern neural networks are shown to produce overconfident and therefore untrustworthy predictions for unknown OOD inputs [3]. To mitigate the issue, recent works have explored training with an auxiliary outlier dataset, where the model is regularized to learn a more conservative decision boundary around in-distribution (ID) data [4-7]. These methods have demonstrated encouraging OOD detection performance over the counterparts without auxiliary data.
Despite the promise, preparing auxiliary data can be labor-intensive and inflexible, and necessitates careful human intervention, such as data cleaning, to ensure the auxiliary outlier data does not overlap with the ID data. Automating outlier data generation has thus been a long-desired alternative. Despite the appeal, generating photo-realistic outliers has been extremely challenging due to the high dimensional space. Recent works including VOS and NPOS [1, 2] proposed sampling outliers in the low-dimensional feature space and directly employed the latent-space outliers to regularize the model. However, these latent-space methods do not allow us to understand the outliers in a human-compatible way. Today, the field still lacks an automatic mechanism to generate high-resolution outliers in the pixel space.
In this paper, we propose a new framework DREAM-OOD that enables imagining photo-realistic outliers by way of diffusion models, provided with only ID data and classes (see Figure 1). Harnessing the power of diffusion models for outlier imagination is non-trivial, since one cannot easily describe
ID data
Imagined OOD
Figure 1: Top: Original ID training data in IMAGENET [8]. Bottom: Samples generated by our method DREAM-OOD, which deviate from the ID data.
the exponentially many possibilities of outliers using text prompts. It can be particularly challenging to characterize informative outliers that lie on the boundary of ID data, which have been shown to be the most effective in regularizing the ID classifier and its decision boundary [7]. After all, it is almost impossible to describe something in words without knowing what it looks like.
Our framework circumvents the above challenges by: (1) learning compact visual representations for the ID data, conditioned on the textual latent space of the diffusion model (Section 3.1), and (2) sampling new visual embeddings in the text-conditioned latent space, which are then decoded to pixel-space images by the diffusion model (Section 3.2). Concretely, to learn the text-conditioned latent space, we train an image classifier to produce image embeddings that have a higher probability to be aligned with the corresponding class token embedding. The resulting feature embeddings thus form a compact and informative distribution that encodes the ID data. Equipped with the text-conditioned latent space, we sample new embeddings from the low-likelihood region, which can be decoded into the images via the diffusion model. The rationale is if the sampled embedding is distributionally far away from the in-distribution embeddings, the generated image will have a large semantic discrepancy from the ID images and vice versa.
We demonstrate that our proposed framework creatively imagines OOD samples conditioned on a given dataset, and as a result, helps improve the OOD detection performance. On IMAGENET dataset, training with samples generated by DREAM-OOD improves the OOD detection on a comprehensive suite of OOD datasets. Different from [1, 2], our method allows visualizing and understanding the imagined outliers, covering a wide spectrum of near-OOD and far-OOD. Note that DREAM-OOD enables leveraging off-the-shelf diffusion models for OOD detection, rather than modifying the diffusion model (which is an actively studied area on its own [9]). In other words, this work's core contribution is to leverage generative modeling to improve discriminative learning, establishing innovative connections between the diffusion model and outlier data generation.
1. To the best of our knowledge, DREAM-OOD is the first to enable the generation of photorealistic high-resolution outliers for OOD detection. DREAM-OOD establishes promising performance on common benchmarks and can benefit OOD detection.
2. We conduct comprehensive analyses to understand the efficacy of DREAM-OOD, both quantitatively and qualitatively. The results provide insights into outlier imagination with diffusion models.
3. As an extension, we show that our synthesis method can be used to automatically generate ID samples, and as a result, improves the generalization performance of the ID task itself.
We consider a training set  $\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^n$ , drawn i.i.d. from the joint data distribution  $P_{\mathcal{X}\mathcal{Y}}$ .  $\mathcal{X}$  denotes the input space and  $\mathcal{Y} \in \{1, 2, \dots, C\}$  denotes the label space. Let  $\mathbb{P}_{\mathrm{in}}$  denote the marginal distribution on  $\mathcal{X}$ , which is also referred to as the in-distribution. Let  $f_\theta: \mathcal{X} \mapsto \mathbb{R}^C$  denote a multi-class classifier, which predicts the label of an input sample with parameter  $\theta$ . To obtain an
Figure 2: Illustration of our proposed outlier imagination framework DREAM-OOD. DREAM-OOD first learns a text-conditioned space to produce compact image embeddings aligned with the token embedding  $\mathcal{T}(y)$  of the diffusion model. It then samples new embeddings in the latent space, which can be decoded into pixel-space outlier images  $\mathbf{x}_{\mathrm{ood}}$  by diffusion model. The newly generated samples can help improve OOD detection. Best viewed in color.
optimal classifier  $f^{*}$ , a standard approach is to perform empirical risk minimization (ERM) [10]:  $f^{*} = \operatorname{argmin}_{f\in \mathcal{F}}\frac{1}{n}\sum_{i = 1}^{n}\ell (f(\mathbf{x}_i),y_i)$  where  $\ell$  is the loss function and  $\mathcal{F}$  is the hypothesis space.
Out-of-distribution detection. When deploying a machine model in the real world, a reliable classifier should not only accurately classify known in-distribution samples, but also identify OOD input from unknown class  $y \notin \mathcal{V}$ . This can be achieved by having an OOD detector, in tandem with the classification model  $f_{\theta}$ . At its core, OOD detection can be formulated as a binary classification problem. At test time, the goal is to decide whether a test-time input is from ID or not (OOD). We denote  $g_{\theta}: \mathcal{X} \mapsto \{\text{in}, \text{out}\}$  as the function mapping for OOD detection.
Denoising diffusion models have emerged as a promising generative modeling framework, pushing the state-of-the-art in image generation [11, 12]. Inspired by non-equilibrium thermodynamics, diffusion probabilistic models [13-15] define a forward Gaussian Markov transition kernel of diffusion steps to gradually corrupt training data until the data distribution is transformed into a simple noisy distribution. The model then learns to reverse this process by learning a denoising transition kernel parameterized by a neural network.
Diffusion models can be conditional, for example, on class labels or text descriptions [11, 16, 17]. In particular, Stable Diffusion [18] is a text-to-image model that enables synthesizing new images guided by the text prompt. The model was trained on 5 billion pairs of images and captions taken from LAION-5B [19], a publicly available dataset derived from Common Crawl data scraped from the web. Given a class name  $y$ , the generation process can be mathematically denoted by:
$$
\mathbf {x} \sim P (\mathbf {x} | \mathbf {z} _ {y}), \tag {1}
$$
where  $\mathbf{z}_y = \mathcal{T}(y)$  is the textual representation of label  $y$  with prompting (e.g., "A high-quality photo of a  $[y]$ "). In Stable Diffusion,  $\mathcal{T}(\cdot)$  is the text encoder of the CLIP model [20].
3 DREAM-OOD: Outlier Imagination with Diffusion Models
In this paper, we propose a novel framework that enables synthesizing photo-realistic outliers with respect to a given ID dataset (see Figure 1). The synthesized outliers can be useful for regularizing the ID classifier to be less confident in the OOD region. Recall that the vanilla diffusion generation takes as input the textual representation. While it is easy to encode the ID classes  $y \in \mathcal{V}$  into textual latent space via  $\mathcal{T}(y)$ , one cannot trivially generate text prompts for outliers. It can be particularly challenging to characterize informative outliers that lie on the boundary of ID data, which have been shown to be most effective in regularizing the ID classifier and its decision boundary [7]. After all, it is almost impossible to concretely describe something in words without knowing what it looks like.
Overview. As illustrated in Figure 2, our framework circumvents the challenge by: (1) learning compact visual representations for the ID data, conditioned on the textual latent space of the diffusion model (Section 3.1), and (2) sampling new visual embeddings in the text-conditioned latent space, which are then decoded into the images by diffusion model (Section 3.2). We demonstrate in Section 4 that, our proposed outlier synthesis framework produces meaningful out-of-distribution samples conditioned on a given dataset, and as a result, significantly improves the OOD detection performance.
3.1 Learning the Text-Conditioned Latent Space
Our key idea is to first train a classifier on ID data  $\mathcal{D}$  that produces image embeddings, conditioned on the token embeddings  $\mathcal{T}(y)$ , with  $y \in \mathcal{V}$ . To learn the text-conditioned visual latent space, we train the image classifier to produce image embeddings that have a higher probability of being aligned with the corresponding class token embedding, and vice versa.
Specifically, denote  $h_{\theta}:\mathcal{X}\mapsto \mathbb{R}^{m}$  as a feature encoder that maps an input  $\mathbf{x}\in \mathcal{X}$  to the image embedding  $h_\theta (\mathbf{x})$  and  $\mathcal{T}:\mathcal{V}\mapsto \mathbb{R}^m$  as the text encoder that takes a class name  $y$  and outputs its token embedding  $\mathcal{T}(y)$ . Here  $\mathcal{T}(\cdot)$  is a fixed text encoder of the diffusion model. Only the image feature encoder needs to be trained, with learnable parameters  $\theta$ . Mathematically, the loss function for learning the visual representations is formulated as follows:
$$
\mathcal {L} = \mathbb {E} _ {(\mathbf {x}, y) \sim \mathcal {D}} [ - \log \frac {\exp (\mathcal {T} (y) ^ {\top} \mathbf {z} / t)}{\sum_ {j = 1} ^ {C} \exp (\mathcal {T} (y _ {j}) ^ {\top} \mathbf {z} / t)} ], \tag {2}
$$
where  $\mathbf{z} = h_{\theta}(\mathbf{x}) / \| h_{\theta}(\mathbf{x})\| _2$  is the  $L_{2}$ -normalized image embedding, and  $t$  is temperature.
Theoretical interpretation of loss. Formally, our loss function directly promotes the class-conditional von Mises Fisher (vMF) distribution [21-23]. vMF is analogous to spherical Gaussian distributions for features with unit
norms  $(\| \mathbf{z}\| ^2 = 1)$ . The probability density function of  $\mathbf{z}\in \mathbb{R}^m$  in class  $c$  is:
Figure 3: TSNE visualization of learned feature embeddings using  $\mathcal{L}$ . Black dots indicate token embeddings, one for each class.
$$
p _ {m} (\mathbf {z}; \boldsymbol {\mu} _ {c}, \kappa) = Z _ {m} (\kappa) \exp \left(\kappa \boldsymbol {\mu} _ {c} ^ {\top} \mathbf {z}\right), \tag {3}
$$
where  $\mu_{c}$  is the class centroid with unit norm,  $\kappa \geq 0$  controls the extent of class concentration, and  $Z_{m}(\kappa)$  is the normalization factor detailed in the Appendix B. The probability of the feature vector  $\mathbf{z}$  belonging to class  $c$  is:
$$
\begin{array}{l} P (y = c | \mathbf {z}; \{\kappa , \boldsymbol {\mu} _ {j} \} _ {j = 1} ^ {C}) = \frac {Z _ {m} (\kappa) \exp (\kappa \boldsymbol {\mu} _ {c} ^ {\top} \mathbf {z})}{\sum_ {j = 1} ^ {C} Z _ {m} (\kappa) \exp (\kappa \boldsymbol {\mu} _ {j} ^ {\top} \mathbf {z})} \\ = \frac {\exp \left(\boldsymbol {\mu} _ {c} ^ {\top} \mathbf {z} / t\right)}{\sum_ {j = 1} ^ {C} \exp \left(\boldsymbol {\mu} _ {j} ^ {\top} \mathbf {z} / t\right)}, \tag {4} \\ \end{array}
$$
where  $\kappa = \frac{1}{t}$ . Therefore, by encouraging features to be aligned with its class token embedding, our loss function  $\mathcal{L}$  (Equation (2)) maximizes the log-likelihood of the class-conditional vMF distributions and promotes compact clusters on the hypersphere (see Figure 3). The highly compact representations can benefit the sampling of new embeddings, as we introduce next in Section 3.2.
3.2 Outlier Imagination via Text-Conditioned Latent
Given the well-trained compact representation space that encodes the information of  $\mathbb{P}_{\mathrm{in}}$ , we propose to generate outliers by sampling new embeddings in the text-conditioned latent space, and then decoding via diffusion model. The rationale is that if the sampled embeddings are distributionally far away from the ID embeddings, the decoded images will have a large semantic discrepancy with the ID images and vice versa.
Recent works [1, 2] proposed sampling outlier embeddings and directly employed the latent-space outliers to regularize the model. In contrast, our method focuses on generating pixel-space photorealistic images, which allows us to directly inspect the generated outliers in a human-compatible way. Despite the appeal, generating high-resolution outliers has been extremely challenging due to the high dimensional space. To tackle the issue, our generation procedure constitutes two steps:
1. Sample OOD in the latent space: draw new embeddings  $\mathbf{v}$  that are in the low-likelihood region of the text-conditioned latent space.
2. Image generation: decode  $\mathbf{v}$  into a pixel-space OOD image via diffusion model.
Algorithm 1 DREAM-OOD: Outlier Imagination with Diffusion Models
Input: In-distribution training data  $\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^n$ , initial model parameters  $\theta$  for learning the text-conditioned latent space, diffusion model.
Output: Synthetic images  $\mathbf{x}_{\mathrm{odd}}$
Phases: Phase 1: Learning the Text-conditioned Latent Space. Phase 2: Outlier Imagination via Text-Conditioned Latent.
while Phase 1 do
1. Extract token embeddings  $\mathcal{T}(y)$  of the ID label  $y\in \mathcal{V}$
2. Learn the text-conditioned latent representation space by Equation (2).
while Phase 2 do
1. Sample a set of outlier embeddings  $V_{i}$  in the low-likelihood region of the text-conditioned latent space as in Section 3.2.
2. Decode the outlier embeddings into the pixel-space OOD images via diffusion model by Equation (6).
Sampling OOD embedding. Our goal here is to sample low-likelihood embeddings based on the learned feature representations (see Figure 4). The sampling procedure can be instantiated by different approaches. For example, a recent work by Tao et.al. [2] proposed a latent non-parametric sampling method, which does not make any distributional assumption on the ID embeddings and offers stronger flexibility compared to the parametric sampling approach [1]. Concretely, we can select the boundary ID anchors by leveraging the non-parametric nearest neighbor distance, and then draw new embeddings around that boundary point.
Denote the  $L_{2}$ -normalized embedding set of training data as  $\mathbb{Z} = (\mathbf{z}_1, \mathbf{z}_2, \dots, \mathbf{z}_n)$ , where  $\mathbf{z}_i = h_\theta(\mathbf{x}_i) / \|h_\theta(\mathbf{x}_i)\|_2$ . For any embedding  $\mathbf{z}' \in \mathbb{Z}$ , we calculate the  $k$ -NN distance w.r.t.  $\mathbb{Z}$ :
$$
d _ {k} \left(\mathbf {z} ^ {\prime}, \mathbb {Z}\right) = \left\| \mathbf {z} ^ {\prime} - \mathbf {z} _ {(k)} \right\| _ {2}, \tag {5}
$$
where  $\mathbf{z}_{(k)}$  is the  $k$ -th nearest neighbor in  $\mathbb{Z}$ . If an embedding has a large  $k$ -NN distance, it is likely to be on the boundary of the ID data and vice versa.
Given a boundary ID point, we then draw new embedding sample  $\mathbf{v} \in \mathbb{R}^m$  from a Gaussian kernel<sup>1</sup> centered at  $\mathbf{z}_i$  with covariance  $\sigma^2\mathbf{I}$ :  $\mathbf{v} \sim \mathcal{N}(\mathbf{z}_i, \sigma^2\mathbf{I})$ . In addition, to ensure that the outliers are sufficiently far away from the ID data, we repeatedly sample multiple outlier embeddings from the Gaussian kernel  $\mathcal{N}(\mathbf{z}_i, \sigma^2\mathbf{I})$ , which produces a set  $V_i$ , and further perform a filtering process by selecting the outlier embedding in  $V_i$  with the largest  $k$ -NN distance w.r.t.  $\mathbb{Z}$ . Detailed ablations on the sampling parameters are provided in Section 4.2.
Figure 4: TSNE visualization of ID embeddings (purple) and the sampled outlier embeddings (orange), for the class "hen" in IMAGENET.
Outlier image generation. Lastly, to obtain the outlier images in the pixel space, we decode the sampled outlier embeddings  $\mathbf{v}$  via the diffusion model. In practice, this can be done by replacing the original token embedding  $\mathcal{T}(y)$  with the sampled new embedding  $\mathbf{v}^2$ . Different from the vanilla prompt-based generation (c.f. Equation (1)), our outlier imagination is mathematically reflected by:
$$
\mathbf {x} _ {\mathrm {o o d}} \sim P (\mathbf {x} | \mathbf {v}), \tag {6}
$$
where  $\mathbf{x}_{\mathrm{ood}}$  denotes the generated outliers in the pixel space. Importantly,  $\mathbf{v} \sim S \circ h_{\theta} \circ (\mathbb{P}_{\mathrm{in}})$  is dependent on the in-distribution data, which enables generating images that deviate from  $\mathbb{P}_{\mathrm{in}}$ .  $S(\cdot)$  denotes the sampling procedure. Our framework DREAM-OOD is summarized in Algorithm 1.
<sup>1</sup>The choice of kernel function form (e.g., Gaussian vs. Epanechnikov) is not influential, while the kernel bandwidth parameter is [24].
2In the implementation, we re-scale  $\mathbf{v}$  by multiplying the norm of the original token embedding to preserve the magnitude.
Methods	OOD Datasets	ID ACC
1NATURALIST	PLACES	SUN	TEXTURES	Average
FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑
MSP [25]	31.80	94.98	47.10	90.84	47.60	90.86	65.80	83.34	48.08	90.01	87.64
ODIN [26]	24.40	95.92	50.30	90.20	44.90	91.55	61.00	81.37	45.15	89.76	87.64
Mahalanobis [27]	91.60	75.16	96.70	60.87	97.40	62.23	36.50	91.43	80.55	72.42	87.64
Energy [6]	32.50	94.82	50.80	90.76	47.60	91.71	63.80	80.54	48.68	89.46	87.64
GODIN [28]	39.90	93.94	59.70	89.20	58.70	90.65	39.90	92.71	49.55	91.62	87.38
KNN [29]	28.67	95.57	65.83	88.72	58.08	90.17	12.92	90.37	41.38	91.20	87.64
ViM [30]	75.50	87.18	88.30	81.25	88.70	81.37	15.60	96.63	67.03	86.61	87.64
ReAct [31]	22.40	96.05	45.10	92.28	37.90	93.04	59.30	85.19	41.17	91.64	87.64
DICE [32]	37.30	92.51	53.80	87.75	45.60	89.21	50.00	83.27	46.67	88.19	87.64
Synthesis-based methods
GAN [33]	83.10	71.35	83.20	69.85	84.40	67.56	91.00	59.16	85.42	66.98	79.52
VOS [1]	43.00	93.77	47.60	91.77	39.40	93.17	66.10	81.42	49.02	90.03	87.50
NPOS [2]	53.84	86.52	59.66	83.50	53.54	87.99	8.98	98.13	44.00	89.04	85.37
DREAM-OOD (Ours)	24.10±0.2	96.10±0.1	39.87±0.1	93.11±0.3	36.88±0.4	93.31±0.4	53.99±0.6	85.56±0.9	38.76±0.2	92.02±0.4	87.54±0.1
Table 1: OOD detection results for IMAGENET-100 as the in-distribution data. We report standard deviations estimated across 3 runs. Bold numbers are superior results.
Learning with imagined outlier images. The generated synthetic OOD images  $\mathbf{x}_{\mathrm{ood}}$  can be used for regularizing the training of the classification model [1]:
$$
\mathcal {L} _ {\mathrm {o o d}} = \mathbb {E} _ {\mathbf {x} _ {\mathrm {o o d}}} \left[ - \log \frac {1}{1 + \exp^ {\phi (E \left(f _ {\theta} (\mathbf {x} _ {\mathrm {o o d}})\right))}} \right] + \mathbb {E} _ {\mathbf {x} \sim \mathbb {P} _ {\mathrm {i n}}} \left[ - \log \frac {\exp^ {\phi (E \left(f _ {\theta} (\mathbf {x})\right))}}{1 + \exp^ {\phi (E \left(f _ {\theta} (\mathbf {x})\right))}} \right], \tag {7}
$$
where  $\phi (\cdot)$  is a three-layer nonlinear MLP function with the same architecture as VOS [1],  $E(\cdot)$  denotes the energy function, and  $f_{\theta}(\mathbf{x})$  denotes the logit output of the classification model. In other words, the loss function takes both the ID and generated OOD images, and learns to separate them explicitly. The overall training objective combines the standard cross-entropy loss, along with an additional loss in terms of OOD regularization  $\mathcal{L}_{\mathrm{CE}} + \beta \cdot \mathcal{L}_{\mathrm{odd}}$ , where  $\beta$  is the weight of the OOD regularization.  $\mathcal{L}_{\mathrm{CE}}$  denotes the cross-entropy loss on the ID training data. In testing, we use the output of the binary logistic classifier for OOD detection.
In this section, we present empirical evidence to validate the effectiveness of our proposed outlier imagination framework. In what follows, we show that DREAM-OOD produces meaningful OOD images, and as a result, significantly improves OOD detection (Section 4.1) performance. We provide comprehensive ablations and qualitative studies in Section 4.2. In addition, we showcase an extension of our framework for improving generalization by leveraging the synthesized inliers (Section 4.3).
4.1 Evaluation on OOD Detection Performance
Datasets. Following [2], we use the CIFAR-100 and the large-scale IMAGENET dataset [8] as the ID training data. For CIFAR-100, we use a suite of natural image datasets as OOD including TEXTURES [34], SVHN [35], PLACES365 [36], ISUN [37] & LSUN [38]. For IMAGENET-100, we adopt the OOD test data as in [39], including subsets of INATURALIST [40], SUN [41], PLACES [36], and TEXTURES [34]. For each OOD dataset, the categories are disjoint from the ID dataset. We provide the details of the datasets and categories in Appendix A.
Training details. We use ResNet-34 [42] as the network architecture for both CIFAR-100 andImagenet-100 datasets. We train the model using stochastic gradient descent for 100 epochs with the cosine learning rate decay schedule, a momentum of 0.9, and a weight decay of  $5e^{-4}$ . The initial learning rate is set to 0.1 and the batch size is set to 160. We generate 1,000 OOD samples per class using Stable Diffusion v1.4, which results in 100,000 synthetic images in total.  $\beta$  is set to 1.0 forImagenet-100 and 2.5 for CIFAR-100. To learn the feature encoder  $h_\theta$ , we set the temperature  $t$  in Equation (2) to 0.1. Extensive ablations on hyperparameters  $\sigma$ ,  $k$  and  $\beta$  are provided in Section 4.2.
Evaluation metrics. We report the following metrics: (1) the false positive rate (FPR95) of OOD samples when the true positive rate of ID samples is  $95\%$ , (2) the area under the receiver operating characteristic curve (AUROC), and (3) ID accuracy (ID ACC).