SlowGuess/NAACL_2022_OCR huggingface dataset:
Eyup Halit Yilmaz and Cagri Toraman
Aselsan Research Center
{ehyilmaz,ctoraman}@aselsan.com.tr
Supervised training with cross-entropy loss implicitly forces models to produce probability distributions that follow a discrete delta distribution. Model predictions in test time are expected to be similar to delta distributions if the classifier determines the class of an input correctly. However, the shape of the predicted probability distribution can become similar to the uniform distribution when the model cannot infer properly. We exploit this observation for detecting out-of-scope (OOS) utterances in conversational systems. Specifically, we propose a zero-shot post-processing step, called Distance-to-Uniform (D2U), exploiting not only the classification confidence score, but the shape of the entire output distribution. We later combine it with a learning procedure that uses D2U for loss calculation in the supervised setup. We conduct experiments using six publicly available datasets. Experimental results show that the performance of OOS detection is improved with our post-processing when there is no OOS training data, as well as with D2U learning procedure when OOS training data is available.
Automated conversational systems have recently received attention from the research community (Dopierre et al., 2021; Mehri et al., 2020; Qin et al., 2021). In applications such as voice assistants, Spoken Language Understanding (Young et al., 2013) aims to extract meaning from the user inputs, called utterances, in order to process and execute desired functionalities. The task of Intent Detection, or Intent Classification, aims to classify user utterance into a set of system-identifiable intents. However, supervised training of such systems can only cover a restricted set of classes, i.e. in-scope (INS) classes. To enhance user experience, the task of Out-of-Scope (OOS) detection (Lin and Xu, 2019a; Xu et al., 2021; Zhan et al., 2021; Shen et al., 2021)
Figure 1: Sample output distributions of an INS classifier predicting the intent of an INS and OOS utterance. Since OOS utterances do not belong to any intent, the prediction gets closer to the uniform distribution.
distinguishes INS utterances from those that do not belong to the scope of the classifier with dedicated model architectures and loss functions.
Existing methods in OOS detection utilize the classifier confidence score for a given utterance with thresholding to classify highly confident predictions as INS and lower confidence predictions as OOS. However, softmax classifiers suffer from overconfident predictions for OOS data (Hendrycks and Gimpel, 2017), which makes it difficult to accurately determine a threshold value. Confidence loss (Lee et al., 2018) mitigates this by calculating the KL Divergence between model prediction and the uniform distribution to decrease the confidence for OOS input. We adapt a similar idea to the zero-shot setup with a novel post-processing step and exploit it jointly in the supervised setup with a learning procedure. The joint application of supervised D2U learning and D2U post-processing forms a novel OOS detection pipeline.
Figure 1 illustrates output probability distributions of a classifier for predicting the intent class for an INS and OOS utterance. The classifier, trained only on INS utterances, is confused when OOS utterance is given. The model assigns closer probabilities for different classes since there is no correct class for this OOS utterance, hence the resulting distribution gets closer to a uniform distribution
Human Language Technologies, pages 2093 - 2108
Figure 2: The histogram of prediction probability scores for INS and OOS utterances (MLE) by using a classifier trained on only INS utterances at the left. Instead of MLE, for the same classifier, cross-entropy score between prediction distribution and uniform distribution (D2U) is given at the right. A vertical decision boundary is more accurately determined with D2U.
than a discrete delta distribution.
Based on this observation, we propose to measure the dissimilarity from or distance to the uniform distribution (D2U). Statistical distance calculations between the prediction and uniform distribution enable the decision boundary to be more accurate. Figure 2 illustrates possible benefits of using distance to the uniform distribution with cross-entropy. The subplot at the left shows the distribution of the number of utterances according to their Maximum Likelihood Estimate (MLE) score. The subplot at the right shows the same distribution according to cross-entropy score between prediction probability and uniform distribution. A decision boundary or threshold can be easily determined using D2U's cross-entropy as a post-processing step without any OOS training data.
When OOS training data is available (Larson et al., 2019), D2U can be used as a loss function to minimize the distance between OOS predictions and the uniform distribution. Such a loss function forces OOS predictions to be less confident, and benefit D2U post-processing further. To test our hypothesis that D2U is a useful method for OOS detection, we answer following research questions:
- RQ1: Does the application of D2U as a post-processing step on INS classifier predictions increase OOS detection performance when there is no OOS training data?
- RQ2: Does incorporating D2U into the training procedure as a particular loss function boost performance when OOS training data is available?
- RQ3: Is the performance of OOS detection significantly improved by D2U over existing state-of-the-art methods?
We divide OOS detection studies into three categories: (i) Confidence-based, (ii) representation-based, and (iii) distance-based methods.
2.1 Confidence-based OOS detection
Threshold-based Methods Thresholding is a common approach in OOS detection (Larson et al., 2019; Feng et al., 2020; Zhang et al., 2020), which reflects the intuition that a classifier output is more confident for a sample that follows its training distribution. The overconfidence problem of softmax classifiers (Hendrycks and Gimpel, 2017), although less apparent in Transformer-based (Vaswani et al., 2017) models (Hendrycks et al., 2020), hinders threshold-based OOS detection performance.
Post-processing Methods The overconfidence problem of softmax classifiers is tackled by post-processing predictions. ODIN (Liang et al., 2018) and SoftMax (Lin and Xu, 2019b) apply temperature scaling for enlarging the confidence gap between INS and OOS instances, since INS logits are ideally further away on the positive axis of the softmax input. Gangal et al. (2020) utilize likelihood ratios with generative classifiers to distinguish OOS predictions. Our method, D2U, employs a confidence-based post-processing method.
2.2 Representation-based OOS detection
Dedicated model architectures or loss functions help represent utterances in a high-dimensional space suitable for OOS detection. Large Margin Cosine Loss (LMCL) ensures that INS intents are tightly clustered (Zeng et al., 2021a), so that OOS utterances are exposed for outlier detection algorithms, such as Local Outlier Factor (Lin and Xu, 2019a). Intent class embeddings (Cavalin et al., 2020) model OOS detection as a reverse dictionary task by mapping intent classes and utterances to the same space. Yilmaz and Toraman (2020) propose a feature representation mechanism that uses KL Divergence to capture the changes in model predictions during sequential processing of utterances.
In order to mitigate data scarcity, Marek et al. (2021) propose a method to generate OOS data with Generative Adversarial Networks. GANs are also utilized to generate high-dimensional representations that are hard to distinguish from that of real utterances, providing adversarial signals to the INS classifier which increases the robustness of the model (Zeng et al., 2021b; Liang et al., 2021).
2.3 Distance-based OOS detection
Distances and divergences are useful tools in OOS detection, since they provide a measure of dissimilarity that can distinguish INS and OOS samples. Xu et al. (2020) utilize Euclidean and Mahalanobis distances with generative classifiers to identify outliers with Gaussian Discriminative Analysis. Mahalanobis distance calculated using representations from the intermediate layers of BERT (Devlin et al., 2019) increases OOS detection performance (Shen et al., 2021). Lee et al. (2018) introduce the confidence loss in Computer Vision for GANs that calculates KL Divergence between the training predictions for OOS samples and uniform distribution.
The idea of measuring the distance between prediction distribution and uniform distribution is utilized in different learning architectures (Lee et al., 2018; Gangal et al., 2020), but not extensively studied for OOS intent detection. Besides, we explore various distance metrics in zero-shot OOS detection and different distance-to-uniform training procedures in supervised setup.
3 Distance-to-Uniform OOS Detection
3.1 D2U post-processing for zero-shot setup
Supervised classifiers trained on INS data model the ground truth labels with a discrete delta function that corresponds to the label, given as follows.
$$
\delta_ {c _ {i}} (x) = \left\{ \begin{array}{l l} 1, & \text {i f} x = c _ {i} \\ 0, & \text {o t h e r w i s e} \end{array} \right. \tag {1}
$$
For the data instance  $i$ ,  $c_{i}$  is the ground-truth label indicating the correct class. The cross-entropy loss between softmax model output and discrete delta function is given as follows.
$$
L _ {C E} = - \frac {1}{N} \sum_ {i = 1} ^ {N} \delta_ {c _ {i}} (x) \log \hat {P} (u _ {i}) \tag {2}
$$
Here,  $\hat{P}(u_i)$  is the output probability distribution of the model for utterance  $u_i$  in a batch of  $N$  utterances, and  $c_i$  is the correct class label for the given utterance. This criterion implicitly forces the model to generate confident predictions for a given data point with maximal confidence score assigned to the ground-truth class label, and low prediction scores for the other classes. When an OOS utterance is given to an intent classifier that is trained using only INS data, the classifier gets confused, i.e., the output probability distribution
is more dissimilar to a delta distribution than what an INS utterance would result in. In other words, output distributions of OOS samples get closer to the uniform distribution than that of INS samples, an observation that we exploit for OOS detection.
The conventional methods for OOS detection make use of a pre-determined threshold value on the Maximum Likelihood Estimate (MLE) score assigned to the predicted label, given as follows.
$$
O O S \left(u _ {i}\right) = \left\{ \begin{array}{l l} 1, & \text {i f} \max  \left(\hat {P} \left(u _ {i}\right)\right) <   \theta \\ 0, & \text {o t h e r w i s e} \end{array} \right. \tag {3}
$$
Here,  $\theta$  is a pre-defined threshold value between 0 and 1, and  $max(\hat{P}(u_i))$  is the MLE score, which considers only the confidence and ignores the shape of the distribution. We exploit the information conveyed by the shape of the entire prediction distribution by first calculating a distance between the output distribution  $\hat{P}$  and the uniform distribution  $U$  before applying the threshold, given as follows.
$$
O O S \left(u _ {i}\right) = \left\{ \begin{array}{l l} 1, & \text {i f} d s t (\hat {P} \left(u _ {i}\right), U) <   \theta \\ 0, & \text {o t h e r w i s e} \end{array} \right. \tag {4}
$$
The distance determined by the  $dst(.)$  function between  $\hat{P} (u_i)$  and  $U$  can be calculated with various distance metrics. We experiment with geometric distance calculations, such as Euclidean distance and Cosine distance; as well as statistical distance calculations, such as Jensen-Shannon distance and symmetrized Kullback-Leibler divergence. The distance value calculated by the  $dst(.)$  function can be intuitively interpreted as the level of confidence of the model. When the distance value is low, the model is less confident and more confused, since the output distribution assigns closer scores for each class.
This is an architecture-agnostic zero-shot post-processing step which can be generalized to any classification model trained with cross-entropy loss with no need for OOS training data. OOS detection in test time is achieved by a function of the prediction distribution given by D2U.
3.2 Distance metrics for post-processing
We examine a number of geometric and statistical distance measures listed as follows.
- Bray Curtis Distance (BC): For two probability distributions,  $u$  and  $v$ , the Bray Curtis distance is given as  $\sum_{i} |u_i - v_i| / \sum_{i} |u_i + v_i|$ .
Figure 3: The supervised learning architecture of D2U. INS utterances are learned with conventional cross-entropy loss against true label distribution, while OOS loss is calculated against the uniform distribution.
- Canberra Distance (Cbr): Canberra distance between  $u$  and  $v$  is  $\sum_{i} \left( \frac{|u_i - v_i|}{u_i + v_i} \right)$ .
- Cosine Distance (Cos): Derived from the Cosine similarity, the Cosine distance is formulated as  $1 - \left( {u \cdot  v/\parallel u{\parallel }_{2}\parallel v{\parallel }_{2}}\right)$  where  $\left| {\cdot ,\cdot }\right|$  is the  ${L}_{2}$  norm.
- Euclidean Distance (Euc): The Euclidean distance between  $u$  and  $v$  is given as  $||u - v||_2$ .
- Hellinger Distance (Helng): The Hellinger distance between  $u$  and  $v$  is  $||\sqrt{u} - \sqrt{v}||_2 / \sqrt{2}$ .
- Cross-Entropy (CE): Cross-Entropy is a measure of dissimilarity between distributions  $u$  and  $v$  given as  $-\sum_{i}u_{i}\log v_{i}$ .
- Symmetrized KL Divergence (KL): The symmetrized Kullback-Leibler divergence is given as  $[KL(u,v) + KL(v,u)] / 2$  where  $KL(u,v) = \sum_{i}u_{i}\log \left(u_{i} / v_{i}\right)$ .
- Jenshen Shannon Distance (JS): JS distance between  $u$  and  $v$  is  $KL(u, m) / 2 + KL(v, m) / 2$  where  $m$  is the mean of two distributions.
3.3 D2U training for supervised setup
When OOS training data is available, we modify the fine-tuning procedure as in Figure 3, to increase the similarity between OOS prediction and uniform distribution. We use pretrained BERT (Devlin et al., 2019) as the classifier network. The loss function for INS utterances,  $L_{ins}$ , is still cross-entropy between true label and prediction, given as follows.
$$
L _ {i n s} = - \frac {1}{N _ {i n s}} \sum_ {i = 1} ^ {N _ {i n s}} \delta \left(c _ {i}\right) \log \hat {P} \left(u _ {i}\right) \tag {5}
$$
For OOS utterances, the loss  $L_{oos}$ , is calculated against the uniform distribution, given as follows.
$$
L _ {o o s} = \frac {1}{N _ {o o s}} \sum_ {i = 1} ^ {N _ {o o s}} d s t (\hat {P} (u _ {i}), U) \tag {6}
$$
Table 1: The statistics of the datasets used in this study.
ACID Banking	CLINC HWU64	SNIPS	TOP
INS	22,172	13,081	22,500	23,431	36,668
OOS	16,000	16,000	16,000	16,000	3,653
Total	38,172	29,081	38,500	39,431	40,321
Vocabulary	25,083	26,702	25,810	26,069	12,610
Avg. Len.	8.95	9.85	8.39	7.25	8.65
Classes	175	77	150	46	7
The total loss is the weighted average over a batch of utterances containing  $N_{ins}$  number of INS utterances and  $N_{oos}$  number of OOS utterances, given as follows.
$$
L _ {t o t a l} = \frac {N _ {i n s} L _ {i n s} + N _ {o o s} L _ {o o s}}{N _ {i n s} + N _ {o o s}} \tag {7}
$$
As the  $dst(.)$  function in Equation 6, we experiment with differentiable functions; such as cross-entropy, KL divergence, and Sinkhorn distance (Cuturei, 2013), named as D2U-CE, D2U-KL, and D2US, respectively. These functions treat the model output and ground truth as probability distributions and provide a differentiable measure. We do not modify the loss calculation for INS utterances so as not to affect the INS classification performance.
Note that this architecture does not model the OOS intent as a separate class. Therefore, post-processing is applied as described in Section 3.1 in test time. Since the loss function incorporates D2U into training, the performance gain by the post-processing is expected to increase.
We use six publicly available intent classification datasets, some of which include labeled OOS data. We give the main statistics of the datasets in Table 1. CLINC (Larson et al., 2019) is a dataset with 150 INS intent classes targeting various domains with curated OOS data. We use the OOS split of CLINC to augment other existing intent detection datasets that do not include labeled OOS data; which are ACID (Acharya and Fung, 2020), Banking (Casanueva et al., 2020), HWU64 (Liu et al., 2019), and SNIPS (Coucke et al., 2018). We observe that HWU64 has many short and noisy utterances, we therefore remove any utterances with length less than or equal to three words.
TOP (Gupta et al., 2018) is an intent detection dataset that generalizes conventional intent labeling with semantic parsing. The intent labels follow a hierarchical structure with potentially many
labels for an utterance. However, we take only root intent class label into account to be consistent with other datasets. The utterances with the intent labels "UNSUPPORTED" and "UNSUPPORTED NAVIGATION" are treated as OOS.
The variety of the number of classes, average length (number of words), and vocabulary size provide a wide spectrum for understanding different OOS detection scenarios. For instance, TOP dataset can be considered a low resource setup since the number of OOS utterances is significantly lower than INS utterances.
To assess the performance of OOS detection, we report the scores of Receiver Operating Curve Area Under Curve (ROC AUC), False Positive Rate at  $90\%$  OOS True Positive Rate (FPR90), and False Negative Rate at  $90\%$  OOS True Negative Rate (FNR90) using sklearn (Pedregosa et al., 2011). These metrics are independent of the threshold value used for decision boundary, providing a means of fair comparison. We also report weighted OOS Recall and weighted OOS F1 based on the threshold value that maximizes the Youden's J statistic (Youden, 1950) on a validation set.
Compared to Precision, Recall is arguably a more critical performance metric for OOS detection; since Recall considers Type II error, meaning that OOS utterances are mislabeled as INS. In this case, the voice assistant would execute a task that the user does not intent to do. We argue that ROC is a more generic measure that considers the performances of varying thresholds, than Recall and F1 considering only a fixed threshold.
4.3 Baseline approaches
In the experiments, BERT (Devlin et al., 2019) with softmax layer is used as the classifier network. For RQ1, the baseline zero-shot post-processing approaches are listed below.
- MLE (Hendrycks and Gimpel, 2017; Hendrycks et al., 2020): The confidence score of a classifier trained only on INS data is used for thresholding.
- Softmax temperature scaling (Temp) (Liang et al., 2018; Lin and Xu, 2019b): As a modification to the MLE setup, the softmax input is applied a temperature value of  $10^{3}$ .
- Standard deviation (Stdev): We use the stdev of the distribution before thresholding since OOS predictions would have lower standard deviation.
Figure 4: Modified leave-one-out 10-fold split strategy that complies with original splits. At each fold, only  $10\%$  of test data is included, while  $90\%$  of training data is retained and the remaining  $10\%$  is used as validation.
- Entropy (Ent) (Shen et al., 2021): The entropy of the prediction distribution,  $H(\hat{P}(u_i))$ , is calculated before applying the threshold, as follows.
$$
O O S \left(u _ {i}\right) = \left\{ \begin{array}{l l} 1, & \text {i f} H \left(\hat {P} \left(u _ {i}\right)\right) > \theta \\ 0, & \text {o t h e r w i s e} \end{array} \right. \tag {8}
$$
For RQ2, we use D2U zero-shot cross-entropy post-processing (D2U-zero) as the baseline method, since we examine any improvement in supervised setup over zero-shot. For RQ3, we compare supervised D2U with the following baselines.
- Large Margin Cosine Loss (LMCL) (Zeng et al., 2021b): Cosine distance among INS class centroids is increased up to a margin. We set the margin as 0.35, and scaling factor as 30.
- **Domain Regularization Module (DRM)** (Shen et al., 2021): DRM introduces domain logits for regularization during INS training. We slightly modify the design and apply sigmoid to domain logits before dividing the classification logits for training stability.
- BERT-Binary (Binary) (Devlin et al., 2019): The "bert-base-uncased" model fine-tuned as a binary classifier for OOS detection.
- Entropy Regularization (Reg.) (Zheng et al., 2020): Entropy of OOS predictions are maximized while minimizing INS training loss.
4.4 Experimental design
To avoid potential annotator-dependent effects as noted by Larson et al. (2019) and comply with the original splits, we modify 10-fold leave-one-out cross-validation as illustrated in Figure 4. The validation splits are used to find confidence threshold values for Recall and F1 calculations. We validate statistically significant differences in the average performances of 10-folds with the two-tailed paired t-test at a  $95\%$  interval with Bonferroni correction.
Table 2: RQ1: D2U-zero with various distance metrics vs. post-processing baselines in zero-shot setup. Row-wise best scores are given in bold.  $(\uparrow)$  and  $(\downarrow)$  indicate that higher and lower scores are better, respectively. "●" indicates statistically significant differences with two-tailed paired t-test at a  $95\%$  interval (with Bonferroni correction  $p<0.0125$ ) in pairwise comparison between D2U-zero and all baselines except the ones marked with "○".
Metric	Dataset	Baselines	D2U-zero
MLE	Temp	Stdev	Ent	CE	BC	Cbr	Cos	JS	Euc	KL	Helng.
ROC AUC (↑)	ACID	90.93	91.83°	91.54	91.98	92.01	91.40	90.18	91.54	92.08	91.54	92.14°	92.13
Banking	94.92	96.15	95.88	96.66	97.03°	96.89	96.09	95.88	96.99	95.88	96.91	96.97
CLINC	95.34	96.16	95.52	95.90	96.32°	96.09	96.26	95.52	96.24	95.52	96.20	96.25
HWU64	79.29	80.46	79.95	80.90	80.32	81.49°	80.49	79.95	81.16	79.95	80.92	81.05
SNIPS	95.45	96.20	95.50	95.70	96.33°	95.61	95.83	95.50	95.86	95.50	96.15	96.04
TOP	74.23	73.23	74.26	74.19	73.24	74.32	74.36°	74.26	74.18	74.26	73.25	73.76
FPR90 (↓)	ACID	25.00	22.10°	20.70°	19.85°	21.40	24.80	28.60	20.70	20.90	20.70	19.70°	20.70
Banking	13.30	10.00	11.30	7.85	7.30	6.60	12.00	11.30	6.20	11.30	6.30	6.10°
CLINC	9.30	7.80°	9.30	8.00°	7.50°	8.10	7.70	9.30	7.70	9.30	7.80	7.60
HWU64	58.90	54.10	55.70	48.12	52.50	51.60	55.00	55.70	52.20	55.70	52.60	51.90
SNIPS	10.40	8.40	10.30	10.71	8.40	10.40	10.30	10.30	9.90	10.30	8.60	8.70
TOP	51.38	53.00	51.38	66.65	53.00	51.38	51.38	51.38	51.50	51.38	53.00	51.75
FNR90 (↓)	ACID	27.30	25.67	26.22	19.70°	21.46°	20.60°	26.70	26.22	20.45°	26.22	21.38°	20.75°
Banking	14.36	10.49	11.37	7.20°	8.01°	8.79°	13.68	11.37	7.88°	11.37	7.69°	7.79°
CLINC	11.80	9.16°	11.60	8.90	8.36	8.11	7.56°	11.60	7.98	11.60	8.67	8.31
HWU64	51.97	52.26	51.75	52.80°	52.01	47.14	47.01°	51.75	48.63	51.75	51.03	49.57
SNIPS	11.14	9.29	11.14	11.80	9.29	11.14	11.14	11.14	10.71	11.14	9.71	10.29
TOP	67.88	69.05	67.89	51.50	69.07	67.89	67.90	67.89	68.17	67.89	69.04	68.46
Note that the test splits do not overlap in order to satisfy the independence criterion of t-test.
The experiments are designed with respect to our research questions (RQ 1-3). First, we fine-tune a BERT classifier (Devlin et al., 2019) using huggingface implementation (Wolf et al., 2019) for INS intent detection with cross-entropy loss, and apply different D2U post-processing methods for RQ1. Then, we fix the post-processing method, and examine the effect of supervised D2U training for RQ2. Lastly, we compare D2U with state-of-the-art baselines for RQ3 to assess the performance gain of our method.
4.5 Experimental results