Uncertainty Estimation of Transformer Predictions for Misclassification Detection (2022)

Abstract: Uncertainty estimation (UE) of model predictions is a crucial step for a variety of tasks such as active learning, misclassification detection, adversarial attack detection, out-of-distribution detection, etc. Most of the works on modeling the uncertainty of deep neural networks evaluate these methods on image classification tasks. Little attention has been paid to UE in natural language processing. To fill this gap, we perform a vast empirical investigation of state-of-the-art UE methods for Transformer models on misclassification detection in named entity recognition and text classification tasks and propose two computationally efficient modifications, one of which approaches or even outperforms computationally intensive methods.

## introduction machine learning methods are naturally prone to errors as they typically have to deal with ambiguous and incomplete data during both training and inference. unreliable predictions hinder the application of these methods in domains, where the price of mistakes is very high, such as clinical medicine. even in more error-tolerant domains and tasks, such as intent recognition in generalpurpose chatbots, one would like to achieve a better trade-off between expressiveness of a model and its computational performance during inference. since mistakes are inevitable, it is crucial to understand whether model predictions can be trusted or not and abstain from unreliable decisions. uncertainty estimation (ue) of model predictions aims to solve this task. ideally, uncertain instances should correspond to erroneous objects and help in misclassification detection. besides misclassification detection, ue is a crucial component for active learning @xcite , adversarial attack detection @xcite , detection of out-of-distribution (ood) instances @xcite , etc. some classical machine learning models, e.g. gaussian processes @xcite , have built-in ue capabilities. modern deep neural networks (dnns) usually take advantage of a softmax layer, which output can be considered as a prediction probability and be used for ue. however, the softmax probabilities are usually unreliable and produce overconfident predictions @xcite . some previously proposed techniques such as deep ensemble @xcite are known for producing good ue scores but require a large additional memory footprint for storing several versions of weights and multiply an amount of computation for conducting several forward passes. reliable ue of dnn predictions that does not introduce high computational overhead is an open research question @xcite . in this work, we investigate methods for ue of dnns based on the transformer architecture @xcite in misclassification detection. we consider two of the most common nlp tasks: text classification and named entity recognition (ner). the latter has been overlooked in the literature on ue. to our knowledge, this work is the first to consider ue for ner. we propose two novel computationally cheap methods for ue of transformer predictions. the first method is the modification of the monte carlo dropout with determinantal point process sampling of dropout masks @xcite . we introduce an additional step for making masks more diverse, which helps to achieve substantial improvements and approach the performance of computationally-intensive methods on ner. the second method leverages mahalanobis distance @xcite but also adds a spectral normalization of the weight matrix in the classification layer @xcite . this method achieves the best results on most of the datasets and even outperforms computationallyintensive methods. we also investigate recently proposed regularization techniques in combination with other ue methods. the contributions of this paper are the following: ## related work it is well known that reliable uncertainty scores can be obtained simply by constructing an ensemble of decorrelated neural networks (deep ensemble) @xcite . however, such a straightforward approach is coupled with substantial computational and memory overhead during training an ensemble, performing inference of all its components, and storing multiple versions of weights. this overhead is a serious obstacle to deploying ensemble-based uncertainty estimation methods in practice. uncertainty estimation is a built-in capability of bayesian neural networks @xcite . however, such models have similar issues as ensembles and also require special training procedures. recently, it was shown by @xcite that dropout, a well-known regularization technique, is formally equivalent to approximate variational inference in a deep gaussian process if it is activated during prediction. this method, known as monte carlo (mc) dropout, uses the approximating variational distribution with bernoulli variables related to network units. mc dropout does not impose any overhead during training, introduces no additional parameters, and thus does not require any additional memory. the main disadvantage of this method is that it usually requires many forward-pass samplings for approximating predictive posterior, which makes it also computationally expensive. recently, many works have investigated the approximate bayesian inference for neural networks using deterministic approaches: @xcite al. (2018); liu et al. (2020); van amersfoort et al. (2020); mukhoti et al. (2021); shen et al. (2021) , etc. these methods do not introduce notable overhead for inference, storing weights, and usually require compatible training time. however, most of the research in this area is accomplished for computer vision tasks. for text classification, a series of works investigates ue methods for the ood detection task @xcite @xcite . in this work, we focus on a more challenging taskmisclassification detection. while ood detection requires to model only the epistemic uncertainty inherent to the model and caused by a lack of training data, misclassification detection also requires to model aleatoric uncertainty caused by noise and ambiguity in data @xcite . we consider recently proposed methods in this area that are evaluated in text processing. three recent works propose techniques for misclassification detection based on an additive regularization of a training loss function. @xcite suggest adding a penalty that reduces the euclidean distance between training instances of the same class and increases the distance between instances of different classes. @xcite suggest using two components in the loss function that reduce the difference between outputs from two versions of a model initialized with different weights. they also use mix-up @xcite to generate additional training instance representations that help to capture aleatoric uncertainty, self-ensembling, mc dropout, and a distinctiveness score to measure the epistemic uncertainty. @xcite introduce a regularizer that penalizes overconfident instances with high loss. in another recent work, @xcite propose to combine mc dropout with a determinantal point process (dpp) to improve the diversity of predictions by considering the correlations between neurons and sampling the diverse neurons for activation in a dropout layer. in this work, we conduct a systematic empirical investigation of ue methods on nlp tasks. we evaluate combinations of methods that have not been tested before and propose two modifications, one of which achieves the best results among computationally cheap methods. the previous work focuses on text classification tasks, while this work is the first to investigate ue also for ner. ## background and methods in this section, we describe the baselines and propose novel uncertainty estimation techniques. ## experimental setup in the experiments, we train a model on a given dataset and perform inference on a separate test set to compute both predictions and ue scores u. we are interested in how the scores correlate with the mistakes ẽ of the model on the test set. for text classification, mistakes are computed in the following way: for ner, we use two evaluation options: tokenlevel and sequence-level. for the token-level evaluation, individual tokens are considered as separate instances as in the text classification. for the sequence-level evaluation, mistakes are computed in the following way: ## conclusion our extensive empirical investigation on text classification and ner tasks shows that computationally cheap ue methods are able to substantially improve misclassification detection for transformers, performing on par or even better than computationally intensive mc dropout and deep ensemble. the proposed in this work method based on the mahalanobis distance and spectral normalization of a weight matrix (md sn) achieves the best results among other computationally cheap methods on text classification datasets and is on par with expensive methods. this method does not require seriously modifying a model architecture, extra memory storage, and introduces only a little amount of additional computation during inference. we also show that our modification of dpp mc dropout that leverages the diversity of generated dropout masks, which is also a computationally cheap method, is able to outperform the softmax response baseline and approach the computationally intensive methods on ner. finally, we find that regularization can slightly improve the results of methods based on mc dropout and the mahalanobis distance in text classification. the spectral normalization is theoretically proven to ensure bi-lipschitz constraint on the transformation defined by the standard residual connection network @xcite . however, the self-attention blocks in transformers have a more complicated architecture than the layers of standard resnets, which means that the theoretical guarantees for them do not hold in general. in future work, we are looking forward to investigating other techniques to ensure bi-lipschitz constraint on self-attention blocks, which might further improve deterministic methods for uncertainty estimation of transformers.