Word-Level Uncertainty Estimation for Black-Box Text Classifiers using RNN s (2020)

Abstract: Estimating uncertainties of Neural Network predictions paves the way towards more reliable and trustful text classifications. However, common uncertainty estimation approaches remain as black-boxes without explaining which features have led to the uncertainty of a prediction. This hinders users from understanding the cause of unreliable model behaviour. We introduce an approach to decompose and visualize the uncertainty of text classifiers at the level of words. Our approach builds on top of Recurrent Neural Networks and Bayesian modelling in order to provide detailed explanations of uncertainties, enabling a deeper reasoning about unreliable model behaviours. We conduct a preliminary experiment to check the impact and correctness of our approach. By explaining and investigating the predictive uncertainties of a sentiment analysis task, we argue that our approach is able to provide a more profound understanding of artificial decision making.

## introduction neural networks or variations of them achieve state-of-the-art accuracy across a wide range of text classification tasks like sentiment analysis @xcite or spam detection @xcite . neural networks are however not interpretable, since they provide no information about why particular decisions were made @xcite . this lack of transparency makes it hard for users to assess the certainty and trustfulness of predictions. in the worst case, an unreliable prediction is considered correct even if it is not. it is thus important to know what a model does not know. this would allow treating particular error prone or unreliable predictions with additional care -enabling a better understanding of why wrong predictions occur. several techniques to assess the uncertainty of individual predictions have been successfully applied to neural networks @xcite @xcite . however, a global uncertainty estimation for the whole input text does not describe which features lead to an uncertain prediction. it is thus unclear, why a neural networks classifier is, e.g., uncertain whether a text represents a positive sentiment or whether it is a spam. as a first step towards a better understanding of neural networks-based text classification tasks, we suggest the decomposition of uncertainties on the level of words. we present a novel uncertainty modelling approach that estimates word-level uncertainties in any text classification task. our approach applies bayesian modelling to a sequence attribution technique for recurrent neural networks (rnns). we implement the approach using tensorflow and keras and demonstrate its effectiveness by investigating word-level uncertainties in a sentiment analysis task. ## word-level uncertainty estimation we first introduce how we model uncertainty in a classification task and then describe how we decompose the prediction uncertainties on the level of words. ## conclusion and further work this paper proposes a simple novel approach to estimate word-level uncertainties in text classification tasks. our approach uses monte carlo dropout in conjuncture with a sequence modelling technique to decompose uncertainties. our approach does not require additional labelling effort beside the original training data for the classifier. we exemplary show that the transparency gained by applying our approach enables a deeper understanding of artificial decision-making. the visualization in figure 3 can, e.g., help a human moderator to understand the classifier uncertainty. we plan several empirical studies to examine the impact and benefits of word-level uncertainty awareness in human-in-the-loop applications @xcite . further, we plan to adapt and compare our approach to additional rnn variations like bidirectional lstms and gated recurrent units (grus).