Efficient, Uncertainty-based Moderation of Neural Networks Text Classifiers (2022)

Abstract: To maximize the accuracy and increase the overall acceptance of text classifiers, we propose a framework for the efficient, in-operation moderation of classifiers’ output. Our framework focuses on use cases in which F1-scores of modern Neural Networks classifiers (ca. 90%) are still inapplicable in practice. We suggest a semi-automated approach that uses prediction uncertainties to pass unconfident, probably incorrect classifications to human moderators. To minimize the workload, we limit the human moderated data to the point where the accuracy gains saturate and further human effort does not lead to substantial improvements. A series of benchmarking experiments based on three different datasets and three state-of-the-art classifiers show that our framework can improve the classification F1-scores by 5.1 to 11.2% (up to approx. 98 to 99%), while reducing the moderation load up to 73.3% compared to a random moderation.

## introduction accurately classifying an overwhelming amount of textual data is a common research challenge @xcite . in recent years, machine learning approaches, particularly neural networks (nns), have received great attention to support textual classification @xcite . however, in practice, fully automated approaches are still rare due to the general lack of top, almost-perfect classification accuracy. if the accuracy of a trained and hyper-tuned state-of-the-art classifier still does not meet the domain requirements, a full manual approach is likely to be the fall-back solution. to prevent classification mistakes and strengthen the overall acceptance of artificial decision making, socio-technical approaches that integrate human domain experts in the decision loop are gaining in importance @xcite . recent research has shown that including the prediction uncertainty of nns can detect more complex text inputs, i.e., either short or very long texts with less informative tokens @xcite , and probably wrong @xcite predictions, which are worth checking manually. since human resources are cost intensive and do not scale well to larger workloads, moderation processes should be designed with human-resource-efficiency in mind. yet, the efficient in-operation integration of human efforts for building semi-automated decisionmaking systems -i.e. moderated classifiers -remains largely unexplored. this paper introduces a novel framework for the efficient moderation of nn text classifiers. our framework extends a given nn with human expertise to create a semi-automated decision-making system. text instances are moderated manually when classifier outcomes are likely to be false. we use the concept of prediction uncertainty @xcite to quantify the reliability of a classification. when a classifier is highly uncertain, we let human moderators intervene. to minimize human efforts, we propose to limit the moderation to the point where the classification accuracy gain saturates. while active learning @xcite aims to limit human efforts during the training of classifiers, our moderation framework seeks to substantially enhance the accuracy of trained and already deployed classifiers -surpassing the maximum achievable accuracy of an automatic classifier to still achieve an almostperfect in-operation accuracy. our contribution is twofold. first, we introduce a novel saturation-based framework for the efficient, in-operation moderation of nn-based text classifiers. second, we empirically evaluate the accuracy improvement and needed moderation load for three english text classification tasks including hate speech detection, sentiment analysis, and topic classification. we run multiple training trails using different predictive uncertainty estimation approaches and compare their initial and post-moderation f1-scores and evaluate their suitability. the remainder of the paper is structured as follows: section 2 introduces our moderation framework and outlines the uncertainty estimation techniques to decide when to moderate. then, section 3 describes the setting to evaluate our framework. afterwards, we report on the experiment results in section 4 and discuss the implications and limitations of our findings in section 5. finally, section 6 discusses related work and section 7 concludes the paper. ## related work the moderation of classifier outputs can clearly be considered as an application of the human-inthe-loop (hil) paradigm @xcite . so far, most hil implementations focus on querying additional labels from humans for the purpose of training in order to reduce a classifier's uncertainty, commonly referred to as active learning @xcite . in comparison, our framework aims to efficiently prevent error-prone classifications during operation and thus to further enhance the accuracy of an already trained classifier. different approaches have been previously discussed to coordinate human involvement in semiautomatic text classification. to our best knowledge, we are the first to investigate the human efficient, moderation of text classifiers. @xcite propose to search for confidence thresholds which maximize a classifier's accuracy, when classification outcomes are moderated manually. however, this approach requires moderators to set the amount of data they are willing to moderate. assessment of the efficiency is not made. another approach to reduce error prone classifications is to let classifiers abstain when no clear decisions can be made @xcite . this can be performed by adding an additional label to the classification task or by training a separate and independent classifier. abstained instances could be, similarly to our approach, passed to human moderators. our work focuses on uncertainty modeling using nns. @xcite propose a classification approach with a reject option which additionally allows practitioners to set a desired level of risk. similar to our approach, they aim to ensure a certain classification performance. in contrast, they do not focus on the efficiency of human involvement. @xcite propose an approach to detect whether an instance is out-of the distribution of the training dataset and thus probably wrongly classified. however, their approach requires an auxiliary dataset representing out-of-distribution samples during training which is difficult to create. @xcite introduce a semi-automated approach which directly optimizes a classifier for different automation levels. however, their approach is only applicable to convex-margin based classifiers and not to nns. @xcite suggest a self-checking mechanism for nn, where the features of the internal layers are used to check the reliability of predictions. in contrast, our approach uses predictive uncertainties obtained via a softmax function, which is rather simple to implement. moderating classifiers' outcomes is also related to the field of explainable ml, in particular explaining individual classification outcomes @xcite . studies indicate that explaining relevant words of a class outcome support human annotation tasks by, e.g., reducing the annotation time needed per instance and increasing user trust @xcite . our approach is likely to benefit from explaining artificial decisionmaking as well as model uncertainties @xcite during the moderation process. ## conclusion this paper contributes to the human-in-the-loop ai paradigm. we particularly present a rather simple, semi-automated text classification framework to efficiently minimize unreliable and error-prone classification outcomes. based on explicit uncertainty modeling, the framework seeks to prevent unconfident classifications by consulting human moderators. at its core, the framework uses a saturation-based moderation strategy, which limits the moderation load and keeps it human-resourceefficient. we conduct several benchmarking experiments including state-of-the-art classifiers and public datasets to examine the effectiveness of the suggested moderated classification. our evaluation shows that a moderated classifier can achieve a major increase in accuracy while limiting the moderation efficiency. with moderation, the f1-score of a convolutional nn for hate speech detection increases from initially 90.6 to 98.10% limiting the manual effort to 31.6%. using distilbert, the benefits of a moderated classifier seems even stronger. here, our framework accomplishes an improvement of the absolute f1score from 94.1% to 99.37% while only moderating 23.9% of the data. across all our experiments, we increased the f1-score from initially ∼89-94% to ∼98-99% by manually moderating between a third to a fourth of the data. our results indicate that an uncertainty-based moderated classification can increase the applicability and reliability of text classifiers, particularly in domains where a top accuracy of ∼99% is required and a full manual classification would be more expensive.