Not So Fast, Classifier – Accuracy and Entropy Reduction in Incremental Intent Classification (2021)

Abstract: Incremental intent classification requires the assignment of intent labels to partial utterances. However, partial utterances do not necessarily contain enough information to be mapped to the intent class of their complete utterance (correctly and with a certain degree of confidence). Using the final interpretation as the ground truth to measure a classifier’s accuracy during intent classification of partial utterances is thus problematic. We release inCLINC, a dataset of partial and full utterances with human annotations of plausible intent labels for different portions of each utterance, as an upper (human) baseline for incremental intent classification. We analyse the incremental annotations and propose entropy reduction as a measure of human annotators’ convergence on an interpretation (i.e. intent label). We argue that, when the annotators do not converge to one or a few possible interpretations and yet the classifier already identifies the final intent class early on, it is a sign of overfitting that can be ascribed to artefacts in the dataset.

## introduction in non-incremental spoken dialogue systems (sds), modules process complete utterances sequentially: the automatic speech recognition (asr) module must detect an end of turn before the transcribed speech can be processed by the natural language understanding (nlu) module, in which utterances are often assigned an intent label. the sequential execution of such systems not only increases response latency, but also affects the perceived naturalness of the interaction. natural conversations typically proceed incrementally: people rely on multiple cues to build partial interpretations of incomplete input, check if the communication is successful, and adapt their production accordingly @xcite , sometimes completing the interlocutor's turn, barging in, or responding before the turn is over @xcite . incremental dialogue systems, on the other hand, make use of incremental processors to enable more efficient, flexible, and effective interactions with users @xcite . for example, the asr module of an incremental, task-oriented sds continuously processes incoming speech signals and posts hypothesized transcriptions. then, the downstream nlu module analyses the increasingly longer portions of the final utterance, before the user finishes their turn. these early hypotheses produced by an incremental nlu (inlu) module can in turn be consumed by downstream modules as soon as they are posted. thanks to incremental processing, an sds can approximate various characteristics of human dialogue, such as timing backchannel responses or optimizing turn-taking between speakers @xcite @xcite . incremental processing can also improve the computational efficiency of an sds, for example by accessing timeconsuming external services (e.g. database queries) before the end of a user's turn. what's more, incremental processing enables new types of interactions between a dialogue system and its users. for example, @xcite developed a personal assistant that could communicate its incremental understanding of the user's ongoing speech and its prediction states by graphically displaying a branching tree. at the start of the interaction, a tree with one branch per supported intent is displayed. once the user's intent is recognized by the system, the tree is adjusted and its associated slots are displayed as subbranches to its node. this visual feedback increases the transparency of both the system's capabilities and current understanding; it also guides the user to provide values for all required slot for the task. intent classification, a common task assigned to the nlu module of a task-oriented sds, poses the problem of identifying the point of the utterance where the intent has been classified with a given degree of certainty. an early identification of the correct intent, however, is not necessarily a sign of an effective classifier: when an inlu module identifies the correct intent label before a human can (for example, after processing a not yet informative partial utterance such as "i'd like to"), then its "success" may likely be caused by the presence of artefacts in the training set. in order to provide an upper baseline for incremental intent classification, we present inclinc foot_0 , a dataset of crowd-sourced incremental intent annotations, where utterances are broken into increasingly longer partial utterances by identifying peaks and troughs in surprisal @xcite as boundaries. we then compare the performance of human annotators to that of a transformer-based classifier @xcite . we propose entropy reduction (between the different intent interpretation hypotheses) as a measure of uncertainty reduction during human incremental intent identification. we show that, for a substantial amount of the partial utterances, the final intent label is not yet identifiable to humans, and that a reduction in uncertainty (as annotators converge on an interpretation) is typically associated with an increase in accuracy for the annotators. we argue that, when the human annotators do not converge to one or a few possible interpretations and yet the classifier already identifies the final intent class early on, it is a sign of overfitting that can be ascribed to artefacts in the dataset. ## proposed approach we propose the theory of entropy reduction as a lens through which to view the problem of incremental intent classification. the theory suggests that more cognitive effort is needed to process an encountered word that greatly reduces the probable interpretations of the complete sentence, as compared to a word which does not. like human listeners, inlu modules form (multiple) early predictions (in this case, intent predictions) for partial utterances during incremental processing, which can be revised or revoked as more words arrive. of course, the computational cost of a typical inlu module does not vary as a function of an input's surprisal or the entropy reduction that it triggers. however, identifying which parts of an utterance trigger a considerable reduction in the set of plausible intent interpretations may be helpful when evaluating the performance of such a module. when the set of interpretations is too open (i.e. a partial utterance could conceivably belong to almost any intent), identifying one correct answer (and potentially acting upon it) does not make sense. as this set narrows, comparing the human hypotheses with the classifier's hypotheses may help understand the classifier's decisions. intent labels for complete utterances do not show how intent interpretations by humans change as an utterance progresses. to the best of our knowledge, however, there is no (publicly-available) dataset with incremental annotations of utterances with intent labels. our first contribution is a dataset of partial and full utterances with human annotations of plausible intent labels at different portions of each utterance, which can provide an upper baseline for incremental intent classification. as a second contribution, we use the collected annotations for an analysis of the performance of a transformerbased classifier on an equivalent task of incremental intent classification @xcite . we hypothesise that humans will outperform the classifier, achieving a higher overall accuracy and higher word savings due to correct early predictions. re-sponses from the humans are then used to examine over-fitting versus under-performance of the classifier. lastly, to showcase the application of our dataset, we present an analysis of the relationship between entropy reduction and increases in accuracy during incremental intent classification. based on previous research, such as that by linzen and jaeger (2016), we hypothesise that a reduction in entropy between two subsequent partial utterances are more frequently associated with increases in accuracy than between partial utterances with no changes/increases in entropy. ## discussion humans as an upper baseline bender and koller (2020) debate how much meaning a trained neural language model understands and argue that, when a system outperforms inter-annotator agreement, the task likely contains artefacts that do not represent meaning. as such, over-fitting in the context of incremental intent prediction can be assessed by examining the cases where distilbert predicts the complete label for an earlier partial utterance than the annotators. conversely, areas where a model could be improved can be studied by looking at utterances where the annotators predicted the complete utterance's intent earlier. we looked at partial utterances whose complete label was predicted by either the annotators or the classifier, but not both. these disagreements, found in 16% of partial utterances, were used to examine over-fitting and under-performance. evidence for overfitting for eight partial utterances, distilbert predicted the complete utterance's label, but the majority of participants did not. these partial utterances included: "tell my", "i have to", and "on the" (complete list in evidence for under-performance 61 utterances were correctly predicted by the participants but not by distilbert. of these, 24 had at least moderate agreement (α ≥ 0.50) between participants (values reported in appendix d). these utterances do not represent the point where intent is identifiable by a vast majority of participants. rather, they represent subtle differences in formulation of utterances that humans, but not the classifier, might associate with a certain intent class, which results in a considerable amount of (but not all) participants predicting the final intent. a few interesting observations can be made. first, "i need milk" was correctly assigned to update/add to shopping list by participants, while dis-tilbert predicted place an order. people are probably more likely to buy perishable items such as milk at a store in-person: a failure to "understand" this real-world knowledge might have caused the classifier to miss this prediction. next, the familiarity of participants with the well-known expression "how's the weather" is visible in the agreement of α = 1.0 for the partial utterance "how's the" and class get weather. the completion "how's the" to "how's the weather" suggests that "how's the" is a high-cloze phrase @xcite and "weather" is a highly-predictable continuation. note that the missing word "weather" is also found in the label for its intent, and that "how's the weather" matches an utterance one would expect to hear in the context of a task-oriented sds with inclinc's supported intents. this example can be considered the humanequivalent of the artefact in entropy reduction during incremental intent classification, not all steps contribute equally to the final interpretation. we identified surprisal peaks and troughs as relevant points to break the utterances into informative incremental chunks, as informed by the token's conditional probability. we then proposed er (computed from the human interpretations) as a metric to demonstrate the incremental narrowing of the set of plausible final intents at these breaking points.utterances accompanied by a reduction in entropy are more frequently associated with an increase in accuracy compared to those with no changes/increases in entropy. er serves as a tangible metric of how much certain (chunks of) words, driven by information content, restrict the set of plausible final intents and is thus accompanied by increased prediction accuracy. limitations changes in er across partial utterances could be affected by a) different representations of the intent classes across participants and/or b) their differing language abilities. this limitations arises as our (relatively small number of) participants were recruited online and their english fluency was self-reported 8 . with six to nine annotations per item, each response has significantly influence the set of annotations for a given item. collecting more annotations would be beneficial to reduce fluctuations in the distribution of responses triggered by individual participants and to allow for a more in-depth study of the role of entropy reduction in incremental intent classification (controlling for e.g. the surprisal and the predictability of the next word, the position in which reductions of entropy are expected, etc.). additionally, when the complete utterance's label was among those with the most votes, then this label was selected as the prediction: this is a biased selection of a favourable answer. 3.5% of stimuli were resolved using this heuristic and annotators' accuracy in ## conclusion assigning an utterance's ground-truth label to its partial utterances is a common oversimplification of incremental intent classification. existing studies report global performance metrics across partial utterances and fail to distinguish between those not containing enough information to be representative of the final intent and those who do. it is then unclear to what degree a classifier's performance is attributable to overfitting to the evaluated dataset versus to a generalisable representation of intent classes. we proposed an alternate lens to view this task: evaluating a model of incremental intent classification should not just be a matter of getting the intent right. rather, it should be about predicting a set of plausible labels for an ongoing utterance and communicating with what certainty the final intent has been classified at this point. as an alternative to simply adopting the complete utterance's label, we presented a new dataset with annotated incrementalised utterances. we proposed a novel method for determining plausible intent annotations at relevant points -from an information-theoretic point of view -in an ongoing utterance. with humans as an upper baseline, the performance of incremental intent classifiers may be evaluated against our labeled annotations. the remainder of our work then focused on the evaluation of a transformer-based intent classifier in an incremental setting. to analyse our dataset, we proposed er as a metric to detect the incremental narrowing of intent interpretations by human annotators. our analysis showed that er more frequently accompanied increased prediction accuracy for the annotators compared to decreases/no changes in accuracy. er has the potential to identify the earliest point of understanding, before which accuracy and word savings are not informative measures to evaluate the performance of an incremental classifier, as before this point the upper performance bound is unknown. while our work did not focus on the training of incremental intent classifiers, inclinc opens up several new possibilities. for example, its annotations could be used to assess and improve the confidence calibration (in the strong sense, see @xcite of multi-class classifiers to ensure that their softmax output more reliably estimates the uncertainty about the (final) intent at a given point in an ongoing utterance. providing strongly-calibrated confidence scores alongside predictions would help increase model interpretability, facilitate its integration into other probabilistic models @xcite , and help coordinated processing across modules in an incremental sds @xcite . alternatively, the annotations could be used as labels in a multiintent classification setting for partial utterances foot_6 . this approach would circumvent the problem of assigning a complete utterance's label to a partial utterance with not yet enough semantic information relevant to the target class: all plausible labels could be trained for partial utterances. finally, er in a sds could be monitored across ongoing utterances to learn points at which the entropy of possible interpretations is low enough for the given task. a dataset with annotations such as inclinc could provide useful labels for this paradigm. all in all, models which are able to overcome the challenges of incremental intent classification could have the potential to learn more robust and generalisable class representations, which will could conceivably improve their performance in a reallife applications.