Annotating Errors and Emotions in Human-Chatbot Interactions in I talian (2020)

Abstract: This paper describes a novel annotation scheme specifically designed for a customer-service context where written interactions take place between a given user and the chatbot of an Italian telecommunication company. More specifically, the scheme aims to detect and highlight two aspects: the presence of errors in the conversation on both sides (i.e. customer and chatbot) and the “emotional load” of the conversation. This can be inferred from the presence of emotions of some kind (especially negative ones) in the customer messages, and from the possible empathic responses provided by the agent. The dataset annotated according to this scheme is currently used to develop the prototype of a rule-based Natural Language Generation system aimed at improving the chatbot responses and the customer experience overall.

## introduction conversational agents, or chatbots, have become a widespread technology used by an ever increasing number of companies. most often they are used to automate tasks such as customer support inquiries, providing a 24/7 livechat service via web-based interfaces or standalone mobile apps and enabling natural language interactions with the users. chatbot systems typically include natural language understanding (nlu) modules aimed at identifying users' intents and extracting the relevant information; these systems either resort to a set of predefined response templates or use natural language generation (nlg) techniques to reply to the user. despite the ever increasing chatbot abilities to recognize user requests and provide appropriate answers, miscommunication is still a common issue in human-chatbot interactions @xcite . such shortcomings go hand-in-hand with the need to strengthen, also and especially in customer support contexts, the ability of chatbots to interact in a human-like fashion, so as to encourage the users to engage more in the conversation and to comply with the agent's requests. an improved interaction quality also passes through the creation of emotion-aware chatbots able to recognize the emotional state of the customer and to respond appropriately, showing some form of empathy @xcite . the work presented here forms part of a wider project consisting in the development of the prototype of a rule-based genuine nlg module @xcite . the aim of this module is to improve the answers provided by the template-based chatbot of an italian online customer service. its main feature is that it takes as input a set of annotated dialogues between the chatbot and the customers, and it substitutes the template responses provided by the chatbot system, taking into account also various dimensions in the generation process, such as possible errors in the conversations and the presence of emotions (especially negative ones) in the user messages. the intended purposes in building this module are the enhancement of the chatbot responses on multiple levels. from a strict content viewpoint, a selection procedure inspired by narrative roles in explanation @xcite has been developed in order to generate more relevant and ah hoc responses to individual customers' requests as described in di @xcite . in addition, an emotional layer has been added in the module design to generate empathic responses, more in line with customers' feelings and expectations. the development of task-oriented dialogue systems for italian has raised great interest (especially from private companies) and there are high expectations for a new generation of dialogue systems that can naturally interact and assist humans in real scenarios. however, work on italian language systems remains limited. some related task with a focus on evaluation has been recently proposed in the context of evalita, the periodic evaluation campaign of nlp and speech tools for italian @xcite . as this highlights, developing synergies between public institutions and private companies is extremely important to improve existing solutions for the italian language, in particular with regard to effectiveness of the dialogue, user satisfaction and societal consequences and risks of dealing with conversational agents. our aim with this contribution is also to partially fill this gap, presenting the result of a corpusbased study of a set of real human-chatbot conversations carried out in the context of a virtous example of collaboration between academia and private companies. in this paper, we describe a multi-layer annotation scheme developed to annotate the aforementioned dialogues in the italian language. the main novelty of this scheme is that it focuses on two dimensions: conversation errors and emotions. the design of the scheme has been driven by two main objectives: 1) to provide a qualitative analysis of these interactions, using a sample dataset to explore the possible interrelations between errors and emotions in conversation, thus providing the company with the opportunity to establish adequate error handling strategies, 2) to feed the nlg module with this additional information so as to improve the newly-generated responses. the distinguishing feature of the proposed scheme is that, in order to make sense of the complexity of the tagset relating to the possible errors, the error types have been clustered in a coarse-grained taxonomy whose classes are partially inspired by the popular gricean maxims upon which the cooperative principle is built @xcite . after outlining past related work, we hereby introduce the dataset used to apply the scheme. we then provide an overview of the scheme itself with some basic statistics and examples. finally, we provide a qualitative analysis of the annotated data, discussing the main issues arisen during the annotation process. ## related work error analysis and classification in both spoken and written human-machine interactions has been a matter of study for decades and its goal is mainly to define an improved set of error recovery strategies or to predict eventual conversation breakdowns. in @xcite , a scheme was developed to establish the degree of comprehension of a task-oriented spoken dialogue system, along with the effects on the user experience. user utterances were thus annotated according to how well they were understood by the system, using the following set of tags: non-understanding, misunderstanding, partial understanding, full understanding. more recently, an improved taxonomy of errors was proposed in @xcite , based on the revision and comparison of two previous taxonomies @xcite built according to a theory-driven, top-down approach the former, and with a data-driven, bottom-up approach the latter. in this revision process, both taxonomies were applied to the datasets used for the shared tasks on automatic breakdown detection @xcite ; these datasets consist of dialogues between humans and chat-oriented systems. the schemes were meant to identify systems' errors within conversations. the revision process and eventual comparison of the schemes showed a higher reliability, in terms of inter-annotator agreement results, of the bottom-up taxonomy. while the above-mentioned contributions focused on system's errors, a past work by @xcite aimed specifically at studying user errors, i.e., those cases where the user fails to comply with the normative model of the dialogue (e.g., not following the system's explicit instructions or deliberately refusing to be cooperative). finally, in @xcite , both parties are taken into account while designing an error annotation scheme within the context of a task-based spoken interaction. in fact, a set of errors on different levels (goal, task, command, concept) was defined for user utterances, but also a recognition-level error was introduced to capture system's mistyping and automatic speech-recognition errors. interestingly, each user-system exchange was also annotated with a label aimed at capturing the consequences of such errors, which may be stagnation, repetition (as a special case of stagnation), regression, or rather partial or complete progress. parallel to error analysis is the annotation of human emotions in dialogues. while a large number of contributions on open-domain, human-human interactions can be found in literature (see @xcite for an overview), relatively fewer works are available on the annotation of human-machine dialogues, and more specifically in task-based domains, such as customer service. in addition, the distinguishing purpose of such systems -namely to complete a given task, be it booking a flight of providing assistance of any kind -often makes traditional schemes, such as those based on ekman o plutchik theories @xcite , partially inadequate to provide a proper emotion representation within such contexts. as a result, work on this subject mostly focuses on negative emotions that may result from partial or complete failure in completing the given task. typically, the range of negative emotions include anger @xcite @xcite , frustration @xcite @xcite -even providing fine-grained distinctions between different degrees of emotions @xcite -but also confusion, disappointment and sadness @xcite . while such schemes usually apply to user emotions only, recent works also include agent's behavior, proposing annotation labels, as well as computational approaches that aim at identifying empathy on agent's responses @xcite . the annotation scheme described in this paper draws inspiration from the works briefly outlined in this section, and its main contribution lies in the attempt to encompass within a single framework all these different, though in a sense complementary, dimensions. ## dataset description the scheme has been applied to a corpus of interactions between humans and a task-oriented dialogue system foot_0 . more specifically, it contains dialogues between customers and a chatbot from the customer care unit of an italian telecommunication service provider. the dialogues, which take place by means of a text chat, mainly deal with requests for commercial assistance, both on landline and mobile phones. as briefly introduced in section 1, the corpus development and its annotation have been designed so as to improve the nlg module of the chatbot, with a particular focus on explanations. therefore, the dataset was created by selecting, from a sample of dialogues held over 24 hours, a reduced subset that included requests for explanations from customers. the resulting corpus consists of 142 dialogues, including 1540 turns -an average of about 11 turns per dialogue -and an average length of 9 tokens in customer turns and 38 tokens in the agent turns. similar to @xcite , consecutive messages of the same party (customer or agent), if not interrupted by another message of the other party, were considered as a single turn. this explains the difference in the average turn lengths, as, especially in the case of the chatbot, a single turn may actually consist of more than one message. this difference is also due to the way the agent's responses are currently structured; agents' responses usually include detailed information (for example, on invoice items or available options), while customers' messages are generally more concise. in some cases, the latter are basic yes/no answers, or digits (1,2,...) corresponding to the options provided by the agent in its previous message, as in the example below: ## annotation scheme this section provides an overview of the tagset adopted for the annotation of both dimensions included in the scheme, i.e. errors and emotions. detailed guidelines with examples can also be found in the following document (in both italian and english): https://cutt.ly/cdmcnym ## agreement results and discussion the first round of the annotation process was carried out by two independent annotators, following the guidelines. the inter-annotator agreement (iaa) statistics, measured using the cohen's k coefficient @xcite , were then computed after this step, and, as expected, they reported from moderate to low results. more specifically, we obtained a @xmath0.46 for error categories, @xmath1.43 for the agent's empathy and @xmath2.28 for customers' emotions. the resulting annotation was then reviewed to solve the cases of disagreement, which were discussed by an extended team including two other partners of the project. the goal of this annotation work was to have the broader coverage possible of the problems encountered in a conversational context such as that of a customer service chat. hence, the need to create a large and varied tagset. however, this variety has been at the expense of annotation consistency, as the iaa results show. despite the presence of detailed guidelines, the subjectivity of the individual annotators proved to be crucial, and not only -as one would have expected -with respect to the choices regarding emotions and empathy, but also in the error annotation. in the latter, in fact, the highest disagreement was reported where, for the same turn, one annotator identified an error while the other did not, thus showing different perceptions of the dialogue state and its deviation from a regular path. with a similar trend, most of the disagreement in the annotation of emotions was detected when an annotator identified an emotion of some kind in the customer's message, whereas the other one perceived the message as completely neutral. the second main cause of disagreement for this dimension was the different perception of anger and frustration between the two annotators. despite an agreed-upon definition of both emotions, in practice, different interpretations were often given to the users' messages and to the conversational contexts in this respect. by contrast, the fine-grained distinction between somewhat/very angry or frustrated did not cause substantial disagreement between the annotators and the conflation of the two labels into a single one, both for anger and frustration, did not significantly improve iaa results, while conflating frustration and anger into a single label increased the agreement rate to @xmath3.40. these findings brought to a revision of the definition of both emotions in the guidelines. it is worth pointing out that when designing the scheme for this dimension, we took into account the fact that customers' interventions could be affected by the awareness of the medium used. for example, customers may consider that politeness is not required or that their messages should be as simple as possible for the machine to understand them and, accordingly, their interventions may result quite straightforward and could be wrongly believed to carry some kind of anger. instead, what we actually observed in the sample set is that most often -maybe due to little familiarity with such tools -customers are not fully aware of this. they keep posing questions or answering to the chatbot's questions as if they were interacting with a real person. we hypothesize that frustration and anger may arise even due to this factor. an in-depth study will be carried out to verify if there is enough evidence to confirm it. agreement results on empathy are also moderate, even more so if compared to the ones obtained in @xcite , who reported a cohen's @xmath4.74, and whose definition of empathy has been used in this work. we attribute such divergence to the differences in the type of data collected, as @xcite focus on human-human spoken interactions, while in this work we deal with conversations between humans and a chatbot system with a relatively limited set of predefined responses. whether the same response, in different conversational contexts, could be considered as sufficiently empathic was thus left at the annotator's discretion. all these issues prove once again the complexity of this task, especially considering that it deals with interactions on a text chat, rather than via spoken conversations, in which other factors such as acoustic and prosodic features can help discriminate. this is also the reason why the reviewing phase to solve these cases was extended to two other group members who had not participated in the first step, so as to reach a final version as widely shared as possible. ## conclusions in this paper, we described the main features of an annotation scheme designed to label a sample set of interactions between customers and the chatbot of an italian telecom customer service unit. the ultimate goal of this scheme and its applications on the sample data is to develop a rule-based nlg module to be integrated within the template-based chatbot system and intended to substitute template responses with more context-dependent responses, with an eye in particular on the improvement of explanations. with this purpose in mind, a set of dialogues was selected, where the customer requests for an explanation of some kind. the specific characteristics of these conversational contexts -errors of various nature from both parties, potential customer dissatisfaction as perceived within the dialogues, etc. -as well as the expected output of the nlg module under development motivated the way the scheme was devised. the annotation experiments carried out to validate the scheme highlighted the significant role played by subjectivity of individual annotators, which is reflected by a moderate iaa, but also the need to further analyze the inherent meaning of some of the labels that proved to be more controversial (as, for example, the ones introduced to define anger and frustration in customers' messages), and to revise their usage accordingly. the annotation layer devoted to make the presence of negative emotions in the customer's message explicit was designed with the aim of improving the chatbot's awareness of the user's emotional state. indeed, this information is essential as a basis for generating an emotion-aware and empathic response, which is the subject of future work. on this line, we also aim at investigating how to manage the cases of frustrated or angry users who insult the chatbot, in order to avoid escalation or responses where the chatbot shows himself unaware of the insult @xcite .