Natural Language Generation in Dialogue Systems for Customer Care
Authors: Mirko Di Lascio, Manuela Sanguinetti, Luca Anselma, Dario Mana, Alessandro Mazzei, Viviana Patti, Rossana Simeoni (Università di Torino, Università di Cagliari, TIM, 2020)
Topic: Chatbots and Dialogue Systems

Abstract
In this paper we discuss the role of natural language generation (NLG) in modern dialogue systems (DSs). In particular, we will study the role that a linguistically sound NLG architecture can have in a DS. Using real examples from a new corpus of dialogue in customer-care domain, we will study how the non-linguistic contextual data can be exploited by using NLG.

Introduction
In this paper we present the first results of an ongoing project on the design of a dialogue system for customer care in the telco field. In most of the dialogue systems (DSs), the generation side of the communication is quite limited to the use of templates AUTHOR. Templates are pre-compiled sentences with empty slots that can be filled with appropriate fillers. Most of commercial DSs, following the classical cascade architecture NLUnderstanding, DialogueManager, NLGeneration AUTHOR, use machine learning-based Natural Language Understanding (NLU) techniques to identify important concepts (e.g., intent and entities in AUTHOR) that will be used by the dialogue manager (i) to update the state of the system and (ii) to produce the next dialogue act AUTHOR, possibly filling the slots in the generation templates. This classical, and quite common, information flow/architecture for dialogue processing has, as a working hypothesis, the assumption that most of necessary information is provided by the user's utterance: we call this information linguistic channel (L-channel). However, especially in the customer-care domain, this assumption is only partially true. For instance, in the sentence ``Scusami ma vorrei sapere come mai mi vengono fatti certi addebiti?'' (``Excuse me, I'd like to know why I'm charged certain fees?''), even a very advanced NLU module can produce only a vague information about the user's request to the DialogueManager. Indeed, in order to provide good enough responses, the DialogueManager resorts to other two sources of information: the domain context channel (DC-channel) and the user model channel (UM-channel). The DC-channel is fundamental to produce the content of the answer, while the UM-channel is necessary to give also the correct form. It is worth noting that both channels, that are often neglected in the design of commercial DSs for customer-care domain, have central roles in the design of (linguistically sound) natural language generation (NLG) systems AUTHOR. In particular, considering the standard architecture for data-to-text NLG systems AUTHOR, the analysis of the DC-channel exactly corresponds to the content selection task and the UM-channel influences both the sentence planning and sentence realization phases. In other words, the central claims of this paper are that in commercial DSs for customer care: (1) L-channel is often not informative enough and one needs to use the DC-channel and the UM-channel for producing a sufficiently good answer, (2) DC-channel and UM-channel can be exploited by using standard symbolic in neural networks deters their use in real-world NLG AUTHOR NLG techniques and methods}. The remainder of the paper supports both of these claims while presenting our ongoing project on the development of a rule-based NLG prototype to be used in a customer care domain. Section sec::corpus presents the corpus developed in the first stage of this project, consisting of real dialogues containing explanation requests in telco customer-care domain. Section sec::nlg presents an NLG architecture for managing the L-DC-UM channels that can be adopted in a DS for customer care. Finally, Section sec::conclusion concludes the paper with few remarks on the current state of the project and on future work.

Conclusion
In this paper we have discussed the main features of the design of a DS system for telco customer care. In particular, we outlined the peculiarities of this domain, describing the construction of a specifically-designed dialogue corpus and discussing a possible integration of standard DS and NLG architectures in order to manage these peculiarities. This is an ongoing project and we are considering various enhancements: (1) we will integrate emoji prediction capabilities into the proposed architecture in order to allow the DS to automatically attach an appropriate emoji at the end of the generated response, relying on previous work for Italian (Ronzano et al., 2018); we would also take into account the current user emotions, while generating an appropriate emoji – it may be the case that an emoji that is adequate when the conversation is characterized by a neutral tone, suddenly becomes inappropriate if the user is frustrated or angry (Pamungkas, 2019; Cercas Curry and Rieser, 2019); (2) we would like to enhance the system so as to adapt the generated responses to other aspects of the users, such as their mental models, levels of domain expertise, and personality traits; (3) we want to evaluate the DS following the user-based comparative schema adopted in (Demberg et al., 2011). Finally, we add some closing remarks on the corpus availability and its anonymization. The publication of a dataset of conversations between customers and a company virtual assistant is a great opportunity for the company and for its surrounding communities of academics, designers, and developers. However, it entails a number of obstacles to overcome. Rules and laws by regulating bodies must be strictly followed – see, for example, the GDPR regulation. This means, first of all, including within the to-be-published dataset only those conversations made by customers who have given their consent to this type of treatment of their data. Moreover, it is mandatory to obscure both personal and sensitive customer data. Such obfuscation activities are particularly difficult in the world of chatbots, where customers are free to input unrestricted text in the conversations. Regular expressions can be used in order to recognize the pieces of data to be obscured, such as email addresses, telephone numbers, social security numbers, bank account identifiers, dates of birth, etc. More sophisticated techniques needed be adopted to identify and obscure, within the text entered by customers, names, surnames, home and work addresses. Even more complex and open is the problem of anonymizing sensitive customer data. For example, consider the case of a disabled customer who reveals his/her sanitary condition to the virtual assistant, in order to obtain a legitimate better treatment from the company: the text revealing the health condition of the customer must be obscured. Other relevant sensitive data include racial or ethnic origins, religious or philosophical beliefs, political opinions, etc. Some of these techniques, used for identifying certain types of data to be obscured, have a certain degree of precision that may even be far, given the current state of the art, from what a trained human analyst could do. Therefore, it is also necessary to consider the need for the dataset being published to be reviewed and edited by specialized personnel before the actual publication. With this in mind, the techniques of data recognition mentioned above - regular expressions, Named Entity Recognition, etc. - could also be exploited to develop tools that can speed up the task of completing and verifying the accurate anonymization of the dataset.