pwc-archive/papers-with-abstracts huggingface dataset:
Using Interaction Style Dimensions to Characterize Spoken Dialog Corpora
Authors: Nigel Ward
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9

The construction of spoken dialog systems today relies heavily on appropriate corpora, but corpus selection is more an art than a science. As interaction style properties govern many aspects of dialog, they have the potential to be useful for relating and comparing corpora. This paper overviews a recently-developed model of interaction styles and shows how it can be used to identify relevant corpus differences, estimate corpus similarity, and flag likely outlier dialogs.
Multi-Domain Dialogue State Tracking with Top-K Slot Self Attention
Authors: Longfei Yang, Jiyi Li, Sheng Li, Takahiro Shinozaki
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Dialogue State Tracking, Multi-domain Dialogue State Tracking, Task-Oriented Dialogue Systems

As an important component of task-oriented dialogue systems, dialogue state tracking is designed to track the dialogue state through the conversations between users and systems. Multi-domain dialogue state tracking is a challenging task, in which the correlation among different domains and slots needs to consider. Recently, slot self-attention is proposed to provide a data-driven manner to handle it. However, a full-support slot self-attention may involve redundant information interchange. In this paper, we propose a top-k attention-based slot self-attention for multi-domain dialogue state tracking. In the slot self-attention layers, we force each slot to involve information from the other k prominent slots and mask the rest out. The experimental results on two mainstream multi-domain task-oriented dialogue datasets, MultiWOZ 2.0 and MultiWOZ 2.4, present that our proposed approach is effective to improve the performance of multi-domain dialogue state tracking. We also find that the best result is obtained when each slot interchanges information with only a few slots.
Building a Knowledge-Based Dialogue System with Text Infilling
Authors: Qiang Xue, Tetsuya Takiguchi, Yasuo Ariki
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Sentence, Text Infilling

In recent years, generation-based dialogue systems using state-of-the-art (SoTA) transformer-based models have demonstrated impressive performance in simulating human-like conversations. To improve the coherence and knowledge utilization capabilities of dialogue systems, knowledge-based dialogue systems integrate retrieved graph knowledge into transformer-based models. However, knowledge-based dialog systems sometimes generate responses without using the retrieved knowledge.In this work, we propose a method in which the knowledge-based dialogue system can constantly utilize the retrieved knowledge using text infilling . Text infilling is the task of predicting missing spans of a sentence or paragraph. We utilize this text infilling to enable dialog systems to fill incomplete responses with the retrieved knowledge. Our proposed dialogue system has been proven to generate significantly more correct responses than baseline dialogue systems.
Generating Meaningful Topic Descriptions with Sentence Embeddings and LDA
Authors: Javier Miguel Sastre Martinez, Sean Gorman, Aisling Nugent, Anandita Pal
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Sentence, Sentence Embeddings

A major part of business operations is interacting with customers. Traditionally this was done by human agents, face to face or over telephone calls within customer support centers. There is now a move towards automation in this field using chatbots and virtual assistants, as well as an increased focus on analyzing recorded conversations to gather insights. Determining the different services that a human agent provides and estimating the incurred call handling costs per service are key to prioritizing service automation. We propose a new technique, ELDA (Embedding based LDA), based on a combination of LDA topic modeling and sentence embeddings, that can take a dataset of customer-agent dialogs and extract key utterances instead of key words. The aim is to provide more meaningful and contextual topic descriptions required for interpreting and labeling the topics, reducing the need for manually reviewing dialog transcripts.
Inferring Ranked Dialog Flows from Human-to-Human Conversations
Authors: Javier Miguel Sastre Martinez, Aisling Nugent
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9

We present a novel technique to infer ranked dialog flows from human-to-human conversations that can be used as an initial conversation design or to analyze the complexities of the conversations in a call center. This technique aims to identify, for a given service, the most common sequences of questions and responses from the human agent. Multiple dialog flows for different ranges of top paths can be produced so they can be reviewed in rank order and be refined in successive iterations until additional flows have the desired level of detail. The system ingests historical conversations and efficiently condenses them into a weighted deterministic finite-state automaton, which is then used to export dialog flow designs that can be readily used by conversational agents. A proof-of-concept experiment was conducted with the MultiWoz data set, a sample output is presented and future directions are outlined.
“Do you follow me?”: A Survey of Recent Approaches in Dialogue State Tracking
Authors: Léo Jacqmin, Lina M. Rojas Barahona, Benoit Favre
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Dialogue State Tracking

While communicating with a user, a task-oriented dialogue system has to track the user’s needs at each turn according to the conversation history. This process called dialogue state tracking (DST) is crucial because it directly informs the downstream dialogue policy. DST has received a lot of interest in recent years with the text-to-text paradigm emerging as the favored approach. In this review paper, we first present the task and its associated datasets. Then, considering a large number of recent publications, we identify highlights and advances of research in 2021-2022. Although neural approaches have enabled significant progress, we argue that some critical aspects of dialogue systems such as generalizability are still underexplored. To motivate future studies, we propose several research avenues.
The Duration of a Turn Cannot be Used to Predict When It Ends
Authors: Charles Threlkeld, Jp de Ruiter
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9

Turn taking in conversation is a complex process. We still don’t know how listeners are able to anticipate the end of a speaker’s turn. Previous work focuses on prosodic, semantic, and non-verbal cues that a turn is coming to an end. In this paper, we look at simple measures of duration — time, word count, and syllable count — to see if we can exploit the duration of turns as a cue. We find strong evidence that these metrics are useless.
Getting Better Dialogue Context for Knowledge Identification by Leveraging Document-level Topic Shift
Authors: Nhat Tran, Diane Litman
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Retrieval

To build a goal-oriented dialogue system that can generate responses given a knowledge base, identifying the relevant pieces of information to be grounded in is vital. When the number of documents in the knowledge base is large, retrieval approaches are typically used to identify the top relevant documents. However, most prior work simply uses an entire dialogue history to guide retrieval, rather than exploiting a dialogue’s topical structure. In this work, we examine the importance of building the proper contextualized dialogue history when document-level topic shifts are present. Our results suggest that excluding irrelevant turns from the dialogue history (e.g., excluding turns not grounded in the same document as the current turn) leads to better retrieval results. We also propose a cascading approach utilizing the topical nature of a knowledge-grounded conversation to further manipulate the dialogue history used as input to the retrieval models.
DeepCon: An End-to-End Multilingual Toolkit for Automatic Minuting of Multi-Party Dialogues
Authors: Aakash Bhatnagar, Nidhir Bhavsar, Muskaan Singh
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Machine Translation, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, speech-recognition, Speech Recognition, Translation

In this paper, we present our minuting tool DeepCon, an end-to-end toolkit for minuting the multiparty dialogues of meetings. It provides technological support for (multilingual) communication and collaboration, with a specific focus on Natural Language Processing (NLP) technologies: Automatic Speech Recognition (ASR), Machine Translation (MT), Automatic Minuting (AM), Topic Modelling (TM) and Named Entity Recognition (NER). To the best of our knowledge, there is no such tool available. Further, this tool follows a microservice architecture, and we release the tool as open-source, deployed on Amazon Web Services (AWS). We release our tool open-source here http://www.deepcon.in.
ICM : Intent and Conversational Mining from Conversation Logs
Authors: Sayantan Mitra, Roshni Ramnani, Sumit Ranjan, Shubhashis Sengupta
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9

Building conversation agents requires a large amount of manual effort in creating training data for intents / entities as well as mapping out extensive conversation flows. In this demonstration, we present ICM (Intent and conversation Mining), a tool which can be used to analyze existing conversation logs and help a bot designer analyze customer intents, train a custom intent model as well as map and optimize conversation flows. The tool can be used for first time deployment or subsequent deployments of chatbots.
Entity-based De-noising Modeling for Controllable Dialogue Summarization
Authors: Zhengyuan Liu, Nancy Chen
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Abstractive Text Summarization, Hallucination, Text Generation

Although fine-tuning pre-trained backbones produces fluent and grammatically-correct text in various language generation tasks, factual consistency in abstractive summarization remains challenging. This challenge is especially thorny for dialogue summarization, where neural models often make inaccurate associations between personal named entities and their respective actions. To tackle this type of hallucination, we present an entity-based de-noising model via text perturbation on reference summaries. We then apply this proposed approach in beam search validation, conditional training augmentation, and inference post-editing. Experimental results on the SAMSum corpus show that state-of-the-art models equipped with our proposed method achieve generation quality improvement in both automatic evaluation and human assessment.
iEval: Interactive Evaluation Framework for Open-Domain Empathetic Chatbots
Authors: Ekaterina Svikhnushina, Anastasiia Filippova, Pearl Pu
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Chatbot

Building an empathetic chatbot is an important objective in dialog generation research, with evaluation being one of the most challenging parts. By empathy, we mean the ability to understand and relate to the speakers’ emotions, and respond to them appropriately. Human evaluation has been considered as the current standard for measuring the performance of open-domain empathetic chatbots. However, existing evaluation procedures suffer from a number of limitations we try to address in our current work. In this paper, we describe iEval, a novel interactive evaluation framework where the person chatting with the bots also rates them on different conversational aspects, as well as ranking them, resulting in greater consistency of the scores. We use iEval to benchmark several state-of-the-art empathetic chatbots, allowing us to discover some intricate details in their performance in different emotional contexts. Based on these results, we present key implications for further improvement of such chatbots. To facilitate other researchers using the iEval framework, we will release our dataset consisting of collected chat logs and human scores.
Unsupervised Domain Adaptation on Question-Answering System with Conversation Data
Authors: Amalia Adiba, Takeshi Homma, Yasuhiro Sogawa
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Domain Adaptation, Machine Reading Comprehension, Question Answering, Reading Comprehension, Unsupervised Domain Adaptation

Machine reading comprehension (MRC) is a task for question answering that finds answers to questions from documents of knowledge. Most studies on the domain adaptation of MRC require documents describing knowledge of the target domain. However, it is sometimes difficult to prepare such documents. The goal of this study was to transfer an MRC model to another domain without documents in an unsupervised manner. Therefore, unlike previous studies, we propose a domain-adaptation framework of MRC under the assumption that the only available data in the target domain are human conversations between a user asking questions and an expert answering the questions. The framework consists of three processes: (1) training an MRC model on the source domain, (2) converting conversations into documents using document generation (DG), a task we developed for retrieving important information from several human conversations and converting it to an abstractive document text, and (3) transferring the MRC model to the target domain with unsupervised domain adaptation. To the best of our knowledge, our research is the first to use conversation data to train MRC models in an unsupervised manner. We show that the MRC model successfully obtains question-answering ability from conversations in the target domain.
Disruptive Talk Detection in Multi-Party Dialogue within Collaborative Learning Environments with a Regularized User-Aware Network
Authors: Kyungjin Park, Hyunwoo Sohn, Wookhee Min, Bradford Mott, Krista Glazewski, Cindy E. Hmelo-Silver, James Lester
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9

Accurate detection and appropriate handling of disruptive talk in multi-party dialogue is essential for users to achieve shared goals. In collaborative game-based learning environments, detecting and attending to disruptive talk holds significant potential since it can cause distraction and produce negative learning experiences for students. We present a novel attention-based user-aware neural architecture for disruptive talk detection that uses a sequence dropout-based regularization mechanism. The disruptive talk detection models are evaluated with multi-party dialogue collected from 72 middle school students who interacted with a collaborative game-based learning environment. Our proposed disruptive talk detection model significantly outperforms competitive baseline approaches and shows significant potential for helping to support effective collaborative learning experiences.
Generating Discourse Connectives with Pre-trained Language Models: Conditioning on Discourse Relations Helps Reconstruct the PDTB
Authors: Symon Stevens-Guille, Aleksandre Maskharashvili, Xintong Li, Michael White
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Relation, Text Generation

We report results of experiments using BART (Lewis et al., 2019) and the Penn Discourse Tree Bank (Webber et al., 2019) (PDTB) to generate texts with correctly realized discourse relations. We address a question left open by previous research (Yung et al., 2021; Ko and Li, 2020) concerning whether conditioning the model on the intended discourse relation—which corresponds to adding explicit discourse relation information into the input to the model—improves its performance. Our results suggest that including discourse relation information in the input of the model significantly improves the consistency with which it produces a correctly realized discourse relation in the output. We compare our models’ performance to known results concerning the discourse structures found in written text and their possible explanations in terms of discourse interpretation strategies hypothesized in the psycholinguistics literature. Our findings suggest that natural language generation models based on current pre-trained Transformers will benefit from infusion with discourse level information if they aim to construct discourses with the intended relations.
Combining Structured and Unstructured Knowledge in an Interactive Search Dialogue System
Authors: Svetlana Stoyanchev, Suraj Pandey, Simon Keizer, Norbert Braunschweiler, Rama Sanand Doddipatla
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Semantic Similarity, Semantic Textual Similarity

Users of interactive search dialogue systems specify their preferences with natural language utterances. However, a schema-driven system is limited to handling the preferences that correspond to the predefined database content. In this work, we present a methodology for extending a schema-driven interactive search dialogue system with the ability to handle unconstrained user preferences. Using unsupervised semantic similarity metrics and the text snippets associated with the search items, the system identifies suitable items for the user’s unconstrained natural language query. In crowd-sourced evaluation, the users chat with our extended restaurant search system. Based on objective metrics and subjective user ratings, we demonstrate the feasibility of using an unsupervised low latency approach to extend a schema-driven search dialogue system to handle unconstrained user preferences.
Evaluating N-best Calibration of Natural Language Understanding for Dialogue Systems
Authors: Ranim Khojah, Alexander Berman, Staffan Larsson
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Classification, intent-classification, Intent Classification, Natural Language Understanding

A Natural Language Understanding (NLU) component can be used in a dialogue system to perform intent classification, returning an N-best list of hypotheses with corresponding confidence estimates. We perform an in-depth evaluation of 5 NLUs, focusing on confidence estimation. We measure and visualize calibration for the 10 best hypotheses on model level and rank level, and also measure classification performance. The results indicate a trade-off between calibration and performance. In particular, Rasa (with Sklearn classifier) had the best calibration but the lowest performance scores, while Watson Assistant had the best performance but a poor calibration.
User Satisfaction Modeling with Domain Adaptation in Task-oriented Dialogue Systems
Authors: Yan Pan, Mingyang Ma, Bernhard Pflugfelder, Georg Groh
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Action Recognition, Domain Adaptation, Task-Oriented Dialogue Systems, Unsupervised Domain Adaptation

User Satisfaction Estimation (USE) is crucial in helping measure the quality of a task-oriented dialogue system. However, the complex nature of implicit responses poses challenges in detecting user satisfaction, and most datasets are limited in size or not available to the public due to user privacy policies. Unlike task-oriented dialogue, large-scale annotated chitchat with emotion labels is publicly available. Therefore, we present a novel user satisfaction model with domain adaptation (USMDA) to utilize this chitchat. We adopt a dialogue Transformer encoder to capture contextual features from the dialogue. And we reduce domain discrepancy to learn dialogue-related invariant features. Moreover, USMDA jointly learns satisfaction signals in the chitchat context with user satisfaction estimation, and user actions in task-oriented dialogue with dialogue action recognition. Experimental results on two benchmarks show that our proposed framework for the USE task outperforms existing unsupervised domain adaptation methods. To the best of our knowledge, this is the first work to study user satisfaction estimation with unsupervised domain adaptation from chitchat to task-oriented dialogue.
A Visually-Aware Conversational Robot Receptionist
Authors: Nancie Gunson, Daniel Hernandez Garcia, Weronika Sieińska, Angus Addlesee, Christian Dondrup, Oliver Lemon, Jose L. Part, Yanchao Yu
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Question Answering

Socially Assistive Robots (SARs) have the potential to play an increasingly important role in a variety of contexts including healthcare, but most existing systems have very limited interactive capabilities. We will demonstrate a robot receptionist that not only supports task-based and social dialogue via natural spoken conversation but is also capable of visually grounded dialogue; able to perceive and discuss the shared physical environment (e.g. helping users to locate personal belongings or objects of interest). Task-based dialogues include check-in, navigation and FAQs about facilities, alongside social features such as chit-chat, access to the latest news and a quiz game to play while waiting. We also show how visual context (objects and their spatial relations) can be combined with linguistic representations of dialogue context, to support visual dialogue and question answering. We will demonstrate the system on a humanoid ARI robot, which is being deployed in a hospital reception area.
Demonstrating EMMA: Embodied MultiModal Agent for Language-guided Action Execution in 3D Simulated Environments
Authors: Alessandro Suglia, Bhathiya Hemanthage, Malvina Nikandrou, George Pantazopoulos, Amit Parekh, Arash Eshghi, Claudio Greco, Ioannis Konstas, Oliver Lemon, Verena Rieser
Date: 2022-09-01 00:00:00
Proceeding: SIGDIAL (ACL) 2022 9
Tasks: Conditional Text Generation, Text Generation

We demonstrate EMMA, an embodied multimodal agent which has been developed for the Alexa Prize SimBot challenge. The agent acts within a 3D simulated environment for household tasks. EMMA is a unified and multimodal generative model aimed at solving embodied tasks. In contrast to previous work, our approach treats multiple multimodal tasks as a single multimodal conditional text generation problem, where a model learns to output text given both language and visual input. Furthermore, we showcase that a single generative agent can solve tasks with visual inputs of varying length, such as answering questions about static images, or executing actions given a sequence of previous frames and dialogue utterances. The demo system will allow users to interact conversationally with EMMA in embodied dialogues in different 3D environments from the TEACh dataset.