pwc-archive/papers-with-abstracts huggingface dataset:
Leveraging Non-Conversational Tasks for Low Resource Slot Filling: Does it help?
Authors: Samuel Louvan, Bernardo Magnini
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Multi-Task Learning, slot-filling, Slot Filling, Task-Oriented Dialogue Systems, Transfer Learning

Slot filling is a core operation for utterance understanding in task-oriented dialogue systems. Slots are typically domain-specific, and adding new domains to a dialogue system involves data and time-intensive processes. A popular technique to address the problem is transfer learning, where it is assumed the availability of a large slot filling dataset for the source domain, to be used to help slot filling on the target domain, with fewer data. In this work, instead, we propose to leverage source tasks based on semantically related non-conversational resources (e.g., semantic sequence tagging datasets), as they are both cheaper to obtain and reusable to several slot filling domains. We show that using auxiliary non-conversational tasks in a multi-task learning setup consistently improves low resource slot filling performance.
Graph2Bots, Unsupervised Assistance for Designing Chatbots
Authors: Jean-Leon Bouraoui, Sonia Le Meitour, Romain Carbou, Lina M. Rojas Barahona, Vincent Lemaire
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9

We present Graph2Bots, a tool for assisting conversational agent designers. It extracts a graph representation from human-human conversations by using unsupervised learning. The generated graph contains the main stages of the dialogue and their inner transitions. The graphical user interface (GUI) then allows graph editing.
On a Chatbot Conducting Dialogue-in-Dialogue
Authors: Boris Galitsky, Dmitry Ilvovsky, Elizaveta Goncharova
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Chatbot, Form

We demo a chatbot that delivers content in the form of virtual dialogues automatically produced from plain texts extracted and selected from documents. This virtual dialogue content is provided in the form of answers derived from the found and selected documents split into fragments, and questions are automatically generated for these answers.
Unsupervised Dialogue Spectrum Generation for Log Dialogue Ranking
Authors: Xinnuo Xu, Yizhe Zhang, Lars Liden, Sungjin Lee
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9

Although the data-driven approaches of some recent bot building platforms make it possible for a wide range of users to easily create dialogue systems, those platforms don{'}t offer tools for quickly identifying which log dialogues contain problems. This is important since corrections to log dialogues provide a means to improve performance after deployment. A log dialogue ranker, which ranks problematic dialogues higher, is an essential tool due to the sheer volume of log dialogues that could be generated. However, training a ranker typically requires labelling a substantial amount of data, which is not feasible for most users. In this paper, we present a novel unsupervised approach for dialogue ranking using GANs and release a corpus of labelled dialogues for evaluation and comparison with supervised methods. The evaluation result shows that our method compares favorably to supervised methods without any labelled data.
FriendsQA: Open-Domain Question Answering on TV Show Transcripts
Authors: Zhengzhe Yang, Jinho D. Choi
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Open-Domain Question Answering, Question Answering, Reading Comprehension

This paper presents FriendsQA, a challenging question answering dataset that contains 1,222 dialogues and 10,610 open-domain questions, to tackle machine comprehension on everyday conversations. Each dialogue, involving multiple speakers, is annotated with several types of questions regarding the dialogue contexts, and the answers are annotated with certain spans in the dialogue. A series of crowdsourcing tasks are conducted to ensure good annotation quality, resulting a high inter-annotator agreement of 81.82{\%}. A comprehensive annotation analytics is provided for a deeper understanding in this dataset. Three state-of-the-art QA systems are experimented, R-Net, QANet, and BERT, and evaluated on this dataset. BERT in particular depicts promising results, an accuracy of 74.2{\%} for answer utterance selection and an F1-score of 64.2{\%} for answer span selection, suggesting that the FriendsQA task is hard yet has a great potential of elevating QA research on multiparty dialogue to another level.
Foundations of Collaborative Task-Oriented Dialogue: What's in a Slot?
Authors: Philip Cohen
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Task-Oriented Dialogue Systems

In this paper, we examine the foundations of task-oriented dialogues, in which systems are requested to perform tasks for humans. We argue that the way this dialogue task has been framed has limited its applicability to processing simple requests with atomic {``}slot-fillers{''}. However, real task-oriented dialogues can contain more complex utterances that provide non-atomic constraints on slot values. For example, in response to the system{'}s question {``}What time do you want me to reserve the restaurant?{''}, a user should be able to say {``}the earliest time available,{''} which cannot be handled by classic {``}intent + slots{''} approaches that do not incorporate expressive logical form meaning representations. Furthermore, situations for which it would be desirable to build task-oriented dialogue systems, e.g., to engage in mixed-initiative, collaborative or multiparty dialogues, will require a more general approach. In order to overcome these limitations and to provide such an approach, we give a logical analysis of the {``}intent+slot{''} dialogue setting using a modal logic of intention and including a more expansive notion of {``}dialogue state{''}. Finally, we briefly discuss our program of research to build a next generation of plan-based dialogue systems that goes beyond {``}intent + slots{''}.
Speaker-adapted neural-network-based fusion for multimodal reference resolution
Authors: Diana Kleingarn, Nima Nabizadeh, Martin Heckmann, Dorothea Kolossa
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9

Humans use a variety of approaches to reference objects in the external world, including verbal descriptions, hand and head gestures, eye gaze or any combination of them. The amount of useful information from each modality, however, may vary depending on the specific person and on several other factors. For this reason, it is important to learn the correct combination of inputs for inferring the best-fitting reference. In this paper, we investigate appropriate speaker-dependent and independent fusion strategies in a multimodal reference resolution task. We show that without any change in the modality models, only through an optimized fusion technique, it is possible to reduce the error rate of the system on a reference resolution task by more than 50{\%}.
Zero-shot transfer for implicit discourse relation classification
Authors: Murathan Kurfal{\i}, Robert {\"O}stling
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Classification, General Classification, Implicit Discourse Relation Classification, Relation, Relation Classification, Transfer Learning

Automatically classifying the relation between sentences in a discourse is a challenging task, in particular when there is no overt expression of the relation. It becomes even more challenging by the fact that annotated training data exists only for a small number of languages, such as English and Chinese. We present a new system using zero-shot transfer learning for implicit discourse relation classification, where the only resource used for the target language is unannotated parallel text. This system is evaluated on the discourse-annotated TED-MDB parallel corpus, where it obtains good results for all seven languages using only English training data.
A Quantitative Analysis of Patients' Narratives of Heart Failure
Authors: Sabita Acharya, Barbara Di Eugenio, Andrew Boyd, Richard Cameron, Karen Dunn Lopez, Pamela Martyn-Nemeth, Debaleena Chattopadhyay, Pantea Habibi, Carolyn Dickens, Haleh Vatani, Amer Ardati
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9

Patients with chronic conditions like heart failure are the most likely to be re-hospitalized. One step towards avoiding re-hospitalization is to devise strategies for motivating patients to take care of their own health. In this paper, we perform a quantitative analysis of patients{'} narratives of their experience with heart failure and explore the different topics that patients talk about. We compare two different groups of patients- those unable to take charge of their illness, and those who make efforts to improve their health. We will use the findings from our analysis to refine and personalize the summaries of hospitalizations that our system automatically generates.
TDDiscourse: A Dataset for Discourse-Level Temporal Ordering of Events
Authors: Aakanksha Naik, Luke Breitfeller, Carolyn Rose
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Articles, Relation Classification, Sentence, Temporal Relation Classification

Prior work on temporal relation classification has focused extensively on event pairs in the same or adjacent sentences (local), paying scant attention to discourse-level (global) pairs. This restricts the ability of systems to learn temporal links between global pairs, since reliance on local syntactic features suffices to achieve reasonable performance on existing datasets. However, systems should be capable of incorporating cues from document-level structure to assign temporal relations. In this work, we take a first step towards discourse-level temporal ordering by creating TDDiscourse, the first dataset focusing specifically on temporal links between event pairs which are more than one sentence apart. We create TDDiscourse by augmenting TimeBank-Dense, a corpus of English news articles, manually annotating global pairs that cannot be inferred automatically from existing annotations. Our annotations double the number of temporal links in TimeBank-Dense, while possessing several desirable properties such as focusing on long-distance pairs and not being automatically inferable. We adapt and benchmark the performance of three state-of-the-art models on TDDiscourse and observe that existing systems indeed find discourse-level temporal ordering harder.
Real Life Application of a Question Answering System Using BERT Language Model
Authors: Francesca Alloatti, Luigi di Caro, Gianpiero Sportelli
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Language Modeling, Language Modelling, Question Answering, text-classification, Text Classification

It is often hard to apply the newest advances in research to real life scenarios. They usually require the resolution of some specific task applied to a restricted domain, all the while providing small amounts of data to begin with. In this study we apply one of the newest innovations in Deep Learning to a task of text classification. We created a question answering system in Italian that provides information about a specific subject, e-invoicing and digital billing. Italy recently introduced a new legislation about e-invoicing and people have some legit doubts, therefore a large share of professionals could benefit from this tool.
Cross-Corpus Data Augmentation for Acoustic Addressee Detection
Authors: Oleg Akhtiamov, Ingo Siegert, Alexey Karpov, Wolfgang Minker
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Cross-corpus, Data Augmentation, General Classification

Acoustic addressee detection (AD) is a modern paralinguistic and dialogue challenge that especially arises in voice assistants. In the present study, we distinguish addressees in two settings (a conversation between several people and a spoken dialogue system, and a conversation between several adults and a child) and introduce the first competitive baseline (unweighted average recall equals 0.891) for the Voice Assistant Conversation Corpus that models the first setting. We jointly solve both classification problems, using three models: a linear support vector machine dealing with acoustic functionals and two neural networks utilising raw waveforms alongside with acoustic low-level descriptors. We investigate how different corpora influence each other, applying the mixup approach to data augmentation. We also study the influence of various acoustic context lengths on AD. Two-second speech fragments turn out to be sufficient for reliable AD. Mixup is shown to be beneficial for merging acoustic data (extracted features but not raw waveforms) from different domains that allows us to reach a higher classification performance on human-machine AD and also for training a multipurpose neural network that is capable of solving both human-machine and adult-child AD problems.
A Scalable Method for Quantifying the Role of Pitch in Conversational Turn-Taking
Authors: Kornel Laskowski, Marcin Wlodarczak, Mattias Heldner
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Activity Prediction

Pitch has long been held as an important signalling channel when planning and deploying speech in conversation, and myriad studies have been undertaken to determine the extent to which it actually plays this role. Unfortunately, these studies have required considerable human investment in data preparation and analysis, and have therefore often been limited to a handful of specific conversational contexts. The current article proposes a framework which addresses these limitations, by enabling a scalable, quantitative characterization of the role of pitch throughout an entire conversation, requiring only the raw signal and speech activity references. The framework is evaluated on the Switchboard dialogue corpus. Experiments indicate that pitch trajectories of both parties are predictive of their incipient speech activity; that pitch should be expressed on a logarithmic scale and Z-normalized, as well as accompanied by a binary voicing variable; and that only the most recent 400 ms of the pitch trajectory are useful in incipient speech activity prediction.
A Large-Scale User Study of an Alexa Prize Chatbot: Effect of TTS Dynamism on Perceived Quality of Social Dialog
Authors: Michelle Cohn, Chun-Yen Chen, Zhou Yu
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Chatbot, text-to-speech, Text to Speech

This study tests the effect of cognitive-emotional expression in an Alexa text-to-speech (TTS) voice on users{'} experience with a social dialog system. We systematically introduced emotionally expressive interjections (e.g., {``}Wow!{''}) and filler words (e.g., {``}um{''}, {``}mhmm{''}) in an Amazon Alexa Prize socialbot, Gunrock. We tested whether these TTS manipulations improved users{'} ratings of their conversation across thousands of real user interactions (n=5,527). Results showed that interjections and fillers each improved users{'} holistic ratings, an improvement that further increased if the system used both manipulations. A separate perception experiment corroborated the findings from the user study, with improved social ratings for conversations including interjections; however, no positive effect was observed for fillers, suggesting that the role of the rater in the conversation{---}as active participant or external listener{---}is an important factor in assessing social dialogs.
Influence of Time and Risk on Response Acceptability in a Simple Spoken Dialogue System
Authors: Andisheh Partovi, Ingrid Zukerman
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9

We describe a longitudinal user study conducted in the context of a Spoken Dialogue System for a household robot, where we examined the influence of time displacement and situational risk on users{'} preferred responses. To this effect, we employed a corpus of spoken requests that asked a robot to fetch or move objects in a room. In the first stage of our study, participants selected among four response types to these requests under two risk conditions: low and high. After some time, the same participants rated several responses to the previous requests {---} these responses were instantiated from the four response types. Our results show that participants did not rate highly their own response types; moreover, they rated their own response types similarly to different ones. This suggests that, at least in this context, people{'}s preferences at a particular point in time may not reflect their general attitudes, and that various reasonable response types may be equally acceptable. Our study also reveals that situational risk influences the acceptability of some response types.
Characterizing the Response Space of Questions: a Corpus Study for English and Polish
Authors: Jonathan Ginzburg, Zulipiye Yusupujiang, Chuyuan Li, Kexin Ren, Pawe{\l} {\L}upkowski
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Dialogue Management, Management

The main aim of this paper is to provide a characterization of the response space for questions using a taxonomy grounded in a dialogical formal semantics. As a starting point we take the typology for responses in the form of questions provided in (Lupkowski and Ginzburg, 2016). This work develops a wide coverage taxonomy for question/question sequences observable in corpora including the BNC, CHILDES, and BEE, as well as formal modelling of all the postulated classes. Our aim is to extend this work to cover all responses to questions. We present the extended typology of responses to questions based on a corpus studies of BNC, BEE and Maptask with include 506, 262, and 467 question/response pairs respectively. We compare the data for English with data from Polish using the Spokes corpus (205 question/response pairs). We discuss annotation reliability and disagreement analysis. We sketch how each class can be formalized using a dialogical semantics appropriate for dialogue management.
From Explainability to Explanation: Using a Dialogue Setting to Elicit Annotations with Justifications
Authors: Nazia Attari, Martin Heckmann, David Schlangen
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9

Despite recent attempts in the field of explainable AI to go beyond black box prediction models, typically already the training data for supervised machine learning is collected in a manner that treats the annotator as a {``}black box{''}, the internal workings of which remains unobserved. We present an annotation method where a task is given to a pair of annotators who collaborate on finding the best response. With this we want to shed light on the questions if the collaboration increases the quality of the responses and if this {``}thinking together{''} provides useful information in itself, as it at least partially reveals their reasoning steps. Furthermore, we expect that this setting puts the focus on explanation as a linguistic act, vs. explainability as a property of models. In a crowd-sourcing experiment, we investigated three different annotation tasks, each in a collaborative dialogical (two annotators) and monological (one annotator) setting. Our results indicate that our experiment elicits collaboration and that this collaboration increases the response accuracy. We see large differences in the annotators{'} behavior depending on the task. Similarly, we also observe that the dialog patterns emerging from the collaboration vary significantly with the task.
Prediction of User Emotion and Dialogue Success Using Audio Spectrograms and Convolutional Neural Networks
Authors: Athanasios Lykartsis, Margarita Kotti
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9

In this paper we aim to predict dialogue success and user satisfaction as well as emotion on a turn level. To achieve this, we investigate the use of spectrogram representations, extracted from audio files, in combination with several types of convolutional neural networks. The experiments were performed on the Let{'}s Go V2 database, comprising 5065 audio files and having labels for subjective and objective dialogue turn success, as well as the emotional state of the user. Results show that by using only audio, it is possible to predict turn success with very high accuracy for all three labels (90{\%}). The best performing input representation were 1s long mel-spectrograms in combination with a CNN with a bottleneck architecture. The resulting system has the potential to be used real-time. Our results significantly surpass the state of the art for dialogue success prediction based only on audio.
Modelling Adaptive Presentations in Human-Robot Interaction using Behaviour Trees
Authors: Nils Axelsson, Gabriel Skantze
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9

In dialogue, speakers continuously adapt their speech to accommodate the listener, based on the feedback they receive. In this paper, we explore the modelling of such behaviours in the context of a robot presenting a painting. A Behaviour Tree is used to organise the behaviour on different levels, and allow the robot to adapt its behaviour in real-time; the tree organises engagement, joint attention, turn-taking, feedback and incremental speech processing. An initial implementation of the model is presented, and the system is evaluated in a user study, where the adaptive robot presenter is compared to a non-adaptive version. The adaptive version is found to be more engaging by the users, although no effects are found on the retention of the presented material.
Coached Conversational Preference Elicitation: A Case Study in Understanding Movie Preferences
Authors: Filip Radlinski, Krisztian Balog, Bill Byrne, Karthik Krishnamoorthi
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Conversational Recommendation

Conversational recommendation has recently attracted significant attention. As systems must understand users{'} preferences, training them has called for conversational corpora, typically derived from task-oriented conversations. We observe that such corpora often do not reflect how people naturally describe preferences. We present a new approach to obtaining user preferences in dialogue: Coached Conversational Preference Elicitation. It allows collection of natural yet structured conversational preferences. Studying the dialogues in one domain, we present a brief quantitative analysis of how people describe movie preferences at scale. Demonstrating the methodology, we release the CCPE-M dataset to the community with over 500 movie preference dialogues expressing over 10,000 preferences.
Dialogue Act Classification in Team Communication for Robot Assisted Disaster Response
Authors: Tatiana Anikina, Ivana Kruijff-Korbayova
Date: 2019-09-01 00:00:00
Proceeding: WS 2019 9
Tasks: Dialogue Act Classification, Disaster Response, General Classification

We present the results we obtained on the classification of dialogue acts in a corpus of human-human team communication in the domain of robot-assisted disaster response. We annotated dialogue acts according to the ISO 24617-2 standard scheme and carried out experiments using the FastText linear classifier as well as several neural architectures, including feed-forward, recurrent and convolutional neural models with different types of embeddings, context and attention mechanism. The best performance was achieved with a {''}Divide {\&} Merge{''} architecture presented in the paper, using trainable GloVe embeddings and a structured dialogue history. This model learns from the current utterance and the preceding context separately and then combines the two generated representations. Average accuracy of 10-fold cross-validation is 79.8{\%}, F-score 71.8{\%}.