pwc-archive/papers-with-abstracts huggingface dataset:
Evaluating Natural Language Understanding Services for Conversational Question Answering Systems
Authors: Daniel Braun, Hern, Adrian ez Mendez, Florian Matthes, Manfred Langen
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Chatbot, Conversational Question Answering, Dialogue Management, General Classification, Natural Language Understanding, Question Answering

Conversational interfaces recently gained a lot of attention. One of the reasons for the current hype is the fact that chatbots (one particularly popular form of conversational interfaces) nowadays can be created without any programming knowledge, thanks to different toolkits and so-called Natural Language Understanding (NLU) services. While these NLU services are already widely used in both, industry and science, so far, they have not been analysed systematically. In this paper, we present a method to evaluate the classification performance of NLU services. Moreover, we present two new corpora, one consisting of annotated questions and one consisting of annotated questions with the corresponding answers. Based on these corpora, we conduct an evaluation of some of the most popular NLU services. Thereby we want to enable both, researchers and companies to make more educated decisions about which service they should use.
VOILA: An Optimised Dialogue System for Interactively Learning Visually-Grounded Word Meanings (Demonstration System)
Authors: Yanchao Yu, Arash Eshghi, Oliver Lemon
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Active Learning

We present VOILA: an optimised, multi-modal dialogue agent for interactive learning of visually grounded word meanings from a human user. VOILA is: (1) able to learn new visual categories interactively from users from scratch; (2) trained on real human-human dialogues in the same domain, and so is able to conduct natural spontaneous dialogue; (3) optimised to find the most effective trade-off between the accuracy of the visual categories it learns and the cost it incurs to users. VOILA is deployed on Furhat, a human-like, multi-modal robot head with back-projection of the face, and a graphical virtual character.
Towards a General, Continuous Model of Turn-taking in Spoken Dialogue using LSTM Recurrent Neural Networks
Authors: Gabriel Skantze
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Feature Engineering, Spoken Dialogue Systems

Previous models of turn-taking have mostly been trained for specific turn-taking decisions, such as discriminating between turn shifts and turn retention in pauses. In this paper, we present a predictive, continuous model of turn-taking using Long Short-Term Memory (LSTM) Recurrent Neural Networks (RNN). The model is trained on human-human dialogue data to predict upcoming speech activity in a future time window. We show how this general model can be applied to two different tasks that it was not specifically trained for. First, to predict whether a turn-shift will occur or not in pauses, where the model achieves a better performance than human observers, and better than results achieved with more traditional models. Second, to make a prediction at speech onset whether the utterance will be a short backchannel or a longer utterance. Finally, we show how the hidden layer in the network can be used as a feature vector for turn-taking decisions in a human-robot interaction scenario.
Beyond On-hold Messages: Conversational Time-buying in Task-oriented Dialogue
Authors: Soledad L{\'o}pez Gambino, Sina Zarrie{\ss}, David Schlangen
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8

A common convention in graphical user interfaces is to indicate a {``}wait state{''}, for example while a program is preparing a response, through a changed cursor state or a progress bar. What should the analogue be in a spoken conversational system? To address this question, we set up an experiment in which a human information provider (IP) was given their information only in a delayed and incremental manner, which systematically created situations where the IP had the turn but could not provide task-related information. Our data analysis shows that 1) IPs bridge the gap until they can provide information by re-purposing a whole variety of task- and grounding-related communicative actions (e.g. echoing the user{'}s request, signaling understanding, asserting partially relevant information), rather than being silent or explicitly asking for time (e.g. {``}please wait{''}), and that 2) IPs combined these actions productively to ensure an ongoing conversation. These results, we argue, indicate that natural conversational interfaces should also be able to manage their time flexibly using a variety of conversational resources.
Predicting Success in Goal-Driven Human-Human Dialogues
Authors: Michael Noseworthy, Jackie Chi Kit Cheung, Joelle Pineau
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Diversity

In goal-driven dialogue systems, success is often defined based on a structured definition of the goal. This requires that the dialogue system be constrained to handle a specific class of goals and that there be a mechanism to measure success with respect to that goal. However, in many human-human dialogues the diversity of goals makes it infeasible to define success in such a way. To address this scenario, we consider the task of automatically predicting success in goal-driven human-human dialogues using only the information communicated between participants in the form of text. We build a dataset from stackoverflow.com which consists of exchanges between two users in the technical domain where ground-truth success labels are available. We then propose a turn-based hierarchical neural network model that can be used to predict success without requiring a structured goal definition. We show this model outperforms rule-based heuristics and other baselines as it is able to detect patterns over the course of a dialogue and capture notions such as gratitude.
Generating and Evaluating Summaries for Partial Email Threads: Conversational Bayesian Surprise and Silver Standards
Authors: Jordon Johnson, Vaden Masrani, Giuseppe Carenini, Raymond Ng
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8

We define and motivate the problem of summarizing partial email threads. This problem introduces the challenge of generating reference summaries for partial threads when human annotation is only available for the threads as a whole, particularly when the human-selected sentences are not uniformly distributed within the threads. We propose an oracular algorithm for generating these reference summaries with arbitrary length, and we are making the resulting dataset publicly available. In addition, we apply a recent unsupervised method based on Bayesian Surprise that incorporates background knowledge into partial thread summarization, extend it with conversational features, and modify the mechanism by which it handles redundancy. Experiments with our method indicate improved performance over the baseline for shorter partial threads; and our results suggest that the potential benefits of background knowledge to partial thread summarization should be further investigated with larger datasets.
Enabling robust and fluid spoken dialogue with cognitively impaired users
Authors: Ramin Yaghoubzadeh, Stefan Kopp
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Dialogue Management, Management, Speech Recognition

We present the flexdiam dialogue management architecture, which was developed in a series of projects dedicated to tailoring spoken interaction to the needs of users with cognitive impairments in an everyday assistive domain, using a multimodal front-end. This hybrid DM architecture affords incremental processing of uncertain input, a flexible, mixed-initiative information grounding process that can be adapted to users{'} cognitive capacities and interactive idiosyncrasies, and generic mechanisms that foster transitions in the joint discourse state that are understandable and controllable by those users, in order to effect a robust interaction for users with varying capacities.
Adversarial evaluation for open-domain dialogue generation
Authors: Elia Bruni, Raquel Fern{\'a}ndez
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Chatbot, Dialogue Generation

We investigate the potential of adversarial evaluation methods for open-domain dialogue generation systems, comparing the performance of a discriminative agent to that of humans on the same task. Our results show that the task is hard, both for automated models and humans, but that a discriminative agent can learn patterns that lead to above-chance performance.
Exploring Joint Neural Model for Sentence Level Discourse Parsing and Sentiment Analysis
Authors: Bita Nejat, Giuseppe Carenini, Raymond Ng
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Discourse Parsing, Prediction, Relation Prediction, Sentence, Sentiment Analysis

Discourse Parsing and Sentiment Analysis are two fundamental tasks in Natural Language Processing that have been shown to be mutually beneficial. In this work, we design and compare two Neural Based models for jointly learning both tasks. In the proposed approach, we first create a vector representation for all the text segments in the input sentence. Next, we apply three different Recursive Neural Net models: one for discourse structure prediction, one for discourse relation prediction and one for sentiment analysis. Finally, we combine these Neural Nets in two different joint models: Multi-tasking and Pre-training. Our results on two standard corpora indicate that both methods result in improvements in each task but Multi-tasking has a bigger impact than Pre-training. Specifically for Discourse Parsing, we see improvements in the prediction of the set of contrastive relations.
Finding Structure in Figurative Language: Metaphor Detection with Topic-based Frames
Authors: Hyeju Jang, Keith Maki, Eduard Hovy, Carolyn Ros{\'e}
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: BIG-bench Machine Learning, Machine Translation

In this paper, we present a novel and highly effective method for induction and application of metaphor frame templates as a step toward detecting metaphor in extended discourse. We infer implicit facets of a given metaphor frame using a semi-supervised bootstrapping approach on an unlabeled corpus. Our model applies this frame facet information to metaphor detection, and achieves the state-of-the-art performance on a social media dataset when building upon other proven features in a nonlinear machine learning model. In addition, we illustrate the mechanism through which the frame and topic information enable the more accurate metaphor detection.
Using Reinforcement Learning to Model Incrementality in a Fast-Paced Dialogue Game
Authors: Ramesh Manuvinakurike, David DeVault, Kallirroi Georgila
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Spoken Dialogue Systems

We apply Reinforcement Learning (RL) to the problem of incremental dialogue policy learning in the context of a fast-paced dialogue game. We compare the policy learned by RL with a high-performance baseline policy which has been shown to perform very efficiently (nearly as well as humans) in this dialogue game. The RL policy outperforms the baseline policy in offline simulations (based on real user data). We provide a detailed comparison of the RL policy and the baseline policy, including information about how much effort and time it took to develop each one of them. We also highlight the cases where the RL policy performs better, and show that understanding the RL policy can provide valuable insights which can inform the creation of an even better rule-based policy.
Lessons in Dialogue System Deployment
Authors: Anton Leuski, Ron artstein
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8

We analyze deployment of an interactive dialogue system in an environment where deep technical expertise might not be readily available. The initial version was created using a collection of research tools. We summarize a number of challenges with its deployment at two museums and describe a new system that simplifies the installation and user interface; reduces reliance on 3rd-party software; and provides a robust data collection mechanism.
Information Navigation System with Discovering User Interests
Authors: Koichiro Yoshino, Yu Suzuki, Satoshi Nakamura
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Semantic Textual Similarity, Speech Recognition

We demonstrate an information navigation system for sightseeing domains that has a dialogue interface for discovering user interests for tourist activities. The system discovers interests of a user with focus detection on user utterances, and proactively presents related information to the discovered user interest. A partially observable Markov decision process (POMDP)-based dialogue manager, which is extended with user focus states, controls the behavior of the system to provide information with several dialogue acts for providing information. We transferred the belief-update function and the policy of the manager from other system trained on a different domain to show the generality of defined dialogue acts for our information navigation system.
SHIHbot: A Facebook chatbot for Sexual Health Information on HIV/AIDS
Authors: Jacqueline Brixey, Rens Hoegen, Wei Lan, Joshua Rusow, Karan Singla, Xusen Yin, Ron artstein, Anton Leuski
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Chatbot, Retrieval

We present the implementation of an autonomous chatbot, SHIHbot, deployed on Facebook, which answers a wide variety of sexual health questions on HIV/AIDS. The chatbot{'}s response database is com-piled from professional medical and public health resources in order to provide reliable information to users. The system{'}s backend is NPCEditor, a response selection platform trained on linked questions and answers; to our knowledge this is the first retrieval-based chatbot deployed on a large public social network.
How Would You Say It? Eliciting Lexically Diverse Dialogue for Supervised Semantic Parsing
Authors: Ravich, Abhilasha er, Thomas Manzini, Matthias Grabmair, Graham Neubig, Jonathan Francis, Eric Nyberg
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Semantic Parsing

Building dialogue interfaces for real-world scenarios often entails training semantic parsers starting from zero examples. How can we build datasets that better capture the variety of ways users might phrase their queries, and what queries are actually realistic? Wang et al. (2015) proposed a method to build semantic parsing datasets by generating canonical utterances using a grammar and having crowdworkers paraphrase them into natural wording. A limitation of this approach is that it induces bias towards using similar language as the canonical utterances. In this work, we present a methodology that elicits meaningful and lexically diverse queries from users for semantic parsing tasks. Starting from a seed lexicon and a generative grammar, we pair logical forms with mixed text-image representations and ask crowdworkers to paraphrase and confirm the plausibility of the queries that they generated. We use this method to build a semantic parsing dataset from scratch for a dialog agent in a smart-home simulation. We find evidence that this dataset, which we have named SmartHome, is demonstrably more lexically diverse and difficult to parse than existing domain-specific semantic parsing datasets.
A data-driven model of explanations for a chatbot that helps to practice conversation in a foreign language
Authors: Sviatlana H{\"o}hn
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Chatbot, Language Acquisition, Retrieval

This article describes a model of other-initiated self-repair for a chatbot that helps to practice conversation in a foreign language. The model was developed using a corpus of instant messaging conversations between German native and non-native speakers. Conversation Analysis helped to create computational models from a small number of examples. The model has been validated in an AIML-based chatbot. Unlike typical retrieval-based dialogue systems, the explanations are generated at run-time from a linguistic database.
Building a Better Bitext for Structurally Different Languages through Self-training
Authors: Jungyeul Park, Lo{\"\i}c Dugast, Jeen-Pyo Hong, Chang-Uk Shin, Jeong-Won Cha
Date: 2017-11-01 00:00:00
Proceeding: WS 2017 11
Tasks: Information Retrieval, Machine Translation, Sentence, Translation

We propose a novel method to bootstrap the construction of parallel corpora for new pairs of structurally different languages. We do so by combining the use of a pivot language and self-training. A pivot language enables the use of existing translation models to bootstrap the alignment and a self-training procedure enables to achieve better alignment, both at the document and sentence level. We also propose several evaluation methods for the resulting alignment.
MultiNews: A Web collection of an Aligned Multimodal and Multilingual Corpus
Authors: Haithem Afli, Pintu Lohar, Andy Way
Date: 2017-11-01 00:00:00
Proceeding: WS 2017 11
Tasks: Articles, Content-Based Image Retrieval, Image Retrieval, Machine Translation, Multimodal Machine Translation

Integrating Natural Language Processing (NLP) and computer vision is a promising effort. However, the applicability of these methods directly depends on the availability of a specific multimodal data that includes images and texts. In this paper, we present a collection of a Multimodal corpus of comparable texts and their images in 9 languages from the web news articles of Euronews website. This corpus has found widespread use in the NLP community in Multilingual and multimodal tasks. Here, we focus on its acquisition of the images and text data and their multilingual alignment.
Overview of the 4th Workshop on Asian Translation
Authors: Toshiaki Nakazawa, Shohei Higashiyama, Chenchen Ding, Hideya Mino, Isao Goto, Hideto Kazawa, Yusuke Oda, Graham Neubig, Sadao Kurohashi
Date: 2017-11-01 00:00:00
Proceeding: WS 2017 11
Tasks: Machine Translation, Translation

This paper presents the results of the shared tasks from the 4th workshop on Asian translation (WAT2017) including J↔E, J↔C scientific paper translation subtasks, C↔J, K↔J, E↔J patent translation subtasks, H↔E mixed domain subtasks, J↔E newswire subtasks and J↔E recipe subtasks. For the WAT2017, 12 institutions participated in the shared tasks. About 300 translation results have been submitted to the automatic evaluation server, and selected submissions were manually evaluated.
Controlling Target Features in Neural Machine Translation via Prefix Constraints
Authors: Shunsuke Takeno, Masaaki Nagata, Kazuhide Yamamoto
Date: 2017-11-01 00:00:00
Proceeding: WS 2017 11
Tasks: Domain Adaptation, Machine Translation, Sentence, Translation

We propose \textit{prefix constraints}, a novel method to enforce constraints on target sentences in neural machine translation. It places a sequence of special tokens at the beginning of target sentence (target prefix), while side constraints places a special token at the end of source sentence (source suffix). Prefix constraints can be predicted from source sentence jointly with target sentence, while side constraints (Sennrich et al., 2016) must be provided by the user or predicted by some other methods. In both methods, special tokens are designed to encode arbitrary features on target-side or metatextual information. We show that prefix constraints are more flexible than side constraints and can be used to control the behavior of neural machine translation, in terms of output length, bidirectional decoding, domain adaptation, and unaligned target word generation.
Improving Japanese-to-English Neural Machine Translation by Paraphrasing the Target Language
Authors: Yuuki Sekizawa, Tomoyuki Kajiwara, Mamoru Komachi
Date: 2017-11-01 00:00:00
Proceeding: WS 2017 11
Tasks: Machine Translation, NMT, Translation

Neural machine translation (NMT) produces sentences that are more fluent than those produced by statistical machine translation (SMT). However, NMT has a very high computational cost because of the high dimensionality of the output layer. Generally, NMT restricts the size of vocabulary, which results in infrequent words being treated as out-of-vocabulary (OOV) and degrades the performance of the translation. In evaluation, we achieved a statistically significant BLEU score improvement of 0.55-0.77 over the baselines including the state-of-the-art method.
Improving Low-Resource Neural Machine Translation with Filtered Pseudo-Parallel Corpus
Authors: Aizhan Imankulova, Takayuki Sato, Mamoru Komachi
Date: 2017-11-01 00:00:00
Proceeding: WS 2017 11
Tasks: Language Modeling, Language Modelling, Low Resource Neural Machine Translation, Low-Resource Neural Machine Translation, Machine Translation, Sentence, Translation

Large-scale parallel corpora are indispensable to train highly accurate machine translators. However, manually constructed large-scale parallel corpora are not freely available in many language pairs. In previous studies, training data have been expanded using a pseudo-parallel corpus obtained using machine translation of the monolingual corpus in the target language. However, in low-resource language pairs in which only low-accuracy machine translation systems can be used, translation quality is reduces when a pseudo-parallel corpus is used naively. To improve machine translation performance with low-resource language pairs, we propose a method to expand the training data effectively via filtering the pseudo-parallel corpus using a quality estimation based on back-translation. As a result of experiments with three language pairs using small, medium, and large parallel corpora, language pairs with fewer training data filtered out more sentence pairs and improved BLEU scores more significantly.
Japanese to English/Chinese/Korean Datasets for Translation Quality Estimation and Automatic Post-Editing
Authors: Atsushi Fujita, Eiichiro Sumita
Date: 2017-11-01 00:00:00
Proceeding: WS 2017 11
Tasks: Automatic Post-Editing, Benchmarking, Machine Translation, Sentence, Translation

Aiming at facilitating the research on quality estimation (QE) and automatic post-editing (APE) of machine translation (MT) outputs, especially for those among Asian languages, we have created new datasets for Japanese to English, Chinese, and Korean translations. As the source text, actual utterances in Japanese were extracted from the log data of our speech translation service. MT outputs were then given by phrase-based statistical MT systems. Finally, human evaluators were employed to grade the quality of MT outputs and to post-edit them. This paper describes the characteristics of the created datasets and reports on our benchmarking experiments on word-level QE, sentence-level QE, and APE conducted using the created datasets.
NTT Neural Machine Translation Systems at WAT 2017
Authors: Makoto Morishita, Jun Suzuki, Masaaki Nagata
Date: 2017-11-01 00:00:00
Proceeding: WS 2017 11
Tasks: Machine Translation, Translation

In this year, we participated in four translation subtasks at WAT 2017. Our model structure is quite simple but we used it with well-tuned hyper-parameters, leading to a significant improvement compared to the previous state-of-the-art system. We also tried to make use of the unreliable part of the provided parallel corpus by back-translating and making a synthetic corpus. Our submitted system achieved the new state-of-the-art performance in terms of the BLEU score, as well as human evaluation.