pwc-archive/papers-with-abstracts huggingface dataset:
Latent Retrieval for Weakly Supervised Open Domain Question Answering
Authors: Kenton Lee, Ming-Wei Chang, Kristina Toutanova
Date: 2019-06-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Information Retrieval, Open-Domain Question Answering, Question Answering, Retrieval

Recent work on open domain question answering (QA) assumes strong supervision of the supporting evidence and/or assumes a blackbox information retrieval (IR) system to retrieve evidence candidates. We argue that both are suboptimal, since gold evidence is not always available, and QA is fundamentally different from IR. We show for the first time that it is possible to jointly learn the retriever and reader from question-answer string pairs and without any IR system. In this setting, evidence retrieval from all of Wikipedia is treated as a latent variable. Since this is impractical to learn from scratch, we pre-train the retriever with an Inverse Cloze Task. We evaluate on open versions of five QA datasets. On datasets where the questioner already knows the answer, a traditional IR system such as BM25 is sufficient. On datasets where a user is genuinely seeking an answer, we show that learned retrieval is crucial, outperforming BM25 by up to 19 points in exact match.
Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation
Authors: Ronghang Hu, Daniel Fried, Anna Rohrbach, Dan Klein, Trevor Darrell, Kate Saenko
Date: 2019-06-02 00:00:00
Proceeding: ACL 2019 7
Tasks: Vision and Language Navigation

Vision-and-Language Navigation (VLN) requires grounding instructions, such as "turn right and stop at the door", to routes in a visual environment. The actual grounding can connect language to the environment through multiple modalities, e.g. "stop at the door" might ground into visual objects, while "turn right" might rely only on the geometric structure of a route. We investigate where the natural language empirically grounds under two recent state-of-the-art VLN models. Surprisingly, we discover that visual features may actually hurt these models: models which only use route structure, ablating visual features, outperform their visual counterparts in unseen new environments on the benchmark Room-to-Room dataset. To better use all the available modalities, we propose to decompose the grounding procedure into a set of expert models with access to different modalities (including object detections) and ensemble them at prediction time, improving the performance of state-of-the-art models on the VLN task.
Domain Adaptation of Neural Machine Translation by Lexicon Induction
Authors: Junjie Hu, Mengzhou Xia, Graham Neubig, Jaime Carbonell
Date: 2019-06-02 00:00:00
Proceeding: ACL 2019 7
Tasks: Domain Adaptation, Machine Translation, NMT, Translation

It has been previously noted that neural machine translation (NMT) is very sensitive to domain shift. In this paper, we argue that this is a dual effect of the highly lexicalized nature of NMT, resulting in failure for sentences with large numbers of unknown words, and lack of supervision for domain-specific words. To remedy this problem, we propose an unsupervised adaptation method which fine-tunes a pre-trained out-of-domain NMT model using a pseudo-in-domain corpus. Specifically, we perform lexicon induction to extract an in-domain lexicon, and construct a pseudo-parallel in-domain corpus by performing word-for-word back-translation of monolingual in-domain target sentences. In five domains over twenty pairwise adaptation settings and two model architectures, our method achieves consistent improvements without using any in-domain parallel sentences, improving up to 14 BLEU over unadapted models, and up to 2 BLEU over strong back-translation baselines.
Domain Adaptive Inference for Neural Machine Translation
Authors: Danielle Saunders, Felix Stahlberg, Adria de Gispert, Bill Byrne
Date: 2019-06-02 00:00:00
Proceeding: ACL 2019 7
Tasks: Machine Translation, NMT, Translation

We investigate adaptive ensemble weighting for Neural Machine Translation, addressing the case of improving performance on a new and potentially unknown domain without sacrificing performance on the original domain. We adapt sequentially across two Spanish-English and three English-German tasks, comparing unregularized fine-tuning, L2 and Elastic Weight Consolidation. We then report a novel scheme for adaptive NMT ensemble decoding by extending Bayesian Interpolation with source information, and show strong improvements across test domains without access to the domain label.
Plain English Summarization of Contracts
Authors: Laura Manor, Junyi Jessy Li
Date: 2019-06-02 00:00:00
Proceeding: WS 2019 6
Tasks: Extractive Summarization, Text Summarization, Unsupervised Extractive Summarization

Unilateral contracts, such as terms of service, play a substantial role in modern digital life. However, few users read these documents before accepting the terms within, as they are too long and the language too complicated. We propose the task of summarizing such legal documents in plain English, which would enable users to have a better understanding of the terms they are accepting. We propose an initial dataset of legal text snippets paired with summaries written in plain English. We verify the quality of these summaries manually and show that they involve heavy abstraction, compression, and simplification. Initial experiments show that unsupervised extractive summarization methods do not perform well on this task due to the level of abstraction and style differences. We conclude with a call for resource and technique development for simplification and style transfer for legal language.
Deep Unknown Intent Detection with Margin Loss
Authors: Ting-En Lin, Hua Xu
Date: 2019-06-02 00:00:00
Proceeding: ACL 2019 7
Tasks: Intent Detection, Novelty Detection, Open Intent Detection

Identifying the unknown (novel) user intents that have never appeared in the training set is a challenging task in the dialogue system. In this paper, we present a two-stage method for detecting unknown intents. We use bidirectional long short-term memory (BiLSTM) network with the margin loss as the feature extractor. With margin loss, we can learn discriminative deep features by forcing the network to maximize inter-class variance and to minimize intra-class variance. Then, we feed the feature vectors to the density-based novelty detection algorithm, local outlier factor (LOF), to detect unknown intents. Experiments on two benchmark datasets show that our method can yield consistent improvements compared with the baseline methods.
A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions
Authors: Sashank Santhanam, Samira Shaikh
Date: 2019-06-02 00:00:00
Tasks: Articles, Text Generation, World Knowledge

One of the hardest problems in the area of Natural Language Processing and Artificial Intelligence is automatically generating language that is coherent and understandable to humans. Teaching machines how to converse as humans do falls under the broad umbrella of Natural Language Generation. Recent years have seen unprecedented growth in the number of research articles published on this subject in conferences and journals both by academic and industry researchers. There have also been several workshops organized alongside top-tier NLP conferences dedicated specifically to this problem. All this activity makes it hard to clearly define the state of the field and reason about its future directions. In this work, we provide an overview of this important and thriving area, covering traditional approaches, statistical approaches and also approaches that use deep neural networks. We provide a comprehensive review towards building open domain dialogue systems, an important application of natural language generation. We find that, predominantly, the approaches for building dialogue systems use seq2seq or language models architecture. Notably, we identify three important areas of further research towards building more effective dialogue systems: 1) incorporating larger context, including conversation context and world knowledge; 2) adding personae or personality in the NLG system; and 3) overcoming dull and generic responses that affect the quality of system-produced responses. We provide pointers on how to tackle these open problems through the use of cognitive architectures that mimic human language understanding and generation capabilities.
Know More about Each Other: Evolving Dialogue Strategy via Compound Assessment
Authors: Siqi Bao, Huang He, Fan Wang, Rongzhong Lian, Hua Wu
Date: 2019-06-03 00:00:00
Proceeding: ACL 2019 7
Tasks: Informativeness, Reinforcement Learning

In this paper, a novel Generation-Evaluation framework is developed for multi-turn conversations with the objective of letting both participants know more about each other. For the sake of rational knowledge utilization and coherent conversation flow, a dialogue strategy which controls knowledge selection is instantiated and continuously adapted via reinforcement learning. Under the deployed strategy, knowledge grounded conversations are conducted with two dialogue agents. The generated dialogues are comprehensively evaluated on aspects like informativeness and coherence, which are aligned with our objective and human instinct. These assessments are integrated as a compound reward to guide the evolution of dialogue strategy via policy gradient. Comprehensive experiments have been carried out on the publicly available dataset, demonstrating that the proposed method outperforms the other state-of-the-art approaches significantly.
Global Textual Relation Embedding for Relational Understanding
Authors: Zhiyu Chen, Hanwen Zha, Honglei Liu, Wenhu Chen, Xifeng Yan, Yu Su
Date: 2019-06-03 00:00:00
Proceeding: ACL 2019 7
Tasks: Action Classification, Relation, Sentence, Sentence Embeddings, Word Embeddings

Pre-trained embeddings such as word embeddings and sentence embeddings are fundamental tools facilitating a wide range of downstream NLP tasks. In this work, we investigate how to learn a general-purpose embedding of textual relations, defined as the shortest dependency path between entities. Textual relation embedding provides a level of knowledge between word/phrase level and sentence level, and we show that it can facilitate downstream tasks requiring relational understanding of the text. To learn such an embedding, we create the largest distant supervision dataset by linking the entire English ClueWeb09 corpus to Freebase. We use global co-occurrence statistics between textual and knowledge base relations as the supervision signal to train the embedding. Evaluation on two relational understanding tasks demonstrates the usefulness of the learned textual relation embedding. The data and code can be found at https://github.com/czyssrs/GloREPlus
Fluent Translations from Disfluent Speech in End-to-End Speech Translation
Authors: Elizabeth Salesky, Matthias Sperber, Alex Waibel
Date: 2019-06-03 00:00:00
Proceeding: NAACL 2019 6
Tasks: Machine Translation, speech-recognition, Speech Recognition, Translation

Spoken language translation applications for speech suffer due to conversational speech phenomena, particularly the presence of disfluencies. With the rise of end-to-end speech translation models, processing steps such as disfluency removal that were previously an intermediate step between speech recognition and machine translation need to be incorporated into model architectures. We use a sequence-to-sequence model to translate from noisy, disfluent speech to fluent text with disfluencies removed using the recently collected `copy-edited' references for the Fisher Spanish-English dataset. We are able to directly generate fluent translations and introduce considerations about how to evaluate success on this task. This work provides a baseline for a new task, the translation of conversational speech with joint removal of disfluencies.
Controllable Paraphrase Generation with a Syntactic Exemplar
Authors: Mingda Chen, Qingming Tang, Sam Wiseman, Kevin Gimpel
Date: 2019-06-03 00:00:00
Proceeding: ACL 2019 7
Tasks: Attribute, Paraphrase Generation, Representation Learning, Sentence, Text Generation

Prior work on controllable text generation usually assumes that the controlled attribute can take on one of a small set of values known a priori. In this work, we propose a novel task, where the syntax of a generated sentence is controlled rather by a sentential exemplar. To evaluate quantitatively with standard metrics, we create a novel dataset with human annotations. We also develop a variational model with a neural module specifically designed for capturing syntactic knowledge and several multitask training objectives to promote disentangled representation learning. Empirically, the proposed model is observed to achieve improvements over baselines and learn to capture desirable characteristics.
Jointly Learning Semantic Parser and Natural Language Generator via Dual Information Maximization
Authors: Hai Ye, Wenjie Li, Lu Wang
Date: 2019-06-03 00:00:00
Proceeding: ACL 2019 7
Tasks: Code Generation, Dialogue Management, Management, Semantic Parsing

Semantic parsing aims to transform natural language (NL) utterances into formal meaning representations (MRs), whereas an NL generator achieves the reverse: producing a NL description for some given MRs. Despite this intrinsic connection, the two tasks are often studied separately in prior work. In this paper, we model the duality of these two tasks via a joint learning framework, and demonstrate its effectiveness of boosting the performance on both tasks. Concretely, we propose a novel method of dual information maximization (DIM) to regularize the learning process, where DIM empirically maximizes the variational lower bounds of expected joint distributions of NL and MRs. We further extend DIM to a semi-supervision setup (SemiDIM), which leverages unlabeled data of both tasks. Experiments on three datasets of dialogue management and code generation (and summarization) show that performance on both semantic parsing and NL generation can be consistently improved by DIM, in both supervised and semi-supervised setups.
Listening while Speaking and Visualizing: Improving ASR through Multimodal Chain
Authors: Johanes Effendi, Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
Date: 2019-06-03 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Data Augmentation, Image Captioning, Image Retrieval, speech-recognition, Speech Recognition, Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis

Previously, a machine speech chain, which is based on sequence-to-sequence deep learning, was proposed to mimic speech perception and production behavior. Such chains separately processed listening and speaking by automatic speech recognition (ASR) and text-to-speech synthesis (TTS) and simultaneously enabled them to teach each other in semi-supervised learning when they received unpaired data. Unfortunately, this speech chain study is limited to speech and textual modalities. In fact, natural communication is actually multimodal and involves both auditory and visual sensory systems. Although the said speech chain reduces the requirement of having a full amount of paired data, in this case we still need a large amount of unpaired data. In this research, we take a further step and construct a multimodal chain and design a closely knit chain architecture that combines ASR, TTS, image captioning, and image production models into a single framework. The framework allows the training of each component without requiring a large number of parallel multimodal data. Our experimental results also show that an ASR can be further trained without speech and text data and cross-modal data augmentation remains possible through our proposed chain, which improves the ASR performance.
A Semi-Supervised Approach for Low-Resourced Text Generation
Authors: Hongyu Zang, Xiaojun Wan
Date: 2019-06-03 00:00:00
Tasks: Decoder, Denoising, Language Modeling, Language Modelling, Reinforcement Learning, Reinforcement Learning (RL), Text Generation

Recently, encoder-decoder neural models have achieved great success on text generation tasks. However, one problem of this kind of models is that their performances are usually limited by the scale of well-labeled data, which are very expensive to get. The low-resource (of labeled data) problem is quite common in different task generation tasks, but unlabeled data are usually abundant. In this paper, we propose a method to make use of the unlabeled data to improve the performance of such models in the low-resourced circumstances. We use denoising auto-encoder (DAE) and language model (LM) based reinforcement learning (RL) to enhance the training of encoder and decoder with unlabeled data. Our method shows adaptability for different text generation tasks, and makes significant improvements over basic text generation models.
Evaluating Gender Bias in Machine Translation
Authors: Gabriel Stanovsky, Noah A. Smith, Luke Zettlemoyer
Date: 2019-06-03 00:00:00
Proceeding: ACL 2019 7
Tasks: coreference-resolution, Coreference Resolution, Machine Translation, Morphological Analysis, Translation

We present the first challenge set and evaluation protocol for the analysis of gender bias in machine translation (MT). Our approach uses two recent coreference resolution datasets composed of English sentences which cast participants into non-stereotypical gender roles (e.g., "The doctor asked the nurse to help her in the operation"). We devise an automatic gender bias evaluation method for eight target languages with grammatical gender, based on morphological analysis (e.g., the use of female inflection for the word "doctor"). Our analyses show that four popular industrial MT systems and two recent state-of-the-art academic MT models are significantly prone to gender-biased translation errors for all tested target languages. Our data and code are made publicly available.
Semantically Constrained Multilayer Annotation: The Case of Coreference
Authors: Jakob Prange, Nathan Schneider, Omri Abend
Date: 2019-06-03 00:00:00
Proceeding: WS 2019 8

We propose a coreference annotation scheme as a layer on top of the Universal Conceptual Cognitive Annotation foundational layer, treating units in predicate-argument structure as a basis for entity and event mentions. We argue that this allows coreference annotators to sidestep some of the challenges faced in other schemes, which do not enforce consistency with predicate-argument structure and vary widely in what kinds of mentions they annotate and how. The proposed approach is examined with a pilot annotation study and compared with annotations from other schemes.
Multi-task Pairwise Neural Ranking for Hashtag Segmentation
Authors: Mounica Maddela, Wei Xu, Daniel Preoţiuc-Pietro
Date: 2019-06-03 00:00:00
Proceeding: ACL 2019 7
Tasks: Segmentation, Sentiment Analysis

Hashtags are often employed on social media and beyond to add metadata to a textual utterance with the goal of increasing discoverability, aiding search, or providing additional semantics. However, the semantic content of hashtags is not straightforward to infer as these represent ad-hoc conventions which frequently include multiple words joined together and can include abbreviations and unorthodox spellings. We build a dataset of 12,594 hashtags split into individual segments and propose a set of approaches for hashtag segmentation by framing it as a pairwise ranking problem between candidate segmentations. Our novel neural approaches demonstrate 24.6% error reduction in hashtag segmentation accuracy compared to the current state-of-the-art method. Finally, we demonstrate that a deeper understanding of hashtag semantics obtained through segmentation is useful for downstream applications such as sentiment analysis, for which we achieved a 2.6% increase in average recall on the SemEval 2017 sentiment analysis dataset.
Gendered Ambiguous Pronouns Shared Task: Boosting Model Confidence by Evidence Pooling
Authors: Sandeep Attree
Date: 2019-06-03 00:00:00
Tasks: coreference-resolution, Coreference Resolution

This paper presents a strong set of results for resolving gendered ambiguous pronouns on the Gendered Ambiguous Pronouns shared task. The model presented here draws upon the strengths of state-of-the-art language and coreference resolution models, and introduces a novel evidence-based deep learning architecture. Injecting evidence from the coreference models compliments the base architecture, and analysis shows that the model is not hindered by their weaknesses, specifically gender bias. The modularity and simplicity of the architecture make it very easy to extend for further improvement and applicable to other NLP problems. Evaluation on GAP test data results in a state-of-the-art performance at 92.5% F1 (gender bias of 0.97), edging closer to the human performance of 96.6%. The end-to-end solution presented here placed 1st in the Kaggle competition, winning by a significant lead. The code is available at https://github.com/sattree/gap.
Biophysically grounded mean-field models of neural populations under electrical stimulation
Authors: Caglar Cakan, Klaus Obermayer
Date: 2019-06-03 00:00:00

Electrical stimulation of neural systems is a key tool for understanding neural dynamics and ultimately for developing clinical treatments. Many applications of electrical stimulation affect large populations of neurons. However, computational models of large networks of spiking neurons are inherently hard to simulate and analyze. We evaluate a reduced mean-field model of excitatory and inhibitory adaptive exponential integrate-and-fire (AdEx) neurons which can be used to efficiently study the effects of electrical stimulation on large neural populations. The rich dynamical properties of this basic cortical model are described in detail and validated using large network simulations. Bifurcation diagrams reflecting the network's state reveal asynchronous up- and down-states, bistable regimes, and oscillatory regions corresponding to fast excitation-inhibition and slow excitation-adaptation feedback loops. The biophysical parameters of the AdEx neuron can be coupled to an electric field with realistic field strengths which then can be propagated up to the population description.We show how on the edge of bifurcation, direct electrical inputs cause network state transitions, such as turning on and off oscillations of the population rate. Oscillatory input can frequency-entrain and phase-lock endogenous oscillations. Relatively weak electric field strengths on the order of 1 V/m are able to produce these effects, indicating that field effects are strongly amplified in the network. The effects of time-varying external stimulation are well-predicted by the mean-field model, further underpinning the utility of low-dimensional neural mass models.
Poisson multi-Bernoulli mixture trackers: continuity through random finite sets of trajectories
Authors: Karl Granström, Lennart Svensson, Yuxuan Xia, Jason Williams, Angel F Garcia-Fernandez
Date: 2018-12-12 00:00:00

The Poisson multi-Bernoulli mixture (PMBM) is an unlabelled multi-target
distribution for which the prediction and update are closed. It has a Poisson
birth process, and new Bernoulli components are generated on each new
measurement as a part of the Bayesian measurement update. The PMBM filter is
similar to the multiple hypothesis tracker (MHT), but seemingly does not
provide explicit continuity between time steps. This paper considers a recently
developed formulation of the multi-target tracking problem as a random finite
set (RFS) of trajectories, and derives two trajectory RFS filters, called PMBM
trackers. The PMBM trackers efficiently estimate the set of trajectories, and
share hypothesis structure with the PMBM filter. By showing that the prediction
and update in the PMBM filter can be viewed as an efficient method for
calculating the time marginals of the RFS of trajectories, continuity in the
same sense as MHT is established for the PMBM filter.