pwc-archive/papers-with-abstracts huggingface dataset:
Question Condensing Networks for Answer Selection in Community Question Answering
Authors: Wei Wu, Xu sun, Houfeng Wang
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Answer Selection, Community Question Answering, Question Answering

Answer selection is an important subtask of community question answering (CQA). In a real-world CQA forum, a question is often represented as two parts: a subject that summarizes the main points of the question, and a body that elaborates on the subject in detail. Previous researches on answer selection usually ignored the difference between these two parts and concatenated them as the question representation. In this paper, we propose the Question Condensing Networks (QCN) to make use of the subject-body relationship of community questions. In our model, the question subject is the primary part of the question representation, and the question body information is aggregated based on similarity and disparity with the question subject. Experimental results show that QCN outperforms all existing models on two CQA datasets.
How Much Attention Do You Need? A Granular Analysis of Neural Machine Translation Architectures
Authors: Tobias Domhan
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Decoder, Machine Translation, NMT, Translation

With recent advances in network architectures for Neural Machine Translation (NMT) recurrent models have effectively been replaced by either convolutional or self-attentional approaches, such as in the Transformer. While the main innovation of the Transformer architecture is its use of self-attentional layers, there are several other aspects, such as attention with multiple heads and the use of many attention layers, that distinguish the model from previous baselines. In this work we take a fine-grained look at the different architectures for NMT. We introduce an Architecture Definition Language (ADL) allowing for a flexible combination of common building blocks. Making use of this language we show in experiments that one can bring recurrent and convolutional models very close to the Transformer performance by borrowing concepts from the Transformer architecture, but not using self-attention. Additionally, we find that self-attention is much more important on the encoder side than on the decoder side, where it can be replaced by a RNN or CNN without a loss in performance in most settings. Surprisingly, even a model without any target side self-attention performs well.
Weakly Supervised Semantic Parsing with Abstract Examples
Authors: Omer Goldman, Veronica Latcinnik, Ehud Nave, Amir Globerson, Jonathan Berant
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Semantic Parsing, Visual Reasoning

Training semantic parsers from weak supervision (denotations) rather than strong supervision (programs) complicates training in two ways. First, a large search space of potential programs needs to be explored at training time to find a correct program. Second, spurious programs that accidentally lead to a correct denotation add noise to training. In this work we propose that in closed worlds with clear semantic types, one can substantially alleviate these problems by utilizing an abstract representation, where tokens in both the language utterance and program are lifted to an abstract form. We show that these abstractions can be defined with a handful of lexical rules and that they result in sharing between different examples that alleviates the difficulties in training. To test our approach, we develop the first semantic parser for CNLVR, a challenging visual reasoning dataset, where the search space is large and overcoming spuriousness is critical, because denotations are either TRUE or FALSE, and thus random programs are likely to lead to a correct denotation. Our method substantially improves performance, and reaches 82.5{\%} accuracy, a 14.7{\%} absolute accuracy improvement compared to the best reported accuracy so far.
Sequence-to-sequence Models for Cache Transition Systems
Authors: Xiaochang Peng, Linfeng Song, Daniel Gildea, Giorgio Satta
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: AMR Parsing, Decoder, Hard Attention, Sentence Compression, Text Summarization

In this paper, we present a sequence-to-sequence based approach for mapping natural language sentences to AMR semantic graphs. We transform the sequence to graph mapping problem to a word sequence to transition action sequence problem using a special transition system called a cache transition system. To address the sparsity issue of neural AMR parsing, we feed feature embeddings from the transition state to provide relevant local information for each decoder state. We present a monotonic hard attention model for the transition framework to handle the strictly left-to-right alignment between each transition state and the current buffer input focus. We evaluate our neural transition model on the AMR parsing task, and our parser outperforms other sequence-to-sequence approaches and achieves competitive results in comparison with the best-performing models.
Batch IS NOT Heavy: Learning Word Representations From All Samples
Authors: Xin Xin, Fajie Yuan, Xiangnan He, Joemon M. Jose
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: All, Document Classification, Named Entity Recognition (NER), Word Embeddings

Stochastic Gradient Descent (SGD) with negative sampling is the most prevalent approach to learn word representations. However, it is known that sampling methods are biased especially when the sampling distribution deviates from the true data distribution. Besides, SGD suffers from dramatic fluctuation due to the one-sample learning scheme. In this work, we propose AllVec that uses batch gradient learning to generate word representations from all training samples. Remarkably, the time complexity of AllVec remains at the same level as SGD, being determined by the number of positive samples rather than all samples. We evaluate AllVec on several benchmark tasks. Experiments show that AllVec outperforms sampling-based SGD methods with comparable efficiency, especially for small training corpora.
Learning How to Actively Learn: A Deep Imitation Learning Approach
Authors: Ming Liu, Wray Buntine, Gholamreza Haffari
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Active Learning, General Classification, Imitation Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), text-classification, Text Classification

Heuristic-based active learning (AL) methods are limited when the data distribution of the underlying learning problems vary. We introduce a method that learns an AL {``}policy{''} using {``}imitation learning{''} (IL). Our IL-based approach makes use of an efficient and effective {``}algorithmic expert{''}, which provides the policy learner with good actions in the encountered AL situations. The AL strategy is then learned with a feedforward network, mapping situations to most informative query datapoints. We evaluate our method on two different tasks: text classification and named entity recognition. Experimental results show that our IL-based AL strategy is more effective than strong previous methods using heuristics and reinforcement learning.
Language Generation via DAG Transduction
Authors: Yajie Ye, Weiwei Sun, Xiaojun Wan
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Semantic Parsing, Text Generation

A DAG automaton is a formal device for manipulating graphs. By augmenting a DAG automaton with transduction rules, a DAG transducer has potential applications in fundamental NLP tasks. In this paper, we propose a novel DAG transducer to perform graph-to-program transformation. The target structure of our transducer is a program licensed by a declarative programming language rather than linguistic structures. By executing such a program, we can easily get a surface string. Our transducer is designed especially for natural language generation (NLG) from type-logical semantic graphs. Taking Elementary Dependency Structures, a format of English Resource Semantics, as input, our NLG system achieves a BLEU-4 score of 68.07. This remarkable result demonstrates the feasibility of applying a DAG transducer to resolve NLG, as well as the effectiveness of our design.
A Distributional and Orthographic Aggregation Model for English Derivational Morphology
Authors: Daniel Deutsch, John Hewitt, Dan Roth
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: abstractive question answering, Machine Translation, Question Answering, Text Generation, Translation

Modeling derivational morphology to generate words with particular semantics is useful in many text generation tasks, such as machine translation or abstractive question answering. In this work, we tackle the task of derived word generation. That is, we attempt to generate the word {``}runner{''} for {``}someone who runs.{''} We identify two key problems in generating derived words from root words and transformations. We contribute a novel aggregation model of derived word generation that learns derivational transformations both as orthographic functions using sequence-to-sequence models and as functions in distributional word embedding space. The model then learns to choose between the hypothesis of each system. We also present two ways of incorporating corpus information into derived word generation.
Stock Movement Prediction from Tweets and Historical Prices
Authors: Yumo Xu, Shay B. Cohen
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Feature Engineering, Prediction, Stock Market Prediction, Stock Trend Prediction, Time Series Analysis, Variational Inference

Stock movement prediction is a challenging problem: the market is highly stochastic, and we make temporally-dependent predictions from chaotic data. We treat these three complexities and present a novel deep generative model jointly exploiting text and price signals for this task. Unlike the case with discriminative or topic modeling, our model introduces recurrent, continuous latent variables for a better treatment of stochasticity, and uses neural variational inference to address the intractable posterior inference. We also provide a hybrid objective with temporal auxiliary to flexibly capture predictive dependencies. We demonstrate the state-of-the-art performance of our proposed model on a new stock movement prediction dataset which we collected.
Rumor Detection on Twitter with Tree-structured Recursive Neural Networks
Authors: Jing Ma, Wei Gao, Kam-Fai Wong
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Feature Engineering, General Classification, Representation Learning, Rumour Detection

Automatic rumor detection is technically very challenging. In this work, we try to learn discriminative features from tweets content by following their non-sequential propagation structure and generate more powerful representations for identifying different type of rumors. We propose two recursive neural models based on a bottom-up and a top-down tree-structured neural networks for rumor representation learning and classification, which naturally conform to the propagation layout of tweets. Results on two public Twitter datasets demonstrate that our recursive neural models 1) achieve much better performance than state-of-the-art approaches; 2) demonstrate superior capacity on detecting rumors at very early stage.
Visual Attention Model for Name Tagging in Multimodal Social Media
Authors: Di Lu, Leonardo Neves, Vitor Carvalho, Ning Zhang, Heng Ji
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Natural Language Understanding, Question Answering

Everyday billions of multimodal posts containing both images and text are shared in social media sites such as Snapchat, Twitter or Instagram. This combination of image and text in a single message allows for more creative and expressive forms of communication, and has become increasingly common in such sites. This new paradigm brings new challenges for natural language understanding, as the textual component tends to be shorter, more informal, and often is only understood if combined with the visual context. In this paper, we explore the task of name tagging in multimodal social media posts. We start by creating two new multimodal datasets: the first based on Twitter posts and the second based on Snapchat captions (exclusively submitted to public and crowd-sourced stories). We then propose a novel model architecture based on Visual Attention that not only provides deeper visual understanding on the decisions of the model, but also significantly outperforms other state-of-the-art baseline methods for this task.
Multimodal Named Entity Disambiguation for Noisy Social Media Posts
Authors: Seungwhan Moon, Leonardo Neves, Vitor Carvalho
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Entity Disambiguation, Image Captioning, Knowledge Graph Embeddings, Opinion Mining

We introduce the new Multimodal Named Entity Disambiguation (MNED) task for multimodal social media posts such as Snapchat or Instagram captions, which are composed of short captions with accompanying images. Social media posts bring significant challenges for disambiguation tasks because 1) ambiguity not only comes from polysemous entities, but also from inconsistent or incomplete notations, 2) very limited context is provided with surrounding words, and 3) there are many emerging entities often unseen during training. To this end, we build a new dataset called SnapCaptionsKB, a collection of Snapchat image captions submitted to public and crowd-sourced stories, with named entity mentions fully annotated and linked to entities in an external knowledge base. We then build a deep zeroshot multimodal network for MNED that 1) extracts contexts from both text and image, and 2) predicts correct entity in the knowledge graph embeddings space, allowing for zeroshot disambiguation of entities unseen in training set as well. The proposed model significantly outperforms the state-of-the-art text-only NED models, showing efficacy and potentials of the MNED task.
Document Modeling with External Attention for Sentence Extraction
Authors: Shashi Narayan, Ronald Cardenas, Nikos Papasarantopoulos, Shay B. Cohen, Mirella Lapata, Jiangsheng Yu, Yi Chang
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Answer Selection, Document Summarization, Extractive Document Summarization, Extractive Text Summarization, Image Captioning, Informativeness, Language Modeling, Language Modelling, Machine Reading Comprehension, Natural Language Understanding, Reading Comprehension, Sentence

Document modeling is essential to a variety of natural language understanding tasks. We propose to use external information to improve document modeling for problems that can be framed as sentence extraction. We develop a framework composed of a hierarchical document encoder and an attention-based extractor with attention over external information. We evaluate our model on extractive document summarization (where the external information is image captions and the title of the document) and answer selection (where the external information is a question). We show that our model consistently outperforms strong baselines, in terms of both informativeness and fluency (for CNN document summarization) and achieves state-of-the-art results for answer selection on WikiQA and NewsQA.
Syntax for Semantic Role Labeling, To Be, Or Not To Be
Authors: Shexia He, Zuchao Li, Hai Zhao, Hongxiao Bai
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Dependency Parsing, Feature Engineering, Machine Translation, Question Answering, Semantic Parsing, Semantic Role Labeling, Sentence

Semantic role labeling (SRL) is dedicated to recognizing the predicate-argument structure of a sentence. Previous studies have shown syntactic information has a remarkable contribution to SRL performance. However, such perception was challenged by a few recent neural SRL models which give impressive performance without a syntactic backbone. This paper intends to quantify the importance of syntactic information to dependency SRL in deep learning framework. We propose an enhanced argument labeling model companying with an extended korder argument pruning algorithm for effectively exploiting syntactic information. Our model achieves state-of-the-art results on the CoNLL-2008, 2009 benchmarks for both English and Chinese, showing the quantitative significance of syntax to neural SRL together with a thorough empirical survey over existing models.
To Attend or not to Attend: A Case Study on Syntactic Structures for Semantic Relatedness
Authors: Amulya Gupta, Zhu Zhang
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Machine Translation, Paraphrase Identification, Question Answering, Sentence, Translation

With the recent success of Recurrent Neural Networks (RNNs) in Machine Translation (MT), attention mechanisms have become increasingly popular. The purpose of this paper is two-fold; firstly, we propose a novel attention model on Tree Long Short-Term Memory Networks (Tree-LSTMs), a tree-structured generalization of standard LSTM. Secondly, we study the interaction between attention and syntactic structures, by experimenting with three LSTM variants: bidirectional-LSTMs, Constituency Tree-LSTMs, and Dependency Tree-LSTMs. Our models are evaluated on two semantic relatedness tasks: semantic relatedness scoring for sentence pairs (SemEval 2012, Task 6 and SemEval 2014, Task 1) and paraphrase detection for question pairs (Quora, 2017).
What you can cram into a single \$\&!\#* vector: Probing sentence embeddings for linguistic properties
Authors: Alexis Conneau, German Kruszewski, Guillaume Lample, Lo{\"\i}c Barrault, Marco Baroni
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: General Classification, Machine Translation, Sentence, Sentence Classification, Sentence Embedding, Sentence Embeddings

Although much effort has recently been devoted to training high-quality sentence embeddings, we still have a poor understanding of what they are capturing. {``}Downstream{''} tasks, often based on sentence classification, are commonly used to evaluate the quality of sentence representations. The complexity of the tasks makes it however difficult to infer what kind of information is present in the representations. We introduce here 10 probing tasks designed to capture simple linguistic features of sentences, and we use them to study embeddings generated by three different encoders trained in eight distinct ways, uncovering intriguing properties of both encoders and training methods.
Recursive Neural Structural Correspondence Network for Cross-domain Aspect and Opinion Co-Extraction
Authors: Wenya Wang, Sinno Jialin Pan
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Domain Adaptation, Extract Aspect, Fine-Grained Opinion Analysis, Opinion Summarization, Sentence, Sentiment Analysis, Unsupervised Domain Adaptation

Fine-grained opinion analysis aims to extract aspect and opinion terms from each sentence for opinion summarization. Supervised learning methods have proven to be effective for this task. However, in many domains, the lack of labeled data hinders the learning of a precise extraction model. In this case, unsupervised domain adaptation methods are desired to transfer knowledge from the source domain to any unlabeled target domain. In this paper, we develop a novel recursive neural network that could reduce domain shift effectively in word level through syntactic relations. We treat these relations as invariant {``}pivot information{''} across domains to build structural correspondences and generate an auxiliary task to predict the relation between any two adjacent words in the dependency tree. In the end, we demonstrate state-of-the-art results on three benchmark datasets.
Efficient Large-Scale Neural Domain Classification with Personalized Attention
Authors: Young-Bum Kim, Dongchan Kim, Anjishnu Kumar, Ruhi Sarikaya
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Classification, domain classification, General Classification, Natural Language Understanding, Spoken Language Understanding

In this paper, we explore the task of mapping spoken language utterances to one of thousands of natural language understanding domains in intelligent personal digital assistants (IPDAs). This scenario is observed in mainstream IPDAs in industry that allow third parties to develop thousands of new domains to augment built-in first party domains to rapidly increase domain coverage and overall IPDA capabilities. We propose a scalable neural model architecture with a shared encoder, a novel attention mechanism that incorporates personalization information and domain-specific classifiers that solves the problem efficiently. Our architecture is designed to efficiently accommodate incremental domain additions achieving two orders of magnitude speed up compared to full model retraining. We consider the practical constraints of real-time production systems, and design to minimize memory footprint and runtime latency. We demonstrate that incorporating personalization significantly improves domain classification accuracy in a setting with thousands of overlapping domains.
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Authors: AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, Louis-Philippe Morency
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Emotion Recognition, Language Modeling, Language Modelling, Multimodal Sentiment Analysis, Multi-Task Learning, Sentiment Analysis

Analyzing human multimodal language is an emerging area of research in NLP. Intrinsically this language is multimodal (heterogeneous), sequential and asynchronous; it consists of the language (words), visual (expressions) and acoustic (paralinguistic) modalities all in the form of asynchronous coordinated sequences. From a resource perspective, there is a genuine need for large scale datasets that allow for in-depth studies of this form of language. In this paper we introduce CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI), the largest dataset of sentiment analysis and emotion recognition to date. Using data from CMU-MOSEI and a novel multimodal fusion technique called the Dynamic Fusion Graph (DFG), we conduct experimentation to exploit how modalities interact with each other in human multimodal language. Unlike previously proposed fusion techniques, DFG is highly interpretable and achieves competative performance when compared to the previous state of the art.
Discourse Coherence: Concurrent Explicit and Implicit Relations
Authors: Hannah Rohde, Alex Johnson, er, Nathan Schneider, Bonnie Webber
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Discourse Parsing, Implicit Relations

Theories of discourse coherence posit relations between discourse segments as a key feature of coherent text. Our prior work suggests that multiple discourse relations can be simultaneously operative between two segments for reasons not predicted by the literature. Here we test how this joint presence can lead participants to endorse seemingly divergent conjunctions (e.g., BUT and SO) to express the link they see between two segments. These apparent divergences are not symptomatic of participant naivety or bias, but arise reliably from the concurrent availability of multiple relations between segments {--} some available through explicit signals and some via inference. We believe that these new results can both inform future progress in theoretical work on discourse coherence and lead to higher levels of performance in discourse parsing.