pwc-archive/papers-with-abstracts huggingface dataset:
Towards Improving Neural Named Entity Recognition with Gazetteers
Authors: Tianyu Liu, Jin-Ge Yao, Chin-Yew Lin
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Most of the recently proposed neural models for named entity recognition have been purely data-driven, with a strong emphasis on getting rid of the efforts for collecting external resources or designing hand-crafted features. This could increase the chance of overfitting since the models cannot access any supervision signal beyond the small amount of annotated data, limiting their power to generalize beyond the annotated entities. In this work, we show that properly utilizing external gazetteers could benefit segmental neural NER models. We add a simple module on the recently proposed hybrid semi-Markov CRF architecture and observe some promising results.
Span-Level Model for Relation Extraction
Authors: Kalpit Dixit, Yaser Al-Onaizan
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: model, Relation, Relation Extraction

Relation Extraction is the task of identifying entity mention spans in raw text and then identifying relations between pairs of the entity mentions. Recent approaches for this span-level task have been token-level models which have inherent limitations. They cannot easily define and implement span-level features, cannot model overlapping entity mentions and have cascading errors due to the use of sequential decoding. To address these concerns, we present a model which directly models all possible spans and performs joint entity mention detection and relation extraction. We report a new state-of-the-art performance of 62.83 F1 (prev best was 60.49) on the ACE2005 dataset.
Enhancing Unsupervised Generative Dependency Parser with Contextual Information
Authors: Wenjuan Han, Yong Jiang, Kewei Tu
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Constituency Grammar Induction, Dependency Grammar Induction, Sentence, Unsupervised Dependency Parsing

Most of the unsupervised dependency parsers are based on probabilistic generative models that learn the joint distribution of the given sentence and its parse. Probabilistic generative models usually explicit decompose the desired dependency tree into factorized grammar rules, which lack the global features of the entire sentence. In this paper, we propose a novel probabilistic model called discriminative neural dependency model with valence (D-NDMV) that generates a sentence and its parse from a continuous latent representation, which encodes global contextual information of the generated sentence. We propose two approaches to model the latent representation: the first deterministically summarizes the representation from the sentence and the second probabilistically models the representation conditioned on the sentence. Our approach can be regarded as a new type of autoencoder model to unsupervised dependency parsing that combines the benefits of both generative and discriminative techniques. In particular, our approach breaks the context-free independence assumption in previous generative approaches and therefore becomes more expressive. Our extensive experimental results on seventeen datasets from various sources show that our approach achieves competitive accuracy compared with both generative and discriminative state-of-the-art unsupervised dependency parsers.
Online Infix Probability Computation for Probabilistic Finite Automata
Authors: Marco Cognetta, Yo-Sub Han, Soon Chan Kwon
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Language Modeling, Language Modelling

Probabilistic finite automata (PFAs) are com- mon statistical language model in natural lan- guage and speech processing. A typical task for PFAs is to compute the probability of all strings that match a query pattern. An impor- tant special case of this problem is computing the probability of a string appearing as a pre- fix, suffix, or infix. These problems find use in many natural language processing tasks such word prediction and text error correction. Recently, we gave the first incremental algorithm to efficiently compute the infix probabilities of each prefix of a string (Cognetta et al., 2018). We develop an asymptotic improvement of that algorithm and solve the open problem of computing the infix probabilities of PFAs from streaming data, which is crucial when process- ing queries online and is the ultimate goal of the incremental approach.
PTB Graph Parsing with Tree Approximation
Authors: Yoshihide Kato, Shigeki Matsubara
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Constituency Parsing

The Penn Treebank (PTB) represents syntactic structures as graphs due to nonlocal dependencies. This paper proposes a method that approximates PTB graph-structured representations by trees. By our approximation method, we can reduce nonlocal dependency identification and constituency parsing into single tree-based parsing. An experimental result demonstrates that our approximation method with an off-the-shelf tree-based constituency parser significantly outperforms the previous methods in nonlocal dependency identification.
A Prism Module for Semantic Disentanglement in Name Entity Recognition
Authors: Kun Liu, Shen Li, Daqi Zheng, Zhengdong Lu, Sheng Gao, Si Li
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Disentanglement, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Natural Language Processing has been perplexed for many years by the problem that multiple semantics are mixed inside a word, even with the help of context. To solve this problem, we propose a prism module to disentangle the semantic aspects of words and reduce noise at the input layer of a model. In the prism module, some words are selectively replaced with task-related semantic aspects, then these denoised word representations can be fed into downstream tasks to make them easier. Besides, we also introduce a structure to train this module jointly with the downstream model without additional data. This module can be easily integrated into the downstream model and significantly improve the performance of baselines on named entity recognition (NER) task. The ablation analysis demonstrates the rationality of the method. As a side effect, the proposed method also provides a way to visualize the contribution of each word.
Collaborative Dialogue in Minecraft
Authors: Anjali Narayan-Chen, Prashant Jayannavar, Julia Hockenmaier
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Minecraft

We wish to develop interactive agents that can communicate with humans to collaboratively solve tasks in grounded scenarios. Since computer games allow us to simulate such tasks without the need for physical robots, we define a Minecraft-based collaborative building task in which one player (A, the Architect) is shown a target structure and needs to instruct the other player (B, the Builder) to build this structure. Both players interact via a chat interface. A can observe B but cannot place blocks. We present the Minecraft Dialogue Corpus, a collection of 509 conversations and game logs. As a first step towards our goal of developing fully interactive agents for this task, we consider the subtask of Architect utterance generation, and show how challenging it is.
Ordinal and Attribute Aware Response Generation in a Multimodal Dialogue System
Authors: Hardik Chauhan, Mauajama Firdaus, Asif Ekbal, Pushpak Bhattacharyya
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Attribute, Goal-Oriented Dialogue Systems, Position, Response Generation, text similarity

Multimodal dialogue systems have opened new frontiers in the traditional goal-oriented dialogue systems. The state-of-the-art dialogue systems are primarily based on unimodal sources, predominantly the text, and hence cannot capture the information present in the other sources such as videos, audios, images etc. With the availability of large scale multimodal dialogue dataset (MMD) (Saha et al., 2018) on the fashion domain, the visual appearance of the products is essential for understanding the intention of the user. Without capturing the information from both the text and image, the system will be incapable of generating correct and desirable responses. In this paper, we propose a novel position and attribute aware attention mechanism to learn enhanced image representation conditioned on the user utterance. Our evaluation shows that the proposed model can generate appropriate responses while preserving the position and attribute information. Experimental results also prove that our proposed approach attains superior performance compared to the baseline models, and outperforms the state-of-the-art approaches on text similarity based evaluation metrics.
Reading Turn by Turn: Hierarchical Attention Architecture for Spoken Dialogue Comprehension
Authors: Zhengyuan Liu, Nancy Chen
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Reading Comprehension

Comprehending multi-turn spoken conversations is an emerging research area, presenting challenges different from reading comprehension of passages due to the interactive nature of information exchange from at least two speakers. Unlike passages, where sentences are often the default semantic modeling unit, in multi-turn conversations, a turn is a topically coherent unit embodied with immediately relevant context, making it a linguistically intuitive segment for computationally modeling verbal interactions. Therefore, in this work, we propose a hierarchical attention neural network architecture, combining turn-level and word-level attention mechanisms, to improve spoken dialogue comprehension performance. Experiments are conducted on a multi-turn conversation dataset, where nurses inquire and discuss symptom information with patients. We empirically show that the proposed approach outperforms standard attention baselines, achieves more efficient learning outcomes, and is more robust to lengthy and out-of-distribution test samples.
Rationally Reappraising ATIS-based Dialogue Systems
Authors: Jingcheng Niu, Gerald Penn
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: slot-filling, Slot Filling, Spoken Language Understanding

The Air Travel Information Service (ATIS) corpus has been the most common benchmark for evaluating Spoken Language Understanding (SLU) tasks for more than three decades since it was released. Recent state-of-the-art neural models have obtained F1-scores near 98{\%} on the task of slot filling. We developed a rule-based grammar for the ATIS domain that achieves a 95.82{\%} F1-score on our evaluation set. In the process, we furthermore discovered numerous shortcomings in the ATIS corpus annotation, which we have fixed. This paper presents a detailed account of these shortcomings, our proposed repairs, our rule-based grammar and the neural slot-filling architectures associated with ATIS. We also rationally reappraise the motivations for choosing a neural architecture in view of this account. Fixing the annotation errors results in a relative error reduction of between 19.4 and 52{\%} across all architectures. We nevertheless argue that neural models must play a different role in ATIS dialogues because of the latter{'}s lack of variety.
Neural-based Chinese Idiom Recommendation for Enhancing Elegance in Essay Writing
Authors: Yuanchao Liu, Bo Pang, Bingquan Liu
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Machine Translation, Translation

Although the proper use of idioms can enhance the elegance of writing, the active use of various expressions is a challenge because remembering idioms is difficult. In this study, we address the problem of idiom recommendation by leveraging a neural machine translation framework, in which we suppose that idioms are written with one pseudo target language. Two types of real-life datasets are collected to support this study. Experimental results show that the proposed approach achieves promising performance compared with other baseline methods.
Better Exploiting Latent Variables in Text Modeling
Authors: Canasai Kruengkrai
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7

We show that sampling latent variables multiple times at a gradient step helps in improving a variational autoencoder and propose a simple and effective method to better exploit these latent variables through hidden state averaging. Consistent gains in performance on two different datasets, Penn Treebank and Yahoo, indicate the generalizability of our method.
Reversing Gradients in Adversarial Domain Adaptation for Question Deduplication and Textual Entailment Tasks
Authors: Anush Kamath, Sparsh Gupta, Vitor Carvalho
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Domain Adaptation, Natural Language Inference, RTE

Adversarial domain adaptation has been recently proposed as an effective technique for textual matching tasks, such as question deduplication. Here we investigate the use of gradient reversal on adversarial domain adaptation to explicitly learn both shared and unshared (domain specific) representations between two textual domains. In doing so, gradient reversal learns features that explicitly compensate for domain mismatch, while still distilling domain specific knowledge that can improve target domain accuracy. We evaluate reversing gradients for adversarial adaptation on multiple domains, and demonstrate that it significantly outperforms other methods on question deduplication as well as on recognizing textual entailment (RTE) tasks, achieving up to 7{\%} absolute boost in base model accuracy on some datasets.
An Empirical Investigation of Structured Output Modeling for Graph-based Neural Dependency Parsing
Authors: Zhisong Zhang, Xuezhe Ma, Eduard Hovy
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Dependency Parsing, Sentence

In this paper, we investigate the aspect of structured output modeling for the state-of-the-art graph-based neural dependency parser (Dozat and Manning, 2017). With evaluations on 14 treebanks, we empirically show that global output-structured models can generally obtain better performance, especially on the metric of sentence-level Complete Match. However, probably because neural models already learn good global views of the inputs, the improvement brought by structured output modeling is modest.
Zero-shot Word Sense Disambiguation using Sense Definition Embeddings
Authors: Sawan Kumar, Sharmistha Jat, Karan Saxena, Partha Talukdar
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Generalized Zero-Shot Learning, Graph Embedding, Knowledge Graph Embedding, Sentence, Word Sense Disambiguation, Zero-Shot Learning

Word Sense Disambiguation (WSD) is a long-standing but open problem in Natural Language Processing (NLP). WSD corpora are typically small in size, owing to an expensive annotation process. Current supervised WSD methods treat senses as discrete labels and also resort to predicting the Most-Frequent-Sense (MFS) for words unseen during training. This leads to poor performance on rare and unseen senses. To overcome this challenge, we propose Extended WSD Incorporating Sense Embeddings (EWISE), a supervised model to perform WSD by predicting over a continuous sense embedding space as opposed to a discrete label space. This allows EWISE to generalize over both seen and unseen senses, thus achieving generalized zero-shot learning. To obtain target sense embeddings, EWISE utilizes sense definitions. EWISE learns a novel sentence encoder for sense definitions by using WordNet relations and also ConvE, a recently proposed knowledge graph embedding method. We also compare EWISE against other sentence encoders pretrained on large corpora to generate definition embeddings. EWISE achieves new state-of-the-art WSD performance.
Word2Sense: Sparse Interpretable Word Embeddings
Authors: Abhishek Panigrahi, Harsha Vardhan Simhadri, Chiranjib Bhattacharyya
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Word Embeddings, Word Similarity

We present an unsupervised method to generate Word2Sense word embeddings that are interpretable {---} each dimension of the embedding space corresponds to a fine-grained sense, and the non-negative value of the embedding along the j-th dimension represents the relevance of the j-th sense to the word. The underlying LDA-based generative model can be extended to refine the representation of a polysemous word in a short context, allowing us to use the embedings in contextual tasks. On computational NLP tasks, Word2Sense embeddings compare well with other word embeddings generated by unsupervised methods. Across tasks such as word similarity, entailment, sense induction, and contextual interpretation, Word2Sense is competitive with the state-of-the-art method for that task. Word2Sense embeddings are at least as sparse and fast to compute as prior art.
Predicting Humorousness and Metaphor Novelty with Gaussian Process Preference Learning
Authors: Edwin Simpson, Erik-L{\^a}n Do Dinh, Tristan Miller, Iryna Gurevych
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Natural Language Understanding, Word Embeddings

The inability to quantify key aspects of creative language is a frequent obstacle to natural language understanding. To address this, we introduce novel tasks for evaluating the creativeness of language{---}namely, scoring and ranking text by humorousness and metaphor novelty. To sidestep the difficulty of assigning discrete labels or numeric scores, we learn from pairwise comparisons between texts. We introduce a Bayesian approach for predicting humorousness and metaphor novelty using Gaussian process preference learning (GPPL), which achieves a Spearman{'}s Ï� of 0.56 against gold using word embeddings and linguistic features. Our experiments show that given sparse, crowdsourced annotation data, ranking using GPPL outperforms best{--}worst scaling. We release a new dataset for evaluating humour containing 28,210 pairwise comparisons of 4,030 texts, and make our software freely available.
Empirical Linguistic Study of Sentence Embeddings
Authors: Katarzyna Krasnowska-Kiera{\'s}, Alina Wr{\'o}blewska
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Sentence, Sentence Embeddings

The purpose of the research is to answer the question whether linguistic information is retained in vector representations of sentences. We introduce a method of analysing the content of sentence embeddings based on universal probing tasks, along with the classification datasets for two contrasting languages. We perform a series of probing and downstream experiments with different types of sentence embeddings, followed by a thorough analysis of the experimental results. Aside from dependency parser-based embeddings, linguistic information is retained best in the recently proposed LASER sentence embeddings.
Deep Neural Model Inspection and Comparison via Functional Neuron Pathways
Authors: James Fiacco, Samridhi Choudhary, Carolyn Rose
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Natural Language Inference

We introduce a general method for the interpretation and comparison of neural models. The method is used to factor a complex neural model into its functional components, which are comprised of sets of co-firing neurons that cut across layers of the network architecture, and which we call neural pathways. The function of these pathways can be understood by identifying correlated task level and linguistic heuristics in such a way that this knowledge acts as a lens for approximating what the network has learned to apply to its intended task. As a case study for investigating the utility of these pathways, we present an examination of pathways identified in models trained for two standard tasks, namely Named Entity Recognition and Recognizing Textual Entailment.
Collocation Classification with Unsupervised Relation Vectors
Authors: Luis Espinosa Anke, Steven Schockaert, Leo Wanner
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Classification, General Classification, Relation, Relation Classification, Word Embeddings

Lexical relation classification is the task of predicting whether a certain relation holds between a given pair of words. In this paper, we explore to which extent the current distributional landscape based on word embeddings provides a suitable basis for classification of collocations, i.e., pairs of words between which idiosyncratic lexical relations hold. First, we introduce a novel dataset with collocations categorized according to lexical functions. Second, we conduct experiments on a subset of this benchmark, comparing it in particular to the well known DiffVec dataset. In these experiments, in addition to simple word vector arithmetic operations, we also investigate the role of unsupervised relation vectors as a complementary input. While these relation vectors indeed help, we also show that lexical function classification poses a greater challenge than the syntactic and semantic relations that are typically used for benchmarks in the literature.
Corpus-based Check-up for Thesaurus
Authors: Natalia Loukachevitch
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7

In this paper we discuss the usefulness of applying a checking procedure to existing thesauri. The procedure is based on the analysis of discrepancies of corpus-based and thesaurus-based word similarities. We applied the procedure to more than 30 thousand words of the Russian wordnet and found some serious errors in word sense description, including inaccurate relationships and missing senses of ambiguous words.
Confusionset-guided Pointer Networks for Chinese Spelling Check
Authors: Dingmin Wang, Yi Tay, Li Zhong
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Sentence

This paper proposes Confusionset-guided Pointer Networks for Chinese Spell Check (CSC) task. More concretely, our approach utilizes the off-the-shelf confusionset for guiding the character generation. To this end, our novel Seq2Seq model jointly learns to copy a correct character from an input sentence through a pointer network, or generate a character from the confusionset rather than the entire vocabulary. We conduct experiments on three human-annotated datasets, and results demonstrate that our proposed generative model outperforms all competitor models by a large margin of up to 20{\%} F1 score, achieving state-of-the-art performance on three datasets.
Better OOV Translation with Bilingual Terminology Mining
Authors: Matthias Huck, Viktor Hangya, Alex Fraser, er
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Machine Translation, NMT, Translation, Word Embeddings

Unseen words, also called out-of-vocabulary words (OOVs), are difficult for machine translation. In neural machine translation, byte-pair encoding can be used to represent OOVs, but they are still often incorrectly translated. We improve the translation of OOVs in NMT using easy-to-obtain monolingual data. We look for OOVs in the text to be translated and translate them using simple-to-construct bilingual word embeddings (BWEs). In our MT experiments we take the 5-best candidates, which is motivated by intrinsic mining experiments. Using all five of the proposed target language words as queries we mine target-language sentences. We then back-translate, forcing the back-translation of each of the five proposed target-language OOV-translation-candidates to be the original source-language OOV. We show that by using this synthetic data to fine-tune our system the translation of OOVs can be dramatically improved. In our experiments we use a system trained on Europarl and mine sentences containing medical terms from monolingual data.