pwc-archive/papers-with-abstracts huggingface dataset:
A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation
Authors: Massimo Poesio, Jon Chamberlain, Silviu Paun, Juntao Yu, Alex Uma, ra, Udo Kruschwitz
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6

We present a corpus of anaphoric information (coreference) crowdsourced through a game-with-a-purpose. The corpus, containing annotations for about 108,000 markables, is one of the largest corpora for coreference for English, and one of the largest crowdsourced NLP corpora, but its main feature is the large number of judgments per markable: 20 on average, and over 2.2M in total. This characteristic makes the corpus a unique resource for the study of disagreements on anaphoric interpretation. A second distinctive feature is its rich annotation scheme, covering singletons, expletives, and split-antecedent plurals. Finally, the corpus also comes with labels inferred using a recently proposed probabilistic model of annotation for coreference. The labels are of high quality and make it possible to successfully train a state of the art coreference resolver, including training on singletons and non-referring expressions. The annotation model can also result in more than one label, or no label, being proposed for a markable, thus serving as a baseline method for automatically identifying ambiguous markables. A preliminary analysis of the results is presented.
A Streamlined Method for Sourcing Discourse-level Argumentation Annotations from the Crowd
Authors: Tristan Miller, Maria Sukhareva, Iryna Gurevych
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Argument Mining

The study of argumentation and the development of argument mining tools depends on the availability of annotated data, which is challenging to obtain in sufficient quantity and quality. We present a method that breaks down a popular but relatively complex discourse-level argument annotation scheme into a simpler, iterative procedure that can be applied even by untrained annotators. We apply this method in a crowdsourcing setup and report on the reliability of the annotations obtained. The source code for a tool implementing our annotation method, as well as the sample data we obtained (4909 gold-standard annotations across 982 documents), are freely released to the research community. These are intended to serve the needs of qualitative research into argumentation, as well as of data-driven approaches to argument mining.
Modeling Document-level Causal Structures for Event Causal Relation Identification
Authors: Lei Gao, Prafulla Kumar Choubey, Ruihong Huang
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Relation, Sentence

We aim to comprehensively identify all the event causal relations in a document, both within a sentence and across sentences, which is important for reconstructing pivotal event structures. The challenges we identified are two: 1) event causal relations are sparse among all possible event pairs in a document, in addition, 2) few causal relations are explicitly stated. Both challenges are especially true for identifying causal relations between events across sentences. To address these challenges, we model rich aspects of document-level causal structures for achieving comprehensive causal relation identification. The causal structures include heavy involvements of document-level main events in causal relations as well as several types of fine-grained constraints that capture implications from certain sentential syntactic relations and discourse relations as well as interactions between event causal relations and event coreference relations. Our experimental results show that modeling the global and fine-grained aspects of causal structures using Integer Linear Programming (ILP) greatly improves the performance of causal relation identification, especially in identifying cross-sentence causal relations.
Hierarchical User and Item Representation with Three-Tier Attention for Recommendation
Authors: Chuhan Wu, Fangzhao Wu, Junxin Liu, Yongfeng Huang
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Informativeness, Recommendation Systems, Sentence

Utilizing reviews to learn user and item representations is useful for recommender systems. Existing methods usually merge all reviews from the same user or for the same item into a long document. However, different reviews, sentences and even words usually have different informativeness for modeling users and items. In this paper, we propose a hierarchical user and item representation model with three-tier attention to learn user and item representations from reviews for recommendation. Our model contains three major components, i.e., a sentence encoder to learn sentence representations from words, a review encoder to learn review representations from sentences, and a user/item encoder to learn user/item representations from reviews. In addition, we incorporate a three-tier attention network in our model to select important words, sentences and reviews. Besides, we combine the user and item representations learned from the reviews with user and item embeddings based on IDs as the final representations to capture the latent factors of individual users and items. Extensive experiments on four benchmark datasets validate the effectiveness of our approach.
Text Similarity Estimation Based on Word Embeddings and Matrix Norms for Targeted Marketing
Authors: Tim vor der Br{\"u}ck, Marc Pouly
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Marketing, text similarity, Word Embeddings

The prevalent way to estimate the similarity of two documents based on word embeddings is to apply the cosine similarity measure to the two centroids obtained from the embedding vectors associated with the words in each document. Motivated by an industrial application from the domain of youth marketing, where this approach produced only mediocre results, we propose an alternative way of combining the word vectors using matrix norms. The evaluation shows superior results for most of the investigated matrix norms in comparison to both the classical cosine measure and several other document similarity estimates.
Glocal: Incorporating Global Information in Local Convolution for Keyphrase Extraction
Authors: Animesh Prasad, Min-Yen Kan
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Clustering, Keyphrase Extraction

Graph Convolutional Networks (GCNs) are a class of spectral clustering techniques that leverage localized convolution filters to perform supervised classification directly on graphical structures. While such methods model nodes{'} local pairwise importance, they lack the capability to model global importance relative to other nodes of the graph. This causes such models to miss critical information in tasks where global ranking is a key component for the task, such as in keyphrase extraction. We address this shortcoming by allowing the proper incorporation of global information into the GCN family of models through the use of scaled node weights. In the context of keyphrase extraction, incorporating global random walk scores obtained from TextRank boosts performance significantly. With our proposed method, we achieve state-of-the-art results, bettering a strong baseline by an absolute 2{\%} increase in F1 score.
A Study of Latent Structured Prediction Approaches to Passage Reranking
Authors: Iryna Haponchyk, Aless Moschitti, ro
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Learning-To-Rank, Passage Reranking, Reranking, Structured Prediction

The structured output framework provides a helpful tool for learning to rank problems. In this paper, we propose a structured output approach which regards rankings as latent variables. Our approach addresses the complex optimization of Mean Average Precision (MAP) ranking metric. We provide an inference procedure to find the max-violating ranking based on the decomposition of the corresponding loss. The results of our experiments on WikiQA and TREC13 datasets show that our reranking based on structured prediction is a promising research direction.
Tweet Stance Detection Using an Attention based Neural Ensemble Model
Authors: Umme Aymun Siddiqua, Abu Nowshed Chy, Masaki Aono
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Stance Detection

Stance detection in twitter aims at mining user stances expressed in a tweet towards a single or multiple target entities. To tackle this problem, most of the prior studies have been explored the traditional deep learning models, e.g., LSTM and GRU. However, in compared to these traditional approaches, recently proposed densely connected Bi-LSTM and nested LSTMs architectures effectively address the vanishing-gradient and overfitting problems as well as dealing with long-term dependencies. In this paper, we propose a neural ensemble model that adopts the strengths of these two LSTM variants to learn better long-term dependencies, where each module coupled with an attention mechanism that amplifies the contribution of important elements in the final representation. We also employ a multi-kernel convolution on top of them to extract the higher-level tweet representations. Results of extensive experiments on single and multi-target stance detection datasets show that our proposed method achieves substantial improvement over the current state-of-the-art deep learning based methods.
Word Embedding-Based Automatic MT Evaluation Metric using Word Position Information
Authors: Hiroshi Echizen{'}ya, Kenji Araki, Eduard Hovy
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Machine Translation, Position, Sentence, Translation, Word Embeddings

We propose a new automatic evaluation metric for machine translation. Our proposed metric is obtained by adjusting the Earth Mover{'}s Distance (EMD) to the evaluation task. The EMD measure is used to obtain the distance between two probability distributions consisting of some signatures having a feature and a weight. We use word embeddings, sentence-level tf-idf, and cosine similarity between two word embeddings, respectively, as the features, weight, and the distance between two features. Results show that our proposed metric can evaluate machine translation based on word meaning. Moreover, for distance, cosine similarity and word position information are used to address word-order differences. We designate this metric as Word Embedding-Based automatic MT evaluation using Word Position Information (WE{\_}WPI). A meta-evaluation using WMT16 metrics shared task set indicates that our WE{\_}WPI achieves the highest correlation with human judgment among several representative metrics.
Learning Unsupervised Multilingual Word Embeddings with Incremental Multilingual Hubs
Authors: Geert Heyman, Bregt Verreet, Ivan Vuli{\'c}, Marie-Francine Moens
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Bilingual Lexicon Induction, Cross-Lingual Word Embeddings, Dependency Parsing, Document Classification, Multilingual Word Embeddings, Self-Learning, Word Embeddings

Recent research has discovered that a shared bilingual word embedding space can be induced by projecting monolingual word embedding spaces from two languages using a self-learning paradigm without any bilingual supervision. However, it has also been shown that for distant language pairs such fully unsupervised self-learning methods are unstable and often get stuck in poor local optima due to reduced isomorphism between starting monolingual spaces. In this work, we propose a new robust framework for learning unsupervised multilingual word embeddings that mitigates the instability issues. We learn a shared multilingual embedding space for a variable number of languages by incrementally adding new languages one by one to the current multilingual space. Through the gradual language addition the method can leverage the interdependencies between the new language and all other languages in the current multilingual space. We find that it is beneficial to project more distant languages later in the iterative process. Our fully unsupervised multilingual embedding spaces yield results that are on par with the state-of-the-art methods in the bilingual lexicon induction (BLI) task, and simultaneously obtain state-of-the-art scores on two downstream tasks: multilingual document classification and multilingual dependency parsing, outperforming even supervised baselines. This finding also accentuates the need to establish evaluation protocols for cross-lingual word embeddings beyond the omnipresent intrinsic BLI task in future work.
Online Distilling from Checkpoints for Neural Machine Translation
Authors: Hao-Ran Wei, Shu-Jian Huang, Ran Wang, Xin-yu Dai, Jia-Jun Chen
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Knowledge Distillation, Machine Translation, NMT, Reading Comprehension, Translation

Current predominant neural machine translation (NMT) models often have a deep structure with large amounts of parameters, making these models hard to train and easily suffering from over-fitting. A common practice is to utilize a validation set to evaluate the training process and select the best checkpoint. Average and ensemble techniques on checkpoints can lead to further performance improvement. However, as these methods do not affect the training process, the system performance is restricted to the checkpoints generated in original training procedure. In contrast, we propose an online knowledge distillation method. Our method on-the-fly generates a teacher model from checkpoints, guiding the training process to obtain better performance. Experiments on several datasets and language pairs show steady improvement over a strong self-attention-based baseline system. We also provide analysis on data-limited setting against over-fitting. Furthermore, our method leads to an improvement in a machine reading experiment as well.
Shifting the Baseline: Single Modality Performance on Visual Navigation \& QA
Authors: Jesse Thomason, Daniel Gordon, Yonatan Bisk
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Visual Navigation

We demonstrate the surprising strength of unimodal baselines in multimodal domains, and make concrete recommendations for best practices in future research. Where existing work often compares against random or majority class baselines, we argue that unimodal approaches better capture and reflect dataset biases and therefore provide an important comparison when assessing the performance of multimodal techniques. We present unimodal ablations on three recent datasets in visual navigation and QA, seeing an up to 29{\%} absolute gain in performance over published baselines.
MuST-C: a Multilingual Speech Translation Corpus
Authors: Mattia A. Di Gangi, Roldano Cattoni, Luisa Bentivogli, Matteo Negri, Marco Turchi
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Machine Translation, Sentence, speech-recognition, Speech Recognition, Translation

Current research on spoken language translation (SLT) has to confront with the scarcity of sizeable and publicly available training corpora. This problem hinders the adoption of neural end-to-end approaches, which represent the state of the art in the two parent tasks of SLT: automatic speech recognition and machine translation. To fill this gap, we created MuST-C, a multilingual speech translation corpus whose size and quality will facilitate the training of end-to-end systems for SLT from English into 8 languages. For each target language, MuST-C comprises at least 385 hours of audio recordings from English TED Talks, which are automatically aligned at the sentence level with their manual transcriptions and translations. Together with a description of the corpus creation methodology (scalable to add new data and cover new languages), we provide an empirical verification of its quality and SLT results computed with a state-of-the-art approach on each language direction.
A Robust Abstractive System for Cross-Lingual Summarization
Authors: Jessica Ouyang, Boya Song, Kathy Mckeown
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Abstractive Text Summarization, Machine Translation, Translation

We present a robust neural abstractive summarization system for cross-lingual summarization. We construct summarization corpora for documents automatically translated from three low-resource languages, Somali, Swahili, and Tagalog, using machine translation and the New York Times summarization corpus. We train three language-specific abstractive summarizers and evaluate on documents originally written in the source languages, as well as on a fourth, unseen language: Arabic. Our systems achieve significantly higher fluency than a standard copy-attention summarizer on automatically translated input documents, as well as comparable content selection.
Improving Neural Machine Translation with Neural Syntactic Distance
Authors: Chunpeng Ma, Akihiro Tamura, Masao Utiyama, Eiichiro Sumita, Tiejun Zhao
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Machine Translation, NMT, Sentence, Translation

The explicit use of syntactic information has been proved useful for neural machine translation (NMT). However, previous methods resort to either tree-structured neural networks or long linearized sequences, both of which are inefficient. Neural syntactic distance (NSD) enables us to represent a constituent tree using a sequence whose length is identical to the number of words in the sentence. NSD has been used for constituent parsing, but not in machine translation. We propose five strategies to improve NMT with NSD. Experiments show that it is not trivial to improve NMT with NSD; however, the proposed strategies are shown to improve translation performance of the baseline model (+2.1 (En{--}Ja), +1.3 (Ja{--}En), +1.2 (En{--}Ch), and +1.0 (Ch{--}En) BLEU).
Measuring Immediate Adaptation Performance for Neural Machine Translation
Authors: Patrick Simianer, Joern Wuebker, John DeNero
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Domain Adaptation, Machine Translation, One-Shot Learning, Translation

Incremental domain adaptation, in which a system learns from the correct output for each input immediately after making its prediction for that input, can dramatically improve system performance for interactive machine translation. Users of interactive systems are sensitive to the speed of adaptation and how often a system repeats mistakes, despite being corrected. Adaptation is most commonly assessed using corpus-level BLEU- or TER-derived metrics that do not explicitly take adaptation speed into account. We find that these metrics often do not capture immediate adaptation effects, such as zero-shot and one-shot learning of domain-specific lexical items. To this end, we propose new metrics that directly evaluate immediate adaptation performance for machine translation. We use these metrics to choose the most suitable adaptation method from a range of different adaptation techniques for neural machine translation systems.
Overcoming Catastrophic Forgetting During Domain Adaptation of Neural Machine Translation
Authors: Brian Thompson, Jeremy Gwinnup, Huda Khayrallah, Kevin Duh, Philipp Koehn
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: BIG-bench Machine Learning, Domain Adaptation, Machine Translation, Translation

Continued training is an effective method for domain adaptation in neural machine translation. However, in-domain gains from adaptation come at the expense of general-domain performance. In this work, we interpret the drop in general-domain performance as catastrophic forgetting of general-domain knowledge. To mitigate it, we adapt Elastic Weight Consolidation (EWC){---}a machine learning method for learning a new task without forgetting previous tasks. Our method retains the majority of general-domain performance lost in continued training without degrading in-domain performance, outperforming the previous state-of-the-art. We also explore the full range of general-domain performance available when some in-domain degradation is acceptable.
Incorporating Emoji Descriptions Improves Tweet Classification
Authors: Abhishek Singh, Eduardo Blanco, Wei Jin
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Classification, General Classification, Sentiment Analysis, Word Embeddings

Tweets are short messages that often include specialized language such as hashtags and emojis. In this paper, we present a simple strategy to process emojis: replace them with their natural language description and use pretrained word embeddings as normally done with standard words. We show that this strategy is more effective than using pretrained emoji embeddings for tweet classification. Specifically, we obtain new state-of-the-art results in irony detection and sentiment analysis despite our neural network is simpler than previous proposals.
Modeling Personal Biases in Language Use by Inducing Personalized Word Embeddings
Authors: Daisuke Oba, Naoki Yoshinaga, Shoetsu Sato, Satoshi Akasaki, Masashi Toyoda
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Multi-class Classification, Multi-Task Learning, Sentiment Analysis, Word Embeddings

There exist biases in individual{'}s language use; the same word (e.g., cool) is used for expressing different meanings (e.g., temperature range) or different words (e.g., cloudy, hazy) are used for describing the same meaning. In this study, we propose a method of modeling such personal biases in word meanings (hereafter, semantic variations) with personalized word embeddings obtained by solving a task on subjective text while regarding words used by different individuals as different words. To prevent personalized word embeddings from being contaminated by other irrelevant biases, we solve a task of identifying a review-target (objective output) from a given review. To stabilize the training of this extreme multi-class classification, we perform a multi-task learning with metadata identification. Experimental results with reviews retrieved from RateBeer confirmed that the obtained personalized word embeddings improved the accuracy of sentiment analysis as well as the target task. Analysis of the obtained personalized word embeddings revealed trends in semantic variations related to frequent and adjective words.
An annotated dataset of literary entities
Authors: David Bamman, Sejal Popat, Sheng Shen
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6

We present a new dataset comprised of 210,532 tokens evenly drawn from 100 different English-language literary texts annotated for ACE entity categories (person, location, geo-political entity, facility, organization, and vehicle). These categories include non-named entities (such as {``}the boy{''}, {``}the kitchen{''}) and nested structure (such as [[the cook]{'}s sister]). In contrast to existing datasets built primarily on news (focused on geo-political entities and organizations), literary texts offer strikingly different distributions of entity categories, with much stronger emphasis on people and description of settings. We present empirical results demonstrating the performance of nested entity recognition models in this domain; training natively on in-domain literary data yields an improvement of over 20 absolute points in F-score (from 45.7 to 68.3), and mitigates a disparate impact in performance for male and female entities present in models trained on news data.
On the Importance of Distinguishing Word Meaning Representations: A Case Study on Reverse Dictionary Mapping
Authors: Mohammad Taher Pilehvar
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Reverse Dictionary

Meaning conflation deficiency is one of the main limiting factors of word representations which, given their widespread use at the core of many NLP systems, can lead to inaccurate semantic understanding of the input text and inevitably hamper the performance. Sense representations target this problem. However, their potential impact has rarely been investigated in downstream NLP applications. Through a set of experiments on a state-of-the-art reverse dictionary system based on neural networks, we show that a simple adjustment aimed at addressing the meaning conflation deficiency can lead to substantial improvements.