pwc-archive/papers-with-abstracts huggingface dataset:
A Semi-Markov Structured Support Vector Machine Model for High-Precision Named Entity Recognition
Authors: Ravneet Arora, Chen-Tse Tsai, Ketevan Tsereteli, Prabhanjan Kambadur, Yi Yang
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Named entity recognition (NER) is the backbone of many NLP solutions. F1 score, the harmonic mean of precision and recall, is often used to select/evaluate the best models. However, when precision needs to be prioritized over recall, a state-of-the-art model might not be the best choice. There is little in literature that directly addresses training-time modifications to achieve higher precision information extraction. In this paper, we propose a neural semi-Markov structured support vector machine model that controls the precision-recall trade-off by assigning weights to different types of errors in the loss-augmented inference during training. The semi-Markov property provides more accurate phrase-level predictions, thereby improving performance. We empirically demonstrate the advantage of our model when high precision is required by comparing against strong baselines based on CRF. In our experiments with the CoNLL 2003 dataset, our model achieves a better precision-recall trade-off at various precision levels.
Using Human Attention to Extract Keyphrase from Microblog Post
Authors: Yingyi Zhang, Chengzhi Zhang
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Keyphrase Extraction

This paper studies automatic keyphrase extraction on social media. Previous works have achieved promising results on it, but they neglect human reading behavior during keyphrase annotating. The human attention is a crucial element of human reading behavior. It reveals the relevance of words to the main topics of the target text. Thus, this paper aims to integrate human attention into keyphrase extraction models. First, human attention is represented by the reading duration estimated from eye-tracking corpus. Then, we merge human attention with neural network models by an attention mechanism. In addition, we also integrate human attention into unsupervised models. To the best of our knowledge, we are the first to utilize human attention on keyphrase extraction tasks. The experimental results show that our models have significant improvements on two Twitter datasets.
Model-Agnostic Meta-Learning for Relation Classification with Limited Supervision
Authors: Abiola Obamuyide, Andreas Vlachos
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Classification, General Classification, Meta-Learning, Relation, Relation Classification

In this paper we frame the task of supervised relation classification as an instance of meta-learning. We propose a model-agnostic meta-learning protocol for training relation classifiers to achieve enhanced predictive performance in limited supervision settings. During training, we aim to not only learn good parameters for classifying relations with sufficient supervision, but also learn model parameters that can be fine-tuned to enhance predictive performance for relations with limited supervision. In experiments conducted on two relation classification datasets, we demonstrate that the proposed meta-learning approach improves the predictive performance of two state-of-the-art supervised relation classification models.
Task Refinement Learning for Improved Accuracy and Stability of Unsupervised Domain Adaptation
Authors: Yftah Ziser, Roi Reichart
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Domain Adaptation, Language Modeling, Language Modelling, Sentiment Analysis, Sentiment Classification, Unsupervised Domain Adaptation

Pivot Based Language Modeling (PBLM) (Ziser and Reichart, 2018a), combining LSTMs with pivot-based methods, has yielded significant progress in unsupervised domain adaptation. However, this approach is still challenged by the large pivot detection problem that should be solved, and by the inherent instability of LSTMs. In this paper we propose a Task Refinement Learning (TRL) approach, in order to solve these problems. Our algorithms iteratively train the PBLM model, gradually increasing the information exposed about each pivot. TRL-PBLM achieves stateof- the-art accuracy in six domain adaptation setups for sentiment classification. Moreover, it is much more stable than plain PBLM across model configurations, making the model much better fitted for practical use.
Interpolated Spectral NGram Language Models
Authors: Ariadna Quattoni, Xavier Carreras
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Language Modeling, Language Modelling

Spectral models for learning weighted non-deterministic automata have nice theoretical and algorithmic properties. Despite this, it has been challenging to obtain competitive results in language modeling tasks, for two main reasons. First, in order to capture long-range dependencies of the data, the method must use statistics from long substrings, which results in very large matrices that are difficult to decompose. The second is that the loss function behind spectral learning, based on moment matching, differs from the probabilistic metrics used to evaluate language models. In this work we employ a technique for scaling up spectral learning, and use interpolated predictions that are optimized to maximize perplexity. Our experiments in character-based language modeling show that our method matches the performance of state-of-the-art ngram models, while being very fast to train.
Barack's Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling
Authors: Robert Logan, Nelson F. Liu, Matthew E. Peters, Matt Gardner, Sameer Singh
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Knowledge Graphs, Language Modeling, Language Modelling

Modeling human language requires the ability to not only generate fluent text but also encode factual knowledge. However, traditional language models are only capable of remembering facts seen at training time, and often have difficulty recalling them. To address this, we introduce the knowledge graph language model (KGLM), a neural language model with mechanisms for selecting and copying facts from a knowledge graph that are relevant to the context. These mechanisms enable the model to render information it has never seen before, as well as generate out-of-vocabulary tokens. We also introduce the Linked WikiText-2 dataset, a corpus of annotated text aligned to the Wikidata knowledge graph whose contents (roughly) match the popular WikiText-2 benchmark. In experiments, we demonstrate that the KGLM achieves significantly better performance than a strong baseline language model. We additionally compare different language model{'}s ability to complete sentences requiring factual knowledge, showing that the KGLM outperforms even very large language models in generating facts.
Towards Comprehensive Description Generation from Factual Attribute-value Tables
Authors: Tianyu Liu, Fuli Luo, Pengcheng Yang, Wei Wu, Baobao Chang, Zhifang Sui
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Attribute, Reinforcement Learning

The comprehensive descriptions for factual attribute-value tables, which should be accurate, informative and loyal, can be very helpful for end users to understand the structured data in this form. However previous neural generators might suffer from key attributes missing, less informative and groundless information problems, which impede the generation of high-quality comprehensive descriptions for tables. To relieve these problems, we first propose force attention (FA) method to encourage the generator to pay more attention to the uncovered attributes to avoid potential key attributes missing. Furthermore, we propose reinforcement learning for information richness to generate more informative as well as more loyal descriptions for tables. In our experiments, we utilize the widely used WIKIBIO dataset as a benchmark. Besides, we create WB-filter based on WIKIBIO to test our model in the simulated user-oriented scenarios, in which the generated descriptions should accord with particular user interests. Experimental results show that our model outperforms the state-of-the-art baselines on both automatic and human evaluation.
Learning to Control the Fine-grained Sentiment for Story Ending Generation
Authors: Fuli Luo, Damai Dai, Pengcheng Yang, Tianyu Liu, Baobao Chang, Zhifang Sui, Xu sun
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Decoder, Text Generation

Automatic story ending generation is an interesting and challenging task in natural language generation. Previous studies are mainly limited to generate coherent, reasonable and diversified story endings, and few works focus on controlling the sentiment of story endings. This paper focuses on generating a story ending which meets the given fine-grained sentiment intensity. There are two major challenges to this task. First is the lack of story corpus which has fine-grained sentiment labels. Second is the difficulty of explicitly controlling sentiment intensity when generating endings. Therefore, we propose a generic and novel framework which consists of a sentiment analyzer and a sentimental generator, respectively addressing the two challenges. The sentiment analyzer adopts a series of methods to acquire sentiment intensities of the story dataset. The sentimental generator introduces the sentiment intensity into decoder via a Gaussian Kernel Layer to control the sentiment of the output. To the best of our knowledge, this is the first endeavor to control the fine-grained sentiment for story ending generation without manually annotating sentiment labels. Experiments show that our proposed framework can generate story endings which are not only more coherent and fluent but also able to meet the given sentiment intensity better.
Self-Attention Architectures for Answer-Agnostic Neural Question Generation
Authors: Thomas Scialom, Benjamin Piwowarski, Jacopo Staiano
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Question Generation, Question-Generation, Word Embeddings

Neural architectures based on self-attention, such as Transformers, recently attracted interest from the research community, and obtained significant improvements over the state of the art in several tasks. We explore how Transformers can be adapted to the task of Neural Question Generation without constraining the model to focus on a specific answer passage. We study the effect of several strategies to deal with out-of-vocabulary words such as copy mechanisms, placeholders, and contextual word embeddings. We report improvements obtained over the state-of-the-art on the SQuAD dataset according to automated metrics (BLEU, ROUGE), as well as qualitative human assessments of the system outputs.
Storyboarding of Recipes: Grounded Contextual Generation
Authors: Ch, Khyathi u, Eric Nyberg, Alan W. black
Date: 2019-07-01 00:00:00
Proceeding: ICLR Workshop DeepGenStruct 2019
Tasks: Decoder, Text Generation

Information need of humans is essentially multimodal in nature, enabling maximum exploitation of situated context. We introduce a dataset for sequential procedural (how-to) text generation from images in cooking domain. The dataset consists of 16,441 cooking recipes with 160,479 photos associated with different steps. We setup a baseline motivated by the best performing model in terms of human evaluation for the Visual Story Telling (ViST) task. In addition, we introduce two models to incorporate high level structure learnt by a Finite State Machine (FSM) in neural sequential generation process by: (1) Scaffolding Structure in Decoder (SSiD) (2) Scaffolding Structure in Loss (SSiL). Our best performing model (SSiL) achieves a METEOR score of 0.31, which is an improvement of 0.6 over the baseline model. We also conducted human evaluation of the generated grounded recipes, which reveal that 61{\%} found that our proposed (SSiL) model is better than the baseline model in terms of overall recipes. We also discuss analysis of the output highlighting key important NLP issues for prospective directions.
Negative Lexically Constrained Decoding for Paraphrase Generation
Authors: Tomoyuki Kajiwara
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Machine Translation, Paraphrase Generation, Sentence, Text Simplification, Translation

Paraphrase generation can be regarded as monolingual translation. Unlike bilingual machine translation, paraphrase generation rewrites only a limited portion of an input sentence. Hence, previous methods based on machine translation often perform conservatively to fail to make necessary rewrites. To solve this problem, we propose a neural model for paraphrase generation that first identifies words in the source sentence that should be paraphrased. Then, these words are paraphrased by the negative lexically constrained decoding that avoids outputting these words as they are. Experiments on text simplification and formality transfer show that our model improves the quality of paraphrasing by making necessary rewrites to an input sentence.
Large-Scale Transfer Learning for Natural Language Generation
Authors: Sergey Golovanov, Rauf Kurbanov, Sergey Nikolenko, Kyryl Truskovskyi, Alex Tselousov, er, Thomas Wolf
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Open-Domain Dialog, Text Generation, Transfer Learning

Large-scale pretrained language models define state of the art in natural language processing, achieving outstanding performance on a variety of tasks. We study how these architectures can be applied and adapted for natural language generation, comparing a number of architectural and training schemes. We focus in particular on open-domain dialog as a typical high entropy generation task, presenting and comparing different architectures for adapting pretrained models with state of the art results.
Automatic Grammatical Error Correction for Sequence-to-sequence Text Generation: An Empirical Study
Authors: Tao Ge, Xingxing Zhang, Furu Wei, Ming Zhou
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Formality Style Transfer, Grammatical Error Correction, Machine Translation, Sentence, Sentence Compression, Style Transfer, Text Generation, Translation

Sequence-to-sequence (seq2seq) models have achieved tremendous success in text generation tasks. However, there is no guarantee that they can always generate sentences without grammatical errors. In this paper, we present a preliminary empirical study on whether and how much automatic grammatical error correction can help improve seq2seq text generation. We conduct experiments across various seq2seq text generation tasks including machine translation, formality style transfer, sentence compression and simplification. Experiments show the state-of-the-art grammatical error correction system can improve the grammaticality of generated text and can bring task-oriented improvements in the tasks where target sentences are in a formal style.
Improving the Robustness of Question Answering Systems to Question Paraphrasing
Authors: Wee Chung Gan, Hwee Tou Ng
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Data Augmentation, Question Answering

Despite the advancement of question answering (QA) systems and rapid improvements on held-out test sets, their generalizability is a topic of concern. We explore the robustness of QA models to question paraphrasing by creating two test sets consisting of paraphrased SQuAD questions. Paraphrased questions from the first test set are very similar to the original questions designed to test QA models{'} over-sensitivity, while questions from the second test set are paraphrased using context words near an incorrect answer candidate in an attempt to confuse QA models. We show that both paraphrased test sets lead to significant decrease in performance on multiple state-of-the-art QA models. Using a neural paraphrasing model trained to generate multiple paraphrased questions for a given source question and a set of paraphrase suggestions, we propose a data augmentation approach that requires no human intervention to re-train the models for improved robustness to question paraphrasing.
Combining Knowledge Hunting and Neural Language Models to Solve the Winograd Schema Challenge
Authors: Ashok Prakash, Arpit Sharma, Arindam Mitra, Chitta Baral
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Language Modeling, Language Modelling

Winograd Schema Challenge (WSC) is a pronoun resolution task which seems to require reasoning with commonsense knowledge. The needed knowledge is not present in the given text. Automatic extraction of the needed knowledge is a bottleneck in solving the challenge. The existing state-of-the-art approach uses the knowledge embedded in their pre-trained language model. However, the language models only embed part of the knowledge, the ones related to frequently co-existing concepts. This limits the performance of such models on the WSC problems. In this work, we build-up on the language model based methods and augment them with a commonsense knowledge hunting (using automatic extraction from text) module and an explicit reasoning module. Our end-to-end system built in such a manner improves on the accuracy of two of the available language model based approaches by 5.53{\%} and 7.7{\%} respectively. Overall our system achieves the state-of-the-art accuracy of 71.06{\%} on the WSC dataset, an improvement of 7.36{\%} over the previous best.
Modeling Intra-Relation in Math Word Problems with Different Functional Multi-Head Attentions
Authors: Jierui Li, Lei Wang, Jipeng Zhang, Yan Wang, Bing Tian Dai, Dongxiang Zhang
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Deep Learning, Math, Math Word Problem Solving, Relation

Several deep learning models have been proposed for solving math word problems (MWPs) automatically. Although these models have the ability to capture features without manual efforts, their approaches to capturing features are not specifically designed for MWPs. To utilize the merits of deep learning models with simultaneous consideration of MWPs{'} specific features, we propose a group attention mechanism to extract global features, quantity-related features, quantity-pair features and question-related features in MWPs respectively. The experimental results show that the proposed approach performs significantly better than previous state-of-the-art methods, and boost performance from 66.9{\%} to 69.5{\%} on Math23K with training-test split, from 65.8{\%} to 66.9{\%} on Math23K with 5-fold cross-validation and from 69.2{\%} to 76.1{\%} on MAWPS.
Are Red Roses Red? Evaluating Consistency of Question-Answering Models
Authors: Marco Tulio Ribeiro, Carlos Guestrin, Sameer Singh
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Question Answering, valid, Visual Question Answering (VQA)

Although current evaluation of question-answering systems treats predictions in isolation, we need to consider the relationship between predictions to measure true understanding. A model should be penalized for answering {``}no{''} to {``}Is the rose red?{''} if it answers {``}red{''} to {``}What color is the rose?{''}. We propose a method to automatically extract such implications for instances from two QA datasets, VQA and SQuAD, which we then use to evaluate the consistency of models. Human evaluation shows these generated implications are well formed and valid. Consistency evaluation provides crucial insights into gaps in existing models, while retraining with implication-augmented data improves consistency on both synthetic and human-generated implications.
MC\^2: Multi-perspective Convolutional Cube for Conversational Machine Reading Comprehension
Authors: Xuanyu Zhang
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Conversational Question Answering, Machine Reading Comprehension, Question Answering, Reading Comprehension

Conversational machine reading comprehension (CMRC) extends traditional single-turn machine reading comprehension (MRC) by multi-turn interactions, which requires machines to consider the history of conversation. Most of models simply combine previous questions for conversation understanding and only employ recurrent neural networks (RNN) for reasoning. To comprehend context profoundly and efficiently from different perspectives, we propose a novel neural network model, Multi-perspective Convolutional Cube (MC{\^{}}2). We regard each conversation as a cube. 1D and 2D convolutions are integrated with RNN in our model. To avoid models previewing the next turn of conversation, we also extend causal convolution partially to 2D. Experiments on the Conversational Question Answering (CoQA) dataset show that our model achieves state-of-the-art results.
Reducing Word Omission Errors in Neural Machine Translation: A Contrastive Learning Approach
Authors: Zonghan Yang, Yong Cheng, Yang Liu, Maosong Sun
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Contrastive Learning, Machine Translation, NMT, Translation

While neural machine translation (NMT) has achieved remarkable success, NMT systems are prone to make word omission errors. In this work, we propose a contrastive learning approach to reducing word omission errors in NMT. The basic idea is to enable the NMT model to assign a higher probability to a ground-truth translation and a lower probability to an erroneous translation, which is automatically constructed from the ground-truth translation by omitting words. We design different types of negative examples depending on the number of omitted words, word frequency, and part of speech. Experiments on Chinese-to-English, German-to-English, and Russian-to-English translation tasks show that our approach is effective in reducing word omission errors and achieves better translation performance than three baseline methods.
Wetin dey with these comments? Modeling Sociolinguistic Factors Affecting Code-switching Behavior in Nigerian Online Discussions
Authors: Innocent Ndubuisi-Obi, Sayan Ghosh, David Jurgens
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Articles

Multilingual individuals code switch between languages as a part of a complex communication process. However, most computational studies have examined only one or a handful of contextual factors predictive of switching. Here, we examine Naija-English code switching in a rich contextual environment to understand the social and topical factors eliciting a switch. We introduce a new corpus of 330K articles and accompanying 389K comments labeled for code switching behavior. In modeling whether a comment will switch, we show that topic-driven variation, tribal affiliation, emotional valence, and audience design all play complementary roles in behavior.
Accelerating Sparse Matrix Operations in Neural Networks on Graphics Processing Units
Authors: Arturo Argueta, David Chiang
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: GPU

Graphics Processing Units (GPUs) are commonly used to train and evaluate neural networks efficiently. While previous work in deep learning has focused on accelerating operations on dense matrices/tensors on GPUs, efforts have concentrated on operations involving sparse data structures. Operations using sparse structures are common in natural language models at the input and output layers, because these models operate on sequences over discrete alphabets. We present two new GPU algorithms: one at the input layer, for multiplying a matrix by a few-hot vector (generalizing the more common operation of multiplication by a one-hot vector) and one at the output layer, for a fused softmax and top-N selection (commonly used in beam search). Our methods achieve speedups over state-of-the-art parallel GPU baselines of up to 7x and 50x, respectively. We also illustrate how our methods scale on different GPU architectures.