pwc-archive/papers-with-abstracts huggingface dataset:
GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction
Authors: Tsu-Jui Fu, Peng-Hsuan Li, Wei-Yun Ma
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Joint Entity and Relation Extraction, Relation, Relation Extraction

In this paper, we present GraphRel, an end-to-end relation extraction model which uses graph convolutional networks (GCNs) to jointly learn named entities and relations. In contrast to previous baselines, we consider the interaction between named entities and relations via a 2nd-phase relation-weighted GCN to better extract relations. Linear and dependency structures are both used to extract both sequential and regional features of the text, and a complete word graph is further utilized to extract implicit features among all word pairs of the text. With the graph-based approach, the prediction for overlapping relations is substantially improved over previous sequential approaches. We evaluate GraphRel on two public datasets: NYT and WebNLG. Results show that GraphRel maintains high precision while increasing recall substantially. Also, GraphRel outperforms previous work by 3.2{\%} and 5.8{\%} (F1 score), achieving a new state-of-the-art for relation extraction.
A Neural Multi-digraph Model for Chinese NER with Gazetteers
Authors: Ruixue Ding, Pengjun Xie, Xiaoyan Zhang, Wei Lu, Linlin Li, Luo Si
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Gazetteers were shown to be useful resources for named entity recognition (NER). Many existing approaches to incorporating gazetteers into machine learning based NER systems rely on manually defined selection strategies or handcrafted templates, which may not always lead to optimal effectiveness, especially when multiple gazetteers are involved. This is especially the case for the task of Chinese NER, where the words are not naturally tokenized, leading to additional ambiguities. To automatically learn how to incorporate multiple gazetteers into an NER system, we propose a novel approach based on graph neural networks with a multi-digraph structure that captures the information that the gazetteers offer. Experiments on various datasets show that our model is effective in incorporating rich gazetteer information while resolving ambiguities, outperforming previous approaches.
Training Hybrid Language Models by Marginalizing over Segmentations
Authors: Edouard Grave, Sainbayar Sukhbaatar, Piotr Bojanowski, Arm Joulin
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Language Modeling, Language Modelling

In this paper, we study the problem of hybrid language modeling, that is using models which can predict both characters and larger units such as character ngrams or words. Using such models, multiple potential segmentations usually exist for a given string, for example one using words and one using characters only. Thus, the probability of a string is the sum of the probabilities of all the possible segmentations. Here, we show how it is possible to marginalize over the segmentations efficiently, in order to compute the true probability of a sequence. We apply our technique on three datasets, comprising seven languages, showing improvements over a strong character level language model.
On the Robustness of Self-Attentive Models
Authors: Yu-Lun Hsieh, Minhao Cheng, Da-Cheng Juan, Wei Wei, Wen-Lian Hsu, Cho-Jui Hsieh
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Machine Translation, Sentiment Analysis, Translation

This work examines the robustness of self-attentive neural networks against adversarial input perturbations. Specifically, we investigate the attention and feature extraction mechanisms of state-of-the-art recurrent neural networks and self-attentive architectures for sentiment analysis, entailment and machine translation under adversarial attacks. We also propose a novel attack algorithm for generating more natural adversarial examples that could mislead neural models but not humans. Experimental results show that, compared to recurrent neural models, self-attentive models are more robust against adversarial perturbation. In addition, we provide theoretical explanations for their superior robustness to support our claims.
Soft Representation Learning for Sparse Transfer
Authors: Haeju Park, Jinyoung Yeo, Gengyu Wang, Seung-won Hwang
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Multi-Task Learning, Representation Learning, Transfer Learning

Transfer learning is effective for improving the performance of tasks that are related, and Multi-task learning (MTL) and Cross-lingual learning (CLL) are important instances. This paper argues that hard-parameter sharing, of hard-coding layers shared across different tasks or languages, cannot generalize well, when sharing with a loosely related task. Such case, which we call sparse transfer, might actually hurt performance, a phenomenon known as negative transfer. Our contribution is using adversarial training across tasks, to {``}soft-code{''} shared and private spaces, to avoid the shared space gets too sparse. In CLL, our proposed architecture considers another challenge of dealing with low-quality input.
Using LSTMs to Assess the Obligatoriness of Phonological Distinctive Features for Phonotactic Learning
Authors: Nicole Mirea, Klinton Bicknell
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7

To ascertain the importance of phonetic information in the form of phonological distinctive features for the purpose of segment-level phonotactic acquisition, we compare the performance of two recurrent neural network models of phonotactic learning: one that has access to distinctive features at the start of the learning process, and one that does not. Though the predictions of both models are significantly correlated with human judgments of non-words, the feature-naive model significantly outperforms the feature-aware one in terms of probability assigned to a held-out test set of English words, suggesting that distinctive features are not obligatory for learning phonotactic patterns at the segment level.
Historical Text Normalization with Delayed Rewards
Authors: Simon Flachs, Marcel Bollmann, Anders S{\o}gaard
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Text Normalization

Training neural sequence-to-sequence models with simple token-level log-likelihood is now a standard approach to historical text normalization, albeit often outperformed by phrase-based models. Policy gradient training enables direct optimization for exact matches, and while the small datasets in historical text normalization are prohibitive of from-scratch reinforcement learning, we show that policy gradient fine-tuning leads to significant improvements across the board. Policy gradient training, in particular, leads to more accurate normalizations for long or unseen words.
Stochastic Tokenization with a Language Model for Neural Text Classification
Authors: Tatsuya Hiraoka, Hiroyuki Shindo, Yuji Matsumoto
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Classification, General Classification, Language Modeling, Language Modelling, Segmentation, Sentence, Sentiment Analysis, text-classification, Text Classification

For unsegmented languages such as Japanese and Chinese, tokenization of a sentence has a significant impact on the performance of text classification. Sentences are usually segmented with words or subwords by a morphological analyzer or byte pair encoding and then encoded with word (or subword) representations for neural networks. However, segmentation is potentially ambiguous, and it is unclear whether the segmented tokens achieve the best performance for the target task. In this paper, we propose a method to simultaneously learn tokenization and text classification to address these problems. Our model incorporates a language model for unsupervised tokenization into a text classifier and then trains both models simultaneously. To make the model robust against infrequent tokens, we sampled segmentation for each sentence stochastically during training, which resulted in improved performance of text classification. We conducted experiments on sentiment analysis as a text classification task and show that our method achieves better performance than previous methods.
A Transparent Framework for Evaluating Unintended Demographic Bias in Word Embeddings
Authors: Chris Sweeney, Maryam Najafian
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Fairness, Word Embeddings

Word embedding models have gained a lot of traction in the Natural Language Processing community, however, they suffer from unintended demographic biases. Most approaches to evaluate these biases rely on vector space based metrics like the Word Embedding Association Test (WEAT). While these approaches offer great geometric insights into unintended biases in the embedding vector space, they fail to offer an interpretable meaning for how the embeddings could cause discrimination in downstream NLP applications. In this work, we present a transparent framework and metric for evaluating discrimination across protected groups with respect to their word embedding bias. Our metric (Relative Negative Sentiment Bias, RNSB) measures fairness in word embeddings via the relative negative sentiment associated with demographic identity terms from various protected groups. We show that our framework and metric enable useful analysis into the bias in word embeddings.
The Risk of Racial Bias in Hate Speech Detection
Authors: Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, Noah A. Smith
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Hate Speech Detection

We investigate how annotators{'} insensitivity to differences in dialect can lead to racial bias in automatic hate speech detection models, potentially amplifying harm against minority populations. We first uncover unexpected correlations between surface markers of African American English (AAE) and ratings of toxicity in several widely-used hate speech datasets. Then, we show that models trained on these corpora acquire and propagate these biases, such that AAE tweets and tweets by self-identified African Americans are up to two times more likely to be labelled as offensive compared to others. Finally, we propose *dialect* and *race priming* as ways to reduce the racial bias in annotation, showing that when annotators are made explicitly aware of an AAE tweet{'}s dialect they are significantly less likely to label the tweet as offensive.
LSTMEmbed: Learning Word and Sense Representations from a Large Semantically Annotated Corpus with Long Short-Term Memories
Authors: Ignacio Iacobacci, Roberto Navigli
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Word Embeddings

While word embeddings are now a de facto standard representation of words in most NLP tasks, recently the attention has been shifting towards vector representations which capture the different meanings, i.e., senses, of words. In this paper we explore the capabilities of a bidirectional LSTM model to learn representations of word senses from semantically annotated corpora. We show that the utilization of an architecture that is aware of word order, like an LSTM, enables us to create better representations. We assess our proposed model on various standard benchmarks for evaluating semantic representations, reaching state-of-the-art performance on the SemEval-2014 word-to-sense similarity task. We release the code and the resulting word and sense embeddings at http://lcl.uniroma1.it/LSTMEmbed.
SphereRE: Distinguishing Lexical Relations with Hyperspherical Relation Embeddings
Authors: Chengyu Wang, Xiaofeng He, Aoying Zhou
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Relation, Representation Learning

Lexical relations describe how meanings of terms relate to each other. Typical examples include hypernymy, synonymy, meronymy, etc. Automatic distinction of lexical relations is vital for NLP applications, and also challenging due to the lack of contextual signals to discriminate between such relations. In this work, we present a neural representation learning model to distinguish lexical relations among term pairs based on Hyperspherical Relation Embeddings (SphereRE). Rather than learning embeddings for individual terms, the model learns representations of relation triples by mapping them to the hyperspherical embedding space, where relation triples of different lexical relations are well separated. Experiments over several benchmarks confirm SphereRE outperforms state-of-the-arts.
Learning Morphosyntactic Analyzers from the Bible via Iterative Annotation Projection across 26 Languages
Authors: Garrett Nicolai, David Yarowsky
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Lemmatization, Morphological Analysis, Reranking

A large percentage of computational tools are concentrated in a very small subset of the planet{'}s languages. Compounding the issue, many languages lack the high-quality linguistic annotation necessary for the construction of such tools with current machine learning methods. In this paper, we address both issues simultaneously: leveraging the high accuracy of English taggers and parsers, we project morphological information onto translations of the Bible in 26 varied test languages. Using an iterative discovery, constraint, and training process, we build inflectional lexica in the target languages. Through a combination of iteration, ensembling, and reranking, we see double-digit relative error reductions in lemmatization and morphological analysis over a strong initial system.
Neural Machine Translation with Reordering Embeddings
Authors: Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Decoder, Machine Translation, Sentence, Translation

The reordering model plays an important role in phrase-based statistical machine translation. However, there are few works that exploit the reordering information in neural machine translation. In this paper, we propose a reordering mechanism to learn the reordering embedding of a word based on its contextual information. These learned reordering embeddings are stacked together with self-attention networks to learn sentence representation for machine translation. The reordering mechanism can be easily integrated into both the encoder and the decoder in the Transformer translation system. Experimental results on WMT{'}14 English-to-German, NIST Chinese-to-English, and WAT Japanese-to-English translation tasks demonstrate that the proposed methods can significantly improve the performance of the Transformer.
Neural Fuzzy Repair: Integrating Fuzzy Matches into Neural Machine Translation
Authors: Bram Bulte, Arda Tezcan
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Data Augmentation, Machine Translation, NMT, Translation

We present a simple yet powerful data augmentation method for boosting Neural Machine Translation (NMT) performance by leveraging information retrieved from a Translation Memory (TM). We propose and test two methods for augmenting NMT training data with fuzzy TM matches. Tests on the DGT-TM data set for two language pairs show consistent and substantial improvements over a range of baseline systems. The results suggest that this method is promising for any translation environment in which a sizeable TM is available and a certain amount of repetition across translations is to be expected, especially considering its ease of implementation.
Generating Diverse Translations with Sentence Codes
Authors: Raphael Shu, Hideki Nakayama, Kyunghyun Cho
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Diversity, Machine Translation, Sentence, Translation

Users of machine translation systems may desire to obtain multiple candidates translated in different ways. In this work, we attempt to obtain diverse translations by using sentence codes to condition the sentence generation. We describe two methods to extract the codes, either with or without the help of syntax information. For diverse generation, we sample multiple candidates, each of which conditioned on a unique code. Experiments show that the sampled translations have much higher diversity scores when using reasonable sentence codes, where the translation quality is still on par with the baselines even under strong constraint imposed by the codes. In qualitative analysis, we show that our method is able to generate paraphrase translations with drastically different structures. The proposed approach can be easily adopted to existing translation systems as no modification to the model is required.
Self-Supervised Neural Machine Translation
Authors: Dana Ruiter, Cristina Espa{\~n}a-Bonet, Josef van Genabith
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Machine Translation, NMT, Translation

We present a simple new method where an emergent NMT system is used for simultaneously selecting training data and learning internal NMT representations. This is done in a self-supervised way without parallel data, in such a way that both tasks enhance each other during training. The method is language independent, introduces no additional hyper-parameters, and achieves BLEU scores of 29.21 (en2fr) and 27.36 (fr2en) on newstest2014 using English and French Wikipedia data for training.
Reinforced Training Data Selection for Domain Adaptation
Authors: Miaofeng Liu, Yan Song, Hongbin Zou, Tong Zhang
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Dependency Parsing, Domain Adaptation, Part-Of-Speech Tagging, Reinforcement Learning, Reinforcement Learning (RL), Sentiment Analysis

Supervised models suffer from the problem of domain shifting where distribution mismatch in the data across domains greatly affect model performance. To solve the problem, training data selection (TDS) has been proven to be a prospective solution for domain adaptation in leveraging appropriate data. However, conventional TDS methods normally requires a predefined threshold which is neither easy to set nor can be applied across tasks, and models are trained separately with the TDS process. To make TDS self-adapted to data and task, and to combine it with model training, in this paper, we propose a reinforcement learning (RL) framework that synchronously searches for training instances relevant to the target domain and learns better representations for them. A selection distribution generator (SDG) is designed to perform the selection and is updated according to the rewards computed from the selected data, where a predictor is included in the framework to ensure a task-specific model can be trained on the selected data and provides feedback to rewards. Experimental results from part-of-speech tagging, dependency parsing, and sentiment analysis, as well as ablation studies, illustrate that the proposed framework is not only effective in data selection and representation, but also generalized to accommodate different NLP tasks.
Rhetorically Controlled Encoder-Decoder for Modern Chinese Poetry Generation
Authors: Zhi-Qiang Liu, Zuohui Fu, Jie Cao, Gerard de Melo, Yik-Cheung Tam, Cheng Niu, Jie zhou
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Decoder

Rhetoric is a vital element in modern poetry, and plays an essential role in improving its aesthetics. However, to date, it has not been considered in research on automatic poetry generation. In this paper, we propose a rhetorically controlled encoder-decoder for modern Chinese poetry generation. Our model relies on a continuous latent variable as a rhetoric controller to capture various rhetorical patterns in an encoder, and then incorporates rhetoric-based mixtures while generating modern Chinese poetry. For metaphor and personification, an automated evaluation shows that our model outperforms state-of-the-art baselines by a substantial margin, while human evaluation shows that our model generates better poems than baseline methods in terms of fluency, coherence, meaningfulness, and rhetorical aesthetics.
Enhancing Topic-to-Essay Generation with External Commonsense Knowledge
Authors: Pengcheng Yang, Lei LI, Fuli Luo, Tianyu Liu, Xu sun
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Concept-To-Text Generation

Automatic topic-to-essay generation is a challenging task since it requires generating novel, diverse, and topic-consistent paragraph-level text with a set of topics as input. Previous work tends to perform essay generation based solely on the given topics while ignoring massive commonsense knowledge. However, this commonsense knowledge provides additional background information, which can help to generate essays that are more novel and diverse. Towards filling this gap, we propose to integrate commonsense from the external knowledge base into the generator through dynamic memory mechanism. Besides, the adversarial training based on a multi-label discriminator is employed to further improve topic-consistency. We also develop a series of automatic evaluation metrics to comprehensively assess the quality of the generated essay. Experiments show that with external commonsense knowledge and adversarial training, the generated essays are more novel, diverse, and topic-consistent than existing methods in terms of both automatic and human evaluation.
Towards Fine-grained Text Sentiment Transfer
Authors: Fuli Luo, Peng Li, Pengcheng Yang, Jie zhou, Yutong Tan, Baobao Chang, Zhifang Sui, Xu sun
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Decoder, Reinforcement Learning

In this paper, we focus on the task of fine-grained text sentiment transfer (FGST). This task aims to revise an input sequence to satisfy a given sentiment intensity, while preserving the original semantic content. Different from the conventional sentiment transfer task that only reverses the sentiment polarity (positive/negative) of text, the FTST task requires more nuanced and fine-grained control of sentiment. To remedy this, we propose a novel Seq2SentiSeq model. Specifically, the numeric sentiment intensity value is incorporated into the decoder via a Gaussian kernel layer to finely control the sentiment intensity of the output. Moreover, to tackle the problem of lacking parallel data, we propose a cycle reinforcement learning algorithm to guide the model training. In this framework, the elaborately designed rewards can balance both sentiment transformation and content preservation, while not requiring any ground truth output. Experimental results show that our approach can outperform existing methods by a large margin in both automatic evaluation and human evaluation.
Improving Abstractive Document Summarization with Salient Information Modeling
Authors: Yongjian You, Weijia Jia, Tianyi Liu, Wenmian Yang
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Decoder, Document Summarization

Comprehensive document encoding and salient information selection are two major difficulties for generating summaries with adequate salient information. To tackle the above difficulties, we propose a Transformer-based encoder-decoder framework with two novel extensions for abstractive document summarization. Specifically, (1) to encode the documents comprehensively, we design a focus-attention mechanism and incorporate it into the encoder. This mechanism models a Gaussian focal bias on attention scores to enhance the perception of local context, which contributes to producing salient and informative summaries. (2) To distinguish salient information precisely, we design an independent saliency-selection network which manages the information flow from encoder to decoder. This network effectively reduces the influences of secondary information on the generated summaries. Experimental results on the popular CNN/Daily Mail benchmark demonstrate that our model outperforms other state-of-the-art baselines on the ROUGE metrics.