pwc-archive/papers-with-abstracts huggingface dataset:
Dataset Creation for Ranking Constructive News Comments
Authors: Soichiro Fujita, Hayato Kobayashi, Manabu Okumura
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Articles

Ranking comments on an online news service is a practically important task for the service provider, and thus there have been many studies on this task. However, most of them considered users{'} positive feedback, such as {``}Like{''}-button clicks, as a quality measure. In this paper, we address directly evaluating the quality of comments on the basis of {``}constructiveness,{''} separately from user feedback. To this end, we create a new dataset including 100K+ Japanese comments with constructiveness scores (C-scores). Our experiments clarify that C-scores are not always related to users{'} positive feedback, and the performance of pairwise ranking models tends to be enhanced by the variation of comments rather than articles.
Enhancing Air Quality Prediction with Social Media and Natural Language Processing
Authors: Jyun-Yu Jiang, Xue Sun, Wei Wang, Sean Young
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Prediction

Accompanied by modern industrial developments, air pollution has already become a major concern for human health. Hence, air quality measures, such as the concentration of PM2.5, have attracted increasing attention. Even some studies apply historical measurements into air quality forecast, the changes of air quality conditions are still hard to monitor. In this paper, we propose to exploit social media and natural language processing techniques to enhance air quality prediction. Social media users are treated as social sensors with their findings and locations. After filtering noisy tweets using word selection and topic modeling, a deep learning model based on convolutional neural networks and over-tweet-pooling is proposed to enhance air quality prediction. We conduct experiments on 7-month real-world Twitter datasets in the five most heavily polluted states in the USA. The results show that our approach significantly improves air quality prediction over the baseline that does not use social media by 6.9{\%} to 17.7{\%} in macro-F1 scores.
Twitter Homophily: Network Based Prediction of User's Occupation
Authors: Jiaqi Pan, Rishabh Bhardwaj, Wei Lu, Hai Leong Chieu, Xinghao Pan, Ni Yi Puay
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Prediction

In this paper, we investigate the importance of social network information compared to content information in the prediction of a Twitter user{'}s occupational class. We show that the content information of a user{'}s tweets, the profile descriptions of a user{'}s follower/following community, and the user{'}s social network provide useful information for classifying a user{'}s occupational group. In our study, we extend an existing data set for this problem, and we achieve significantly better performance by using social network homophily that has not been fully exploited in previous work. In our analysis, we found that by using the graph convolutional network to exploit social homophily, we can achieve competitive performance on this data set with just a small fraction of the training data.
A Simple Recipe towards Reducing Hallucination in Neural Surface Realisation
Authors: Feng Nie, Jin-Ge Yao, Jinpeng Wang, Rong pan, Chin-Yew Lin
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Hallucination, Text Generation

Recent neural language generation systems often \textit{hallucinate} contents (i.e., producing irrelevant or contradicted facts), especially when trained on loosely corresponding pairs of the input structure and text. To mitigate this issue, we propose to integrate a language understanding module for data refinement with self-training iterations to effectively induce strong equivalence between the input data and the paired text. Experiments on the E2E challenge dataset show that our proposed framework can reduce more than 50{\%} relative unaligned noise from the original data-text pairs. A vanilla sequence-to-sequence neural NLG model trained on the refined data has improved on content correctness compared with the current state-of-the-art ensemble generator.
Cross-Modal Commentator: Automatic Machine Commenting Based on Cross-Modal Information
Authors: Pengcheng Yang, Zhihan Zhang, Fuli Luo, Lei LI, Chengyang Huang, Xu sun
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Articles, Comment Generation

Automatic commenting of online articles can provide additional opinions and facts to the reader, which improves user experience and engagement on social media platforms. Previous work focuses on automatic commenting based solely on textual content. However, in real-scenarios, online articles usually contain multiple modal contents. For instance, graphic news contains plenty of images in addition to text. Contents other than text are also vital because they are not only more attractive to the reader but also may provide critical information. To remedy this, we propose a new task: cross-model automatic commenting (CMAC), which aims to make comments by integrating multiple modal contents. We construct a large-scale dataset for this task and explore several representative methods. Going a step further, an effective co-attention model is presented to capture the dependency between textual and visual information. Evaluation results show that our proposed model can achieve better performance than competitive baselines.
A Working Memory Model for Task-oriented Dialog Response Generation
Authors: Xiuyi Chen, Jiaming Xu, Bo Xu
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Response Generation, World Knowledge

Recently, to incorporate external Knowledge Base (KB) information, one form of world knowledge, several end-to-end task-oriented dialog systems have been proposed. These models, however, tend to confound the dialog history with KB tuples and simply store them into one memory. Inspired by the psychological studies on working memory, we propose a working memory model (WMM2Seq) for dialog response generation. Our WMM2Seq adopts a working memory to interact with two separated long-term memories, which are the episodic memory for memorizing dialog history and the semantic memory for storing KB tuples. The working memory consists of a central executive to attend to the aforementioned memories, and a short-term storage system to store the {``}activated{''} contents from the long-term memories. Furthermore, we introduce a context-sensitive perceptual process for the token representations of dialog history, and then feed them into the episodic memory. Extensive experiments on two task-oriented dialog datasets demonstrate that our WMM2Seq significantly outperforms the state-of-the-art results in several evaluation metrics.
Sentence Mover's Similarity: Automatic Evaluation for Multi-Sentence Texts
Authors: Elizabeth Clark, Asli Celikyilmaz, Noah A. Smith
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Reinforcement Learning, Semantic Similarity, Semantic Textual Similarity, Sentence, Sentence Embeddings

For evaluating machine-generated texts, automatic methods hold the promise of avoiding collection of human judgments, which can be expensive and time-consuming. The most common automatic metrics, like BLEU and ROUGE, depend on exact word matching, an inflexible approach for measuring semantic similarity. We introduce methods based on sentence mover{'}s similarity; our automatic metrics evaluate text in a continuous space using word and sentence embeddings. We find that sentence-based metrics correlate with human judgments significantly better than ROUGE, both on machine-generated summaries (average length of 3.4 sentences) and human-authored essays (average length of 7.5). We also show that sentence mover{'}s similarity can be used as a reward when learning a generation model via reinforcement learning; we present both automatic and human evaluations of summaries learned in this way, finding that our approach outperforms ROUGE.
Deep Dominance - How to Properly Compare Deep Neural Models
Authors: Rotem Dror, Segev Shlomov, Roi Reichart
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7

Comparing between Deep Neural Network (DNN) models based on their performance on unseen data is crucial for the progress of the NLP field. However, these models have a large number of hyper-parameters and, being non-convex, their convergence point depends on the random values chosen at initialization and during training. Proper DNN comparison hence requires a comparison between their empirical score distributions on unseen data, rather than between single evaluation scores as is standard for more simple, convex models. In this paper, we propose to adapt to this problem a recently proposed test for the Almost Stochastic Dominance relation between two distributions. We define the criteria for a high quality comparison method between DNNs, and show, both theoretically and through analysis of extensive experimental results with leading DNN models for sequence tagging tasks, that the proposed test meets all criteria while previously proposed methods fail to do so. We hope the test we propose here will set a new working practice in the NLP community.
We Need to Talk about Standard Splits
Authors: Kyle Gorman, Steven Bedrick
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7

It is standard practice in speech {\&} language technology to rank systems according to their performance on a test set held out for evaluation. However, few researchers apply statistical tests to determine whether differences in performance are likely to arise by chance, and few examine the stability of system ranking across multiple training-testing splits. We conduct replication and reproduction experiments with nine part-of-speech taggers published between 2000 and 2018, each of which claimed state-of-the-art performance on a widely-used {``}standard split{''}. While we replicate results on the standard split, we fail to reliably reproduce some rankings when we repeat this analysis with randomly generated training-testing splits. We argue that randomly generated splits should be used in system evaluation.
Aiming beyond the Obvious: Identifying Non-Obvious Cases in Semantic Similarity Datasets
Authors: Nicole Peinelt, Maria Liakata, Dong Nguyen
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Semantic Similarity, Semantic Textual Similarity

Existing datasets for scoring text pairs in terms of semantic similarity contain instances whose resolution differs according to the degree of difficulty. This paper proposes to distinguish obvious from non-obvious text pairs based on superficial lexical overlap and ground-truth labels. We characterise existing datasets in terms of containing difficult cases and find that recently proposed models struggle to capture the non-obvious cases of semantic similarity. We describe metrics that emphasise cases of similarity which require more complex inference and propose that these are used for evaluating systems for semantic similarity.
Putting Evaluation in Context: Contextual Embeddings Improve Machine Translation Evaluation
Authors: Nitika Mathur, Timothy Baldwin, Trevor Cohn
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Machine Translation, Sentence, Translation, Word Embeddings

Accurate, automatic evaluation of machine translation is critical for system tuning, and evaluating progress in the field. We proposed a simple unsupervised metric, and additional supervised metrics which rely on contextual word embeddings to encode the translation and reference sentences. We find that these models rival or surpass all existing metrics in the WMT 2017 sentence-level and system-level tracks, and our trained model has a substantially higher correlation with human judgements than all existing metrics on the WMT 2017 to-English sentence level dataset.
CONAN - COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech
Authors: Yi-Ling Chung, Elizaveta Kuzmenko, Serra Sinem Tekiroglu, Marco Guerini
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Data Augmentation, Translation

Although there is an unprecedented effort to provide adequate responses in terms of laws and policies to hate content on social media platforms, dealing with hatred online is still a tough problem. Tackling hate speech in the standard way of content deletion or user suspension may be charged with censorship and overblocking. One alternate strategy, that has received little attention so far by the research community, is to actually oppose hate content with counter-narratives (i.e. informed textual responses). In this paper, we describe the creation of the first large-scale, multilingual, expert-based dataset of hate-speech/counter-narrative pairs. This dataset has been built with the effort of more than 100 operators from three different NGOs that applied their training and expertise to the task. Together with the collected data we also provide additional annotations about expert demographics, hate and response type, and data augmentation through translation and paraphrasing. Finally, we provide initial experiments to assess the quality of our data.
Categorizing and Inferring the Relationship between the Text and Image of Twitter Posts
Authors: Alakan Vempala, a, Daniel Preo{\c{t}}iuc-Pietro
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Image Captioning

Text in social media posts is frequently accompanied by images in order to provide content, supply context, or to express feelings. This paper studies how the meaning of the entire tweet is composed through the relationship between its textual content and its image. We build and release a data set of image tweets annotated with four classes which express whether the text or the image provides additional information to the other modality. We show that by combining the text and image information, we can build a machine learning approach that accurately distinguishes between the relationship types. Further, we derive insights into how these relationships are materialized through text and image content analysis and how they are impacted by user demographic traits. These methods can be used in several downstream applications including pre-training image tagging models, collecting distantly supervised data for image captioning, and can be directly used in end-user applications to optimize screen estate.
Who Sides with Whom? Towards Computational Construction of Discourse Networks for Political Debates
Authors: Sebastian Pad{\'o}, Andre Blessing, Nico Blokker, Erenay Dayanik, Sebastian Haunss, Jonas Kuhn
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Decision Making, Knowledge Base Population

Understanding the structures of political debates (which actors make what claims) is essential for understanding democratic political decision making. The vision of computational construction of such discourse networks from newspaper reports brings together political science and natural language processing. This paper presents three contributions towards this goal: (a) a requirements analysis, linking the task to knowledge base population; (b) an annotated pilot corpus of migration claims based on German newspaper reports; (c) initial modeling results.
Analyzing Linguistic Differences between Owner and Staff Attributed Tweets
Authors: Daniel Preo{\c{t}}iuc-Pietro, Rita Devlin Marier
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7

Research on social media has to date assumed that all posts from an account are authored by the same person. In this study, we challenge this assumption and study the linguistic differences between posts signed by the account owner or attributed to their staff. We introduce a novel data set of tweets posted by U.S. politicians who self-reported their tweets using a signature. We analyze the linguistic topics and style features that distinguish the two types of tweets. Predictive results show that we are able to predict owner and staff attributed tweets with good accuracy, even when not using any training data from that account.
Look Harder: A Neural Machine Translation Model with Hard Attention
Authors: Sathish Reddy Indurthi, Insoo Chung, Sangha Kim
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Hard Attention, Machine Translation, NMT, Reinforcement Learning, Reinforcement Learning (RL), Translation

Soft-attention based Neural Machine Translation (NMT) models have achieved promising results on several translation tasks. These models attend all the words in the source sequence for each target token, which makes them ineffective for long sequence translation. In this work, we propose a hard-attention based NMT model which selects a subset of source tokens for each target token to effectively handle long sequence translation. Due to the discrete nature of the hard-attention mechanism, we design a reinforcement learning algorithm coupled with reward shaping strategy to efficiently train it. Experimental results show that the proposed model performs better on long sequences and thereby achieves significant BLEU score improvement on English-German (EN-DE) and English-French (ENFR) translation tasks compared to the soft attention based NMT.
A Simple and Effective Approach to Automatic Post-Editing with Transfer Learning
Authors: Gon{\c{c}}alo M. Correia, Andr{\'e} F. T. Martins
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Automatic Post-Editing, Decoder, GPU, Machine Translation, Transfer Learning, Translation

Automatic post-editing (APE) seeks to automatically refine the output of a black-box machine translation (MT) system through human post-edits. APE systems are usually trained by complementing human post-edited data with large, artificial data generated through back-translations, a time-consuming process often no easier than training a MT system from scratch. in this paper, we propose an alternative where we fine-tune pre-trained BERT models on both the encoder and decoder of an APE system, exploring several parameter sharing strategies. By only training on a dataset of 23K sentences for 3 hours on a single GPU we obtain results that are competitive with systems that were trained on 5M artificial sentences. When we add this artificial data our method obtains state-of-the-art results.
Leveraging Local and Global Patterns for Self-Attention Networks
Authors: Mingzhou Xu, Derek F. Wong, Baosong Yang, Yue Zhang, Lidia S. Chao
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Machine Translation, Sentence, Translation

Self-attention networks have received increasing research attention. By default, the hidden states of each word are hierarchically calculated by attending to all words in the sentence, which assembles global information. However, several studies pointed out that taking all signals into account may lead to overlooking neighboring information (e.g. phrase pattern). To address this argument, we propose a hybrid attention mechanism to dynamically leverage both of the local and global information. Specifically, our approach uses a gating scalar for integrating both sources of the information, which is also convenient for quantifying their contributions. Experiments on various neural machine translation tasks demonstrate the effectiveness of the proposed method. The extensive analyses verify that the two types of contexts are complementary to each other, and our method gives highly effective improvements in their integration.
Sentence-Level Agreement for Neural Machine Translation
Authors: Mingming Yang, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Min Zhang, Tiejun Zhao
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Machine Translation, NMT, Sentence, Translation

The training objective of neural machine translation (NMT) is to minimize the loss between the words in the translated sentences and those in the references. In NMT, there is a natural correspondence between the source sentence and the target sentence. However, this relationship has only been represented using the entire neural network and the training objective is computed in word-level. In this paper, we propose a sentence-level agreement module to directly minimize the difference between the representation of source and target sentence. The proposed agreement module can be integrated into NMT as an additional training objective function and can also be used to enhance the representation of the source sentences. Empirical results on the NIST Chinese-to-English and WMT English-to-German tasks show the proposed agreement module can significantly improve the NMT performance.
Multilingual Unsupervised NMT using Shared Encoder and Language-Specific Decoders
Authors: Sukanta Sen, Kamal Kumar Gupta, Asif Ekbal, Pushpak Bhattacharyya
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Denoising, NMT, Translation

In this paper, we propose a multilingual unsupervised NMT scheme which jointly trains multiple languages with a shared encoder and multiple decoders. Our approach is based on denoising autoencoding of each language and back-translating between English and multiple non-English languages. This results in a universal encoder which can encode any language participating in training into an inter-lingual representation, and language-specific decoders. Our experiments using only monolingual corpora show that multilingual unsupervised model performs better than the separately trained bilingual models achieving improvement of up to 1.48 BLEU points on WMT test sets. We also observe that even if we do not train the network for all possible translation directions, the network is still able to translate in a many-to-many fashion leveraging encoder{'}s ability to generate interlingual representation.
Unsupervised Multilingual Word Embedding with Limited Resources using Neural Language Models
Authors: Takashi Wada, Tomoharu Iwata, Yuji Matsumoto
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Word Alignment, Word Embeddings

Recently, a variety of unsupervised methods have been proposed that map pre-trained word embeddings of different languages into the same space without any parallel data. These methods aim to find a linear transformation based on the assumption that monolingual word embeddings are approximately isomorphic between languages. However, it has been demonstrated that this assumption holds true only on specific conditions, and with limited resources, the performance of these methods decreases drastically. To overcome this problem, we propose a new unsupervised multilingual embedding method that does not rely on such assumption and performs well under resource-poor scenarios, namely when only a small amount of monolingual data (i.e., 50k sentences) are available, or when the domains of monolingual data are different across languages. Our proposed model, which we call {`}Multilingual Neural Language Models{'}, shares some of the network parameters among multiple languages, and encodes sentences of multiple languages into the same space. The model jointly learns word embeddings of different languages in the same space, and generates multilingual embeddings without any parallel data or pre-training. Our experiments on word alignment tasks have demonstrated that, on the low-resource condition, our model substantially outperforms existing unsupervised and even supervised methods trained with 500 bilingual pairs of words. Our model also outperforms unsupervised methods given different-domain corpora across languages. Our code is publicly available.
CogNet: A Large-Scale Cognate Database
Authors: Khuyagbaatar Batsuren, Gabor Bella, Fausto Giunchiglia
Date: 2019-07-01 00:00:00
Proceeding: ACL 2019 7

This paper introduces CogNet, a new, large-scale lexical database that provides cognates -words of common origin and meaning- across languages. The database currently contains 3.1 million cognate pairs across 338 languages using 35 writing systems. The paper also describes the automated method by which cognates were computed from publicly available wordnets, with an accuracy evaluated to 94{\%}. Finally, it presents statistics about the cognate data and some initial insights into it, hinting at a possible future exploitation of the resource by various fields of lingustics.