pwc-archive/papers-with-abstracts huggingface dataset:
Mode Effects' Challenge to Authorship Attribution
Authors: Haining Wang, Allen Riddell, Patrick Juola
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Authorship Attribution, Feature Engineering, Sentence

The success of authorship attribution relies on the presence of linguistic features specific to individual authors. There is, however, limited research assessing to what extent authorial style remains constant when individuals switch from one writing modality to another. We measure the effect of writing mode on writing style in the context of authorship attribution research using a corpus of documents composed online (in a web browser) and documents composed offline using a traditional word processor. The results confirm the existence of a {``}mode effect{''} on authorial style. Online writing differs systematically from offline writing in terms of sentence length, word use, readability, and certain part-of-speech ratios. These findings have implications for research design and feature engineering in authorship attribution studies.
Automatic Data Acquisition for Event Coreference Resolution
Authors: Prafulla Kumar Choubey, Ruihong Huang
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Articles, coreference-resolution, Coreference Resolution, Event Coreference Resolution

We propose to leverage lexical paraphrases and high precision rules informed by news discourse structure to automatically collect coreferential and non-coreferential event pairs from unlabeled English news articles. We perform both manual validation and empirical evaluation on multiple evaluation datasets with different event domains and text genres to assess the quality of our acquired event pairs. We found that a model trained on our acquired event pairs performs comparably as the supervised model when applied to new data out of the training data domains. Further, augmenting human-annotated data with the acquired event pairs provides empirical performance gains on both in-domain and out-of-domain evaluation datasets.
Joint Learning of Representations for Web-tables, Entities and Types using Graph Convolutional Network
Authors: Aniket Pramanick, Indrajit Bhattacharya
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Table annotation

Existing approaches for table annotation with entities and types either capture the structure of table using graphical models, or learn embeddings of table entries without accounting for the complete syntactic structure. We propose TabGCN, that uses Graph Convolutional Networks to capture the complete structure of tables, knowledge graph and the training annotations, and jointly learns embeddings for table elements as well as the entities and types. To account for knowledge incompleteness, TabGCN{'}s embeddings can be used to discover new entities and types. Using experiments on 5 benchmark datasets, we show that TabGCN significantly outperforms multiple state-of-the-art baselines for table annotation, while showing promising performance on downstream table-related applications.
Enriching Non-Autoregressive Transformer with Syntactic and Semantic Structures for Neural Machine Translation
Authors: Ye Liu, Yao Wan, JianGuo Zhang, Wenting Zhao, Philip Yu
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Machine Translation, Translation

The non-autoregressive models have boosted the efficiency of neural machine translation through parallelized decoding at the cost of effectiveness, when comparing with the autoregressive counterparts. In this paper, we claim that the syntactic and semantic structures among natural language are critical for non-autoregressive machine translation and can further improve the performance. However, these structures are rarely considered in the existing non-autoregressive models. Inspired by this intuition, we propose to incorporate the explicit syntactic and semantic structure of languages into a non-autoregressive Transformer, for the task of neural machine translation. Moreover, we also consider the intermediate latent alignment within target sentences to better learn the long-term token dependencies. Experimental results on two real-world datasets (i.e., WMT14 En-De and WMT16 En- Ro) show that our model achieves a significantly faster speed, as well as keeps the translation quality when compared with several state-of-the-art non-autoregressive models.
NLQuAD: A Non-Factoid Long Question Answering Data Set
Authors: Amir Soleimani, Christof Monz, Marcel Worring
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Descriptive, Position, Question Answering, Sentence

We introduce NLQuAD, the first data set with baseline methods for non-factoid long question answering, a task requiring document-level language understanding. In contrast to existing span detection question answering data sets, NLQuAD has non-factoid questions that are not answerable by a short span of text and demanding multiple-sentence descriptive answers and opinions. We show the limitation of the F1 score for evaluation of long answers and introduce Intersection over Union (IoU), which measures position-sensitive overlap between the predicted and the target answer spans. To establish baseline performances, we compare BERT, RoBERTa, and Longformer models. Experimental results and human evaluations show that Longformer outperforms the other architectures, but results are still far behind a human upper bound, leaving substantial room for improvements. NLQuAD{'}s samples exceed the input limitation of most pre-trained Transformer-based models, encouraging future research on long sequence language models.
Memorization vs. Generalization : Quantifying Data Leakage in NLP Performance Evaluation
Authors: Aparna Elangovan, Jiayuan He, Karin Verspoor
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Memorization, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Relation Extraction

Public datasets are often used to evaluate the efficacy and generalizability of state-of-the-art methods for many tasks in natural language processing (NLP). However, the presence of overlap between the train and test datasets can lead to inflated results, inadvertently evaluating the model{'}s ability to memorize and interpreting it as the ability to generalize. In addition, such data sets may not provide an effective indicator of the performance of these methods in real world scenarios. We identify leakage of training data into test data on several publicly available datasets used to evaluate NLP tasks, including named entity recognition and relation extraction, and study them to assess the impact of that leakage on the model{'}s ability to memorize versus generalize.
An Expert Annotated Dataset for the Detection of Online Misogyny
Authors: Ella Guest, Bertie Vidgen, Alexandros Mittos, Nishanth Sastry, Gareth Tyson, Helen Margetts
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Binary Classification, Classification

Online misogyny is a pernicious social problem that risks making online platforms toxic and unwelcoming to women. We present a new hierarchical taxonomy for online misogyny, as well as an expert labelled dataset to enable automatic classification of misogynistic content. The dataset consists of 6567 labels for Reddit posts and comments. As previous research has found untrained crowdsourced annotators struggle with identifying misogyny, we hired and trained annotators and provided them with robust annotation guidelines. We report baseline classification performance on the binary classification task, achieving accuracy of 0.93 and F1 of 0.43. The codebook and datasets are made freely available for future researchers.
ChEMU-Ref: A Corpus for Modeling Anaphora Resolution in the Chemical Domain
Authors: Biaoyan Fang, Christian Druckenbrodt, Saber A Akhondi, Jiayuan He, Timothy Baldwin, Karin Verspoor
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2

Chemical patents contain rich coreference and bridging links, which are the target of this research. Specially, we introduce a novel annotation scheme, based on which we create the ChEMU-Ref dataset from reaction description snippets in English-language chemical patents. We propose a neural approach to anaphora resolution, which we show to achieve strong results, especially when jointly trained over coreference and bridging links.
Quantifying Appropriateness of Summarization Data for Curriculum Learning
Authors: Ryuji Kano, Takumi Takahashi, Toru Nishino, Motoki Taniguchi, Tomoki Taniguchi, Tomoko Ohkuma
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Translation

Much research has reported the training data of summarization models are noisy; summaries often do not reflect what is written in the source texts. We propose an effective method of curriculum learning to train summarization models from such noisy data. Curriculum learning is used to train sequence-to-sequence models with noisy data. In translation tasks, previous research quantified noise of the training data using two models trained with noisy and clean corpora. Because such corpora do not exist in summarization fields, we propose a model that can quantify noise from a single noisy corpus. We conduct experiments on three summarization models; one pretrained model and two non-pretrained models, and verify our method improves the performance. Furthermore, we analyze how different curricula affect the performance of pretrained and non-pretrained summarization models. Our result on human evaluation also shows our method improves the performance of summarization models.
Evaluating language models for the retrieval and categorization of lexical collocations
Authors: Luis Espinosa Anke, Joan Codina-Filba, Leo Wanner
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Retrieval, valid

Lexical collocations are idiosyncratic combinations of two syntactically bound lexical items (e.g., {``}heavy rain{''} or {``}take a step{''}). Understanding their degree of compositionality and idiosyncrasy, as well their underlying semantics, is crucial for language learners, lexicographers and downstream NLP applications. In this paper, we perform an exhaustive analysis of current language models for collocation understanding. We first construct a dataset of apparitions of lexical collocations in context, categorized into 17 representative semantic categories. Then, we perform two experiments: (1) unsupervised collocate retrieval using BERT, and (2) supervised collocation classification in context. We find that most models perform well in distinguishing light verb constructions, especially if the collocation{'}s first argument acts as subject, but often fail to distinguish, first, different syntactic structures within the same semantic category, and second, fine-grained semantic categories which restrict the use of small sets of valid collocates for a given base.
BART-TL: Weakly-Supervised Topic Label Generation
Authors: Cristian Popa, Traian Rebedea
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Topic Models

We propose a novel solution for assigning labels to topic models by using multiple weak labelers. The method leverages generative transformers to learn accurate representations of the most important topic terms and candidate labels. This is achieved by fine-tuning pre-trained BART models on a large number of potential labels generated by state of the art non-neural models for topic labeling, enriched with different techniques. The proposed BART-TL model is able to generate valuable and novel labels in a weakly-supervised manner and can be improved by adding other weak labelers or distant supervision on similar tasks.
Dynamic Graph Transformer for Implicit Tag Recognition
Authors: Yi-Ting Liou, Chung-Chi Chen, Hen-Hsen Huang, Hsin-Hsi Chen
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Relation Extraction, TAG

Textual information extraction is a typical research topic in the NLP community. Several NLP tasks such as named entity recognition and relation extraction between entities have been well-studied in previous work. However, few works pay their attention to the implicit information. For example, a financial news article mentioned {``}Apple Inc.{''} may be also related to Samsung, even though Samsung is not explicitly mentioned in this article. This work presents a novel dynamic graph transformer that distills the textual information and the entity relations on the fly. Experimental results confirm the effectiveness of our approach to implicit tag recognition.
Generating Weather Comments from Meteorological Simulations
Authors: Soichiro Murakami, Sora Tanaka, Masatsugu Hangyo, Hidetaka Kamigaito, Kotaro Funakoshi, Hiroya Takamura, Manabu Okumura
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Informativeness

The task of generating weather-forecast comments from meteorological simulations has the following requirements: (i) the changes in numerical values for various physical quantities need to be considered, (ii) the weather comments should be dependent on delivery time and area information, and (iii) the comments should provide useful information for users. To meet these requirements, we propose a data-to-text model that incorporates three types of encoders for numerical forecast maps, observation data, and meta-data. We also introduce weather labels representing weather information, such as sunny and rain, for our model to explicitly describe useful information. We conducted automatic and human evaluations. The results indicate that our model performed best against baselines in terms of informativeness. We make our code and data publicly available.
SICK-NL: A Dataset for Dutch Natural Language Inference
Authors: Gijs Wijnholds, Michael Moortgat
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Multilingual NLP, Natural Language Inference, Translation

We present SICK-NL (read: signal), a dataset targeting Natural Language Inference in Dutch. SICK-NL is obtained by translating the SICK dataset of (Marelli et al., 2014) from English into Dutch. Having a parallel inference dataset allows us to compare both monolingual and multilingual NLP models for English and Dutch on the two tasks. In the paper, we motivate and detail the translation process, perform a baseline evaluation on both the original SICK dataset and its Dutch incarnation SICK-NL, taking inspiration from Dutch skipgram embeddings and contextualised embedding models. In addition, we encapsulate two phenomena encountered in the translation to formulate stress tests and verify how well the Dutch models capture syntactic restructurings that do not affect semantics. Our main finding is all models perform worse on SICK-NL than on SICK, indicating that the Dutch dataset is more challenging than the English original. Results on the stress tests show that models don{'}t fully capture word order freedom in Dutch, warranting future systematic studies.
Cross-Topic Rumor Detection using Topic-Mixtures
Authors: Xiaoying Ren, Jing Jiang, Ling Min Serena Khoo, Hai Leong Chieu
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Mixture-of-Experts

There has been much interest in rumor detection using deep learning models in recent years. A well-known limitation of deep learning models is that they tend to learn superficial patterns, which restricts their generalization ability. We find that this is also true for cross-topic rumor detection. In this paper, we propose a method inspired by the {``}mixture of experts{''} paradigm. We assume that the prediction of the rumor class label given an instance is dependent on the topic distribution of the instance. After deriving a vector representation for each topic, given an instance, we derive a {``}topic mixture{''} vector for the instance based on its topic distribution. This topic mixture is combined with the vector representation of the instance itself to make rumor predictions. Our experiments show that our proposed method can outperform two baseline debiasing methods in a cross-topic setting. In a synthetic setting when we removed topic-specific words, our method also works better than the baselines, showing that our method does not rely on superficial features.
RelWalk - A Latent Variable Model Approach to Knowledge Graph Embedding
Authors: Danushka Bollegala, Huda Hakami, Yuichi Yoshida, Ken-ichi Kawarabayashi
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Graph Embedding, Knowledge Graph Embedding, Knowledge Graph Embeddings, Word Embeddings

Embedding entities and relations of a knowledge graph in a low-dimensional space has shown impressive performance in predicting missing links between entities. Although progresses have been achieved, existing methods are heuristically motivated and theoretical understanding of such embeddings is comparatively underdeveloped. This paper extends the random walk model of word embeddings to Knowledge Graph Embeddings (KGEs) to derive a scoring function that evaluates the strength of a relation R between two entities h (head) and t (tail). Moreover, we show that marginal loss minimisation, a popular objective used in much prior work in KGE, follows naturally from the log-likelihood ratio maximisation under the probabilities estimated from the KGEs according to our theoretical relationship. We propose a learning objective motivated by the theoretical analysis to learn KGEs from a given knowledge graph.Using the derived objective, accurate KGEs are learnt from FB15K237 and WN18RR benchmark datasets, providing empirical evidence in support of the theory.
Exploiting Position and Contextual Word Embeddings for Keyphrase Extraction from Scientific Papers
Authors: Krutarth Patel, Cornelia Caragea
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Keyphrase Extraction, Position, Word Embeddings

Keyphrases associated with research papers provide an effective way to find useful information in the large and growing scholarly digital collections. In this paper, we present KPRank, an unsupervised graph-based algorithm for keyphrase extraction that exploits both positional information and contextual word embeddings into a biased PageRank. Our experimental results on five benchmark datasets show that KPRank that uses contextual word embeddings with additional position signal outperforms previous approaches and strong baselines for this task.
NewsMTSC: A Dataset for (Multi-)Target-dependent Sentiment Classification in Political News Articles
Authors: Felix Hamborg, Karsten Donnay
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Articles, Decision Making, Language Modeling, Language Modelling, Sentiment Analysis, Sentiment Classification

Previous research on target-dependent sentiment classification (TSC) has mostly focused on reviews, social media, and other domains where authors tend to express sentiment explicitly. In this paper, we investigate TSC in news articles, a much less researched TSC domain despite the importance of news as an essential information source in individual and societal decision making. We introduce NewsMTSC, a high-quality dataset for TSC on news articles with key differences compared to established TSC datasets, including, for example, different means to express sentiment, longer texts, and a second test-set to measure the influence of multi-target sentences. We also propose a model that uses a BiGRU to interact with multiple embeddings, e.g., from a language model and external knowledge sources. The proposed model improves the performance of the prior state-of-the-art from F1{\_}m=81.7 to 83.1 (real-world sentiment distribution) and from F1{\_}m=81.2 to 82.5 (multi-target sentences).
Dependency parsing with structure preserving embeddings
Authors: {\'A}kos K{\'a}d{\'a}r, Lan Xiao, Mete Kemertas, Federico Fancellu, Allan Jepson, Afsaneh Fazly
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Dependency Parsing, Sentence

Modern neural approaches to dependency parsing are trained to predict a tree structure by jointly learning a contextual representation for tokens in a sentence, as well as a head{--}dependent scoring function. Whereas this strategy results in high performance, it is difficult to interpret these representations in relation to the geometry of the underlying tree structure. Our work seeks instead to learn interpretable representations by training a parser to explicitly preserve structural properties of a tree. We do so by casting dependency parsing as a tree embedding problem where we incorporate geometric properties of dependency trees in the form of training losses within a graph-based parser. We provide a thorough evaluation of these geometric losses, showing that a majority of them yield strong tree distance preservation as well as parsing performance on par with a competitive graph-based parser (Qi et al., 2018). Finally, we show where parsing errors lie in terms of tree relationship in order to guide future work.
MultiHumES: Multilingual Humanitarian Dataset for Extractive Summarization
Authors: Jenny Paola Yela-Bello, Ewan Oglethorpe, Navid Rekabsaz
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Decision Making, Extractive Summarization, Humanitarian

When responding to a disaster, humanitarian experts must rapidly process large amounts of secondary data sources to derive situational awareness and guide decision-making. While these documents contain valuable information, manually processing them is extremely time-consuming when an expedient response is necessary. To improve this process, effective summarization models are a valuable tool for humanitarian response experts as they provide digestible overviews of essential information in secondary data. This paper focuses on extractive summarization for the humanitarian response domain and describes and makes public a new multilingual data collection for this purpose. The collection {--} called MultiHumES{--} provides multilingual documents coupled with informative snippets that have been annotated by humanitarian analysts over the past four years. We report the performance results of a recent neural networks-based summarization model together with other baselines. We hope that the released data collection can further grow the research on multilingual extractive summarization in the humanitarian response domain.
BERT Prescriptions to Avoid Unwanted Headaches: A Comparison of Transformer Architectures for Adverse Drug Event Detection
Authors: Beatrice Portelli, Edoardo Lenzi, Emmanuele Chersoni, Giuseppe Serra, Enrico Santus
Date: 2021-04-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Event Detection

Pretrained transformer-based models, such as BERT and its variants, have become a common choice to obtain state-of-the-art performances in NLP tasks. In the identification of Adverse Drug Events (ADE) from social media texts, for example, BERT architectures rank first in the leaderboard. However, a systematic comparison between these models has not yet been done. In this paper, we aim at shedding light on the differences between their performance analyzing the results of 12 models, tested on two standard benchmarks. SpanBERT and PubMedBERT emerged as the best models in our evaluation: this result clearly shows that span-based pretraining gives a decisive advantage in the precise recognition of ADEs, and that in-domain language pretraining is particularly useful when the transformer model is trained just on biomedical text from scratch.