pwc-archive/papers-with-abstracts huggingface dataset:
How coherent are neural models of coherence?
Authors: Leila Pishdad, Federico Fancellu, Ran Zhang, Afsaneh Fazly
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8

Despite the recent advances in coherence modelling, most such models including state-of-the-art neural ones, are evaluated on either contrived proxy tasks such as the standard order discrimination benchmark, or tasks that require special expert annotation. Moreover, most evaluations are conducted on small newswire corpora. To address these shortcomings, in this paper we propose four generic evaluation tasks that draw on different aspects of coherence at both the lexical and document levels, and can be applied to any corpora. In designing these tasks, we aim at capturing coherence-specific properties, such as the correct use of discourse connectives, lexical cohesion, as well as the overall temporal and causal consistency among events and participants in a story. Importantly, our proposed tasks either rely on automatically-generated data, or data annotated for other purposes, hence alleviating the need for annotation specifically targeted to the task of coherence modelling. We perform experiments with several existing state-of-the-art neural models of coherence on these tasks, across large corpora from different domains, including newswire, dialogue, as well as narrative and instructional text. Our findings point to a strong need for revisiting the common practices in the development and evaluation of coherence models.
Fact vs. Opinion: the Role of Argumentation Features in News Classification
Authors: Tariq Alhindi, Smaranda Muresan, Daniel Preotiuc-Pietro
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Articles, Event Extraction, Fact Checking, News Classification

A 2018 study led by the Media Insight Project showed that most journalists think that a clearmarking of what is news reporting and what is commentary or opinion (e.g., editorial, op-ed)is essential for gaining public trust. We present an approach to classify news articles into newsstories (i.e., reporting of factual information) and opinion pieces using models that aim to sup-plement the article content representation with argumentation features. Our hypothesis is thatthe nature of argumentative discourse is important in distinguishing between news stories andopinion articles. We show that argumentation features outperform linguistic features used previ-ously and improve on fine-tuned transformer-based models when tested on data from publishersunseen in training. Automatically flagging opinion pieces vs. news stories can aid applicationssuch as fact-checking or event extraction.
Text Classification by Contrastive Learning and Cross-lingual Data Augmentation for Alzheimer's Disease Detection
Authors: Zhiqiang Guo, Zhaoci Liu, ZhenHua Ling, Shijin Wang, Lingjing Jin, Yunxia Li
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Alzheimer's Disease Detection, Contrastive Learning, Data Augmentation, text-classification, Text Classification

Data scarcity is always a constraint on analyzing speech transcriptions for automatic Alzheimer{'}s disease (AD) detection, especially when the subjects are non-English speakers. To deal with this issue, this paper first proposes a contrastive learning method to obtain effective representations for text classification based on monolingual embeddings of BERT. Furthermore, a cross-lingual data augmentation method is designed by building autoencoders to learn the text representations shared by both languages. Experiments on a Mandarin AD corpus show that the contrastive learning method can achieve better detection accuracy than conventional CNN-based and BERTbased methods. Our cross-lingual data augmentation method also outperforms other compared methods when using another English AD corpus for augmentation. Finally, a best detection accuracy of 81.6{\%} is obtained by our proposed methods on the Mandarin AD corpus.
Multilingual Epidemiological Text Classification: A Comparative Study
Authors: Stephen Mutuvi, Emanuela Boros, Antoine Doucet, Adam Jatowt, Ga{\"e}l Lejeune, Moses Odeo
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Articles, Classification, Multilingual text classification, text-classification, Text Classification

In this paper, we approach the multilingual text classification task in the context of the epidemiological field. Multilingual text classification models tend to perform differently across different languages (low- or high-resourced), more particularly when the dataset is highly imbalanced, which is the case for epidemiological datasets. We conduct a comparative study of different machine and deep learning text classification models using a dataset comprising news articles related to epidemic outbreaks from six languages, four low-resourced and two high-resourced, in order to analyze the influence of the nature of the language, the structure of the document, and the size of the data. Our findings indicate that the performance of the models based on fine-tuned language models exceeds by more than 50{\%} the chosen baseline models that include a specialized epidemiological news surveillance system and several machine learning models. Also, low-resource languages are highly influenced not only by the typology of the languages on which the models have been pre-trained or/and fine-tuned but also by their size. Furthermore, we discover that the beginning and the end of documents provide the most salient features for this task and, as expected, the performance of the models was proportionate to the training data size.
Pointing to Select: A Fast Pointer-LSTM for Long Text Classification
Authors: Jinhua Du, Yan Huang, Karo Moilanen
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: text-classification, Text Classification

Recurrent neural networks (RNNs) suffer from well-known limitations and complications which include slow inference and vanishing gradients when processing long sequences in text classification. Recent studies have attempted to accelerate RNNs via various ad hoc mechanisms to skip irrelevant words in the input. However, word skipping approaches proposed to date effectively stop at each or a given time step to decide whether or not a given input word should be skipped, breaking the coherence of input processing in RNNs. Furthermore, current methods cannot change skip rates during inference and are consequently unable to support different skip rates in demanding real-world conditions. To overcome these limitations, we propose Pointer- LSTM, a novel LSTM framework which relies on a pointer network to select important words for target prediction. The model maintains a coherent input process for the LSTM modules and makes it possible to change the skip rate during inference. Our evaluation on four public data sets demonstrates that Pointer-LSTM (a) is 1.1x∼3.5x faster than the standard LSTM architecture; (b) is more accurate than Leap-LSTM (the state-of-the-art LSTM skipping model) at high skip rates; and (c) reaches robust accuracy levels even when the skip rate is changed during inference.
Explainable and Sparse Representations of Academic Articles for Knowledge Exploration
Authors: Keng-Te Liao, Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, PoChun Chen, Kuansan Wang, Shou-De Lin
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Articles

We focus on a recently deployed system built for summarizing academic articles by concept tagging. The system has shown great coverage and high accuracy of concept identification which could be contributed by the knowledge acquired from millions of publications. Provided with the interpretable concepts and knowledge encoded in a pre-trained neural model, we investigate whether the tagged concepts can be applied to a broader class of applications. We propose transforming the tagged concepts into sparse vectors as representations of academic documents. The effectiveness of the representations is analyzed theoretically by a proposed framework. We also empirically show that the representations can have advantages on academic topic discovery and paper recommendation. On these applications, we reveal that the knowledge encoded in the tagging system can be effectively utilized and can help infer additional features from data with limited information.
``What is on your mind?'' Automated Scoring of Mindreading in Childhood and Early Adolescence
Authors: Venelin Kovatchev, Phillip Smith, Mark Lee, Imogen Grumley Traynor, Irene Luque Aguilera, Rory Devine
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8

In this paper we present the first work on the automated scoring of mindreading ability in middle childhood and early adolescence. We create MIND-CA, a new corpus of 11,311 question-answer pairs in English from 1,066 children aged from 7 to 14. We perform machine learning experiments and carry out extensive quantitative and qualitative evaluation. We obtain promising results, demonstrating the applicability of state-of-the-art NLP solutions to a new domain and task.
A Deep Metric Learning Method for Biomedical Passage Retrieval
Authors: Andr{\'e}s Rosso-Mateus, Fabio A. Gonz{\'a}lez, Manuel Montes-y-G{\'o}mez
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Metric Learning, Passage Retrieval, Retrieval, Triplet, valid

Passage retrieval is the task of identifying text snippets that are valid answers for a natural language posed question. One way to address this problem is to look at it as a metric learning problem, where we want to induce a metric between questions and passages that assign smaller distances to more relevant passages. In this work, we present a novel method for passage retrieval that learns a metric for questions and passages based on their internal semantic interactions. The method uses a similar approach to that of triplet networks, where the training samples are composed of one anchor (the question) and two positive and negative samples (passages). However,and in contrast with triplet networks, the proposed method uses a novel deep architecture that better exploits the particularities of text and takes into consideration complementary relatedness measures. Besides, the paper presents a sampling strategy that selects both easy and hard negative samples which improves the accuracy of the trained model. The method is particularly well suited for domain-specific passage retrieval where it is very important to take into account different sources of information. The proposed approach was evaluated in a biomedical passage retrieval task, the BioASQ challenge, outperforming standard triplet loss substantially by 10{\%},and state-of-the-art performance by 26{\%}.
Hierarchical Text Segmentation for Medieval Manuscripts
Authors: Amir Hazem, Beatrice Daille, Dominique Stutzmann, Christopher Kermorvant, Louis Chevalier
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Hierarchical Text Segmentation, Segmentation, Sentence, Text Segmentation

In this paper, we address the segmentation of books of hours, Latin devotional manuscripts of the late Middle Ages, that exhibit challenging issues: a complex hierarchical entangled structure, variable content, noisy transcriptions with no sentence markers, and strong correlations between sections for which topical information is no longer sufficient to draw segmentation boundaries. We show that the main state-of-the-art segmentation methods are either inefficient or inapplicable for books of hours and propose a bottom-up greedy approach that considerably enhances the segmentation results. We stress the importance of such hierarchical segmentation of books of hours for historians to explore their overarching differences underlying conception about Church.
Are We Ready for this Disaster? Towards Location Mention Recognition from Crisis Tweets
Authors: Reem Suwaileh, Muhammad Imran, Tamer Elsayed, Hassan Sajjad
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Articles, Management

The widespread usage of Twitter during emergencies has provided a new opportunity and timely resource to crisis responders for various disaster management tasks. Geolocation information of pertinent tweets is crucial for gaining situational awareness and delivering aid. However, the majority of tweets do not come with geoinformation. In this work, we focus on the task of location mention recognition from crisis-related tweets. Specifically, we investigate the influence of different types of labeled training data on the performance of a BERT-based classification model. We explore several training settings such as combing in- and out-domain data from news articles and general-purpose and crisis-related tweets. Furthermore, we investigate the effect of geospatial proximity while training on near or far-away events from the target event. Using five different datasets, our extensive experiments provide answers to several critical research questions that are useful for the research community to foster research in this important direction. For example, results show that, for training a location mention recognition model, Twitter-based data is preferred over general-purpose data; and crisis-related data is preferred over general-purpose Twitter data. Furthermore, training on data from geographically-nearby disaster events to the target event boosts the performance compared to training on distant events.
Mining Crowdsourcing Problems from Discussion Forums of Workers
Authors: Zahra Nouri, Henning Wachsmuth, Gregor Engels
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8

Crowdsourcing is used in academia and industry to solve tasks that are easy for humans but hard for computers, in natural language processing mostly to annotate data. The quality of annotations is affected by problems in the task design, task operation, and task evaluation that workers face with requesters in crowdsourcing processes. To learn about the major problems, we provide a short but comprehensive survey based on two complementary studies: (1) a literature review where we collect and organize problems known from interviews with workers, and (2) an empirical data analysis where we use topic modeling to mine workers{'} complaints from a new English corpus of workers{'} forum discussions. While literature covers all process phases, problems in the task evaluation are prevalent, including unfair rejections, late payments, and unjustified blockings of workers. According to the data, however, poor task design in terms of malfunctioning environments, bad workload estimation, and privacy violations seems to bother the workers most. Our findings form the basis for future research on how to improve crowdsourcing processes.
A Survey of Automatic Personality Detection from Texts
Authors: Sanja Stajner, Seren Yenikent
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Survey

Personality profiling has long been used in psychology to predict life outcomes. Recently, automatic detection of personality traits from written messages has gained significant attention in computational linguistics and natural language processing communities, due to its applicability in various fields. In this survey, we show the trajectory of research towards automatic personality detection from purely psychology approaches, through psycholinguistics, to the recent purely natural language processing approaches on large datasets automatically extracted from social media. We point out what has been gained and what lost during that trajectory, and show what can be realistic expectations in the field.
Mama/Papa, Is this Text for Me?
Authors: Rashedur Rahman, Gw{\'e}nol{\'e} Lecorv{\'e}, Aline {\'E}tienne, Delphine Battistelli, Nicolas B{\'e}chet, Jonathan Chevelu
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Sentence

Children have less linguistic skills than adults, which makes it more difficult for them to understand some texts, for instance when browsing the Internet. In this context, we present a novel method which predicts the minimal age from which a text can be understood. This method analyses each sentence of a text using a recurrent neural network, and then aggregates this information to provide the text-level prediction. Different approaches are proposed and compared to baseline models, at sentence and text levels. Experiments are carried out on a corpus of 1, 500 texts and 160K sentences. Our best model, based on LSTMs, outperforms state-of-the-art results and achieves mean absolute errors of 1.86 and 2.28, at sentence and text levels, respectively.
HateGAN: Adversarial Generative-Based Data Augmentation for Hate Speech Detection
Authors: Rui Cao, Roy Ka-Wei Lee
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Data Augmentation, Hate Speech Detection

Academia and industry have developed machine learning and natural language processing models to detect online hate speech automatically. However, most of these existing methods adopt a supervised approach that heavily depends on labeled datasets for training. This results in the methods{'} poor detection performance of the hate speech class as the training datasets are highly imbalanced. In this paper, we propose HateGAN, a deep generative reinforcement learning model, which addresses the challenge of imbalance class by augmenting the dataset with hateful tweets. We conduct extensive experiments to augment two commonly-used hate speech detection datasets with the HateGAN generated tweets. Our experiment results show that HateGAN improves the detection performance of the hate speech class regardless of the classifiers and datasets used in the detection task. Specifically, we observe an average 5{\%} improvement for the hate class F1 scores across all state-of-the-art hate speech classifiers. We also conduct case studies to empirically examine the HateGAN generated hate speeches and show that the generated tweets are diverse, coherent, and relevant to hate speech detection.
Evaluating Unsupervised Representation Learning for Detecting Stances of Fake News
Authors: Maike Guderlei, Matthias A{\ss}enmacher
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Representation Learning, Stance Detection, Transfer Learning

Our goal is to evaluate the usefulness of unsupervised representation learning techniques for detecting stances of Fake News. Therefore we examine several pre-trained language models with respect to their performance on two Fake News related data sets, both consisting of instances with a headline, an associated news article and the stance of the article towards the respective headline. Specifically, the aim is to understand how much hyperparameter tuning is necessary when fine-tuning the pre-trained architectures, how well transfer learning works in this specific case of stance detection and how sensitive the models are to changes in hyperparameters like batch size, learning rate (schedule), sequence length as well as the freezing technique. The results indicate that the computationally more expensive autoregression approach of XLNet (Yanget al., 2019) is outperformed by BERT-based models, notably by RoBERTa (Liu et al., 2019).While the learning rate seems to be the most important hyperparameter, experiments with different freezing techniques indicate that all evaluated architectures had already learned powerful language representations that pose a good starting point for fine-tuning them.
XHate-999: Analyzing and Detecting Abusive Language Across Domains and Languages
Authors: Goran Glava{\v{s}}, Mladen Karan, Ivan Vuli{\'c}
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Abusive Language, Disentanglement, Language Modeling, Language Modelling, Masked Language Modeling

We present XHate-999, a multi-domain and multilingual evaluation data set for abusive language detection. By aligning test instances across six typologically diverse languages, XHate-999 for the first time allows for disentanglement of the domain transfer and language transfer effects in abusive language detection. We conduct a series of domain- and language-transfer experiments with state-of-the-art monolingual and multilingual transformer models, setting strong baseline results and profiling XHate-999 as a comprehensive evaluation resource for abusive language detection. Finally, we show that domain- and language-adaption, via intermediate masked language modeling on abusive corpora in the target language, can lead to substantially improved abusive language detection in the target language in the zero-shot transfer setups.
Modeling Evolution of Message Interaction for Rumor Resolution
Authors: Lei Chen, Zhongyu Wei, Jing Li, Baohua Zhou, Qi Zhang, Xuanjing Huang
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Time Series, Time Series Analysis

Previous work for rumor resolution concentrates on exploiting time-series characteristics or modeling topology structure separately. However, how local interactive pattern affects global information assemblage has not been explored. In this paper, we attempt to address the problem by learning evolution of message interaction. We model confrontation and reciprocity between message pairs via discrete variational autoencoders which effectively reflects the diversified opinion interactivity. Moreover, we capture the variation of message interaction using a hierarchical framework to better integrate information flow of a rumor cascade. Experiments on PHEME dataset demonstrate our proposed model achieves higher accuracy than existing methods.
Graph Convolution over Multiple Dependency Sub-graphs for Relation Extraction
Authors: Angrosh Mandya, Danushka Bollegala, Frans Coenen
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Relation, Relation Extraction, Sentence

We propose a contextualised graph convolution network over multiple dependency-based sub-graphs for relation extraction. A novel method to construct multiple sub-graphs using words in shortest dependency path and words linked to entities in the dependency parse is proposed. Graph convolution operation is performed over the resulting multiple sub-graphs to obtain more informative features useful for relation extraction. Our experimental results show that the proposed method achieves superior performance over the existing GCN-based models achieving state-of-the-art performance on cross-sentence n-ary relation extraction dataset and SemEval 2010 Task 8 sentence-level relation extraction dataset. Our model also achieves a comparable performance to the SoTA on the TACRED dataset.
Multi-choice Relational Reasoning for Machine Reading Comprehension
Authors: Wuya Chen, Xiaojun Quan, Chunyu Kit, Zhengcheng Min, Jiahai Wang
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Machine Reading Comprehension, Reading Comprehension, Relational Reasoning

This paper presents our study of cloze-style reading comprehension by imitating human reading comprehension, which normally involves tactical comparing and reasoning over candidates while choosing the best answer. We propose a multi-choice relational reasoning (McR$^2$) model with an aim to enable relational reasoning on candidates based on fusion representations of document, query and candidates. For the fusion representations, we develop an efficient encoding architecture by integrating the schemes of bidirectional attention flow, self-attention and document-gated query reading. Then, comparing and inferring over candidates are executed by a novel relational reasoning network. We conduct extensive experiments on four datasets derived from two public corpora, Children{'}s Book Test and Who DiD What, to verify the validity and advantages of our model. The results show that it outperforms all baseline models significantly on the four benchmark datasets. The effectiveness of its key components is also validated by an ablation study.