ymoslem/acl-abstracts huggingface dataset:
Analysing concatenation approaches to document-level NMT in two different domains (2019)

In this paper, we investigate how different aspects of discourse context affect the performance of recent neural MT systems. We describe two popular datasets covering news and movie subtitles and we provide a thorough analysis of the distribution of various document-level features in their domains. Furthermore, we train a set of context-aware MT models on both datasets and propose a comparative evaluation scheme that contrasts coherent context with artificially scrambled documents and absent context, arguing that the impact of discourse-aware MT models will become visible in this way. Our results show that the models are indeed affected by the manipulation of the test data, providing a different view on document-level translation quality than absolute sentence-level scores.
The FEVER2.0 Shared Task (2019)

We present the results of the second Fact Extraction and VERification (FEVER2.0) Shared Task. The task challenged participants to both build systems to verify factoid claims using evidence retrieved from Wikipedia and to generate adversarial attacks against other participant`s systems. The shared task had three phases: \textit{building, breaking and fixing}. There were 8 systems in the builder`s round, three of which were new qualifying submissions for this shared task, and 5 adversaries generated instances designed to induce classification errors and one builder submitted a fixed system which had higher FEVER score and resilience than their first submission. All but one newly submitted systems attained FEVER scores higher than the best performing system from the first shared task and under adversarial evaluation, all systems exhibited losses in FEVER score. There was a great variety in adversarial attack types as well as the techniques used to generate the attacks, In this paper, we present the results of the shared task and a summary of the systems, highlighting commonalities and innovations among participating systems.
Fact Checking or Psycholinguistics: How to Distinguish Fake and True Claims? (2019)

The goal of our paper is to compare psycholinguistic text features with fact checking approaches to distinguish lies from true statements. We examine both methods using data from a large ongoing study on deception and deception detection covering a mixture of factual and opinionated topics that polarize public opinion. We conclude that fact checking approaches based on Wikipedia are too limited for this task, as only a few percent of sentences from our study has enough evidence to become supported or refuted. Psycholinguistic features turn out to outperform both fact checking and human baselines, but the accuracy is not high. Overall, it appears that deception detection applicable to less-than-obvious topics is a difficult task and a problem to be solved.
Neural Multi-Task Learning for Stance Prediction (2019)

We present a multi-task learning model that leverages large amount of textual information from existing datasets to improve stance prediction. In particular, we utilize multiple NLP tasks under both unsupervised and supervised settings for the target stance prediction task. Our model obtains state-of-the-art performance on a public benchmark dataset, Fake News Challenge, outperforming current approaches by a wide margin.
GEM: Generative Enhanced Model for adversarial attacks (2019)

We present our Generative Enhanced Model (GEM) that we used to create samples awarded the first prize on the FEVER 2.0 Breakers Task. GEM is the extended language model developed upon GPT-2 architecture. The addition of novel target vocabulary input to the already existing context input enabled controlled text generation. The training procedure resulted in creating a model that inherited the knowledge of pretrained GPT-2, and therefore was ready to generate natural-like English sentences in the task domain with some additional control. As a result, GEM generated malicious claims that mixed facts from various articles, so it became difficult to classify their truthfulness.
Unsupervised Natural Question Answering with a Small Model (2019)

The recent demonstration of the power of huge language models such as GPT-2 to memorise the answers to factoid questions raises questions about the extent to which knowledge is being embedded directly within these large models. This short paper describes an architecture through which much smaller models can also answer such questions - by making use of {\textquoteleft}raw' external knowledge. The contribution of this work is that the methods presented here rely on unsupervised learning techniques, complementing the unsupervised training of the Language Model. The goal of this line of research is to be able to add knowledge explicitly, without extensive training.
Scalable Knowledge Graph Construction from Text Collections (2019)

We present a scalable, open-source platform that {\textquotedblleft}distills{\textquotedblright} a potentially large text collection into a knowledge graph. Our platform takes documents stored in Apache Solr and scales out the Stanford CoreNLP toolkit via Apache Spark integration to extract mentions and relations that are then ingested into the Neo4j graph database. The raw knowledge graph is then enriched with facts extracted from an external knowledge graph. The complete product can be manipulated by various applications using Neo4j`s native Cypher query language: We present a subgraph-matching approach to align extracted relations with external facts and show that fact verification, locating textual support for asserted facts, detecting inconsistent and missing facts, and extracting distantly-supervised training data can all be performed within the same framework.
Relation Extraction among Multiple Entities Using a Dual Pointer Network with a Multi-Head Attention Mechanism (2019)

Many previous studies on relation extrac-tion have been focused on finding only one relation between two entities in a single sentence. However, we can easily find the fact that multiple entities exist in a single sentence and the entities form multiple relations. To resolve this prob-lem, we propose a relation extraction model based on a dual pointer network with a multi-head attention mechanism. The proposed model finds n-to-1 subject-object relations by using a forward de-coder called an object decoder. Then, it finds 1-to-n subject-object relations by using a backward decoder called a sub-ject decoder. In the experiments with the ACE-05 dataset and the NYT dataset, the proposed model achieved the state-of-the-art performances (F1-score of 80.5{\%} in the ACE-05 dataset, F1-score of 78.3{\%} in the NYT dataset)
Unsupervised Question Answering for Fact-Checking (2019)

Recent Deep Learning (DL) models have succeeded in achieving human-level accuracy on various natural language tasks such as question-answering, natural language inference (NLI), and textual entailment. These tasks not only require the contextual knowledge but also the reasoning abilities to be solved efficiently. In this paper, we propose an unsupervised question-answering based approach for a similar task, fact-checking. We transform the FEVER dataset into a Cloze-task by masking named entities provided in the claims. To predict the answer token, we utilize pre-trained Bidirectional Encoder Representations from Transformers (BERT). The classifier computes label based on the correctly answered questions and a threshold. Currently, the classifier is able to classify the claims as {\textquotedblleft}SUPPORTS{\textquotedblright} and {\textquotedblleft}MANUAL{\_}REVIEW{\textquotedblright}. This approach achieves a label accuracy of 80.2{\%} on the development set and 80.25{\%} on the test set of the transformed dataset.
Improving Evidence Detection by Leveraging Warrants (2019)

Recognizing the implicit link between a claim and a piece of evidence (i.e. warrant) is the key to improving the performance of evidence detection. In this work, we explore the effectiveness of automatically extracted warrants for evidence detection. Given a claim and candidate evidence, our proposed method extracts multiple warrants via similarity search from an existing, structured corpus of arguments. We then attentively aggregate the extracted warrants, considering the consistency between the given argument and the acquired warrants. Although a qualitative analysis on the warrants shows that the extraction method needs to be improved, our results indicate that our method can still improve the performance of evidence detection.
Hybrid Models for Aspects Extraction without Labelled Dataset (2019)

One of the important tasks in opinion mining is to extract aspects of the opinion target. Aspects are features or characteristics of the opinion target that are being reviewed, which can be categorised into explicit and implicit aspects. Extracting aspects from opinions is essential in order to ensure accurate information about certain attributes of an opinion target is retrieved. For instance, a professional camera receives a positive feedback in terms of its functionalities in a review, but its overly high price receives negative feedback. Most of the existing solutions focus on explicit aspects. However, sentences in reviews normally do not state the aspects explicitly. In this research, two hybrid models are proposed to identify and extract both explicit and implicit aspects, namely TDM-DC and TDM-TED. The proposed models combine topic modelling and dictionary-based approach. The models are unsupervised as they do not require any labelled dataset. The experimental results show that TDM-DC achieves F1-measure of 58.70{\%}, where it outperforms both the baseline topic model and dictionary-based approach. In comparison to other existing unsupervised techniques, the proposed models are able to achieve higher F1-measure by approximately 3{\%}. Although the supervised techniques perform slightly better, the proposed models are domain-independent, and hence more versatile.
Extract and Aggregate: A Novel Domain-Independent Approach to Factual Data Verification (2019)

Triggered by Internet development, a large amount of information is published in online sources. However, it is a well-known fact that publications are inundated with inaccurate data. That is why fact-checking has become a significant topic in the last 5 years. It is widely accepted that factual data verification is a challenge even for the experts. This paper presents a domain-independent fact checking system. It can solve the fact verification problem entirely or at the individual stages. The proposed model combines various advanced methods of text data analysis, such as BERT and Infersent. The theoretical and empirical study of the system features is carried out. Based on FEVER and Fact Checking Challenge test-collections, experimental results demonstrate that our model can achieve the score on a par with state-of-the-art models designed by the specificity of particular datasets.
Interactive Evidence Detection: train state-of-the-art model out-of-domain or simple model interactively? (2019)

Finding evidence is of vital importance in research as well as fact checking and an evidence detection method would be useful in speeding up this process. However, when addressing a new topic there is no training data and there are two approaches to get started. One could use large amounts of out-of-domain data to train a state-of-the-art method, or to use the small data that a person creates while working on the topic. In this paper, we address this problem in two steps. First, by simulating users who read source documents and label sentences they can use as evidence, thereby creating small amounts of training data for an interactively trained evidence detection model; and second, by comparing such an interactively trained model against a pre-trained model that has been trained on large out-of-domain data. We found that an interactively trained model not only often out-performs a state-of-the-art model but also requires significantly lower amounts of computational resources. Therefore, especially when computational resources are scarce, e.g. no GPU available, training a smaller model on the fly is preferable to training a well generalising but resource hungry out-of-domain model.
Veritas Annotator: Discovering the Origin of a Rumour (2019)

Defined as the intentional or unintentionalspread of false information (K et al., 2019)through context and/or content manipulation,fake news has become one of the most seriousproblems associated with online information(Waldrop, 2017). Consequently, it comes asno surprise that Fake News Detection hasbecome one of the major foci of variousfields of machine learning and while machinelearning models have allowed individualsand companies to automate decision-basedprocesses that were once thought to be onlydoable by humans, it is no secret that thereal-life applications of such models are notviable without the existence of an adequatetraining dataset. In this paper we describethe Veritas Annotator, a web application formanually identifying the origin of a rumour. These rumours, often referred as claims,were previously checked for validity byFact-Checking Agencies.
FEVER Breaker`s Run of Team NbAuzDrLqg (2019)

We describe our submission for the Breaker phase of the second Fact Extraction and VERification (FEVER) Shared Task. Our adversarial data can be explained by two perspectives. First, we aimed at testing model`s ability to retrieve evidence, when appropriate query terms could not be easily generated from the claim. Second, we test model`s ability to precisely understand the implications of the texts, which we expect to be rare in FEVER 1.0 dataset. Overall, we suggested six types of adversarial attacks. The evaluation on the submitted systems showed that the systems were only able get both the evidence and label correct in 20{\%} of the data. We also demonstrate our adversarial run analysis in the data development process.
Team DOMLIN: Exploiting Evidence Enhancement for the FEVER Shared Task (2019)

This paper contains our system description for the second Fact Extraction and VERification (FEVER) challenge. We propose a two-staged sentence selection strategy to account for examples in the dataset where evidence is not only conditioned on the claim, but also on previously retrieved evidence. We use a publicly available document retrieval module and have fine-tuned BERT checkpoints for sentence se- lection and as the entailment classifier. We report a FEVER score of 68.46{\%} on the blind testset.
Team GPLSI. Approach for automated fact checking (2019)

Fever Shared 2.0 Task is a challenge meant for developing automated fact checking systems. Our approach for the Fever 2.0 is based on a previous proposal developed by Team Athene UKP TU Darmstadt. Our proposal modifies the sentence retrieval phase, using statement extraction and representation in the form of triplets (subject, object, action). Triplets are extracted from the claim and compare to triplets extracted from Wikipedia articles using semantic similarity. Our results are satisfactory but there is room for improvement.
Apprentissage de plongements de mots dynamiques avec r\'egularisation de la d\'erive (Learning dynamic word embeddings with drift regularisation) (2019)

L`usage, le sens et la connotation des mots peuvent changer au cours du temps. Les plongements lexicaux diachroniques permettent de mod{\'e}liser ces changements de mani{\`e}re non supervis{\'e}e. Dans cet article nous {\'e}tudions l`impact de plusieurs fonctions de co{\^u}t sur l`apprentissage de plongements dynamiques, en comparant les comportements de variantes du mod{\`e}le Dynamic Bernoulli Embeddings. Les plongements dynamiques sont estim{\'e}s sur deux corpus couvrant les m{\^e}mes deux d{\'e}cennies, le New York Times Annotated Corpus en anglais et une s{\'e}lection d`articles du journal Le Monde en fran{\c{c}}ais, ce qui nous permet de mettre en place un processus d`analyse bilingue de l'{\'e}volution de l`usage des mots.
Apprentissage de plongements lexicaux par une approche r\'eseaux complexes (Complex networks based word embeddings) (2019)

La litt{\'e}rature des r{\'e}seaux complexes a montr{\'e} la pertinence de l'{\'e}tude de la langue sous forme de r{\'e}seau pour diff{\'e}rentes applications : d{\'e}sambigu{\"i}sation, r{\'e}sum{\'e} automatique, classification des langues, etc. Cette m{\^e}me litt{\'e}rature a d{\'e}montr{\'e} que les r{\'e}seaux de co-occurrences de mots poss{\`e}dent une structure de communaut{\'e}s latente. Nous formulons l`hypoth{\`e}se que cette structuration du r{\'e}seau sous forme de communaut{\'e}s est utile pour travailler sur la s{\'e}mantique d`une langue et introduisons donc dans cet article une m{\'e}thode d`apprentissage de plongements originale bas{\'e}e sur cette hypoth{\`e}se. Cette hypoth{\`e}se est coh{\'e}rente avec la proximit{\'e} qui existe entre la d{\'e}tection de communaut{\'e}s sur un r{\'e}seau de co-occurrences et la factorisation d`une matrice de co-occurrences, m{\'e}thode couramment utilis{\'e}e pour l`apprentissage de plongements lexicaux. Nous d{\'e}crivons notre m{\'e}thode structur{\'e}e en trois {\'e}tapes : construction et pr{\'e}-traitement du r{\'e}seau, d{\'e}tection de la structure de communaut{\'e}s, construction des plongements de mots {\`a} partir de cette structure. Apr{\`e}s avoir d{\'e}crit cette nouvelle m{\'e}thodologie, nous montrons la pertinence de notre approche avec des premiers r{\'e}sultats d'{\'e}valuation sur les t{\^a}ches de cat{\'e}gorisation et de similarit{\'e}. Enfin, nous discutons des perspectives importantes d`un tel mod{\`e}le issu des r{\'e}seaux complexes : les dimensions du mod{\`e}le (les communaut{\'e}s) semblent interpr{\'e}tables, l`apprentissage est rapide, la construction d`un nouveau plongement est presque instantan{\'e}e, et il est envisageable d`en exp{\'e}rimenter une version incr{\'e}mentale pour travailler sur des corpus textuels temporels.
Comparaison qualitative et extrins\`eque d`analyseurs syntaxiques du fran\ccais : confrontation de mod\`eles distributionnels sur un corpus sp\'ecialis\'e (Extrinsic evaluation of French dependency parsers on a specialised corpus : comparison of distributional thesauri ) (2019)

Nous pr{\'e}sentons une {\'e}tude visant {\`a} comparer 11 diff{\'e}rents analyseurs en d{\'e}pendances du fran{\c{c}}ais sur un corpus sp{\'e}cialis{\'e} (constitu{\'e} des archives des articles de la conf{\'e}rence TALN). En l`absence de gold standard, nous utilisons chacune des sorties de ces analyseurs pour construire des th{\'e}saurus distributionnels en utilisant une m{\'e}thode {\`a} base de fr{\'e}quence. Nous comparons ces 11 th{\'e}saurus afin de proposer un premier aper{\c{c}}u de l`impact du choix d`un analyseur par rapport {\`a} un autre.
Compression de vocabulaire de sens gr\^ace aux relations s\'emantiques pour la d\'esambigu\"isation lexicale (Sense Vocabulary Compression through Semantic Knowledge for Word Sense Disambiguation) (2019)

En D{\'e}sambigu{\"i}sation Lexicale (DL), les syst{\`e}mes supervis{\'e}s dominent largement les campagnes d'{\'e}valuation. La performance et la couverture de ces syst{\`e}mes sont cependant rapidement limit{\'e}s par la faible quantit{\'e} de corpus annot{\'e}s en sens disponibles. Dans cet article, nous pr{\'e}sentons deux nouvelles m{\'e}thodes qui visent {\`a} r{\'e}soudre ce probl{\`e}me en exploitant les relations s{\'e}mantiques entre les sens tels que la synonymie, l`hyperonymie et l`hyponymie, afin de compresser le vocabulaire de sens de WordNet, et ainsi r{\'e}duire le nombre d'{\'e}tiquettes diff{\'e}rentes n{\'e}cessaires pour pouvoir d{\'e}sambigu{\"i}ser tous les mots de la base lexicale. Nos m{\'e}thodes permettent de r{\'e}duire consid{\'e}rablement la taille des mod{\`e}les de DL neuronaux, avec l`avantage d`am{\'e}liorer leur couverture sans donn{\'e}es suppl{\'e}mentaires, et sans impacter leur pr{\'e}cision. En plus de nos m{\'e}thodes, nous pr{\'e}sentons un syst{\`e}me de DL qui tire parti des r{\'e}cents travaux sur les repr{\'e}sentations vectorielles de mots contextualis{\'e}es, afin d`obtenir des r{\'e}sultats qui surpassent largement l'{\'e}tat de l`art sur toutes les t{\^a}ches d'{\'e}valuation de la DL.
Corpus annot\'e de cas cliniques en fran\ccais (Annotated corpus with clinical cases in French) (2019)

Les corpus textuels sont utiles pour diverses applications de traitement automatique des langues (TAL) en fournissant les donn{\'e}es n{\'e}cessaires pour leur cr{\'e}ation, adaptation ou {\'e}valuation. Cependant, dans certains domaines comme le domaine m{\'e}dical, l`acc{\`e}s aux donn{\'e}es est rendu compliqu{\'e}, voire impossible, pour des raisons de confidentialit{\'e} et d'{\'e}thique. Il existe n{\'e}anmoins de r{\'e}els besoins en corpus cliniques pour l`enseignement et la recherche. Pour r{\'e}pondre {\`a} ce d{\'e}fi, nous pr{\'e}sentons dans cet article le corpus CAS contenant des cas cliniques de patients, r{\'e}els ou fictifs, que nous avons compil{\'e}s. Ces cas cliniques en fran{\c{c}}ais couvrent plusieurs sp{\'e}cialit{\'e}s m{\'e}dicales et focalisent donc sur diff{\'e}rentes situations cliniques. Actuellement, le corpus contient 4 300 cas (environ 1,5M d`occurrences de mots). Il est accompagn{\'e} d`informations (discussions des cas cliniques, mots-cl{\'e}s, etc.) et d`annotations que nous avons effectu{\'e}es au regard des besoins de la recherche en TAL dans ce domaine. Nous pr{\'e}sentons {\'e}galement les r{\'e}sultats de premi{\`e}res exp{\'e}riences de recherche et d`extraction d`information qui ont {\'e}t{\'e} effectu{\'e}es avec ce corpus annot{\'e}. Ces exp{\'e}riences peuvent fournir une baseline {\`a} d`autres chercheurs souhaitant travailler avec les donn{\'e}es.
Curriculum d`apprentissage : reconnaissance d`entit\'es nomm\'ees pour l`extraction de concepts s\'emantiques (Curriculum learning : named entity recognition for semantic concept extraction) (2019)

Dans cet article, nous pr{\'e}sentons une approche de bout en bout d`extraction de concepts s{\'e}mantiques de la parole. En particulier, nous mettons en avant l`apport d`une cha{\^i}ne d`apprentissage successif pilot{\'e}e par une strat{\'e}gie de curriculum d`apprentissage. Dans la cha{\^i}ne d`apprentissage mise en place, nous exploitons des donn{\'e}es fran{\c{c}}aises annot{\'e}es en entit{\'e}s nomm{\'e}es que nous supposons {\^e}tre des concepts plus g{\'e}n{\'e}riques que les concepts s{\'e}mantiques li{\'e}s {\`a} une application informatique sp{\'e}cifique. Dans cette {\'e}tude, il s`agit d`extraire des concepts s{\'e}mantiques dans le cadre de la t{\^a}che MEDIA. Pour renforcer le syst{\`e}me propos{\'e}, nous exploitons aussi des strat{\'e}gies d`augmentation de donn{\'e}es, un mod{\`e}le de langage 5-gramme, ainsi qu`un mode {\'e}toile aidant le syst{\`e}me {\`a} se concentrer sur les concepts et leurs valeurs lors de l`apprentissage. Les r{\'e}sultats montrent un int{\'e}r{\^e}t {\`a} l`utilisation des donn{\'e}es d`entit{\'e}s nomm{\'e}es, permettant un gain relatif allant jusqu'{\`a} 6,5 {\%}.
D\'etection des ellipses dans des corpus de sous-titres en anglais (Ellipsis Detection in English Subtitles Corpora ) (2019)

Cet article pr{\'e}sente une m{\'e}thodologie de d{\'e}tection des ellipses en anglais qui repose sur des patrons combinant des informations sur les tokens, leur {\'e}tiquette morphosyntaxique et leur lemme. Les patrons sont {\'e}valu{\'e}s sur deux corpus de sous-titres. Ces travaux constituent une {\'e}tape pr{\'e}alable {\`a} une {\'e}tude contrastive et multi-genres de l`ellipse.