pwc-archive/papers-with-abstracts huggingface dataset:
Understanding the Effect of Textual Adversaries in Multimodal Machine Translation
Authors: Koel Dutta Chowdhury, Desmond Elliott
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Machine Translation, Multimodal Machine Translation, Sentence, Translation

It is assumed that multimodal machine translation systems are better than text-only systems at translating phrases that have a direct correspondence in the image. This assumption has been challenged in experiments demonstrating that state-of-the-art multimodal systems perform equally well in the presence of randomly selected images, but, more recently, it has been shown that masking entities from the source language sentence during training can help to overcome this problem. In this paper, we conduct experiments with both visual and textual adversaries in order to understand the role of incorrect textual inputs to such systems. Our results show that when the source language sentence contains mistakes, multimodal translation systems do not leverage the additional visual signal to produce the correct translation. We also find that the degradation of translation performance caused by textual adversaries is significantly higher than by visual adversaries.
Learning to request guidance in emergent language
Authors: Benjamin Kolb, Leon Lang, Henning Bartsch, Arwin Gansekoele, Raymond Koopmanschap, Leonardo Romor, David Speck, Mathijs Mul, Elia Bruni
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Imitation Learning

Previous research into agent communication has shown that a pre-trained guide can speed up the learning process of an imitation learning agent. The guide achieves this by providing the agent with discrete messages in an emerged language about how to solve the task. We extend this one-directional communication by a one-bit communication channel from the learner back to the guide: It is able to ask the guide for help, and we limit the guidance by penalizing the learner for these requests. During training, the agent learns to control this gate based on its current observation. We find that the amount of requested guidance decreases over time and guidance is requested in situations of high uncertainty. We investigate the agent{'}s performance in cases of open and closed gates and discuss potential motives for the observed gating behavior.
At a Glance: The Impact of Gaze Aggregation Views on Syntactic Tagging
Authors: Sigrid Klerke, Barbara Plank
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Chunking, Part-Of-Speech Tagging, POS, POS Tagging

Readers{'} eye movements used as part of the training signal have been shown to improve performance in a wide range of Natural Language Processing (NLP) tasks. Previous work uses gaze data either at the type level or at the token level and mostly from a single eye-tracking corpus. In this paper, we analyze type vs token-level integration options with eye tracking data from two corpora to inform two syntactic sequence labeling problems: binary phrase chunking and part-of-speech tagging. We show that using globally-aggregated measures that capture the central tendency or variability of gaze data is more beneficial than proposed local views which retain individual participant information. While gaze data is informative for supervised POS tagging, which complements previous findings on unsupervised POS induction, almost no improvement is obtained for binary phrase chunking, except for a single specific setup. Hence, caution is warranted when using gaze data as signal for NLP, as no single view is robust over tasks, modeling choice and gaze corpus.
Seeded self-play for language learning
Authors: Abhinav Gupta, Ryan Lowe, Jakob Foerster, Douwe Kiela, Joelle Pineau
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Deep Reinforcement Learning, Imitation Learning, Meta-Learning, Reinforcement Learning

How can we teach artificial agents to use human language flexibly to solve problems in real-world environments? We have an example of this in nature: human babies eventually learn to use human language to solve problems, and they are taught with an adult human-in-the-loop. Unfortunately, current machine learning methods (e.g. from deep reinforcement learning) are too data inefficient to learn language in this way. An outstanding goal is finding an algorithm with a suitable {`}language learning prior{'} that allows it to learn human language, while minimizing the number of on-policy human interactions. In this paper, we propose to learn such a prior in simulation using an approach we call, Learning to Learn to Communicate (L2C). Specifically, in L2C we train a meta-learning agent in simulation to interact with populations of pre-trained agents, each with their own distinct communication protocol. Once the meta-learning agent is able to quickly adapt to each population of agents, it can be deployed in new populations, including populations speaking human language. Our key insight is that such populations can be obtained via self-play, after pre-training agents with imitation learning on a small amount of off-policy human language data. We call this latter technique Seeded Self-Play (S2P). Our preliminary experiments show that agents trained with L2C and S2P need fewer on-policy samples to learn a compositional language in a Lewis signaling game.
Context-Aware Neural Machine Translation Decoding
Authors: Eva Mart{\'\i}nez Garcia, Carles Creus, Cristina Espa{\~n}a-Bonet
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Language Modeling, Language Modelling, Machine Translation, Translation, Word Embeddings

This work presents a decoding architecture that fuses the information from a neural translation model and the context semantics enclosed in a semantic space language model based on word embeddings. The method extends the beam search decoding process and therefore can be applied to any neural machine translation framework. With this, we sidestep two drawbacks of current document-level systems: (i) we do not modify the training process so there is no increment in training time, and (ii) we do not require document-level an-notated data. We analyze the impact of the fusion system approach and its parameters on the final translation quality for English{--}Spanish. We obtain consistent and statistically significant improvements in terms of BLEU and METEOR and we observe how the fused systems are able to handle synonyms to propose more adequate translations as well as help the system to disambiguate among several translation candidates for a word.
Data augmentation using back-translation for context-aware neural machine translation
Authors: Amane Sugiyama, Naoki Yoshinaga
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Data Augmentation, Machine Translation, NMT, Sentence, Translation

A single sentence does not always convey information that is enough to translate it into other languages. Some target languages need to add or specialize words that are omitted or ambiguous in the source languages (e.g, zero pronouns in translating Japanese to English or epicene pronouns in translating English to French). To translate such ambiguous sentences, we need contexts beyond a single sentence, and have so far explored context-aware neural machine translation (NMT). However, a large amount of parallel corpora is not easily available to train accurate context-aware NMT models. In this study, we first obtain large-scale pseudo parallel corpora by back-translating monolingual data, and then investigate its impact on the translation accuracy of context-aware NMT models. We evaluated context-aware NMT models trained with small parallel corpora and the large-scale pseudo parallel corpora on English-Japanese and English-French datasets to demonstrate the large impact of the data augmentation for context-aware NMT models.
Context-aware Neural Machine Translation with Coreference Information
Authors: Takumi Ohtani, Hidetaka Kamigaito, Masaaki Nagata, Manabu Okumura
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Machine Translation, Sentence, Translation

We present neural machine translation models for translating a sentence in a text by using a graph-based encoder which can consider coreference relations provided within the text explicitly. The graph-based encoder can dynamically encode the source text without attending to all tokens in the text. In experiments, our proposed models provide statistically significant improvement to the previous approach of at most 0.9 points in the BLEU score on the OpenSubtitle2018 English-to-Japanese data set. Experimental results also show that the graph-based encoder can handle a longer text well, compared with the previous approach.
Analysing concatenation approaches to document-level NMT in two different domains
Authors: Yves Scherrer, J{\"o}rg Tiedemann, Sharid Lo{\'a}iciga
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: NMT, Sentence, Translation, Vocal Bursts Valence Prediction

In this paper, we investigate how different aspects of discourse context affect the performance of recent neural MT systems. We describe two popular datasets covering news and movie subtitles and we provide a thorough analysis of the distribution of various document-level features in their domains. Furthermore, we train a set of context-aware MT models on both datasets and propose a comparative evaluation scheme that contrasts coherent context with artificially scrambled documents and absent context, arguing that the impact of discourse-aware MT models will become visible in this way. Our results show that the models are indeed affected by the manipulation of the test data, providing a different view on document-level translation quality than absolute sentence-level scores.
The FEVER2.0 Shared Task
Authors: James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, Arpit Mittal
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Adversarial Attack

We present the results of the second Fact Extraction and VERification (FEVER2.0) Shared Task. The task challenged participants to both build systems to verify factoid claims using evidence retrieved from Wikipedia and to generate adversarial attacks against other participant{'}s systems. The shared task had three phases: \textit{building, breaking and fixing}. There were 8 systems in the builder{'}s round, three of which were new qualifying submissions for this shared task, and 5 adversaries generated instances designed to induce classification errors and one builder submitted a fixed system which had higher FEVER score and resilience than their first submission. All but one newly submitted systems attained FEVER scores higher than the best performing system from the first shared task and under adversarial evaluation, all systems exhibited losses in FEVER score. There was a great variety in adversarial attack types as well as the techniques used to generate the attacks, In this paper, we present the results of the shared task and a summary of the systems, highlighting commonalities and innovations among participating systems.
Fact Checking or Psycholinguistics: How to Distinguish Fake and True Claims?
Authors: Aleks Wawer, er, Grzegorz Wojdyga, Justyna Sarzy{\'n}ska-Wawer
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Deception Detection, Fact Checking

The goal of our paper is to compare psycholinguistic text features with fact checking approaches to distinguish lies from true statements. We examine both methods using data from a large ongoing study on deception and deception detection covering a mixture of factual and opinionated topics that polarize public opinion. We conclude that fact checking approaches based on Wikipedia are too limited for this task, as only a few percent of sentences from our study has enough evidence to become supported or refuted. Psycholinguistic features turn out to outperform both fact checking and human baselines, but the accuracy is not high. Overall, it appears that deception detection applicable to less-than-obvious topics is a difficult task and a problem to be solved.
Neural Multi-Task Learning for Stance Prediction
Authors: Wei Fang, Moin Nadeem, Mitra Mohtarami, James Glass
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Multi-Task Learning, Prediction

We present a multi-task learning model that leverages large amount of textual information from existing datasets to improve stance prediction. In particular, we utilize multiple NLP tasks under both unsupervised and supervised settings for the target stance prediction task. Our model obtains state-of-the-art performance on a public benchmark dataset, Fake News Challenge, outperforming current approaches by a wide margin.
GEM: Generative Enhanced Model for adversarial attacks
Authors: Piotr Niewinski, Maria Pszona, Maria Janicka
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Articles, Language Modeling, Language Modelling, model, Text Generation

We present our Generative Enhanced Model (GEM) that we used to create samples awarded the first prize on the FEVER 2.0 Breakers Task. GEM is the extended language model developed upon GPT-2 architecture. The addition of novel target vocabulary input to the already existing context input enabled controlled text generation. The training procedure resulted in creating a model that inherited the knowledge of pretrained GPT-2, and therefore was ready to generate natural-like English sentences in the task domain with some additional control. As a result, GEM generated malicious claims that mixed facts from various articles, so it became difficult to classify their truthfulness.
Scalable Knowledge Graph Construction from Text Collections
Authors: Ryan Clancy, Ihab F. Ilyas, Jimmy Lin
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Fact Verification, graph construction

We present a scalable, open-source platform that {``}distills{''} a potentially large text collection into a knowledge graph. Our platform takes documents stored in Apache Solr and scales out the Stanford CoreNLP toolkit via Apache Spark integration to extract mentions and relations that are then ingested into the Neo4j graph database. The raw knowledge graph is then enriched with facts extracted from an external knowledge graph. The complete product can be manipulated by various applications using Neo4j{'}s native Cypher query language: We present a subgraph-matching approach to align extracted relations with external facts and show that fact verification, locating textual support for asserted facts, detecting inconsistent and missing facts, and extracting distantly-supervised training data can all be performed within the same framework.
Relation Extraction among Multiple Entities Using a Dual Pointer Network with a Multi-Head Attention Mechanism
Authors: Seong Sik Park, Harksoo Kim
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Decoder, Object, Relation, Relation Extraction, Sentence

Many previous studies on relation extrac-tion have been focused on finding only one relation between two entities in a single sentence. However, we can easily find the fact that multiple entities exist in a single sentence and the entities form multiple relations. To resolve this prob-lem, we propose a relation extraction model based on a dual pointer network with a multi-head attention mechanism. The proposed model finds n-to-1 subject-object relations by using a forward de-coder called an object decoder. Then, it finds 1-to-n subject-object relations by using a backward decoder called a sub-ject decoder. In the experiments with the ACE-05 dataset and the NYT dataset, the proposed model achieved the state-of-the-art performances (F1-score of 80.5{\%} in the ACE-05 dataset, F1-score of 78.3{\%} in the NYT dataset)
Improving Evidence Detection by Leveraging Warrants
Authors: Keshav Singh, Paul Reisert, Naoya Inoue, Pride Kavumba, Kentaro Inui
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11

Recognizing the implicit link between a claim and a piece of evidence (i.e. warrant) is the key to improving the performance of evidence detection. In this work, we explore the effectiveness of automatically extracted warrants for evidence detection. Given a claim and candidate evidence, our proposed method extracts multiple warrants via similarity search from an existing, structured corpus of arguments. We then attentively aggregate the extracted warrants, considering the consistency between the given argument and the acquired warrants. Although a qualitative analysis on the warrants shows that the extraction method needs to be improved, our results indicate that our method can still improve the performance of evidence detection.
Hybrid Models for Aspects Extraction without Labelled Dataset
Authors: Wai-Howe Khong, Lay-Ki Soon, Hui-Ngo Goh
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Opinion Mining

One of the important tasks in opinion mining is to extract aspects of the opinion target. Aspects are features or characteristics of the opinion target that are being reviewed, which can be categorised into explicit and implicit aspects. Extracting aspects from opinions is essential in order to ensure accurate information about certain attributes of an opinion target is retrieved. For instance, a professional camera receives a positive feedback in terms of its functionalities in a review, but its overly high price receives negative feedback. Most of the existing solutions focus on explicit aspects. However, sentences in reviews normally do not state the aspects explicitly. In this research, two hybrid models are proposed to identify and extract both explicit and implicit aspects, namely TDM-DC and TDM-TED. The proposed models combine topic modelling and dictionary-based approach. The models are unsupervised as they do not require any labelled dataset. The experimental results show that TDM-DC achieves F1-measure of 58.70{\%}, where it outperforms both the baseline topic model and dictionary-based approach. In comparison to other existing unsupervised techniques, the proposed models are able to achieve higher F1-measure by approximately 3{\%}. Although the supervised techniques perform slightly better, the proposed models are domain-independent, and hence more versatile.
Extract and Aggregate: A Novel Domain-Independent Approach to Factual Data Verification
Authors: Anton Chernyavskiy, Dmitry Ilvovsky
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Fact Checking, Fact Verification, Specificity

Triggered by Internet development, a large amount of information is published in online sources. However, it is a well-known fact that publications are inundated with inaccurate data. That is why fact-checking has become a significant topic in the last 5 years. It is widely accepted that factual data verification is a challenge even for the experts. This paper presents a domain-independent fact checking system. It can solve the fact verification problem entirely or at the individual stages. The proposed model combines various advanced methods of text data analysis, such as BERT and Infersent. The theoretical and empirical study of the system features is carried out. Based on FEVER and Fact Checking Challenge test-collections, experimental results demonstrate that our model can achieve the score on a par with state-of-the-art models designed by the specificity of particular datasets.
Interactive Evidence Detection: train state-of-the-art model out-of-domain or simple model interactively?
Authors: Chris Stahlhut
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Fact Checking, GPU, model, Small Data Image Classification

Finding evidence is of vital importance in research as well as fact checking and an evidence detection method would be useful in speeding up this process. However, when addressing a new topic there is no training data and there are two approaches to get started. One could use large amounts of out-of-domain data to train a state-of-the-art method, or to use the small data that a person creates while working on the topic. In this paper, we address this problem in two steps. First, by simulating users who read source documents and label sentences they can use as evidence, thereby creating small amounts of training data for an interactively trained evidence detection model; and second, by comparing such an interactively trained model against a pre-trained model that has been trained on large out-of-domain data. We found that an interactively trained model not only often out-performs a state-of-the-art model but also requires significantly lower amounts of computational resources. Therefore, especially when computational resources are scarce, e.g. no GPU available, training a smaller model on the fly is preferable to training a well generalising but resource hungry out-of-domain model.
Veritas Annotator: Discovering the Origin of a Rumour
Authors: Lucas Azevedo, Mohamed Moustafa
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Fake News Detection, Rumour Detection

Defined as the intentional or unintentionalspread of false information (K et al., 2019)through context and/or content manipulation,fake news has become one of the most seriousproblems associated with online information(Waldrop, 2017). Consequently, it comes asno surprise that Fake News Detection hasbecome one of the major foci of variousfields of machine learning and while machinelearning models have allowed individualsand companies to automate decision-basedprocesses that were once thought to be onlydoable by humans, it is no secret that thereal-life applications of such models are notviable without the existence of an adequatetraining dataset. In this paper we describethe Veritas Annotator, a web application formanually identifying the origin of a rumour.These rumours, often referred as claims,were previously checked for validity byFact-Checking Agencies.
FEVER Breaker's Run of Team NbAuzDrLqg
Authors: Youngwoo Kim, James Allan
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11

We describe our submission for the Breaker phase of the second Fact Extraction and VERification (FEVER) Shared Task. Our adversarial data can be explained by two perspectives. First, we aimed at testing model{'}s ability to retrieve evidence, when appropriate query terms could not be easily generated from the claim. Second, we test model{'}s ability to precisely understand the implications of the texts, which we expect to be rare in FEVER 1.0 dataset. Overall, we suggested six types of adversarial attacks. The evaluation on the submitted systems showed that the systems were only able get both the evidence and label correct in 20{\%} of the data. We also demonstrate our adversarial run analysis in the data development process.
Team DOMLIN: Exploiting Evidence Enhancement for the FEVER Shared Task
Authors: Dominik Stammbach, Guenter Neumann
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Retrieval, Sentence

This paper contains our system description for the second Fact Extraction and VERification (FEVER) challenge. We propose a two-staged sentence selection strategy to account for examples in the dataset where evidence is not only conditioned on the claim, but also on previously retrieved evidence. We use a publicly available document retrieval module and have fine-tuned BERT checkpoints for sentence se- lection and as the entailment classifier. We report a FEVER score of 68.46{\%} on the blind testset.
Team GPLSI. Approach for automated fact checking
Authors: Aim{\'e}e Alonso-Reina, Robiert Sep{\'u}lveda-Torres, Estela Saquete, Manuel Palomar
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Articles, Fact Checking, Retrieval, Semantic Similarity, Semantic Textual Similarity, Sentence, Sentence Retrieval

Fever Shared 2.0 Task is a challenge meant for developing automated fact checking systems. Our approach for the Fever 2.0 is based on a previous proposal developed by Team Athene UKP TU Darmstadt. Our proposal modifies the sentence retrieval phase, using statement extraction and representation in the form of triplets (subject, object, action). Triplets are extracted from the claim and compare to triplets extracted from Wikipedia articles using semantic similarity. Our results are satisfactory but there is room for improvement.