pwc-archive/papers-with-abstracts huggingface dataset:
Annotating Claims in the Vaccination Debate
Authors: Benedetta Torsi, Roser Morante
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining

In this paper we present annotation experiments with three different annotation schemes for the identification of argument components in texts related to the vaccination debate. Identifying claims about vaccinations made by participants in the debate is of great societal interest, as the decision to vaccinate or not has impact in public health and safety. Since most corpora that have been annotated with argumentation information contain texts that belong to a specific genre and have a well defined argumentation structure, we needed to adjust the annotation schemes to our corpus, which contains heterogeneous texts from the Web. We started with a complex annotation scheme that had to be simplified due to low IAA. In our final experiment, which focused on annotating claims, annotators reached 57.3{\%} IAA.
Evidence Types, Credibility Factors, and Patterns or Soft Rules for Weighing Conflicting Evidence: Argument Mining in the Context of Legal Rules Governing Evidence Assessment
Authors: Vern R. Walker, Dina Foerster, Julia Monica Ponce, Matthew Rosen
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining, Decision Making

This paper reports on the results of an empirical study of adjudicatory decisions about veterans{'} claims for disability benefits in the United States. It develops a typology of kinds of relevant evidence (argument premises) employed in cases, and it identifies factors that the tribunal considers when assessing the credibility or trustworthiness of individual items of evidence. It also reports on patterns or {``}soft rules{''} that the tribunal uses to comparatively weigh the probative value of conflicting evidence. These evidence types, credibility factors, and comparison patterns are developed to be inter-operable with legal rules governing the evidence assessment process in the U.S. This approach should be transferable to other legal and non-legal domains.
Feasible Annotation Scheme for Capturing Policy Argument Reasoning using Argument Templates
Authors: Paul Reisert, Naoya Inoue, Tatsuki Kuribayashi, Kentaro Inui
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining, Document Summarization, General Classification, Structured Prediction

Most of the existing works on argument mining cast the problem of argumentative structure identification as classification tasks (e.g. attack-support relations, stance, explicit premise/claim). This paper goes a step further by addressing the task of automatically identifying reasoning patterns of arguments using predefined templates, which is called \textit{argument template (AT) instantiation}. The contributions of this work are three-fold. First, we develop a simple, yet expressive set of easily annotatable ATs that can represent a majority of writer{'}s reasoning for texts with diverse policy topics while maintaining the computational feasibility of the task. Second, we create a small, but highly reliable annotated corpus of instantiated ATs on top of reliably annotated support and attack relations and conduct an annotation study. Third, we formulate the task of AT instantiation as structured prediction constrained by a feasible set of templates. Our evaluation demonstrates that we can annotate ATs with a reasonably high inter-annotator agreement, and the use of template-constrained inference is useful for instantiating ATs with only partial reasoning comprehension clues.
Frame- and Entity-Based Knowledge for Common-Sense Argumentative Reasoning
Authors: Teresa Botschen, Daniil Sorokin, Iryna Gurevych
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining, Common Sense Reasoning, Dependency Parsing, Natural Language Inference, Question Answering, Relation Classification, Semantic Parsing, Semantic Role Labeling, Word Embeddings, World Knowledge

Common-sense argumentative reasoning is a challenging task that requires holistic understanding of the argumentation where external knowledge about the world is hypothesized to play a key role. We explore the idea of using event knowledge about prototypical situations from FrameNet and fact knowledge about concrete entities from Wikidata to solve the task. We find that both resources can contribute to an improvement over the non-enriched approach and point out two persisting challenges: first, integration of many annotations of the same type, and second, fusion of complementary annotations. After our explorations, we question the key role of external world knowledge with respect to the argumentative reasoning task and rather point towards a logic-based analysis of the chain of reasoning.
Incorporating Topic Aspects for Online Comment Convincingness Evaluation
Authors: Yunfan Gu, Zhongyu Wei, Maoran Xu, Hao Fu, Yang Liu, Xuanjing Huang
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining

In this paper, we propose to incorporate topic aspects information for online comments convincingness evaluation. Our model makes use of graph convolutional network to utilize implicit topic information within a discussion thread to assist the evaluation of convincingness of each single comment. In order to test the effectiveness of our proposed model, we annotate topic information on top of a public dataset for argument convincingness evaluation. Experimental results show that topic information is able to improve the performance for convincingness evaluation. We also make a move to detect topic aspects automatically.
Proposed Method for Annotation of Scientific Arguments in Terms of Semantic Relations and Argument Schemes
Authors: Nancy Green
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining, Articles

This paper presents a proposed method for annotation of scientific arguments in biological/biomedical journal articles. Semantic entities and relations are used to represent the propositional content of arguments in instances of argument schemes. We describe an experiment in which we encoded the arguments in a journal article to identify issues in this approach. Our catalogue of argument schemes and a copy of the annotated article are now publically available.
Using context to identify the language of face-saving
Authors: Nona Naderi, Graeme Hirst
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining

We created a corpus of utterances that attempt to save face from parliamentary debates and use it to automatically analyze the language of reputation defence. Our proposed model that incorporates information regarding threats to reputation can predict reputation defence language with high confidence. Further experiments and evaluations on different datasets show that the model is able to generalize to new utterances and can predict the language of reputation defence in a new dataset.
Dave the debater: a retrieval-based and generative argumentative dialogue agent
Authors: Dieu Thu Le, Cam-Tu Nguyen, Kim Anh Nguyen
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining, Retrieval, Stance Detection

In this paper, we explore the problem of developing an argumentative dialogue agent that can be able to discuss with human users on controversial topics. We describe two systems that use retrieval-based and generative models to make argumentative responses to the users. The experiments show promising results although they have been trained on a small dataset.
PD3: Better Low-Resource Cross-Lingual Transfer By Combining Direct Transfer and Annotation Projection
Authors: Steffen Eger, Andreas R{\"u}ckl{\'e}, Iryna Gurevych
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining, Cross-Lingual Transfer, Multi-Task Learning, POS, POS Tagging, Sentence, Word Alignment

We consider unsupervised cross-lingual transfer on two tasks, viz., sentence-level argumentation mining and standard POS tagging. We combine direct transfer using bilingual embeddings with annotation projection, which projects labels across unlabeled parallel data. We do so by either merging respective source and target language datasets or alternatively by using multi-task learning. Our combination strategy considerably improves upon both direct transfer and projection with few available parallel sentences, the most realistic scenario for many low-resource target languages.
Cross-Lingual Argumentative Relation Identification: from English to Portuguese
Authors: Gil Rocha, Christian Stab, Henrique Lopes Cardoso, Iryna Gurevych
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining, Relation, Transfer Learning

Argument mining aims to detect and identify argument structures from textual resources. In this paper, we aim to address the task of argumentative relation identification, a subtask of argument mining, for which several approaches have been recently proposed in a monolingual setting. To overcome the lack of annotated resources in less-resourced languages, we present the first attempt to address this subtask in a cross-lingual setting. We compare two standard strategies for cross-language learning, namely: projection and direct-transfer. Experimental results show that by using unsupervised language adaptation the proposed approaches perform at a competitive level when compared with fully-supervised in-language learning settings.
More or less controlled elicitation of argumentative text: Enlarging a microtext corpus via crowdsourcing
Authors: Maria Skeppstedt, Andreas Peldszus, Manfred Stede
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Argument Mining

We present an extension of an annotated corpus of short argumentative texts that had originally been built in a controlled text production experiment. Our extension more than doubles the size of the corpus by means of crowdsourcing. We report on the setup of this experiment and on the consequences that crowdsourcing had for assembling the data, and in particular for annotation. We labeled the argumentative structure by marking claims, premises, and relations between them, following the scheme used in the original corpus, but had to make a few modifications in response to interesting phenomena in the data. Finally, we report on an experiment with the automatic prediction of this argumentation structure: We first replicated the approach of an earlier study on the original corpus, and compare the performance to various settings involving the extension.
Results of the sixth edition of the BioASQ Challenge
Authors: Anastasios Nentidis, Anastasia Krithara, Konstantinos Bougiatiotis, Georgios Paliouras, Ioannis Kakadiaris
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Information Retrieval, Question Answering

This paper presents the results of the sixth edition of the BioASQ challenge. The BioASQ challenge aims at the promotion of systems and methodologies through the organization of a challenge on two tasks: semantic indexing and question answering. In total, 26 teams with more than 90 systems participated in this year{'}s challenge. As in previous years, the best systems were able to outperform the strong baselines. This suggests that state-of-the-art systems are continuously improving, pushing the frontier of research.
Semantic role labeling tools for biomedical question answering: a study of selected tools on the BioASQ datasets
Authors: Fabian Eckert, Mariana Neves
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Named Entity Recognition (NER), Part-Of-Speech Tagging, Question Answering, Semantic Parsing, Semantic Role Labeling

Question answering (QA) systems usually rely on advanced natural language processing components to precisely understand the questions and extract the answers. Semantic role labeling (SRL) is known to boost performance for QA, but its use for biomedical texts has not yet been fully studied. We analyzed the performance of three SRL tools (BioKIT, BIOSMILE and PathLSTM) on 1776 questions from the BioASQ challenge. We compared the systems regarding the coverage of the questions and snippets, as well as based on pre-defined criteria, such as easiness of installation, supported formats and usability. Finally, we integrated two of the tools in a simple QA system to further evaluate their performance over the official BioASQ test sets.
Macquarie University at BioASQ 6b: Deep learning and deep reinforcement learning for query-based summarisation
Authors: Diego Moll{\'a}
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Deep Learning, Deep Reinforcement Learning, Position, Question Answering, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sentence, Word Embeddings

This paper describes Macquarie University{'}s contribution to the BioASQ Challenge (BioASQ 6b, Phase B). We focused on the extraction of the ideal answers, and the task was approached as an instance of query-based multi-document summarisation. In particular, this paper focuses on the experiments related to the deep learning and reinforcement learning approaches used in the submitted runs. The best run used a deep learning model under a regression-based framework. The deep learning architecture used features derived from the output of LSTM chains on word embeddings, plus features based on similarity with the query, and sentence position. The reinforcement learning approach was a proof-of-concept prototype that trained a global policy using REINFORCE. The global policy was implemented as a neural network that used tf.idf features encoding the candidate sentence, question, and context.
MindLab Neural Network Approach at BioASQ 6B
Authors: Andr{\'e}s Rosso-Mateus, Fabio A. Gonz{\'a}lez, Manuel Montes-y-G{\'o}mez
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Information Retrieval, Position, Question Answering, Retrieval, Semantic Similarity, Semantic Textual Similarity

Biomedical Question Answering is concerned with the development of methods and systems that automatically find answers to natural language posed questions. In this work, we describe the system used in the BioASQ Challenge task 6b for document retrieval and snippet retrieval (with particular emphasis in this subtask). The proposed model makes use of semantic similarity patterns that are evaluated and measured by a convolutional neural network architecture. Subsequently, the snippet ranking performance is improved with a pseudo-relevance feedback approach in a later step. Based on the preliminary results, we reached the second position in snippet retrieval sub-task.
AttentionMeSH: Simple, Effective and Interpretable Automatic MeSH Indexer
Authors: Qiao Jin, Bhuwan Dhingra, William Cohen, Xinghua Lu
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Articles, Information Retrieval, Question Answering, Retrieval

There are millions of articles in PubMed database. To facilitate information retrieval, curators in the National Library of Medicine (NLM) assign a set of Medical Subject Headings (MeSH) to each article. MeSH is a hierarchically-organized vocabulary, containing about 28K different concepts, covering the fields from clinical medicine to information sciences. Several automatic MeSH indexing models have been developed to improve the time-consuming and financially expensive manual annotation, including the NLM official tool {--} Medical Text Indexer, and the winner of BioASQ Task5a challenge {--} DeepMeSH. However, these models are complex and not interpretable. We propose a novel end-to-end model, AttentionMeSH, which utilizes deep learning and attention mechanism to index MeSH terms to biomedical text. The attention mechanism enables the model to associate textual evidence with annotations, thus providing interpretability at the word level. The model also uses a novel masking mechanism to enhance accuracy and speed. In the final week of BioASQ Chanllenge Task6a, we ranked 2nd by average MiF using an on-construction model. After the contest, we achieve close to state-of-the-art MiF performance of âˆ¼ 0.684 using our final model. Human evaluations show AttentionMeSH also provides high level of interpretability, retrieving about 90{\%} of all expert-labeled relevant words given an MeSH-article pair at 20 output.
Extraction Meets Abstraction: Ideal Answer Generation for Biomedical Questions
Authors: Yutong Li, Nicholas Gekakis, Qiuze Wu, Boyue Li, Ch, Khyathi u, Eric Nyberg
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Abstractive Text Summarization, Answer Generation, Extractive Summarization, Question Answering, Sentence, Sentence Fusion, Sentence Ordering

The growing number of biomedical publications is a challenge for human researchers, who invest considerable effort to search for relevant documents and pinpointed answers. Biomedical Question Answering can automatically generate answers for a user{'}s topic or question, significantly reducing the effort required to locate the most relevant information in a large document corpus. Extractive summarization techniques, which concatenate the most relevant text units drawn from multiple documents, perform well on automatic evaluation metrics like ROUGE, but score poorly on human readability, due to the presence of redundant text and grammatical errors in the answer. This work moves toward abstractive summarization, which attempts to distill and present the meaning of the original text in a more coherent way. We incorporate a sentence fusion approach, based on Integer Linear Programming, along with three novel approaches for sentence ordering, in an attempt to improve the human readability of ideal answers. Using an open framework for configuration space exploration (BOOM), we tested over 2000 unique system configurations in order to identify the best-performing combinations for the sixth edition of Phase B of the BioASQ challenge.
UNCC QA: Biomedical Question Answering system
Authors: Bh, Abhishek waldar, Wlodek Zadrozny
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Extractive Summarization, Question Answering

In this paper, we detail our submission to the BioASQ competition{'}s Biomedical Semantic Question and Answering task. Our system uses extractive summarization techniques to generate answers and has scored highest ROUGE-2 and Rogue-SU4 in all test batch sets. Our contributions are named-entity based method for answering factoid and list questions, and an extractive summarization techniques for building paragraph-sized summaries, based on lexical chains. Our system got highest ROUGE-2 and ROUGE-SU4 scores for ideal-type answers in all test batch sets. We also discuss the limitations of the described system, such lack of the evaluation on other criteria (e.g. manual). Also, for factoid- and list -type question our system got low accuracy (which suggests that our algorithm needs to improve in the ranking of entities).
An Adaption of BIOASQ Question Answering dataset for Machine Reading systems by Manual Annotations of Answer Spans.
Authors: Sanjay Kamath, Brigitte Grau, Yue Ma
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: All, Domain Adaptation, Question Answering, Reading Comprehension

BIOASQ Task B Phase B challenge focuses on extracting answers from snippets for a given question. The dataset provided by the organizers contains answers, but not all their variants. Henceforth a manual annotation was performed to extract all forms of correct answers. This article shows the impact of using all occurrences of correct answers for training on the evaluation scores which are improved significantly.
Ontology-Based Retrieval \& Neural Approaches for BioASQ Ideal Answer Generation
Authors: Ashwin Naresh Kumar, Harini Kesavamoorthy, Madhura Das, Pramati Kalwad, Ch, Khyathi u, Teruko Mitamura, Eric Nyberg
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Abstractive Text Summarization, Answer Generation, Learning-To-Rank, Question Answering, Retrieval, Text Generation

The ever-increasing magnitude of biomedical information sources makes it difficult and time-consuming for a human researcher to find the most relevant documents and pinpointed answers for a specific question or topic when using only a traditional search engine. Biomedical Question Answering systems automatically identify the most relevant documents and pinpointed answers, given an information need expressed as a natural language question. Generating a non-redundant, human-readable summary that satisfies the information need of a given biomedical question is the focus of the Ideal Answer Generation task, part of the BioASQ challenge. This paper presents a system for ideal answer generation (using ontology-based retrieval and a neural learning-to-rank approach, combined with extractive and abstractive summarization techniques) which achieved the highest ROUGE score of 0.659 on the BioASQ 5b batch 2 test.
When does deep multi-task learning work for loosely related document classification tasks?
Authors: Emma Kerinec, Chlo{\'e} Braud, Anders S{\o}gaard
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Document Classification, General Classification, Machine Translation, Multi-Task Learning, Sentence Compression, Sentiment Analysis, text-classification, Text Classification

This work aims to contribute to our understanding of \textit{when} multi-task learning through parameter sharing in deep neural networks leads to improvements over single-task learning. We focus on the setting of learning from \textit{loosely related} tasks, for which no theoretical guarantees exist. We therefore approach the question empirically, studying which properties of datasets and single-task learning characteristics correlate with improvements from multi-task learning. We are the first to study this in a text classification setting and across more than 500 different task pairs.
Explaining non-linear Classifier Decisions within Kernel-based Deep Architectures
Authors: Danilo Croce, Daniele Rossini, Roberto Basili
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: General Classification, Image Classification, Question Answering, Semantic Role Labeling

Nonlinear methods such as deep neural networks achieve state-of-the-art performances in several semantic NLP tasks. However epistemologically transparent decisions are not provided as for the limited interpretability of the underlying acquired neural models. In neural-based semantic inference tasks epistemological transparency corresponds to the ability of tracing back causal connections between the linguistic properties of a input instance and the produced classification output. In this paper, we propose the use of a methodology, called \textit{Layerwise Relevance Propagation}, over linguistically motivated neural architectures, namely \textit{Kernel-based Deep Architectures} (KDA), to guide argumentations and explanation inferences. In such a way, each decision provided by a KDA can be linked to real examples, linguistically related to the input instance: these can be used to motivate the network output. Quantitative analysis shows that richer explanations about the semantic and syntagmatic structures of the examples characterize more convincing arguments in two tasks, i.e. question classification and semantic role labeling.