pwc-archive/papers-with-abstracts huggingface dataset:
The Power of Character N-grams in Native Language Identification
Authors: Artur Kulmizev, Bo Blankers, Johannes Bjerva, Malvina Nissim, Gertjan Van Noord, Barbara Plank, Martijn Wieling
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Language Identification, Native Language Identification, Text Classification

In this paper, we explore the performance of a linear SVM trained on language independent character features for the NLI Shared Task 2017. Our basic system (GRONINGEN) achieves the best performance (87.56 F1-score) on the evaluation set using only 1-9 character n-grams as features. We compare this against several ensemble and meta-classifiers in order to examine how the linear system fares when combined with other, especially non-linear classifiers. Special emphasis is placed on the topic bias that exists by virtue of the assessment essay prompt distribution.
Classifier Stacking for Native Language Identification
Authors: Wen Li, Liang Zou
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Language Acquisition, Language Identification, Native Language Identification, Text Classification, Vocal Bursts Type Prediction

This paper reports our contribution (team WLZ) to the NLI Shared Task 2017 (essay track). We first extract lexical and syntactic features from the essays, perform feature weighting and selection, and train linear support vector machine (SVM) classifiers each on an individual feature type. The output of base classifiers, as probabilities for each class, are then fed into a multilayer perceptron to predict the native language of the author. We also report the performance of each feature type, as well as the best features of a type. Our system achieves an accuracy of 86.55{\%}, which is among the best performing systems of this shared task.
Native Language Identification using Phonetic Algorithms
Authors: Charese Smiley, S K{\"u}bler, ra
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Language Identification, Native Language Identification

In this paper, we discuss the results of the IUCL system in the NLI Shared Task 2017. For our system, we explore a variety of phonetic algorithms to generate features for Native Language Identification. These features are contrasted with one of the most successful type of features in NLI, character n-grams. We find that although phonetic features do not perform as well as character n-grams alone, they do increase overall F1 score when used together with character n-grams.
A deep-learning based native-language classification by using a latent semantic analysis for the NLI Shared Task 2017
Authors: Yoo Rhee Oh, Hyung-Bae Jeon, Hwa Jeon Song, Yun-Kyung Lee, Jeon-Gue Park, Yun-Keun Lee
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Automatic Speech Recognition (ASR), Dimensionality Reduction, General Classification, Language Identification, Native Language Identification, Speech Recognition

This paper proposes a deep-learning based native-language identification (NLI) using a latent semantic analysis (LSA) as a participant (ETRI-SLP) of the NLI Shared Task 2017 where the NLI Shared Task 2017 aims to detect the native language of an essay or speech response of a standardized assessment of English proficiency for academic purposes. To this end, we use the six unit forms of a text data such as character 4/5/6-grams and word 1/2/3-grams. For each unit form of text data, we convert it into a count-based vector, extract a 2000-rank LSA feature, and perform a linear discriminant analysis (LDA) based dimension reduction. From the count-based vector or the LSA-LDA feature, we also obtain the output prediction values of a support vector machine (SVM) based classifier, the output prediction values of a deep neural network (DNN) based classifier, and the bottleneck values of a DNN based classifier. In order to incorporate the various kinds of text-based features and a speech-based i-vector feature, we design two DNN based ensemble classifiers for late fusion and early fusion, respectively. From the NLI experiments, the F1 (macro) scores are obtained as 0.8601, 0.8664, and 0.9220 for the essay track, the speech track, and the fusion track, respectively. The proposed method has comparable performance to the top-ranked teams for the speech and fusion tracks, although it has slightly lower performance for the essay track.
Fusion of Simple Models for Native Language Identification
Authors: Fabio Kepler, Ramon Astudillo, Alberto Abad
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Information Retrieval, Language Identification, Native Language Identification

In this paper we describe the approaches we explored for the 2017 Native Language Identification shared task. We focused on simple word and sub-word units avoiding heavy use of hand-crafted features. Following recent trends, we explored linear and neural networks models to attempt to compensate for the lack of rich feature use. Initial efforts yielded f1-scores of 82.39{\%} and 83.77{\%} in the development and test sets of the fusion track, and were officially submitted to the task as team L2F. After the task was closed, we carried on further experiments and relied on a late fusion strategy for combining our simple proposed approaches with modifications of the baselines provided by the task. As expected, the i-vectors based sub-system dominates the performance of the system combinations, and results in the major contributor to our achieved scores. Our best combined system achieves 90.1{\%} and 90.2{\%} f1-score in the development and test sets of the fusion track, respectively.
Stacked Sentence-Document Classifier Approach for Improving Native Language Identification
Authors: Andrea Cimino, Felice Dell{'}Orletta
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Document Classification, Language Identification, Native Language Identification, Sentence, Sentence Classification, Text Classification

In this paper, we describe the approach of the ItaliaNLP Lab team to native language identification and discuss the results we submitted as participants to the essay track of NLI Shared Task 2017. We introduce for the first time a 2-stacked sentence-document architecture for native language identification that is able to exploit both local sentence information and a wide set of general-purpose features qualifying the lexical and grammatical structure of the whole document. When evaluated on the official test set, our sentence-document stacked architecture obtained the best result among all the participants of the essay track with an F1 score of 0.8818.
Using Gaze to Predict Text Readability
Authors: Ana Valeria Gonz{\'a}lez-Gardu{\~n}o, Anders S{\o}gaard
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Machine Translation, Multi-Task Learning, regression, Sentence, Text Generation, Text Simplification

We show that text readability prediction improves significantly from hard parameter sharing with models predicting first pass duration, total fixation duration and regression duration. Specifically, we induce multi-task Multilayer Perceptrons and Logistic Regression models over sentence representations that capture various aggregate statistics, from two different text readability corpora for English, as well as the Dundee eye-tracking corpus. Our approach leads to significant improvements over Single task learning and over previous systems. In addition, our improvements are consistent across train sample sizes, making our approach especially applicable to small datasets.
Annotating Orthographic Target Hypotheses in a German L1 Learner Corpus
Authors: Ronja Laarmann-Quante, Katrin Ortmann, Anna Ehlert, Maurice Vogel, Stefanie Dipper
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9

NLP applications for learners often rely on annotated learner corpora. Thereby, it is important that the annotations are both meaningful for the task, and consistent and reliable. We present a new longitudinal L1 learner corpus for German (handwritten texts collected in grade 2{--}4), which is transcribed and annotated with a target hypothesis that strictly only corrects orthographic errors, and is thereby tailored to research and tool development for orthographic issues in primary school. While for most corpora, transcription and target hypothesis are not evaluated, we conducted a detailed inter-annotator agreement study for both tasks. Although we achieved high agreement, our discussion of cases of disagreement shows that even with detailed guidelines, annotators differ here and there for different reasons, which should also be considered when working with transcriptions and target hypotheses of other corpora, especially if no explicit guidelines for their construction are known.
A Large Scale Quantitative Exploration of Modeling Strategies for Content Scoring
Authors: Nitin Madnani, Anastassia Loukina, Aoife Cahill
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Math

We explore various supervised learning strategies for automated scoring of content knowledge for a large corpus of 130 different content-based questions spanning four subject areas (Science, Math, English Language Arts, and Social Studies) and containing over 230,000 responses scored by human raters. Based on our analyses, we provide specific recommendations for content scoring. These are based on patterns observed across multiple questions and assessments and are, therefore, likely to generalize to other scenarios and prove useful to the community as automated content scoring becomes more popular in schools and classrooms.
200K+ Crowdsourced Political Arguments for a New Chilean Constitution
Authors: Constanza Fierro, Claudio Fuentes, Jorge P{\'e}rez, Mauricio Quezada
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, General Classification, Multi-class Classification

In this paper we present the dataset of 200,000+ political arguments produced in the local phase of the 2016 Chilean constitutional process. We describe the human processing of this data by the government officials, and the manual tagging of arguments performed by members of our research group. Afterwards we focus on classification tasks that mimic the human processes, comparing linear methods with neural network architectures. The experiments show that some of the manual tasks are suitable for automatization. In particular, the best methods achieve a 90{\%} top-5 accuracy in a multi-class classification of arguments, and 65{\%} macro-averaged F1-score for tagging arguments according to a three-part argumentation model.
Analyzing the Semantic Types of Claims and Premises in an Online Persuasive Forum
Authors: Christopher Hidey, Elena Musi, Alyssa Hwang, Smar Muresan, a, Kathy Mckeown
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining

Argumentative text has been analyzed both theoretically and computationally in terms of argumentative structure that consists of argument components (e.g., claims, premises) and their argumentative relations (e.g., support, attack). Less emphasis has been placed on analyzing the semantic types of argument components. We propose a two-tiered annotation scheme to label claims and premises and their semantic types in an online persuasive forum, Change My View, with the long-term goal of understanding what makes a message persuasive. Premises are annotated with the three types of persuasive modes: ethos, logos, pathos, while claims are labeled as interpretation, evaluation, agreement, or disagreement, the latter two designed to account for the dialogical nature of our corpus. We aim to answer three questions: 1) can humans reliably annotate the semantic types of argument components? 2) are types of premises/claims positioned in recurrent orders? and 3) are certain types of claims and/or premises more likely to appear in persuasive messages than in non-persuasive messages?
Annotation of argument structure in Japanese legal documents
Authors: Hiroaki Yamada, Simone Teufel, Takenobu Tokunaga
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining

We propose a method for the annotation of Japanese civil judgment documents, with the purpose of creating flexible summaries of these. The first step, described in the current paper, concerns content selection, i.e., the question of which material should be extracted initially for the summary. In particular, we utilize the hierarchical argument structure of the judgment documents. Our main contributions are a) the design of an annotation scheme that stresses the connection between legal points (called issue topics) and argument structure, b) an adaptation of rhetorical status to suit the Japanese legal system and c) the definition of a linked argument structure based on legal sub-arguments. In this paper, we report agreement between two annotators on several aspects of the overall task.
Improving Claim Stance Classification with Lexical Knowledge Expansion and Context Utilization
Authors: Roy Bar-Haim, Lilach Edelstein, Charles Jochim, Noam Slonim
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, Classification, General Classification, Sentiment Analysis, Stance Classification

Stance classification is a core component in on-demand argument construction pipelines. Previous work on claim stance classification relied on background knowledge such as manually-composed sentiment lexicons. We show that both accuracy and coverage can be significantly improved through automatic expansion of the initial lexicon. We also developed a set of contextual features that further improves the state-of-the-art for this task.
Mining Argumentative Structure from Natural Language text using Automatically Generated Premise-Conclusion Topic Models
Authors: John Lawrence, Chris Reed
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, Opinion Mining, Sentiment Analysis, Topic Models

This paper presents a method of extracting argumentative structure from natural language text. The approach presented is based on the way in which we understand an argument being made, not just from the words said, but from existing contextual knowledge and understanding of the broader issues. We leverage high-precision, low-recall techniques in order to automatically build a large corpus of inferential statements related to the text{'}s topic. These statements are then used to produce a matrix representing the inferential relationship between different aspects of the topic. From this matrix, we are able to determine connectedness and directionality of inference between statements in the original text. By following this approach, we obtain results that compare favourably to those of other similar techniques to classify premise-conclusion pairs (with results 22 points above baseline), but without the requirement of large volumes of annotated, domain specific data.
Building an Argument Search Engine for the Web
Authors: Henning Wachsmuth, Martin Potthast, Khalid Al-Khatib, Yamen Ajjour, Jana Puschmann, Jiani Qu, Jonas Dorsch, Viorel Morari, Janek Bevendorff, Benno Stein
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, Decision Making

Computational argumentation is expected to play a critical role in the future of web search. To make this happen, many search-related questions must be revisited, such as how people query for arguments, how to mine arguments from the web, or how to rank them. In this paper, we develop an argument search framework for studying these and further questions. The framework allows for the composition of approaches to acquiring, mining, assessing, indexing, querying, retrieving, ranking, and presenting arguments while relying on standard infrastructure and interfaces. Based on the framework, we build a prototype search engine, called args, that relies on an initial, freely accessible index of nearly 300k arguments crawled from reliable web resources. The framework and the argument search engine are intended as an environment for collaborative research on computational argumentation and its practical evaluation.
Argument Relation Classification Using a Joint Inference Model
Authors: Yufang Hou, Charles Jochim
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, Classification, General Classification, model, Relation, Relation Classification, Stance Classification

In this paper, we address the problem of argument relation classification where argument units are from different texts. We design a joint inference method for the task by modeling argument relation classification and stance classification jointly. We show that our joint model improves the results over several strong baselines.
Projection of Argumentative Corpora from Source to Target Languages
Authors: Ahmet Aker, Huangpan Zhang
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, Machine Translation, Translation

Argumentative corpora are costly to create and are available in only few languages with English dominating the area. In this paper we release the first publicly available Mandarin argumentative corpus. The corpus is created by exploiting the idea of comparable corpora from Statistical Machine Translation. We use existing corpora in English and manually map the claims and premises to comparable corpora in Mandarin. We also implement a simple solution to automate this approach with the view of creating argumentative corpora in other less-resourced languages. In this way we introduce a new task of multi-lingual argument mapping that can be evaluated using our English-Mandarin argumentative corpus. The preliminary results of our automatic argument mapper mirror the simplicity of our approach, but provide a baseline for further improvements.
Manual Identification of Arguments with Implicit Conclusions Using Semantic Rules for Argument Mining
Authors: Nancy Green
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, Articles

This paper describes a pilot study to evaluate human analysts{'} ability to identify the argumentation scheme and premises of an argument having an implicit conclusion. In preparation for the study, argumentation scheme definitions were crafted for genetics research articles. The schemes were defined in semantic terms, following a proposal to use semantic rules to mine arguments in that literature.
Unsupervised corpus--wide claim detection
Authors: Ran Levy, Shai Gretz, Benjamin Sznajder, Shay Hummel, Ranit Aharonov, Noam Slonim
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, Decision Making, Sentence

Automatic claim detection is a fundamental argument mining task that aims to automatically mine claims regarding a topic of consideration. Previous works on mining argumentative content have assumed that a set of relevant documents is given in advance. Here, we present a first corpus{--} wide claim detection framework, that can be directly applied to massive corpora. Using simple and intuitive empirical observations, we derive a claim sentence query by which we are able to directly retrieve sentences in which the prior probability to include topic-relevant claims is greatly enhanced. Next, we employ simple heuristics to rank the sentences, leading to an unsupervised corpus{--}wide claim detection system, with precision that outperforms previously reported results on the task of claim detection given relevant documents and labeled data.
Using Question-Answering Techniques to Implement a Knowledge-Driven Argument Mining Approach
Authors: Patrick Saint-Dizier
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, Question Answering

This short paper presents a first implementation of a knowledge-driven argument mining approach. The major processing steps and language resources of the system are surveyed. An indicative evaluation outlines challenges and improvement directions.
What works and what does not: Classifier and feature analysis for argument mining
Authors: Ahmet Aker, Alfred Sliwa, Yuan Ma, Ruishen Lui, Niravkumar Borad, Seyedeh Ziyaei, Mina Ghobadi
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining

This paper offers a comparative analysis of the performance of different supervised machine learning methods and feature sets on argument mining tasks. Specifically, we address the tasks of extracting argumentative segments from texts and predicting the structure between those segments. Eight classifiers and different combinations of six feature types reported in previous work are evaluated. The results indicate that overall best performing features are the structural ones. Although the performance of classifiers varies depending on the feature combinations and corpora used for training and testing, Random Forest seems to be among the best performing classifiers. These results build a basis for further development of argument mining techniques and can guide an implementation of argument mining into different applications such as argument based search.
Unsupervised Detection of Argumentative Units though Topic Modeling Techniques
Authors: Alfio Ferrara, Stefano Montanelli, Georgios Petasis
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, Opinion Mining, Stance Detection

In this paper we present a new unsupervised approach, {``}Attraction to Topics{''} {--} A2T , for the detection of argumentative units, a sub-task of argument mining. Motivated by the importance of topic identification in manual annotation, we examine whether topic modeling can be used for performing unsupervised detection of argumentative sentences, and to what extend topic modeling can be used to classify sentences as claims and premises. Preliminary evaluation results suggest that topic information can be successfully used for the detection of argumentative sentences, at least for corpora used for evaluation. Our approach has been evaluated on two English corpora, the first of which contains 90 persuasive essays, while the second is a collection of 340 documents from user generated content.
Using Complex Argumentative Interactions to Reconstruct the Argumentative Structure of Large-Scale Debates
Authors: John Lawrence, Chris Reed
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, Decision Making

In this paper we consider the insights that can be gained by considering large scale argument networks and the complex interactions between their constituent propositions. We investigate metrics for analysing properties of these networks, illustrating these using a corpus of arguments taken from the 2016 US Presidential Debates. We present techniques for determining these features directly from natural language text and show that there is a strong correlation between these automatically identified features and the argumentative structure contained within the text. Finally, we combine these metrics with argument mining techniques and show how the identification of argumentative relations can be improved by considering the larger context in which they occur.
Unit Segmentation of Argumentative Texts
Authors: Yamen Ajjour, Wei-Fan Chen, Johannes Kiesel, Henning Wachsmuth, Benno Stein
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Argument Mining, Segmentation

The segmentation of an argumentative text into argument units and their non-argumentative counterparts is the first step in identifying the argumentative structure of the text. Despite its importance for argument mining, unit segmentation has been approached only sporadically so far. This paper studies the major parameters of unit segmentation systematically. We explore the effectiveness of various features, when capturing words separately, along with their neighbors, or even along with the entire text. Each such context is reflected by one machine learning model that we evaluate within and across three domains of texts. Among the models, our new deep learning approach capturing the entire text turns out best within all domains, with an F-score of up to 88.54. While structural features generalize best across domains, the domain transfer remains hard, which points to major challenges of unit segmentation.