pwc-archive/papers-with-abstracts huggingface dataset:
Liner2 --- a Generic Framework for Named Entity Recognition
Authors: Micha{\l} Marci{\'n}czuk, Jan Koco{\'n}, Marcin Oleksy
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Morphological Analysis, Multilingual Named Entity Recognition, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

In the paper we present an adaptation of Liner2 framework to solve the BSNLP 2017 shared task on multilingual named entity recognition. The tool is tuned to recognize and lemmatize named entities for Polish.
Language-Independent Named Entity Analysis Using Parallel Projection and Rule-Based Disambiguation
Authors: James Mayfield, Paul McNamee, Cash Costello
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Clustering, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

The 2017 shared task at the Balto-Slavic NLP workshop requires identifying coarse-grained named entities in seven languages, identifying each entity{'}s base form, and clustering name mentions across the multilingual set of documents. The fact that no training data is provided to systems for building supervised classifiers further adds to the complexity. To complete the task we first use publicly available parallel texts to project named entity recognition capability from English to each evaluation language. We ignore entirely the subtask of identifying non-inflected forms of names. Finally, we create cross-document entity identifiers by clustering named mentions using a procedure-based approach.
Comparison of String Similarity Measures for Obscenity Filtering
Authors: Ekaterina Chernyak
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Information Retrieval, Sentiment Analysis, Spelling Correction

In this paper we address the problem of filtering obscene lexis in Russian texts. We use string similarity measures to find words similar or identical to words from a stop list and establish both a test collection and a baseline for the task. Our experiments show that a novel string similarity measure based on the notion of an annotated suffix tree outperforms some of the other well known measures.
Stylometric Analysis of Parliamentary Speeches: Gender Dimension
Authors: M, Justina ravickait{\.e}, Tomas Krilavi{\v{c}}ius
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4

Relation between gender and language has been studied by many authors, however, there is still some uncertainty left regarding gender influence on language usage in the professional environment. Often, the studied data sets are too small or texts of individual authors are too short in order to capture differences of language usage wrt gender successfully. This study draws from a larger corpus of speeches transcripts of the Lithuanian Parliament (1990-2013) to explore language differences of political debates by gender via stylometric analysis. Experimental set up consists of stylistic features that indicate lexical style and do not require external linguistic tools, namely the most frequent words, in combination with unsupervised machine learning algorithms. Results show that gender differences in the language use remain in professional environment not only in usage of function words, preferred linguistic constructions, but in the presented topics as well.
Towards Never Ending Language Learning for Morphologically Rich Languages
Authors: Kseniya Buraya, Lidia Pivovarova, Sergey Budkov, Andrey Filchenkov
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Information Retrieval, Specificity

This work deals with ontology learning from unstructured Russian text. We implement one of components Never Ending Language Learner and introduce the algorithm extensions aimed to gather specificity of morphologicaly rich free-word-order language. We demonstrate that this method may be successfully applied to Russian data. In addition we perform several additional experiments comparing different settings of the training process. We demonstrate that utilizing of morphological features significantly improves the system precision while using of seed patterns helps to improve the coverage.
Gender Profiling for Slovene Twitter communication: the Influence of Gender Marking, Content and Style
Authors: Ben Verhoeven, Iza {\v{S}}krjanec, Senja Pollak
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Gender Classification, General Classification, LEMMA, Lemmatization

We present results of the first gender classification experiments on Slovene text to our knowledge. Inspired by the TwiSty corpus and experiments (Verhoeven et al., 2016), we employed the Janes corpus (Erjavec et al., 2016) and its gender annotations to perform gender classification experiments on Twitter text comparing a token-based and a lemma-based approach. We find that the token-based approach (92.6{\%} accuracy), containing gender markings related to the author, outperforms the lemma-based approach by about 5{\%}. Especially in the lemmatized version, we also observe stylistic and content-based differences in writing between men (e.g. more profane language, numerals and beer mentions) and women (e.g. more pronouns, emoticons and character flooding). Many of our findings corroborate previous research on other languages.
Enriching Basque Coreference Resolution System using Semantic Knowledge sources
Authors: Ander Soraluze, Olatz Arregi, Xabier Arregi, Arantza D{\'\i}az de Ilarraza
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: coreference-resolution, Coreference Resolution, World Knowledge

In this paper we present a Basque coreference resolution system enriched with semantic knowledge. An error analysis carried out revealed the deficiencies that the system had in resolving coreference cases in which semantic or world knowledge is needed. We attempt to improve the deficiencies using two semantic knowledge sources, specifically Wikipedia and WordNet.
Improving Polish Mention Detection with Valency Dictionary
Authors: Maciej Ogrodniczuk, Bart{\l}omiej Nito{\'n}
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Coreference Resolution

This paper presents results of an experiment integrating information from valency dictionary of Polish into a mention detection system. Two types of information is acquired: positions of syntactic schemata for nominal and verbal constructs and secondary prepositions present in schemata. The syntactic schemata are used to prevent (for verbal realizations) or encourage (for nominal groups) constructing mentions from phrases filling multiple schema positions, the secondary prepositions {--} to filter out artificial mentions created from their nominal components. Mention detection is evaluated against the manual annotation of the Polish Coreference Corpus in two settings: taking into account only mention heads or exact borders.
A Google-Proof Collection of French Winograd Schemas
Authors: Pascal Amsili, Olga Seminck
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Coreference Resolution, World Knowledge

This article presents the first collection of French Winograd Schemas. Winograd Schemas form anaphora resolution problems that can only be resolved with extensive world knowledge. For this reason the Winograd Schema Challenge has been proposed as an alternative to the Turing Test. A very important feature of Winograd Schemas is that it should be impossible to resolve them with statistical information about word co-occurrences: they should be Google-proof. We propose a measure of Google-proofness based on Mutual Information, and demonstrate the method on our collection of French Winograd Schemas.
Using Coreference Links to Improve Spanish-to-English Machine Translation
Authors: Lesly Miculicich Werlen, Andrei Popescu-Belis
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Coreference Resolution, Decoder, Diversity, Document Level Machine Translation, Machine Translation, Sentence, Translation

In this paper, we present a proof-of-concept implementation of a coreference-aware decoder for document-level machine translation. We consider that better translations should have coreference links that are closer to those in the source text, and implement this criterion in two ways. First, we define a similarity measure between source and target coreference structures, by projecting the target ones onto the source and reusing existing coreference metrics. Based on this similarity measure, we re-rank the translation hypotheses of a baseline system for each sentence. Alternatively, to address the lack of diversity of mentions in the MT hypotheses, we focus on mention pairs and integrate their coreference scores with MT ones, resulting in post-editing decisions for mentions. The experimental results for Spanish to English MT on the AnCora-ES corpus show that the second approach yields a substantial increase in the accuracy of pronoun translation, with BLEU scores remaining constant.
Multi-source annotation projection of coreference chains: assessing strategies and testing opportunities
Authors: Yulia Grishina, Manfred Stede
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Coreference Resolution, Named Entity Recognition (NER)

In this paper, we examine the possibility of using annotation projection from multiple sources for automatically obtaining coreference annotations in the target language. We implement a multi-source annotation projection algorithm and apply it on an English-German-Russian parallel corpus in order to transfer coreference chains from two sources to the target side. Operating in two settings {--} a low-resource and a more linguistically-informed one {--} we show that automatic coreference transfer could benefit from combining information from multiple languages, and assess the quality of both the extraction and the linking of target coreference mentions.
CORBON 2017 Shared Task: Projection-Based Coreference Resolution
Authors: Yulia Grishina
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: coreference-resolution, Coreference Resolution, Word Alignment

The CORBON 2017 Shared Task, organised as part of the Coreference Resolution Beyond OntoNotes workshop at EACL 2017, presented a new challenge for multilingual coreference resolution: we offer a projection-based setting in which one is supposed to build a coreference resolver for a new language exploiting little or even no knowledge of it, with our languages of interest being German and Russian. We additionally offer a more traditional setting, targeting the development of a multilingual coreference resolver without any restrictions on the resources and methods used. In this paper, we describe the task setting and provide the results of one participant who successfully completed the task, comparing their results to the closely related previous research. Analysing the task setting and the results, we discuss the major challenges and make suggestions on the future directions of coreference evaluation.
Projection-based Coreference Resolution Using Deep Syntax
Authors: Michal Nov{\'a}k, Anna Nedoluzhko, Zden{\v{e}}k {\v{Z}}abokrtsk{\'y}
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: coreference-resolution, Coreference Resolution

The paper describes the system for coreference resolution in German and Russian, trained exclusively on coreference relations project ed through a parallel corpus from English. The resolver operates on the level of deep syntax and makes use of multiple specialized models. It achieves 32 and 22 points in terms of CoNLL score for Russian and German, respectively. Analysis of the evaluation results show that the resolver for Russian is able to preserve 66{\%} of the English resolver{'}s quality in terms of CoNLL score. The system was submitted to the Closed track of the CORBON 2017 Shared task.
Gender as a Variable in Natural-Language Processing: Ethical Considerations
Authors: Brian Larson
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4

Researchers and practitioners in natural-language processing (NLP) and related fields should attend to ethical principles in study design, ascription of categories/variables to study participants, and reporting of findings or results. This paper discusses theoretical and ethical frameworks for using gender as a variable in NLP studies and proposes four guidelines for researchers and practitioners. The principles outlined here should guide practitioners, researchers, and peer reviewers, and they may be applicable to other social categories, such as race, applied to human beings connected to NLP research.
These are not the Stereotypes You are Looking For: Bias and Fairness in Authorial Gender Attribution
Authors: Corina Koolen, Andreas van Cranenburgh
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Descriptive, Fairness, General Classification, Text Categorization

Stylometric and text categorization results show that author gender can be discerned in texts with relatively high accuracy. However, it is difficult to explain what gives rise to these results and there are many possible confounding factors, such as the domain, genre, and target audience of a text. More fundamentally, such classification efforts risk invoking stereotyping and essentialism. We explore this issue in two datasets of Dutch literary novels, using commonly used descriptive (LIWC, topic modeling) and predictive (machine learning) methods. Our results show the importance of controlling for variables in the corpus and we argue for taking care not to overgeneralize from the results.
A Quantitative Study of Data in the NLP community
Authors: Margot Mieskes
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Speech Recognition

We present results on a quantitative analysis of publications in the NLP domain on collecting, publishing and availability of research data. We find that a wide range of publications rely on data crawled from the web, but few give details on how potentially sensitive data was treated. Additionally, we find that while links to repositories of data are given, they often do not work even a short time after publication. We put together several suggestions on how to improve this situation based on publications from the NLP domain, but also other research areas.
Ethical by Design: Ethics Best Practices for Natural Language Processing
Authors: Jochen L. Leidner, Vassilis Plachouras
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Ethics

Natural language processing (NLP) systems analyze and/or generate human language, typically on users{'} behalf. One natural and necessary question that needs to be addressed in this context, both in research projects and in production settings, is the question how ethical the work is, both regarding the process and its outcome. Towards this end, we articulate a set of issues, propose a set of best practices, notably a process featuring an ethics review board, and sketch and how they could be meaningfully applied. Our main argument is that ethical outcomes ought to be achieved by design, i.e. by following a process aligned by ethical values. We also offer some response options for those facing ethics issues. While a number of previous works exist that discuss ethical issues, in particular around big data and machine learning, to the authors{'} knowledge this is the first account of NLP and ethics from the perspective of a principled process.
Building Better Open-Source Tools to Support Fairness in Automated Scoring
Authors: Nitin Madnani, Anastassia Loukina, Alina von Davier, Jill Burstein, Aoife Cahill
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Fairness

Automated scoring of written and spoken responses is an NLP application that can significantly impact lives especially when deployed as part of high-stakes tests such as the GRE® and the TOEFL®. Ethical considerations require that automated scoring algorithms treat all test-takers fairly. The educational measurement community has done significant research on fairness in assessments and automated scoring systems must incorporate their recommendations. The best way to do that is by making available automated, non-proprietary tools to NLP researchers that directly incorporate these recommendations and generate the analyses needed to help identify and resolve biases in their scoring systems. In this paper, we attempt to provide such a solution.
Gender and Dialect Bias in YouTube's Automatic Captions
Authors: Rachael Tatman
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition, TAG

This project evaluates the accuracy of YouTube{'}s automatically-generated captions across two genders and five dialect groups. Speakers{'} dialect and gender was controlled for by using videos uploaded as part of the {``}accent tag challenge{''}, where speakers explicitly identify their language background. The results show robust differences in accuracy across both gender and dialect, with lower accuracy for 1) women and 2) speakers from Scotland. This finding builds on earlier research finding that speaker{'}s sociolinguistic identity may negatively impact their ability to use automatic speech recognition, and demonstrates the need for sociolinguistically-stratified validation of systems.
Integrating the Management of Personal Data Protection and Open Science with Research Ethics
Authors: Dave Lewis, Joss Moorkens, Kaniz Fatema
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Ethics, Management

We examine the impact of the EU General Data Protection Regulation and the push from research funders to provide open access research data on the current practices in Language Technology Research. We analyse the challenges that arise and the opportunities to address many of them through the use of existing open data practices. We discuss the impact of this also on current practice in research ethics.
Ethical Considerations in NLP Shared Tasks
Authors: Carla Parra Escart{\'\i}n, Wessel Reijers, Teresa Lynn, Joss Moorkens, Andy Way, Chao-Hong Liu
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Ethics, Machine Translation

Shared tasks are increasingly common in our field, and new challenges are suggested at almost every conference and workshop. However, as this has become an established way of pushing research forward, it is important to discuss how we researchers organise and participate in shared tasks, and make that information available to the community to allow further research improvements. In this paper, we present a number of ethical issues along with other areas of concern that are related to the competitive nature of shared tasks. As such issues could potentially impact on research ethics in the Natural Language Processing community, we also propose the development of a framework for the organisation of and participation in shared tasks that can help mitigate against these issues arising.
Social Bias in Elicited Natural Language Inferences
Authors: Rachel Rudinger, Ch May, ler, Benjamin Van Durme
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Language Modeling, Language Modelling, Natural Language Inference, Word Embeddings

We analyze the Stanford Natural Language Inference (SNLI) corpus in an investigation of bias and stereotyping in NLP data. The SNLI human-elicitation protocol makes it prone to amplifying bias and stereotypical associations, which we demonstrate statistically (using pointwise mutual information) and with qualitative examples.
Goal-Oriented Design for Ethical Machine Learning and NLP
Authors: Tyler Schnoebelen
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: BIG-bench Machine Learning

The argument made in this paper is that to act ethically in machine learning and NLP requires focusing on goals. NLP projects are often classificatory systems that deal with human subjects, which means that goals from people affected by the systems should be included. The paper takes as its core example a model that detects criminality, showing the problems of training data, categories, and outcomes. The paper is oriented to the kinds of critiques on power and the reproduction of inequality that are found in social theory, but it also includes concrete suggestions on how to put goal-oriented design into practice.
Ethical Research Protocols for Social Media Health Research
Authors: Adrian Benton, Glen Coppersmith, Mark Dredze
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Decision Making, Ethics

Social media have transformed data-driven research in political science, the social sciences, health, and medicine. Since health research often touches on sensitive topics that relate to ethics of treatment and patient privacy, similar ethical considerations should be acknowledged when using social media data in health research. While much has been said regarding the ethical considerations of social media research, health research leads to an additional set of concerns. We provide practical suggestions in the form of guidelines for researchers working with social media data in health research. These guidelines can inform an IRB proposal for researchers new to social media health research.
Say the Right Thing Right: Ethics Issues in Natural Language Generation Systems
Authors: Charese Smiley, Frank Schilder, Vassilis Plachouras, Jochen L. Leidner
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Ethics, Text Generation

We discuss the ethical implications of Natural Language Generation systems. We use one particular system as a case study to identify and classify issues, and we provide an ethics checklist, in the hope that future system designers may benefit from conducting their own ethics reviews based on our checklist.
ParaDi: Dictionary of Paraphrases of Czech Complex Predicates with Light Verbs
Authors: Petra Baran{\v{c}}{\'\i}kov{\'a}, V{\'a}clava Kettnerov{\'a}
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Information Retrieval, Machine Translation, Question Answering, Translation

We present a new freely available dictionary of paraphrases of Czech complex predicates with light verbs, ParaDi. Candidates for single predicative paraphrases of selected complex predicates have been extracted automatically from large monolingual data using word2vec. They have been manually verified and further refined. We demonstrate one of many possible applications of ParaDi in an experiment with improving machine translation quality.
Multi-word Entity Classification in a Highly Multilingual Environment
Authors: Sophie Chesney, Guillaume Jacquet, Ralf Steinberger, Jakub Piskorski
Date: 2017-04-01 00:00:00
Proceeding: WS 2017 4
Tasks: Classification, General Classification, Named Entity Recognition (NER)

This paper describes an approach for the classification of millions of existing multi-word entities (MWEntities), such as organisation or event names, into thirteen category types, based only on the tokens they contain. In order to classify our very large in-house collection of multilingual MWEntities into an application-oriented set of entity categories, we trained and tested distantly-supervised classifiers in 43 languages based on MWEntities extracted from BabelNet. The best-performing classifier was the multi-class SVM using a TF.IDF-weighted data representation. Interestingly, one unique classifier trained on a mix of all languages consistently performed better than classifiers trained for individual languages, reaching an averaged F1-value of 88.8{\%}. In this paper, we present the training and test data, including a human evaluation of its accuracy, describe the methods used to train the classifiers, and discuss the results.