pwc-archive/papers-with-abstracts huggingface dataset:
The Connection between the Text and Images of News Articles: New Insights for Multimedia Analysis
Authors: Nelleke Oostdijk, Hans van Halteren, Erkan Ba{\textcommabelow{s}}ar, Martha Larson
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Articles

We report on a case study of text and images that reveals the inadequacy of simplistic assumptions about their connection and interplay. The context of our work is a larger effort to create automatic systems that can extract event information from online news articles about flooding disasters. We carry out a manual analysis of 1000 articles containing a keyword related to flooding. The analysis reveals that the articles in our data set cluster into seven categories related to different topical aspects of flooding, and that the images accompanying the articles cluster into five categories related to the content they depict. The results demonstrate that flood-related news articles do not consistently report on a single, currently unfolding flooding event and we should also not assume that a flood-related image will directly relate to a flooding-event described in the corresponding article. In particular, spatiotemporal distance is important. We validate the manual analysis with an automatic classifier demonstrating the technical feasibility of multimedia analysis approaches that admit more realistic relationships between text and images. In sum, our case study confirms that closer attention to the connection between text and images has the potential to improve the collection of multimodal information from news articles.
LifeQA: A Real-life Dataset for Video Question Answering
Authors: Santiago Castro, Mahmoud Azab, Jonathan Stroud, Cristina Noujaim, Ruoyao Wang, Jia Deng, Rada Mihalcea
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Multiple-choice, Question Answering, Video Question Answering

We introduce LifeQA, a benchmark dataset for video question answering that focuses on day-to-day real-life situations. Current video question answering datasets consist of movies and TV shows. However, it is well-known that these visual domains are not representative of our day-to-day lives. Movies and TV shows, for example, benefit from professional camera movements, clean editing, crisp audio recordings, and scripted dialog between professional actors. While these domains provide a large amount of data for training models, their properties make them unsuitable for testing real-life question answering systems. Our dataset, by contrast, consists of video clips that represent only real-life scenarios. We collect 275 such video clips and over 2.3k multiple-choice questions. In this paper, we analyze the challenging but realistic aspects of LifeQA, and we apply several state-of-the-art video question answering models to provide benchmarks for future research. The full dataset is publicly available at https://lit.eecs.umich.edu/lifeqa/.
A Domain-Specific Dataset of Difficulty Ratings for German Noun Compounds in the Domains DIY, Cooking and Automotive
Authors: Julia Bettinger, Anna H{\"a}tty, Michael Dorna, Sabine Schulte im Walde
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

We present a dataset with difficulty ratings for 1,030 German closed noun compounds extracted from domain-specific texts for do-it-ourself (DIY), cooking and automotive. The dataset includes two-part compounds for cooking and DIY, and two- to four-part compounds for automotive. The compounds were identified in text using the Simple Compound Splitter (Weller-Di Marco, 2017); a subset was filtered and balanced for frequency and productivity criteria as basis for manual annotation and fine-grained interpretation. This study presents the creation, the final dataset with ratings from 20 annotators and statistics over the dataset, to provide insight into the perception of domain-specific term difficulty. It is particularly striking that annotators agree on a coarse, binary distinction between easy vs. difficult domain-specific compounds but that a more fine grained distinction of difficulty is not meaningful. We finally discuss the challenges of an annotation for difficulty, which includes both the task description as well as the selection of the data basis.
All That Glitters is Not Gold: A Gold Standard of Adjective-Noun Collocations for German
Authors: Yana Strakatova, Neele Falk, Isabel Fuhrmann, Erhard Hinrichs, Daniela Rossmann
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: All, Word Embeddings

In this paper we present the GerCo dataset of adjective-noun collocations for German, such as alter Freund {`}old friend{'} and tiefe Liebe {`}deep love{'}. The annotation has been performed by experts based on the annotation scheme introduced in this paper. The resulting dataset contains 4,732 positive and negative instances of collocations and covers all the 16 semantic classes of adjectives as defined in the German wordnet GermaNet. The dataset can serve as a reliable empirical basis for comparing different theoretical frameworks concerned with collocations or as material for data-driven approaches to the studies of collocations including different machine learning experiments. This paper addresses the latter issue by using the GerCo dataset for evaluating different models on the task of automatic collocation identification. We compare lexical association measures with static and contextualized word embeddings. The experiments show that word embeddings outperform methods based on statistical association measures by a wide margin.
Variants of Vector Space Reductions for Predicting the Compositionality of English Noun Compounds
Authors: Pegah Alipoor, Sabine Schulte im Walde
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

Predicting the degree of compositionality of noun compounds such as {``}snowball{''} and {``}butterfly{''} is a crucial ingredient for lexicography and Natural Language Processing applications, to know whether the compound should be treated as a whole, or through its constituents, and what it means. Computational approaches for an automatic prediction typically represent and compare compounds and their constituents within a vector space and use distributional similarity as a proxy to predict the semantic relatedness between the compounds and their constituents as the compound{'}s degree of compositionality. This paper provides a systematic evaluation of vector-space reduction variants across kinds, exploring reductions based on part-of-speech next to and also in combination with Principal Components Analysis using Singular Value and word2vec embeddings. We show that word2vec and nouns only dimensionality reductions are the most successful and stable vector space variants for our task.
Varying Vector Representations and Integrating Meaning Shifts into a PageRank Model for Automatic Term Extraction
Authors: Anurag Nigam, Anna H{\"a}tty, Sabine Schulte im Walde
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Term Extraction

We perform a comparative study for automatic term extraction from domain-specific language using a PageRank model with different edge-weighting methods. We vary vector space representations within the PageRank graph algorithm, and we go beyond standard co-occurrence and investigate the influence of measures of association strength and first- vs. second-order co-occurrence. In addition, we incorporate meaning shifts from general to domain-specific language as personalized vectors, in order to distinguish between termhood strengths of ambiguous words across word senses. Our study is performed for two domain-specific English corpora: ACL and do-it-yourself (DIY); and a domain-specific German corpus: cooking. The models are assessed by applying average precision and the roc score as evaluation metrices.
Rigor Mortis: Annotating MWEs with a Gamified Platform
Authors: Kar{\"e}n Fort, Bruno Guillaume, Yann-Alan Pilatte, Mathieu Constant, Nicolas Lef{\`e}bvre
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

We present here Rigor Mortis, a gamified crowdsourcing platform designed to evaluate the intuition of the speakers, then train them to annotate multi-word expressions (MWEs) in French corpora. We previously showed that the speakers{'} intuition is reasonably good (65{\%} in recall on non-fixed MWE). We detail here the annotation results, after a training phase using some of the tests developed in the PARSEME-FR project.
A Multi-word Expression Dataset for Swedish
Authors: Murathan Kurfal{\i}, Robert {\"O}stling, Johan Sjons, Mats Wir{\'e}n
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

We present a new set of 96 Swedish multi-word expressions annotated with degree of (non-)compositionality. In contrast to most previous compositionality datasets we also consider syntactically complex constructions and publish a formal specification of each expression. This allows evaluation of computational models beyond word bigrams, which have so far been the norm. Finally, we use the annotations to evaluate a system for automatic compositionality estimation based on distributional semantics. Our analysis of the disagreements between human annotators and the distributional model reveal interesting questions related to the perception of compositionality, and should be informative to future work in the area.
Dedicated Language Resources for Interdisciplinary Research on Multiword Expressions: Best Thing since Sliced Bread
Authors: Ferdy Hubers, Catia Cucchiarini, Helmer Strik
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Language Acquisition

Multiword expressions such as idioms (beat about the bush), collocations (plastic surgery) and lexical bundles (in the middle of) are challenging for disciplines like Natural Language Processing (NLP), psycholinguistics and second language acquisition, , due to their more or less fixed character. Idiomatic expressions are especially problematic, because they convey a figurative meaning that cannot always be inferred from the literal meanings of the component words. Researchers acknowledge that important properties that characterize idioms such as frequency of exposure, familiarity, transparency, and imageability, should be taken into account in research, but these are typically properties that rely on subjective judgments. This is probably one of the reasons why many studies that investigated idiomatic expressions collected limited information about idiom properties for very small numbers of idioms only. In this paper we report on cross-boundary work aimed at developing a set of tools and language resources that are considered crucial for this kind of multifaceted research. We discuss the results of our research and suggest possible avenues for future research
Introducing RONEC - the Romanian Named Entity Corpus
Authors: Stefan Daniel Dumitrescu, Andrei-Marius Avram
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

We present RONEC - the Named Entity Corpus for the Romanian language. The corpus contains over 26000 entities in {\textasciitilde}5000 annotated sentences, belonging to 16 distinct classes. The sentences have been extracted from a copy-right free newspaper, covering several styles. This corpus represents the first initiative in the Romanian language space specifically targeted for named entity recognition. It is available in BRAT and CoNLL-U Plus formats, and it is free to use and extend at github.com/dumitrescustefan/ronec
A Semi-supervised Approach for De-identification of Swedish Clinical Text
Authors: Hanna Berg, Hercules Dalianis
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: De-identification

An abundance of electronic health records (EHR) is produced every day within healthcare. The records possess valuable information for research and future improvement of healthcare. Multiple efforts have been done to protect the integrity of patients while making electronic health records usable for research by removing personally identifiable information in patient records. Supervised machine learning approaches for de-identification of EHRs need annotated data for training, annotations that are costly in time and human resources. The annotation costs for clinical text is even more costly as the process must be carried out in a protected environment with a limited number of annotators who must have signed confidentiality agreements. In this paper is therefore, a semi-supervised method proposed, for automatically creating high-quality training data. The study shows that the method can be used to improve recall from 84.75{\%} to 89.20{\%} without sacrificing precision to the same extent, dropping from 95.73{\%} to 94.20{\%}. The model{'}s recall is arguably more important for de-identification than precision.
Czech Historical Named Entity Corpus v 1.0
Authors: Helena Hubkov{\'a}, Pavel Kral, Eva Pettersson
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Word Embeddings

As the number of digitized archival documents increases very rapidly, named entity recognition (NER) in historical documents has become very important for information extraction and data mining. For this task an annotated corpus is needed, which has up to now been missing for Czech. In this paper we present a new annotated data collection for historical NER, composed of Czech historical newspapers. This corpus is freely available for research purposes. For this corpus, we have defined relevant domain-specific named entity types and created an annotation manual for corpus labelling. We further conducted some experiments on this corpus using recurrent neural networks. We experimented with randomly initialized embeddings and static and dynamic fastText word embeddings. We achieved 0.73 F1 score with a bidirectional LSTM model using static fastText embeddings.
CodE Alltag 2.0 --- A Pseudonymized German-Language Email Corpus
Authors: Elisabeth Eder, Ulrike Krieg-Holz, Udo Hahn
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: De-identification

The vast amount of social communication distributed over various electronic media channels (tweets, blogs, emails, etc.), so-called user-generated content (UGC), creates entirely new opportunities for today{'}s NLP research. Yet, data privacy concerns implied by the unauthorized use of these text streams as a data resource are often neglected. In an attempt to reconciliate the diverging needs of unconstrained raw data use and preservation of data privacy in digital communication, we here investigate the automatic recognition of privacy-sensitive stretches of text in UGC and provide an algorithmic solution for the protection of personal data via pseudonymization. Our focus is directed at the de-identification of emails where personally identifying information does not only refer to the sender but also to those people, locations, dates, and other identifiers mentioned in greetings, boilerplates and the content-carrying body of emails. We evaluate several de-identification procedures and systems on two hitherto non-anonymized German-language email corpora (CodE AlltagS+d and CodE AlltagXL), and generate fully pseudonymized versions for both (CodE Alltag 2.0) in which personally identifying information of all social actors addressed in these mails has been camouflaged (to the greatest extent possible).
Hedwig: A Named Entity Linker
Authors: Marcus Klang, Pierre Nugues
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Entity Linking

Named entity linking is the task of identifying mentions of named things in text, such as {``}Barack Obama{''} or {``}New York{''}, and linking these mentions to unique identifiers. In this paper, we describe Hedwig, an end-to-end named entity linker, which uses a combination of word and character BILSTM models for mention detection, a Wikidata and Wikipedia-derived knowledge base with global information aggregated over nine language editions, and a PageRank algorithm for entity linking. We evaluated Hedwig on the TAC2017 dataset, consisting of news texts and discussion forums, and we obtained a final score of 59.9{\%} on CEAFmC+, an improvement over our previous generation linker Ugglan, and a trilingual entity link score of 71.9{\%}.
An Experiment in Annotating Animal Species Names from ISTEX Resources
Authors: Sabine Barreaux, Dominique Besagni
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

To exploit scientific publications from global research for TDM purposes, the ISTEX platform enriched its data with value-added information to ease access to its full-text documents. We built an experiment to explore new enrichment possibilities in documents focussing on scientific named entities recognition which could be integrated into ISTEX resources. This led to testing two detection tools for animal species names in a corpus of 100 documents in zoology. This makes it possible to provide the French scientific community with an annotated reference corpus available for use to measure these tools{'} performance.
Where are we in Named Entity Recognition from Speech?
Authors: Antoine Caubri{\`e}re, Sophie Rosset, Yannick Est{\`e}ve, Antoine Laurent, Emmanuel Morin
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Entity Extraction using GAN, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, speech-recognition, Speech Recognition

Named entity recognition (NER) from speech is usually made through a pipeline process that consists in (i) processing audio using an automatic speech recognition system (ASR) and (ii) applying a NER to the ASR outputs. The latest data available for named entity extraction from speech in French were produced during the ETAPE evaluation campaign in 2012. Since the publication of ETAPE{'}s campaign results, major improvements were done on NER and ASR systems, especially with the development of neural approaches for both of these components. In addition, recent studies have shown the capability of End-to-End (E2E) approach for NER / SLU tasks. In this paper, we propose a study of the improvements made in speech recognition and named entity recognition for pipeline approaches. For this type of systems, we propose an original 3-pass approach. We also explore the capability of an E2E system to do structured NER. Finally, we compare the performances of ETAPE{'}s systems (state-of-the-art systems in 2012) with the performances obtained using current technologies. The results show the interest of the E2E approach, which however remains below an updated pipeline approach.
Tagging Location Phrases in Text
Authors: Paul McNamee, James Mayfield, Cash Costello, Caitlyn Bishop, Shelby Anderson
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Humanitarian

For over thirty years researchers have studied the problem of automatically detecting named entities in written language. Throughout this time the majority of such work has focused on detection and classification of entities into coarse-grained types like: PERSON, ORGANIZATION, and LOCATION. Less attention has been focused on non-named mentions of entities, including non-named location phrases such as {``}the medical clinic in Telonge{''} or {``}2 km below the Dolin Maniche bridge{''}. In this work we describe the Location Phrase Detection task to identify such spans. Our key accomplishments include: developing a sequential tagging approach; crafting annotation guidelines; building annotated datasets for English and Russian news; and, conducting experiments in automated detection of location phrases with both statistical and neural taggers. This work is motivated by extracting rich location information to support situational awareness during humanitarian crises such as natural disasters.
Tag Me If You Can! Semantic Annotation of Biodiversity Metadata with the QEMP Corpus and the BiodivTagger
Authors: Felicitas L{\"o}ffler, Nora Abdelmageed, Samira Babalou, Paw Kaur, eep, Birgitta K{\"o}nig-Ries
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Retrieval, TAG

Dataset Retrieval is gaining importance due to a large amount of research data and the great demand for reusing scientific data. Dataset Retrieval is mostly based on metadata, structured information about the primary data. Enriching these metadata with semantic annotations based on Linked Open Data (LOD) enables datasets, publications and authors to be connected and expands the search on semantically related terms. In this work, we introduce the BiodivTagger, an ontology-based Information Extraction pipeline, developed for metadata from biodiversity research. The system recognizes biological, physical and chemical processes, environmental terms, data parameters and phenotypes as well as materials and chemical compounds and links them to concepts in dedicated ontologies. To evaluate our pipeline, we created a gold standard of 50 metadata files (QEMP corpus) selected from five different data repositories in biodiversity research. To the best of our knowledge, this is the first annotated metadata corpus for biodiversity research data. The results reveal a mixed picture. While materials and data parameters are properly matched to ontological concepts in most cases, some ontological issues occurred for processes and environmental terms.
Towards a Versatile Medical-Annotation Guideline Feasible Without Heavy Medical Knowledge: Starting From Critical Lung Diseases
Authors: Shuntaro Yada, Ayami Joh, Ribeka Tanaka, Fei Cheng, Eiji Aramaki, Sadao Kurohashi
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Applying natural language processing (NLP) to medical and clinical texts can bring important social benefits by mining valuable information from unstructured text. A popular application for that purpose is named entity recognition (NER), but the annotation policies of existing clinical corpora have not been standardized across clinical texts of different types. This paper presents an annotation guideline aimed at covering medical documents of various types such as radiography interpretation reports and medical records. Furthermore, the annotation was designed to avoid burdensome requirements related to medical knowledge, thereby enabling corpus development without medical specialists. To achieve these design features, we specifically focus on critical lung diseases to stabilize linguistic patterns in corpora. After annotating around 1100 electronic medical records following the annotation scheme, we demonstrated its feasibility using an NER task. Results suggest that our guideline is applicable to large-scale clinical NLP projects.
Creating a Dataset for Named Entity Recognition in the Archaeology Domain
Authors: Br, Alex sen, Suzan Verberne, Milco Wansleeben, Karsten Lambers
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: BIG-bench Machine Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

In this paper, we present the development of a training dataset for Dutch Named Entity Recognition (NER) in the archaeology domain. This dataset was created as there is a dire need for semantic search within archaeology, in order to allow archaeologists to find structured information in collections of Dutch excavation reports, currently totalling around 60,000 (658 million words) and growing rapidly. To guide this search task, NER is needed. We created rigorous annotation guidelines in an iterative process, then instructed five archaeology students to annotate a number of documents. The resulting dataset contains {\textasciitilde}31k annotations between six entity types (artefact, time period, place, context, species {\&} material). The inter-annotator agreement is 0.95, and when we used this data for machine learning, we observed an increase in F1 score from 0.51 to 0.70 in comparison to a machine learning model trained on a dataset created in prior work. This indicates that the data is of high quality, and can confidently be used to train NER classifiers.