pwc-archive/papers-with-abstracts huggingface dataset:
Modeling Paths for Explainable Knowledge Base Completion
Authors: Josua Stadelmaier, Sebastian Pad{\'o}
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Knowledge Base Completion

A common approach in knowledge base completion (KBC) is to learn representations for entities and relations in order to infer missing facts by generalizing existing ones. A shortcoming of standard models is that they do not explain their predictions to make them verifiable easily to human inspection. In this paper, we propose the Context Path Model (CPM) which generates explanations for new facts in KBC by providing sets of \textit{context paths} as supporting evidence for these triples. For example, a new triple (Theresa May, nationality, Britain) may be explained by the path (Theresa May, born in, Eastbourne, contained in, Britain). The CPM is formulated as a wrapper that can be applied on top of various existing KBC models. We evaluate it for the well-established TransE model. We observe that its performance remains very close despite the added complexity, and that most of the paths proposed as explanations provide meaningful evidence to assess the correctness.
Probing Word and Sentence Embeddings for Long-distance Dependencies Effects in French and English
Authors: Paola Merlo
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Sentence, Sentence Embeddings, Word Embeddings

The recent wide-spread and strong interest in RNNs has spurred detailed investigations of the distributed representations they generate and specifically if they exhibit properties similar to those characterising human languages. Results are at present inconclusive. In this paper, we extend previous work on long-distance dependencies in three ways. We manipulate word embeddings to translate them in a space that is attuned to the linguistic properties under study. We extend the work to sentence embeddings and to new languages. We confirm previous negative results: word embeddings and sentence embeddings do not unequivocally encode fine-grained linguistic properties of long-distance dependencies.
Derivational Morphological Relations in Word Embeddings
Authors: Tom{\'a}{\v{s}} Musil, Jon{\'a}{\v{s}} Vidra, David Mare{\v{c}}ek
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Clustering, Word Embeddings

Derivation is a type of a word-formation process which creates new words from existing ones by adding, changing or deleting affixes. In this paper, we explore the potential of word embeddings to identify properties of word derivations in the morphologically rich Czech language. We extract derivational relations between pairs of words from DeriNet, a Czech lexical network, which organizes almost one million Czech lemmas into derivational trees. For each such pair, we compute the difference of the embeddings of the two words, and perform unsupervised clustering of the resulting vectors. Our results show that these clusters largely match manually annotated semantic categories of the derivational relations (e.g. the relation {`}bake{--}baker{'} belongs to category {`}actor{'}, and a correct clustering puts it into the same cluster as {`}govern{--}governor{'}).
Blackbox Meets Blackbox: Representational Similarity \& Stability Analysis of Neural Language Models and Brains
Authors: Samira Abnar, Lisa Beinborn, Rochelle Choenni, Willem Zuidema
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

In this paper, we define and apply representational stability analysis (ReStA), an intuitive way of analyzing neural language models. ReStA is a variant of the popular representational similarity analysis (RSA) in cognitive neuroscience. While RSA can be used to compare representations in models, model components, and human brains, ReStA compares instances of the same model, while systematically varying single model parameter. Using ReStA, we study four recent and successful neural language models, and evaluate how sensitive their internal representations are to the amount of prior context. Using RSA, we perform a systematic study of how similar the representational spaces in the first and second (or higher) layers of these models are to each other and to patterns of activation in the human brain. Our results reveal surprisingly strong differences between language models, and give insights into where the deep linguistic processing, that integrates information over multiple sentences, is happening in these models. The combination of ReStA and RSA on models and brains allows us to start addressing the important question of what kind of linguistic processes we can hope to observe in fMRI brain imaging data. In particular, our results suggest that the data on story reading from Wehbe et al./ (2014) contains a signal of shallow linguistic processing, but show no evidence on the more interesting deep linguistic processing.
An LSTM Adaptation Study of (Un)grammaticality
Authors: Shammur Absar Chowdhury, Roberto Zamparelli
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: CoLA, Language Modeling, Language Modelling

We propose a novel approach to the study of how artificial neural network perceive the distinction between grammatical and ungrammatical sentences, a crucial task in the growing field of synthetic linguistics. The method is based on performance measures of language models trained on corpora and fine-tuned with either grammatical or ungrammatical sentences, then applied to (different types of) grammatical or ungrammatical sentences. The results show that both in the difficult and highly symmetrical task of detecting subject islands and in the more open CoLA dataset, grammatical sentences give rise to better scores than ungrammatical ones, possibly because they can be better integrated within the body of linguistic structural knowledge that the language model has accumulated.
Finding Hierarchical Structure in Neural Stacks Using Unsupervised Parsing
Authors: William Merrill, Lenny Khazan, Noah Amsel, Yiding Hao, Simon Mendelsohn, Robert Frank
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Language Modeling, Language Modelling

Neural network architectures have been augmented with differentiable stacks in order to introduce a bias toward learning hierarchy-sensitive regularities. It has, however, proven difficult to assess the degree to which such a bias is effective, as the operation of the differentiable stack is not always interpretable. In this paper, we attempt to detect the presence of latent representations of hierarchical structure through an exploration of the unsupervised learning of constituency structure. Using a technique due to Shen et al. (2018a,b), we extract syntactic trees from the pushing behavior of stack RNNs trained on language modeling and classification objectives. We find that our models produce parses that reflect natural language syntactic constituencies, demonstrating that stack RNNs do indeed infer linguistically relevant hierarchical structure.
GEval: Tool for Debugging NLP Datasets and Models
Authors: Filip Grali{\'n}ski, Anna Wr{\'o}blewska, Tomasz Stanis{\l}awek, Kamil Grabowski, Tomasz G{\'o}recki
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

This paper presents a simple but general and effective method to debug the output of machine learning (ML) supervised models, including neural networks. The algorithm looks for features that lower the evaluation metric in such a way that it cannot be ascribed to chance (as measured by their p-values). Using this method {--} implemented as MLEval tool {--} you can find: (1) anomalies in test sets, (2) issues in preprocessing, (3) problems in the ML model itself. It can give you an insight into what can be improved in the datasets and/or the model. The same method can be used to compare ML models or different versions of the same model. We present the tool, the theory behind it and use cases for text-based models of various types.
Learning from the Experience of Doctors: Automated Diagnosis of Appendicitis Based on Clinical Notes
Authors: Steven Kester Yuwono, Hwee Tou Ng, Kee Yuan Ngiam
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Feature Engineering

The objective of this work is to develop an automated diagnosis system that is able to predict the probability of appendicitis given a free-text emergency department (ED) note and additional structured information (e.g., lab test results). Our clinical corpus consists of about 180,000 ED notes based on ten years of patient visits to the Accident and Emergency (A{\&}E) Department of the National University Hospital (NUH), Singapore. We propose a novel neural network approach that learns to diagnose acute appendicitis based on doctors{'} free-text ED notes without any feature engineering. On a test set of 2,000 ED notes with equal number of appendicitis (positive) and non-appendicitis (negative) diagnosis and in which all the negative ED notes only consist of abdominal-related diagnosis, our model is able to achieve a promising F{\_}0.5-score of 0.895 while ED doctors achieve F{\_}0.5-score of 0.900. Visualization shows that our model is able to learn important features, signs, and symptoms of patients from unstructured free-text ED notes, which will help doctors to make better diagnosis.
A Paraphrase Generation System for EHR Question Answering
Authors: Sarvesh Soni, Kirk Roberts
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Decoder, Paraphrase Generation, Question Answering

This paper proposes a dataset and method for automatically generating paraphrases for clinical questions relating to patient-specific information in electronic health records (EHRs). Crowdsourcing is used to collect 10,578 unique questions across 946 semantically distinct paraphrase clusters. This corpus is then used with a deep learning-based question paraphrasing method utilizing variational autoencoder and LSTM encoder/decoder. The ultimate use of such a method is to improve the performance of automatic question answering methods for EHRs.
Analysing Representations of Memory Impairment in a Clinical Notes Classification Model
Authors: Mark Ormerod, Jes{\'u}s Mart{\'\i}nez-del-Rinc{\'o}n, Neil Robertson, Bernadette McGuinness, Barry Devereux
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Document Classification, General Classification, Sentence, Sentence Embedding, Sentence-Embedding

Despite recent advances in the application of deep neural networks to various kinds of medical data, extracting information from unstructured textual sources remains a challenging task. The challenges of training and interpreting document classification models are amplified when dealing with small and highly technical datasets, as are common in the clinical domain. Using a dataset of de-identified clinical letters gathered at a memory clinic, we construct several recurrent neural network models for letter classification, and evaluate them on their ability to build meaningful representations of the documents and predict patients{'} diagnoses. Additionally, we probe sentence embedding models in order to build a human-interpretable representation of the neural network{'}s features, using a simple and intuitive technique based on perturbative approaches to sentence importance. In addition to showing which sentences in a document are most informative about the patient{'}s condition, this method reveals the types of sentences that lead the model to make incorrect diagnoses. Furthermore, we identify clusters of sentences in the embedding space that correlate strongly with importance scores for each clinical diagnosis class.
Combining Structured and Free-text Electronic Medical Record Data for Real-time Clinical Decision Support
Authors: Emilia Apostolova, Tony Wang, Tim Tschampel, Ioannis Koutroulis, Tom Velez
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

The goal of this work is to utilize Electronic Medical Record (EMR) data for real-time Clinical Decision Support (CDS). We present a deep learning approach to combining in real time available diagnosis codes (ICD codes) and free-text notes: Patient Context Vectors. Patient Context Vectors are created by averaging ICD code embeddings, and by predicting the same from free-text notes via a Convolutional Neural Network. The Patient Context Vectors were then simply appended to available structured data (vital signs and lab results) to build prediction models for a specific condition. Experiments on predicting ARDS, a rare and complex condition, demonstrate the utility of Patient Context Vectors as a means of summarizing the patient history and overall condition, and improve significantly the prediction model results.
MoNERo: a Biomedical Gold Standard Corpus for the Romanian Language
Authors: Maria Mitrofan, Verginica Barbu Mititelu, Grigorina Mitrofan
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

In an era when large amounts of data are generated daily in various fields, the biomedical field among others, linguistic resources can be exploited for various tasks of Natural Language Processing. Moreover, increasing number of biomedical documents are available in languages other than English. To be able to extract information from natural language free text resources, methods and tools are needed for a variety of languages. This paper presents the creation of the MoNERo corpus, a gold standard biomedical corpus for Romanian, annotated with both part of speech tags and named entities. MoNERo comprises 154,825 morphologically annotated tokens and 23,188 entity annotations belonging to four entity semantic groups corresponding to UMLS Semantic Groups.
Domain Adaptation of SRL Systems for Biological Processes
Authors: Dheeraj Rajagopal, Nidhi Vyas, Aditya Siddhant, Anirudha Rayasam, T, Niket on, Eduard Hovy
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Domain Adaptation, Semantic Role Labeling

Domain adaptation remains one of the most challenging aspects in the wide-spread use of Semantic Role Labeling (SRL) systems. Current state-of-the-art methods are typically trained on large-scale datasets, but their performances do not directly transfer to low-resource domain-specific settings. In this paper, we propose two approaches for domain adaptation in the biological domain that involves pre-training LSTM-CRF based on existing large-scale datasets and adapting it for a low-resource corpus of biological processes. Our first approach defines a mapping between the source labels and the target labels, and the other approach modifies the final CRF layer in sequence-labeling neural network architecture. We perform our experiments on ProcessBank dataset which contains less than 200 paragraphs on biological processes. We improve over the previous state-of-the-art system on this dataset by 21 F1 points. We also show that, by incorporating event-event relationship in ProcessBank, we are able to achieve an additional 2.6 F1 gain, giving us possible insights into how to improve SRL systems for biological process using richer annotations.
RNN Embeddings for Identifying Difficult to Understand Medical Words
Authors: Hanna Pylieva, Artem Chernodub, Natalia Grabar, Thierry Hamon
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Word Embeddings

Patients and their families often require a better understanding of medical information provided by doctors. We currently address this issue by improving the identification of difficult to understand medical words. We introduce novel embeddings received from RNN - FrnnMUTE (French RNN Medical Understandability Text Embeddings) which allow to reach up to 87.0 F1 score in identification of difficult words. We also note that adding pre-trained FastText word embeddings to the feature set substantially improves the performance of the model which classifies words according to their difficulty. We study the generalizability of different models through three cross-validation scenarios which allow testing classifiers in real-world conditions: understanding of medical words by new users, and classification of new unseen words by the automatic models. The RNN - FrnnMUTE embeddings and the categorization code are being made available for the research.
A distantly supervised dataset for automated data extraction from diagnostic studies
Authors: Christopher Norman, Mariska Leeflang, Ren{\'e} Spijker, Evangelos Kanoulas, Aur{\'e}lie N{\'e}v{\'e}ol
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Articles, Diagnostic

Systematic reviews are important in evidence based medicine, but are expensive to produce. Automating or semi-automating the data extraction of index test, target condition, and reference standard from articles has the potential to decrease the cost of conducting systematic reviews of diagnostic test accuracy, but relevant training data is not available. We create a distantly supervised dataset of approximately 90,000 sentences, and let two experts manually annotate a small subset of around 1,000 sentences for evaluation. We evaluate the performance of BioBERT and logistic regression for ranking the sentences, and compare the performance for distant and direct supervision. Our results suggest that distant supervision can work as well as, or better than direct supervision on this problem, and that distantly trained models can perform as well as, or better than human annotators.
Query selection methods for automated corpora construction with a use case in food-drug interactions
Authors: Georgeta Bordea, R, Tsanta riatsitohaina, Fleur Mougin, Natalia Grabar, Thierry Hamon
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Articles, feature selection

In this paper, we address the problem of automatically constructing a relevant corpus of scientific articles about food-drug interactions. There is a growing number of scientific publications that describe food-drug interactions but currently building a high-coverage corpus that can be used for information extraction purposes is not trivial. We investigate several methods for automating the query selection process using an expert-curated corpus of food-drug interactions. Our experiments show that index term features along with a decision tree classifier are the best approach for this task and that feature selection approaches and in particular gain ratio outperform frequency-based methods for query selection.
Enhancing biomedical word embeddings by retrofitting to verb clusters
Authors: Billy Chiu, Simon Baker, Martha Palmer, Anna Korhonen
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Event Extraction, Relation, Relation Extraction, text-classification, Text Classification, Word Embeddings

Verbs play a fundamental role in many biomed-ical tasks and applications such as relation and event extraction. We hypothesize that performance on many downstream tasks can be improved by aligning the input pretrained embeddings according to semantic verb classes.In this work, we show that by using semantic clusters for verbs, a large lexicon of verbclasses derived from biomedical literature, weare able to improve the performance of common pretrained embeddings in downstream tasks by retrofitting them to verb classes. We present a simple and computationally efficient approach using a widely-available {``}off-the-shelf{''} retrofitting algorithm to align pretrained embeddings according to semantic verb clusters. We achieve state-of-the-art results on text classification and relation extraction tasks.
First Steps towards Building a Medical Lexicon for Spanish with Linguistic and Semantic Information
Authors: Leonardo Campillos-Llanos
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

We report the work-in-progress of collecting MedLexSp, an unified medical lexicon for the Spanish language, featuring terms and inflected word forms mapped to Unified Medical Language System (UMLS) Concept Unique Identifiers (CUIs), semantic types and groups. First, we leveraged a list of term lemmas and forms from a previous project, and mapped them to UMLS terms and CUIs. To enrich the lexicon, we used both domain-corpora (e.g. Summaries of Product Characteristics and MedlinePlus) and natural language processing techniques such as string distance methods or generation of syntactic variants of multi-word terms. We also added term variants by mapping their CUIs to missing items available in the Spanish versions of standard thesauri (e.g. Medical Subject Headings and World Health Organization Adverse Drug Reactions terminology). We enhanced the vocabulary coverage by gathering missing terms from resources such as the Anatomical Therapeutical Classification, the National Cancer Institute (NCI) Dictionary of Cancer Terms, OrphaData, or the Nomencl{\'a}tor de Prescripci{\'o}n for drug names. Part-of-Speech information is being included in the lexicon, and the current version amounts up to 76 454 lemmas and 203 043 inflected forms (including conjugated verbs, number and gender variants), corresponding to 30 647 UMLS CUIs. MedLexSp is distributed freely for research purposes.
Incorporating Figure Captions and Descriptive Text in MeSH Term Indexing
Authors: Xindi Wang, Robert E. Mercer
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Articles, Deep Learning, Descriptive, Document Classification, General Classification, text-classification, Text Classification

The goal of text classification is to automatically assign categories to documents. Deep learning automatically learns effective features from data instead of adopting human-designed features. In this paper, we focus specifically on biomedical document classification using a deep learning approach. We present a novel multichannel TextCNN model for MeSH term indexing. Beyond the normal use of the text from the abstract and title for model training, we also consider figure and table captions, as well as paragraphs associated with the figures and tables. We demonstrate that these latter text sources are important feature sources for our method. A new dataset consisting of these text segments curated from 257,590 full text articles together with the articles{'} MEDLINE/PubMed MeSH terms is publicly available.
BioRelEx 1.0: Biological Relation Extraction Benchmark
Authors: Hrant Khachatrian, Lilit Nersisyan, Karen Hambardzumyan, Tigran Galstyan, Anna Hakobyan, Arsen Arakelyan, Andrey Rzhetsky, Aram Galstyan
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Relation, Relation Extraction

Automatic extraction of relations and interactions between biological entities from scientific literature remains an extremely challenging problem in biomedical information extraction and natural language processing in general. One of the reasons for slow progress is the relative scarcity of standardized and publicly available benchmarks. In this paper we introduce BioRelEx, a new dataset of fully annotated sentences from biomedical literature that capture \textit{binding} interactions between proteins and/or biomolecules. To foster reproducible research on the interaction extraction task, we define a precise and transparent evaluation process, tools for error analysis and significance tests. Finally, we conduct extensive experiments to evaluate several baselines, including SciIE, a recently introduced neural multi-task architecture that has demonstrated state-of-the-art performance on several tasks.
Extraction of Lactation Frames from Drug Labels and LactMed
Authors: Heath Goodrum, Meghana Gudala, Ankita Misra, Kirk Roberts
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Articles

This paper describes a natural language processing (NLP) approach to extracting lactation-specific drug information from two sources: FDA-mandated drug labels and the NLM Drugs and Lactation Database (LactMed). A frame semantic approach is utilized, and the paper describes the selected frames, their annotation on a set of 900 sections from drug labels and LactMed articles, and the NLP system to extract such frame instances automatically. The ultimate goal of the project is to use such a system to identify discrepancies in lactation-related drug information between these resources.