pwc-archive/papers-with-abstracts huggingface dataset:
A mostly unlexicalized model for recognizing textual entailment
Authors: Mithun Paul, Rebecca Sharp, Mihai Surdeanu
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Fake News Detection, Information Retrieval, model, Natural Language Inference, Retrieval, Sentence, Word Embeddings

Many approaches to automatically recognizing entailment relations have employed classifiers over hand engineered lexicalized features, or deep learning models that implicitly capture lexicalization through word embeddings. This reliance on lexicalization may complicate the adaptation of these tools between domains. For example, such a system trained in the news domain may learn that a sentence like {``}Palestinians recognize Texas as part of Mexico{''} tends to be unsupported, but this fact (and its corresponding lexicalized cues) have no value in, say, a scientific domain. To mitigate this dependence on lexicalized information, in this paper we propose a model that reads two sentences, from any given domain, to determine entailment without using lexicalized features. Instead our model relies on features that are either unlexicalized or are domain independent such as proportion of negated verbs, antonyms, or noun overlap. In its current implementation, this model does not perform well on the FEVER dataset, due to two reasons. First, for the information retrieval portion of the task we used the baseline system provided, since this was not the aim of our project. Second, this is work in progress and we still are in the process of identifying more features and gradually increasing the accuracy of our model. In the end, we hope to build a generic end-to-end classifier, which can be used in a domain outside the one in which it was trained, with no or minimal re-training.
Detecting Diabetes Risk from Social Media Activity
Authors: Dane Bell, Egoitz Laparra, Aditya Kousik, Terron Ishihara, Mihai Surdeanu, Stephen Kobourov
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Domain Adaptation

This work explores the detection of individuals{'} risk of type 2 diabetes mellitus (T2DM) directly from their social media (Twitter) activity. Our approach extends a deep learning architecture with several contributions: following previous observations that language use differs by gender, it captures and uses gender information through domain adaptation; it captures recency of posts under the hypothesis that more recent posts are more representative of an individual{'}s current risk status; and, lastly, it demonstrates that in this scenario where activity factors are sparsely represented in the data, a bag-of-word neural network model using custom dictionaries of food and activity words performs better than other neural sequence models. Our best model, which incorporates all these contributions, achieves a risk-detection F1 of 41.9, considerably higher than the baseline rate (36.9).
Treatment Side Effect Prediction from Online User-generated Content
Authors: Van Hoang Nguyen, Kazunari Sugiyama, Min-Yen Kan, Kishaloy Halder
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Feature Engineering, Prediction

With Health 2.0, patients and caregivers increasingly seek information regarding possible drug side effects during their medical treatments in online health communities. These are helpful platforms for non-professional medical opinions, yet pose risk of being unreliable in quality and insufficient in quantity to cover the wide range of potential drug reactions. Existing approaches which analyze such user-generated content in online forums heavily rely on feature engineering of both documents and users, and often overlook the relationships between posts within a common discussion thread. Inspired by recent advancements, we propose a neural architecture that models the textual content of user-generated documents and user experiences in online communities to predict side effects during treatment. Experimental results show that our proposed architecture outperforms baseline models.
Revisiting neural relation classification in clinical notes with external information
Authors: Simon {\v{S}}uster, Madhumita Sushil, Walter Daelemans
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Classification, Feature Engineering, General Classification, Natural Language Inference, Relation, Relation Classification, Relation Extraction, Text Categorization

Recently, segment convolutional neural networks have been proposed for end-to-end relation extraction in the clinical domain, achieving results comparable to or outperforming the approaches with heavy manual feature engineering. In this paper, we analyze the errors made by the neural classifier based on confusion matrices, and then investigate three simple extensions to overcome its limitations. We find that including ontological association between drugs and problems, and data-induced association between medical concepts does not reliably improve the performance, but that large gains are obtained by the incorporation of semantic classes to capture relation triggers.
Supervised Machine Learning for Extractive Query Based Summarisation of Biomedical Data
Authors: M Kaur, eep, Diego Moll{\'a}
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: BIG-bench Machine Learning, Classification, General Classification, regression

The automation of text summarisation of biomedical publications is a pressing need due to the plethora of information available online. This paper explores the impact of several supervised machine learning approaches for extracting multi-document summaries for given queries. In particular, we compare classification and regression approaches for query-based extractive summarisation using data provided by the BioASQ Challenge. We tackled the problem of annotating sentences for training classification systems and show that a simple annotation approach outperforms regression-based summarisation.
Investigating the Challenges of Temporal Relation Extraction from Clinical Text
Authors: Diana Galvan, Naoaki Okazaki, Koji Matsuda, Kentaro Inui
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Named Entity Recognition (NER), Question Answering, Relation, Relation Extraction, Temporal Information Extraction, Temporal Relation Extraction, Text Summarization

Temporal reasoning remains as an unsolved task for Natural Language Processing (NLP), particularly demonstrated in the clinical domain. The complexity of temporal representation in language is evident as results of the 2016 Clinical TempEval challenge indicate: the current state-of-the-art systems perform well in solving mention-identification tasks of event and time expressions but poorly in temporal relation extraction, showing a gap of around 0.25 point below human performance. We explore to adapt the tree-based LSTM-RNN model proposed by Miwa and Bansal (2016) to temporal relation extraction from clinical text, obtaining a five point improvement over the best 2016 Clinical TempEval system and two points over the state-of-the-art. We deliver a deep analysis of the results and discuss the next step towards human-like temporal reasoning.
De-identifying Free Text of Japanese Dummy Electronic Health Records
Authors: Kohei Kajiyama, Hiromasa Horiguchi, Takashi Okumura, Mizuki Morita, Yoshinobu Kano
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: De-identification

A new law was established in Japan to promote utilization of EHRs for research and developments, while de-identification is required to use EHRs. However, studies of automatic de-identification in the healthcare domain is not active for Japanese language, no de-identification tool available in practical performance for Japanese medical domains, as far as we know. Previous work shows that rule-based methods are still effective, while deep learning methods are reported to be better recently. In order to implement and evaluate a de-identification tool in a practical level, we implemented three methods, rule-based, CRF, and LSTM. We prepared three datasets of pseudo EHRs with de-identification tags manually annotated. These datasets are derived from shared task data to compare with previous work, and our new data to increase training data. Our result shows that our LSTM-based method is better and robust, which leads to our future work that plans to apply our system to actual de-identification tasks in hospitals.
Identification of Parallel Sentences in Comparable Monolingual Corpora from Different Registers
Authors: R{\'e}mi Cardon, Natalia Grabar
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Information Retrieval, Machine Translation, STS, Text Simplification

Parallel aligned sentences provide useful information for different NLP applications. Yet, this kind of data is seldom available, especially for languages other than English. We propose to exploit comparable corpora in French which are distinguished by their registers (specialized and simplified versions) to detect and align parallel sentences. These corpora are related to the biomedical area. Our purpose is to state whether a given pair of specialized and simplified sentences is to be aligned or not. Manually created reference data show 0.76 inter-annotator agreement. We exploit a set of features and several automatic classifiers. The automatic alignment reaches up to 0.93 Precision, Recall and F-measure. In order to better evaluate the method, it is applied to data in English from the \textit{SemEval} STS competitions. The same features and models are applied in monolingual and cross-lingual contexts, in which they show up to 0.90 and 0.73 F-measure, respectively.
Evaluation of a Prototype System that Automatically Assigns Subject Headings to Nursing Narratives Using Recurrent Neural Network
Authors: Hans Moen, Kai Hakala, Laura-Maria Peltonen, Henry Suhonen, Petri Loukasm{\"a}ki, Tapio Salakoski, Filip Ginter, Sanna Salanter{\"a}
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Sentence, text-classification, Text Classification

We present our initial evaluation of a prototype system designed to assist nurses in assigning subject headings to nursing narratives {--} written in the context of documenting patient care in hospitals. Currently nurses may need to memorize several hundred subject headings from standardized nursing terminologies when structuring and assigning the right section/subject headings to their text. Our aim is to allow nurses to write in a narrative manner without having to plan and structure the text with respect to sections and subject headings, instead the system should assist with the assignment of subject headings and restructuring afterwards. We hypothesize that this could reduce the time and effort needed for nursing documentation in hospitals. A central component of the system is a text classification model based on a long short-term memory (LSTM) recurrent neural network architecture, trained on a large data set of nursing notes. A simple Web-based interface has been implemented for user interaction. To evaluate the system, three nurses write a set of artificial nursing shift notes in a fully unstructured narrative manner, without planning for or consider the use of sections and subject headings. These are then fed to the system which assigns subject headings to each sentence and then groups them into paragraphs. Manual evaluation is conducted by a group of nurses. The results show that about 70{\%} of the sentences are assigned to correct subject headings. The nurses believe that such a system can be of great help in making nursing documentation in hospitals easier and less time consuming. Finally, various measures and approaches for improving the system are discussed.
Automatically Detecting the Position and Type of Psychiatric Evaluation Report Sections
Authors: Deya Banisakher, Naphtali Rishe, Mark A. Finlayson
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Position, Semantic Parsing, Vocal Bursts Type Prediction

Psychiatric evaluation reports represent a rich and still mostly-untapped source of information for developing systems for automatic diagnosis and treatment of mental health problems. These reports contain free-text structured within sections using a convention of headings. We present a model for automatically detecting the position and type of different psychiatric evaluation report sections. We developed this model using a corpus of 150 sample reports that we gathered from the Web, and used sentences as a processing unit while section headings were used as labels of section type. From these labels we generated a unified hierarchy of labels of section types, and then learned n-gram models of the language found in each section. To model conventions for section order, we integrated these n-gram models with a Hierarchical Hidden Markov Model (HHMM) representing the probabilities of observed section orders found in the corpus, and then used this HHMM n-gram model in a decoding framework to infer the most likely section boundaries and section types for documents with their section labels removed. We evaluated our model over two tasks, namely, identifying section boundaries and identifying section types and orders. Our model significantly outperformed baselines for each task with an F1 of 0.88 for identifying section types, and a 0.26 WindowDiff (Wd) and 0.20 and (Pk) scores, respectively, for identifying section boundaries.
Iterative development of family history annotation guidelines using a synthetic corpus of clinical text
Authors: Taraka Rama, P{\aa}l Brekke, {\O}ystein Nytr{\o}, Lilja {\O}vrelid
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10

In this article, we describe the development of annotation guidelines for family history information in Norwegian clinical text. We make use of incrementally developed synthetic clinical text describing patients{'} family history relating to cases of cardiac disease and present a general methodology which integrates the synthetically produced clinical statements and guideline development. We analyze inter-annotator agreement based on the developed guidelines and present results from experiments aimed at evaluating the validity and applicability of the annotated corpus using machine learning techniques. The resulting annotated corpus contains 477 sentences and 6030 tokens. Both the annotation guidelines and the annotated corpus are made freely available and as such constitutes the first publicly available resource of Norwegian clinical text.
CAS: French Corpus with Clinical Cases
Authors: Natalia Grabar, Vincent Claveau, Cl{\'e}ment Dalloux
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Information Retrieval

Textual corpora are extremely important for various NLP applications as they provide information necessary for creating, setting and testing these applications and the corresponding tools. They are also crucial for designing reliable methods and reproducible results. Yet, in some areas, such as the medical area, due to confidentiality or to ethical reasons, it is complicated and even impossible to access textual data representative of those produced in these areas. We propose the CAS corpus built with clinical cases, such as they are reported in the published scientific literature in French. We describe this corpus, currently containing over 397,000 word occurrences, and the existing linguistic and semantic annotations.
In-domain Context-aware Token Embeddings Improve Biomedical Named Entity Recognition
Authors: Golnar Sheikhshabbafghi, Inanc Birol, Anoop Sarkar
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Language Modeling, Language Modelling, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Question Answering, Sentiment Analysis, Word Embeddings, Word Sense Disambiguation

Rapidly expanding volume of publications in the biomedical domain makes it increasingly difficult for a timely evaluation of the latest literature. That, along with a push for automated evaluation of clinical reports, present opportunities for effective natural language processing methods. In this study we target the problem of named entity recognition, where texts are processed to annotate terms that are relevant for biomedical studies. Terms of interest in the domain include gene and protein names, and cell lines and types. Here we report on a pipeline built on Embeddings from Language Models (ELMo) and a deep learning package for natural language processing (AllenNLP). We trained context-aware token embeddings on a dataset of biomedical papers using ELMo, and incorporated these embeddings in the LSTM-CRF model used by AllenNLP for named entity recognition. We show these representations improve named entity recognition for different types of biomedical named entities. We also achieve a new state of the art in gene mention detection on the BioCreative II gene mention shared task.
Self-training improves Recurrent Neural Networks performance for Temporal Relation Extraction
Authors: Chen Lin, Timothy Miller, Dmitriy Dligach, Hadi Amiri, Steven Bethard, Guergana Savova
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Feature Engineering, Machine Translation, Question Answering, Relation, Relation Extraction, Speech Recognition, Temporal Information Extraction, Temporal Relation Extraction, Word Embeddings

Neural network models are oftentimes restricted by limited labeled instances and resort to advanced architectures and features for cutting edge performance. We propose to build a recurrent neural network with multiple semantically heterogeneous embeddings within a self-training framework. Our framework makes use of labeled, unlabeled, and social media data, operates on basic features, and is scalable and generalizable. With this method, we establish the state-of-the-art result for both in- and cross-domain for a clinical temporal relation extraction task.
Listwise temporal ordering of events in clinical notes
Authors: Serena Jeblee, Graeme Hirst
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Information Retrieval, Relation Extraction

We present metrics for listwise temporal ordering of events in clinical notes, as well as a baseline listwise temporal ranking model that generates a timeline of events that can be used in downstream medical natural language processing tasks.
Time Expressions in Mental Health Records for Symptom Onset Extraction
Authors: Natalia Viani, Lucia Yin, Joyce Kam, Ayunni Alawi, Andr{\'e} Bittar, Rina Dutta, Rashmi Patel, Robert Stewart, Sumithra Velupillai
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Temporal Information Extraction

For psychiatric disorders such as schizophrenia, longer durations of untreated psychosis are associated with worse intervention outcomes. Data included in electronic health records (EHRs) can be useful for retrospective clinical studies, but much of this is stored as unstructured text which cannot be directly used in computation. Natural Language Processing (NLP) methods can be used to extract this data, in order to identify symptoms and treatments from mental health records, and temporally anchor the first emergence of these. We are developing an EHR corpus annotated with time expressions, clinical entities and their relations, to be used for NLP development. In this study, we focus on the first step, identifying time expressions in EHRs for patients with schizophrenia. We developed a gold standard corpus, compared this corpus to other related corpora in terms of content and time expression prevalence, and adapted two NLP systems for extracting time expressions. To the best of our knowledge, this is the first resource annotated for temporal entities in the mental health domain.
Evaluation of a Sequence Tagging Tool for Biomedical Texts
Authors: Julien Tourille, Matthieu Doutreligne, Olivier Ferret, Aur{\'e}lie N{\'e}v{\'e}ol, Nicolas Paris, Xavier Tannier
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Articles, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Part-Of-Speech Tagging, Relation Extraction, Text Classification

Many applications in biomedical natural language processing rely on sequence tagging as an initial step to perform more complex analysis. To support text analysis in the biomedical domain, we introduce Yet Another SEquence Tagger (YASET), an open-source multi purpose sequence tagger that implements state-of-the-art deep learning algorithms for sequence tagging. Herein, we evaluate YASET on part-of-speech tagging and named entity recognition in a variety of text genres including articles from the biomedical literature in English and clinical narratives in French. To further characterize performance, we report distributions over 30 runs and different sizes of training datasets. YASET provides state-of-the-art performance on the CoNLL 2003 NER dataset (F1=0.87), MEDPOST corpus (F1=0.97), MERLoT corpus (F1=0.99) and NCBI disease corpus (F1=0.81). We believe that YASET is a versatile and efficient tool that can be used for sequence tagging in biomedical and clinical texts.
Autonomous Sub-domain Modeling for Dialogue Policy with Hierarchical Deep Reinforcement Learning
Authors: Giovanni Yoko Kristianto, Huiwen Zhang, Bin Tong, Makoto Iwayama, Yoshiyuki Kobayashi
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Deep Reinforcement Learning, Hierarchical Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Solving composites tasks, which consist of several inherent sub-tasks, remains a challenge in the research area of dialogue. Current studies have tackled this issue by manually decomposing the composite tasks into several sub-domains. However, much human effort is inevitable. This paper proposes a dialogue framework that autonomously models meaningful sub-domains and learns the policy over them. Our experiments show that our framework outperforms the baseline without subdomains by 11{\%} in terms of success rate, and is competitive with that with manually defined sub-domains.
Building Dialogue Structure from Discourse Tree of a Question
Authors: Boris Galitsky, Dmitry Ilvovsky
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10
Tasks: Dialogue Management, Management, Sentence

In this section we propose a reasoning-based approach to a dialogue management for a customer support chat bot. To build a dialogue scenario, we analyze the discourse tree (DT) of an initial query of a customer support dialogue that is frequently complex and multi-sentence. We then enforce rhetorical agreement between DT of the initial query and that of the answers, requests and responses. The chat bot finds answers, which are not only relevant by topic but also suitable for a given step of a conversation and match the question by style, communication means, experience level and other domain-independent attributes. We evaluate a performance of proposed algorithm in car repair domain and observe a 5 to 10{\%} improvement for single and three-step dialogues respectively, in comparison with baseline approaches to dialogue management.
A Methodology for Evaluating Interaction Strategies of Task-Oriented Conversational Agents
Authors: Marco Guerini, Sara Falcone, Bernardo Magnini
Date: 2018-10-01 00:00:00
Proceeding: WS 2018 10

In task-oriented conversational agents, more attention has been usually devoted to assessing task effectiveness, rather than to \textit{how} the task is achieved. However, conversational agents are moving towards more complex and human-like interaction capabilities (e.g. the ability to use a formal/informal register, to show an empathetic behavior), for which standard evaluation methodologies may not suffice. In this paper, we provide a novel methodology to assess - in a completely controlled way - the impact on the quality of experience of agent{'}s interaction strategies. The methodology is based on a within subject design, where two slightly different transcripts of the same interaction with a conversational agent are presented to the user. Through a series of pilot experiments we prove that this methodology allows fast and cheap experimentation/evaluation, focusing on aspects that are overlooked by current methods.