pwc-archive/papers-with-abstracts huggingface dataset:
Fast Domain Adaptation of Semantic Parsers via Paraphrase Attention
Authors: Avik Ray, Yilin Shen, Hongxia Jin
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Domain Adaptation

Semantic parsers are used to convert user{'}s natural language commands to executable logical form in intelligent personal agents. Labeled datasets required to train such parsers are expensive to collect, and are never comprehensive. As a result, for effective post-deployment domain adaptation and personalization, semantic parsers are continuously retrained to learn new user vocabulary and paraphrase variety. However, state-of-the art attention based neural parsers are slow to retrain which inhibits real time domain adaptation. Secondly, these parsers do not leverage numerous paraphrases already present in the training dataset. Designing parsers which can simultaneously maintain high accuracy and fast retraining time is challenging. In this paper, we present novel paraphrase attention based sequence-to-sequence/tree parsers which support fast near real time retraining. In addition, our parsers often boost accuracy by jointly modeling the semantic dependencies of paraphrases. We evaluate our model on benchmark datasets to demonstrate upto 9X speedup in retraining time compared to existing parsers, as well as achieving state-of-the-art accuracy.
From Monolingual to Multilingual FAQ Assistant using Multilingual Co-training
Authors: Mayur Patidar, Surabhi Kumari, Manasi Patwardhan, Kar, Shirish e, Puneet Agarwal, Lovekesh Vig, Gautam Shroff
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Cross-Lingual Transfer

Recent research on cross-lingual transfer show state-of-the-art results on benchmark datasets using pre-trained language representation models (PLRM) like BERT. These results are achieved with the traditional training approaches, such as Zero-shot with no data, Translate-train or Translate-test with machine translated data. In this work, we propose an approach of {``}Multilingual Co-training{''} (MCT) where we augment the expert annotated dataset in the source language (English) with the corresponding machine translations in the target languages (e.g. Arabic, Spanish) and fine-tune the PLRM jointly. We observe that the proposed approach provides consistent gains in the performance of BERT for multiple benchmark datasets (e.g. 1.0{\%} gain on MLDocs, and 1.2{\%} gain on XNLI over translate-train with BERT), while requiring a single model for multiple languages. We further consider a FAQ dataset where the available English test dataset is translated by experts into Arabic and Spanish. On such a dataset, we observe an average gain of 4.9{\%} over all other cross-lingual transfer protocols with BERT. We further observe that domain-specific joint pre-training of the PLRM using HR policy documents in English along with the machine translations in the target languages, followed by the joint finetuning, provides a further improvement of 2.8{\%} in average accuracy.
Inject Rubrics into Short Answer Grading System
Authors: Tianqi Wang, Naoya Inoue, Hiroki Ouchi, Tomoya Mizumoto, Kentaro Inui
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11

Short Answer Grading (SAG) is a task of scoring students{'} answers in examinations. Most existing SAG systems predict scores based only on the answers, including the model used as base line in this paper, which gives the-state-of-the-art performance. But they ignore important evaluation criteria such as rubrics, which play a crucial role for evaluating answers in real-world situations. In this paper, we present a method to inject information from rubrics into SAG systems. We implement our approach on top of word-level attention mechanism to introduce the rubric information, in order to locate information in each answer that are highly related to the score. Our experimental results demonstrate that injecting rubric information effectively contributes to the performance improvement and that our proposed model outperforms the state-of-the-art SAG model on the widely used ASAP-SAS dataset under low-resource settings.
Multimodal, Multilingual Grapheme-to-Phoneme Conversion for Low-Resource Languages
Authors: James Route, Steven Hillis, Isak Czeresnia Etinger, Han Zhang, Alan W. black
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Grapheme-to-Phoneme Conversion

Grapheme-to-phoneme conversion (g2p) is the task of predicting the pronunciation of words from their orthographic representation. His- torically, g2p systems were transition- or rule- based, making generalization beyond a mono- lingual (high resource) domain impractical. Recently, neural architectures have enabled multilingual systems to generalize widely; however, all systems to date have been trained only on spelling-pronunciation pairs. We hy- pothesize that the sequences of IPA characters used to represent pronunciation do not capture its full nuance, especially when cleaned to fa- cilitate machine learning. We leverage audio data as an auxiliary modality in a multi-task training process to learn a more optimal inter- mediate representation of source graphemes; this is the first multimodal model proposed for multilingual g2p. Our approach is highly ef- fective: on our in-domain test set, our mul- timodal model reduces phoneme error rate to 2.46{\%}, a more than 65{\%} decrease compared to our implementation of a unimodal spelling- pronunciation model{---}which itself achieves state-of-the-art results on the Wiktionary test set. The advantages of the multimodal model generalize to wholly unseen languages, reduc- ing phoneme error rate on our out-of-domain test set to 6.39{\%} from the unimodal 8.21{\%}, a more than 20{\%} relative decrease. Further- more, our training and test sets are composed primarily of low-resource languages, demon- strating that our multimodal approach remains useful when training data are constrained.
Natural Language Generation for Effective Knowledge Distillation
Authors: Raphael Tang, Yao Lu, Jimmy Lin
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Knowledge Distillation, Linguistic Acceptability, Sentence, Sentence Similarity, Sentiment Analysis, Sentiment Classification, Text Generation, Transfer Learning

Knowledge distillation can effectively transfer knowledge from BERT, a deep language representation model, to traditional, shallow word embedding-based neural networks, helping them approach or exceed the quality of other heavyweight language representation models. As shown in previous work, critical to this distillation procedure is the construction of an unlabeled transfer dataset, which enables effective knowledge transfer. To create transfer set examples, we propose to sample from pretrained language models fine-tuned on task-specific text. Unlike previous techniques, this directly captures the purpose of the transfer set. We hypothesize that this principled, general approach outperforms rule-based techniques. On four datasets in sentiment classification, sentence similarity, and linguistic acceptability, we show that our approach improves upon previous methods. We outperform OpenAI GPT, a deep pretrained transformer, on three of the datasets, while using a single-layer bidirectional LSTM that runs at least ten times faster.
Neural Unsupervised Parsing Beyond English
Authors: Katharina Kann, Anhad Mohananey, Samuel R. Bowman, Kyunghyun Cho
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11

Recently, neural network models which automatically infer syntactic structure from raw text have started to achieve promising results. However, earlier work on unsupervised parsing shows large performance differences between non-neural models trained on corpora in different languages, even for comparable amounts of data. With that in mind, we train instances of the PRPN architecture (Shen et al., 2018){---}one of these unsupervised neural network parsers{---}for Arabic, Chinese, English, and German. We find that (i) the model strongly outperforms trivial baselines and, thus, acquires at least some parsing ability for all languages; (ii) good hyperparameter values seem to be universal; (iii) how the model benefits from larger training set sizes depends on the corpus, with the model achieving the largest performance gains when increasing the number of sentences from 2,500 to 12,500 for English. In addition, we show that, by sharing parameters between the related languages German and English, we can improve the model{'}s unsupervised parsing F1 score by up to 4{\%} in the low-resource setting.
Reevaluating Argument Component Extraction in Low Resource Settings
Authors: Anirudh Joshi, Timothy Baldwin, Richard Sinnott, Cecile Paris
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Multi-Task Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

Argument component extraction is a challenging and complex high-level semantic extraction task. As such, it is both expensive to annotate (meaning training data is limited and low-resource by nature), and hard for current-generation deep learning methods to model. In this paper, we reevaluate the performance of state-of-the-art approaches in both single- and multi-task learning settings using combinations of character-level, GloVe, ELMo, and BERT encodings using standard BiLSTM-CRF encoders. We use evaluation metrics that are more consistent with evaluation practice in named entity recognition to understand how well current baselines address this challenge and compare their performance to lower-level semantic tasks such as CoNLL named entity recognition. We find that performance utilizing various pre-trained representations and training methodologies often leaves a lot to be desired as it currently stands, and suggest future pathways for improvement.
Reinforcement-based denoising of distantly supervised NER with partial annotation
Authors: Farhad Nooralahzadeh, Jan Tore L{\o}nning, Lilja {\O}vrelid
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Denoising, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Reinforcement Learning

Existing named entity recognition (NER) systems rely on large amounts of human-labeled data for supervision. However, obtaining large-scale annotated data is challenging particularly in specific domains like health-care, e-commerce and so on. Given the availability of domain specific knowledge resources, (e.g., ontologies, dictionaries), distant supervision is a solution to generate automatically labeled training data to reduce human effort. The outcome of distant supervision for NER, however, is often noisy. False positive and false negative instances are the main issues that reduce performance on this kind of auto-generated data. In this paper, we explore distant supervision in a supervised setup. We adopt a technique of partial annotation to address false negative cases and implement a reinforcement learning strategy with a neural network policy to identify false positive instances. Our results establish a new state-of-the-art on four benchmark datasets taken from different domains and different languages. We then go on to show that our model reduces the amount of manually annotated data required to perform NER in a new domain.
Samvaadhana: A Telugu Dialogue System in Hospital Domain
Authors: Suma Reddy Duggenpudi, Kusampudi Siva Subrahamanyam Varma, Radhika Mamidi
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: General Classification, Question Answering

In this paper, a dialogue system for Hospital domain in Telugu, which is a resource-poor Dravidian language, has been built. It handles various hospital and doctor related queries. The main aim of this paper is to present an approach for modelling a dialogue system in a resource-poor language by combining linguistic and domain knowledge. Focusing on the question answering aspect of the dialogue system, we identified Question Classification and Query Processing as the two most important parts of the dialogue system. Our method combines deep learning techniques for question classification and computational rule-based analysis for query processing. Human evaluation of the system has been performed as there is no automated evaluation tool for dialogue systems in Telugu. Our system achieves a high overall rating along with a significantly accurate context-capturing method as shown in the results.
Weakly Supervised Attentional Model for Low Resource Ad-hoc Cross-lingual Information Retrieval
Authors: Lingjun Zhao, Rabih Zbib, Zhuolin Jiang, Damianos Karakos, Zhongqiang Huang
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Cross-Lingual Information Retrieval, Information Retrieval, Machine Translation, Retrieval, Sentence, Translation

We propose a weakly supervised neural model for Ad-hoc Cross-lingual Information Retrieval (CLIR) from low-resource languages. Low resource languages often lack relevance annotations for CLIR, and when available the training data usually has limited coverage for possible queries. In this paper, we design a model which does not require relevance annotations, instead it is trained on samples extracted from translation corpora as weak supervision. This model relies on an attention mechanism to learn spans in the foreign sentence that are relevant to the query. We report experiments on two low resource languages: Swahili and Tagalog, trained on less that 100k parallel sentences each. The proposed model achieves 19 MAP points improvement compared to using CNNs for feature extraction, 12 points improvement from machine translation-based CLIR, and up to 6 points improvement compared to probabilistic CLIR models.
Zero-Shot Cross-lingual Name Retrieval for Low-Resource Languages
Authors: Kevin Blissett, Heng Ji
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Retrieval

In this paper we address a challenging cross-lingual name retrieval task. Given an English named entity query, we aim to find all name mentions in documents in low-resource languages. We present a novel method which relies on zero annotation or resources from the target language. By leveraging freely available, cross-lingual resources and a small amount of training data from another language, we are able to perform name retrieval on a new language without any additional training data. Our method proceeds in a multi-step process: first, we pre-train a language-independent orthographic encoder using Wikipedia inter-lingual links from dozens of languages. Next, we gather user expectations about important entities in an English comparable document and compare those expected entities with actual spans of the target language text in order to perform name finding. Our method shows 11.6{\%} absolute F-score improvement over state-of-the-art methods.
Cross-document coreference: An approach to capturing coreference without context
Authors: Kristin Wright-Bettner, Martha Palmer, Guergana Savova, Piet de Groen, Timothy Miller
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11

This paper discusses a cross-document coreference annotation schema that was developed to further automatic extraction of timelines in the clinical domain. Lexical senses and coreference choices are determined largely by context, but cross-document work requires reasoning across contexts that are not necessarily coherent. We found that an annotation approach that relies less on context-guided annotator intuitions and more on schematic rules was most effective in creating meaningful and consistent cross-document relations.
Comparing the Intrinsic Performance of Clinical Concept Embeddings by Their Field of Medicine
Authors: John-Jose Nunez, Giuseppe Carenini
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Word Embeddings

Pre-trained word embeddings are becoming increasingly popular for natural language processing tasks. This includes medical applications, where embeddings are trained for clinical concepts using specific medical data. Recent work continues to improve on these embeddings. However, no one has yet sought to determine whether these embeddings work as well for one field of medicine as they do in others. In this work, we use intrinsic methods to evaluate embeddings from the various fields of medicine as defined by their ICD-9 systems. We find significant differences between fields, and motivate future work to investigate whether extrinsic tasks will follow a similar pattern.
Syntax-aware Multi-task Graph Convolutional Networks for Biomedical Relation Extraction
Authors: Diya Li, Heng Ji
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Dependency Parsing, Drug–drug Interaction Extraction, Multi-Task Learning, Relation, Relation Extraction

In this paper we tackle two unique challenges in biomedical relation extraction. The first challenge is that the contextual information between two entity mentions often involves sophisticated syntactic structures. We propose a novel graph convolutional networks model that incorporates dependency parsing and contextualized embedding to effectively capture comprehensive contextual information. The second challenge is that most of the benchmark data sets for this task are quite imbalanced because more than 80{\%} mention pairs are negative instances (i.e., no relations). We propose a multi-task learning framework to jointly model relation identification and classification tasks to propagate supervision signals from each other and apply a focal loss to focus training on ambiguous mention pairs. By applying these two strategies, experiments show that our model achieves state-of-the-art F-score on the 2013 drug-drug interaction extraction task.
BioReddit: Word Embeddings for User-Generated Biomedical NLP
Authors: Marco Basaldella, Nigel Collier
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Word Embeddings

Word embeddings, in their different shapes and iterations, have changed the natural language processing research landscape in the last years. The biomedical text processing field is no stranger to this revolution; however, scholars in the field largely trained their embeddings on scientific documents only, even when working on user-generated data. In this paper we show how training embeddings from a corpus collected from user-generated text from medical forums heavily influences the performance on downstream tasks, outperforming embeddings trained both on general purpose data or on scientific papers when applied on user-generated content.
Leveraging Hierarchical Category Knowledge for Data-Imbalanced Multi-Label Diagnostic Text Understanding
Authors: Shang-Chi Tsai, Ting-Yun Chang, Yun-Nung Chen
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Descriptive, Diagnostic

Clinical notes are essential medical documents to record each patient{'}s symptoms. Each record is typically annotated with medical diagnostic codes, which means diagnosis and treatment. This paper focuses on predicting diagnostic codes given the descriptive present illness in electronic health records by leveraging domain knowledge. We investigate various losses in a convolutional model to utilize hierarchical category knowledge of diagnostic codes in order to allow the model to share semantics across different labels under the same category. The proposed model not only considers the external domain knowledge but also addresses the issue about data imbalance. The MIMIC3 benchmark experiments show that the proposed methods can effectively utilize category knowledge and provide informative cues to improve the performance in terms of the top-ranked diagnostic codes which is better than the prior state-of-the-art. The investigation and discussion express the potential of integrating the domain knowledge in the current machine learning based models and guiding future research directions.
Experiments with ad hoc ambiguous abbreviation expansion
Authors: Agnieszka Mykowiecka, Malgorzata Marciniak
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: Clustering

The paper addresses experiments to expand ad hoc ambiguous abbreviations in medical notes on the basis of morphologically annotated texts, without using additional domain resources. We work on Polish data but the described approaches can be used for other languages too. We test two methods to select candidates for word abbreviation expansions. The first one automatically selects all words in text which might be an expansion of an abbreviation according to the language rules. The second method uses clustering of abbreviation occurrences to select representative elements which are manually annotated to determine lists of potential expansions. We then train a classifier to assign expansions to abbreviations based on three training sets: automatically obtained, consisting of manual annotation, and concatenation of the two previous ones. The results obtained for the manually annotated training data significantly outperform automatically obtained training data. Adding the automatically obtained training data to the manually annotated data improves the results, in particular for less frequent abbreviations. In this context the proposed a priori data driven selection of possible extensions turned out to be crucial.
Multi-Task, Multi-Channel, Multi-Input Learning for Mental Illness Detection using Social Media Text
Authors: Prasadith Kirinde Gamaarachchige, Diana Inkpen
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11
Tasks: 3D Object Classification, Multi-class Classification, Multi-Task Learning

We investigate the impact of using emotional patterns identified by the clinical practitioners and computational linguists to enhance the prediction capabilities of a mental illness detection (in our case depression and post-traumatic stress disorder) model built using a deep neural network architecture. Over the years, deep learning methods have been successfully used in natural language processing tasks, including a few in the domain of mental illness and suicide ideation detection. We illustrate the effectiveness of using multi-task learning with a multi-channel convolutional neural network as the shared representation and use additional inputs identified by researchers as indicatives in detecting mental disorders to enhance the model predictability. Given the limited amount of unstructured data available for training, we managed to obtain a task-specific AUC higher than 0.90. In comparison to methods such as multi-class classification, we identified multi-task learning with multi-channel convolution neural network and multiple-inputs to be effective in detecting mental disorders.
Extracting relevant information from physician-patient dialogues for automated clinical note taking
Authors: Serena Jeblee, Faiza Khan Khattak, Noah Crampton, Muhammad Mamdani, Frank Rudzicz
Date: 2019-11-01 00:00:00
Proceeding: WS 2019 11

We present a system for automatically extracting pertinent medical information from dialogues between clinicians and patients. The system parses each dialogue and extracts entities such as medications and symptoms, using context to predict which entities are relevant. We also classify the primary diagnosis for each conversation. In addition, we extract topic information and identify relevant utterances. This serves as a baseline for a system that extracts information from dialogues and automatically generates a patient note, which can be reviewed and edited by the clinician.