pwc-archive/papers-with-abstracts huggingface dataset:
Generating Equation by Utilizing Operators : GEO model
Authors: Kyung Seo Ki, Donggeon Lee, Bugeun Kim, Gahgene Gweon
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Decoder, Machine Translation, Math, Math Word Problem Solving, model, Translation

Math word problem solving is an emerging research topic in Natural Language Processing. Recently, to address the math word problem-solving task, researchers have applied the encoder-decoder architecture, which is mainly used in machine translation tasks. The state-of-the-art neural models use hand-crafted features and are based on generation methods. In this paper, we propose the GEO (Generation of Equations by utilizing Operators) model that does not use hand-crafted features and addresses two issues that are present in existing neural models: 1. missing domain-specific knowledge features and 2. losing encoder-level knowledge. To address missing domain-specific feature issue, we designed two auxiliary tasks: operation group difference prediction and implicit pair prediction. To address losing encoder-level knowledge issue, we added an Operation Feature Feed Forward (OP3F) layer. Experimental results showed that the GEO model outperformed existing state-of-the-art models on two datasets, 85.1{\%} in MAWPS, and 62.5{\%} in DRAW-1K, and reached comparable performance of 82.1{\%} in ALG514 dataset.
Improving Abstractive Dialogue Summarization with Graph Structures and Topic Words
Authors: Lulu Zhao, Weiran Xu, Jun Guo
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Abstractive Dialogue Summarization, Graph Attention, Sentence

Recently, people have been beginning paying more attention to the abstractive dialogue summarization task. Since the information flows are exchanged between at least two interlocutors and key elements about a certain event are often spanned across multiple utterances, it is necessary for researchers to explore the inherent relations and structures of dialogue contents. However, the existing approaches often process the dialogue with sequence-based models, which are hard to capture long-distance inter-sentence relations. In this paper, we propose a Topic-word Guided Dialogue Graph Attention (TGDGA) network to model the dialogue as an interaction graph according to the topic word information. A masked graph self-attention mechanism is used to integrate cross-sentence information flows and focus more on the related utterances, which makes it better to understand the dialogue. Moreover, the topic word features are introduced to assist the decoding process. We evaluate our model on the SAMSum Corpus and Automobile Master Corpus. The experimental results show that our method outperforms most of the baselines.
AprilE: Attention with Pseudo Residual Connection for Knowledge Graph Embedding
Authors: Yuzhang Liu, Peng Wang, Yingtai Li, Yizhan Shao, Zhongkai Xu
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Graph Embedding, Knowledge Graph Embedding, Position

Knowledge graph embedding maps entities and relations into low-dimensional vector space. However, it is still challenging for many existing methods to model diverse relational patterns, especially symmetric and antisymmetric relations. To address this issue, we propose a novel model, AprilE, which employs triple-level self-attention and pseudo residual connection to model relational patterns. The triple-level self-attention treats head entity, relation, and tail entity as a sequence and captures the dependency within a triple. At the same time the pseudo residual connection retains primitive semantic features. Furthermore, to deal with symmetric and antisymmetric relations, two schemas of score function are designed via a position-adaptive mechanism. Experimental results on public datasets demonstrate that our model can produce expressive knowledge embedding and significantly outperforms most of the state-of-the-art works.
Variational Autoencoder with Embedded Student-t Mixture Model for Authorship Attribution
Authors: Benedikt Boenninghoff, Steffen Zeiler, Robert Nickel, Dorothea Kolossa
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Authorship Attribution

Traditional computational authorship attribution describes a classification task in a closed-set scenario. Given a finite set of candidate authors and corresponding labeled texts, the objective is to determine which of the authors has written another set of anonymous or disputed texts. In this work, we propose a probabilistic autoencoding framework to deal with this supervised classification task. Variational autoencoders (VAEs) have had tremendous success in learning latent representations. However, existing VAEs are currently still bound by limitations imposed by the assumed Gaussianity of the underlying probability distributions in the latent space. In this work, we are extending a VAE with an embedded Gaussian mixture model to a Student-t mixture model, which allows for an independent control of the {``}heaviness{''} of the respective tails of the implied probability densities. Experiments over an Amazon review dataset indicate superior performance of the proposed method.
Exploiting Node Content for Multiview Graph Convolutional Network and Adversarial Regularization
Authors: Qiuhao Lu, Nisansa de Silva, Dejing Dou, Thien Huu Nguyen, Prithviraj Sen, Berthold Reinwald, Yunyao Li
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Graph Learning, Link Prediction, Network Embedding, Node Clustering, Readmission Prediction, Representation Learning

Network representation learning (NRL) is crucial in the area of graph learning. Recently, graph autoencoders and its variants have gained much attention and popularity among various types of node embedding approaches. Most existing graph autoencoder-based methods aim to minimize the reconstruction errors of the input network while not explicitly considering the semantic relatedness between nodes. In this paper, we propose a novel network embedding method which models the consistency across different views of networks. More specifically, we create a second view from the input network which captures the relation between nodes based on node content and enforce the latent representations from the two views to be consistent by incorporating a multiview adversarial regularization module. The experimental studies on benchmark datasets prove the effectiveness of this method, and demonstrate that our method compares favorably with the state-of-the-art algorithms on challenging tasks such as link prediction and node clustering. We also evaluate our method on a real-world application, i.e., 30-day unplanned ICU readmission prediction, and achieve promising results compared with several baseline methods.
SentiX: A Sentiment-Aware Pre-Trained Model for Cross-Domain Sentiment Analysis
Authors: Jie zhou, Junfeng Tian, Rui Wang, Yuanbin Wu, Wenming Xiao, Liang He
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Language Modeling, Language Modelling, Sentence, Sentiment Analysis

Pre-trained language models have been widely applied to cross-domain NLP tasks like sentiment analysis, achieving state-of-the-art performance. However, due to the variety of users{'} emotional expressions across domains, fine-tuning the pre-trained models on the source domain tends to overfit, leading to inferior results on the target domain. In this paper, we pre-train a sentiment-aware language model (SentiX) via domain-invariant sentiment knowledge from large-scale review datasets, and utilize it for cross-domain sentiment analysis task without fine-tuning. We propose several pre-training tasks based on existing lexicons and annotations at both token and sentence levels, such as emoticons, sentiment words, and ratings, without human interference. A series of experiments are conducted and the results indicate the great advantages of our model. We obtain new state-of-the-art results in all the cross-domain sentiment analysis tasks, and our proposed SentiX can be trained with only 1{\%} samples (18 samples) and it achieves better performance than BERT with 90{\%} samples.
Bayes-enhanced Lifelong Attention Networks for Sentiment Classification
Authors: Hao Wang, Shuai Wang, Sahisnu Mazumder, Bing Liu, Yan Yang, Tianrui Li
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Classification, Sentiment Analysis, Sentiment Classification

The classic deep learning paradigm learns a model from the training data of a single task and the learned model is also tested on the same task. This paper studies the problem of learning a sequence of tasks (sentiment classification tasks in our case). After each sentiment classification task is learned, its knowledge is retained to help future task learning. Following this setting, we explore attention neural networks and propose a Bayes-enhanced Lifelong Attention Network (BLAN). The key idea is to exploit the generative parameters of naive Bayes to learn attention knowledge. The learned knowledge from each task is stored in a knowledge base and later used to build lifelong attentions. The constructed lifelong attentions are then used to enhance the attention of the network to help new task learning. Experimental results on product reviews from Amazon.com show the effectiveness of the proposed model.
Arabizi Language Models for Sentiment Analysis
Authors: Ga{\'e}tan Baert, Souhir Gahbiche, Guillaume Gadek, Alexandre Pauchet
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Language Modeling, Language Modelling, Opinion Mining, Sentiment Analysis

Arabizi is a written form of spoken Arabic, relying on Latin characters and digits. It is informal and does not follow any conventional rules, raising many NLP challenges. In particular, Arabizi has recently emerged as the Arabic language in online social networks, becoming of great interest for opinion mining and sentiment analysis. Unfortunately, only few Arabizi resources exist and state-of-the-art language models such as BERT do not consider Arabizi. In this work, we construct and release two datasets: (i) LAD, a corpus of 7.7M tweets written in Arabizi and (ii) SALAD, a subset of LAD, manually annotated for sentiment analysis. Then, a BERT architecture is pre-trained on LAD, in order to create and distribute an Arabizi language model called BAERT. We show that a language model (BAERT) pre-trained on a large corpus (LAD) in the same language (Arabizi) as that of the fine-tuning dataset (SALAD), outperforms a state-of-the-art multi-lingual pretrained model (multilingual BERT) on a sentiment analysis task.
Modeling Local Contexts for Joint Dialogue Act Recognition and Sentiment Classification with Bi-channel Dynamic Convolutions
Authors: Jingye Li, Hao Fei, Donghong Ji
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Language Modeling, Language Modelling, Multi-Task Learning, Sentiment Analysis, Sentiment Classification

In this paper, we target improving the joint dialogue act recognition (DAR) and sentiment classification (SC) tasks by fully modeling the local contexts of utterances. First, we employ the dynamic convolution network (DCN) as the utterance encoder to capture the dialogue contexts. Further, we propose a novel context-aware dynamic convolution network (CDCN) to better leverage the local contexts when dynamically generating kernels. We extended our frameworks into bi-channel version (i.e., BDCN and BCDCN) under multi-task learning to achieve the joint DAR and SC. Two channels can learn their own feature representations for DAR and SC, respectively, but with latent interaction. Besides, we suggest enhancing the tasks by employing the DiaBERT language model. Our frameworks obtain state-of-the-art performances against all baselines on two benchmark datasets, demonstrating the importance of modeling the local contexts.
Named Entity Recognition for Chinese biomedical patents
Authors: Yuting Hu, Suzan Verberne
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

There is a large body of work on Biomedical Entity Recognition (Bio-NER) for English but there have only been a few attempts addressing NER for Chinese biomedical texts. Because of the growing amount of Chinese biomedical discoveries being patented, and lack of NER models for patent data, we train and evaluate NER models for the analysis of Chinese biomedical patent data, based on BERT. By doing so, we show the value and potential of this domain-specific NER task. For the evaluation of our methods we built our own Chinese biomedical patents NER dataset, and our optimized model achieved an F1 score of 0.54{\mbox{$\pm$}}0.15. Further biomedical analysis indicates that our solution can help detecting meaningful biomedical entities and novel gene-gene interactions, with limited labeled data, training time and computing power.
Learning Health-Bots from Training Data that was Automatically Created using Paraphrase Detection and Expert Knowledge
Authors: Anna Liednikova, Philippe Jolivet, Alexandre Durand-Salmon, Claire Gardent
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Retrieval

A key bottleneck for developing dialog models is the lack of adequate training data. Due to privacy issues, dialog data is even scarcer in the health domain. We propose a novel method for creating dialog corpora which we apply to create doctor-patient interaction data. We use this data to learn both a generation and a hybrid classification/retrieval model and find that the generation model consistently outperforms the hybrid model. We show that our data creation method has several advantages. Not only does it allow for the semi-automatic creation of large quantities of training data. It also provides a natural way of guiding learning and a novel method for assessing the quality of human-machine interactions.
Enhancing Clinical BERT Embedding using a Biomedical Knowledge Base
Authors: Boran Hao, Henghui Zhu, Ioannis Paschalidis
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Language Modeling, Language Modelling, Natural Language Inference

Domain knowledge is important for building Natural Language Processing (NLP) systems for low-resource settings, such as in the clinical domain. In this paper, a novel joint training method is introduced for adding knowledge base information from the Unified Medical Language System (UMLS) into language model pre-training for some clinical domain corpus. We show that in three different downstream clinical NLP tasks, our pre-trained language model outperforms the corresponding model with no knowledge base information and other state-of-the-art models. Specifically, in a natural language inference task applied to clinical texts, our knowledge base pre-training approach improves accuracy by up to 1.7{\%}, whereas in clinical name entity recognition tasks, the F1-score improves by up to 1.0{\%}. The pre-trained models are available at https://github.com/noc-lab/clinical-kb-bert.
TIMBERT: Toponym Identifier For The Medical Domain Based on BERT
Authors: MohammadReza Davari, Leila Kosseim, Tien Bui
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Sentence, Toponym Recognition

In this paper, we propose an approach to automate the process of place name detection in the medical domain to enable epidemiologists to better study and model the spread of viruses. We created a family of Toponym Identification Models based on BERT (TIMBERT), in order to learn in an end-to-end fashion the mapping from an input sentence to the associated sentence labeled with toponyms. When evaluated with the SemEval 2019 task 12 test set (Weissenbacher et al., 2019), our best TIMBERT model achieves an F1 score of 90.85{\%}, a significant improvement compared to the state-of-the-art of 89.13{\%} (Wang et al., 2019).
Extracting Adherence Information from Electronic Health Records
Authors: Jordan Sanders, Meghana Gudala, Kathleen Hamilton, Nishtha Prasad, Jordan Stovall, Eduardo Blanco, Jane E Hamilton, Kirk Roberts
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Sentence

Patient adherence is a critical factor in health outcomes. We present a framework to extract adherence information from electronic health records, including both sentence-level information indicating general adherence information (full, partial, none, etc.) and span-level information providing additional information such as adherence type (medication or nonmedication), reasons and outcomes. We annotate and make publicly available a new corpus of 3,000 de-identified sentences, and discuss the language physicians use to document adherence information. We also explore models based on state-of-the-art transformers to automate both tasks.
French Biomedical Text Simplification: When Small and Precise Helps
Authors: R{\'e}mi Cardon, Natalia Grabar
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Text Simplification

We present experiments on biomedical text simplification in French. We use two kinds of corpora {--} parallel sentences extracted from existing health comparable corpora in French and WikiLarge corpus translated from English to French {--} and a lexicon that associates medical terms with paraphrases. Then, we train neural models on these parallel corpora using different ratios of general and specialized sentences. We evaluate the results with BLEU, SARI and Kandel scores. The results point out that little specialized data helps significantly the simplification.
Summarizing Medical Conversations via Identifying Important Utterances
Authors: Yan Song, Yuanhe Tian, Nan Wang, Fei Xia
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Conversation Summarization

Summarization is an important natural language processing (NLP) task in identifying key information from text. For conversations, the summarization systems need to extract salient contents from spontaneous utterances by multiple speakers. In a special task-oriented scenario, namely medical conversations between patients and doctors, the symptoms, diagnoses, and treatments could be highly important because the nature of such conversation is to find a medical solution to the problem proposed by the patients. Especially consider that current online medical platforms provide millions of public available conversations between real patients and doctors, where the patients propose their medical problems and the registered doctors offer diagnosis and treatment, a conversation in most cases could be too long and the key information is hard to be located. Therefore, summarizations to the patients{'} problems and the doctors{'} treatments in the conversations can be highly useful, in terms of helping other patients with similar problems have a precise reference for potential medical solutions. In this paper, we focus on medical conversation summarization, using a dataset of medical conversations and corresponding summaries which were crawled from a well-known online healthcare service provider in China. We propose a hierarchical encoder-tagger model (HET) to generate summaries by identifying important utterances (with respect to problem proposing and solving) in the conversations. For the particular dataset used in this study, we show that high-quality summaries can be generated by extracting two types of utterances, namely, problem statements and treatment recommendations. Experimental results demonstrate that HET outperforms strong baselines and models from previous studies, and adding conversation-related features can further improve system performance.
Modeling language evolution and feature dynamics in a realistic geographic environment
Authors: Rhea Kapur, Phillip Rogers
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8

Recent, innovative efforts to understand the uneven distribution of languages and linguistic feature values in time and space attest to both the challenge these issues pose and the value in solving them. In this paper, we introduce a model for simulating languages and their features over time in a realistic geographic environment. At its core is a model of language phylogeny and migration whose parameters are chosen to reproduce known language family sizes and geographic dispersions. This foundation in turn is used to explore the dynamics of linguistic features. Languages are assigned feature values that can change randomly or under the influence of nearby languages according to predetermined probabilities. We assess the effects of these settings on resulting geographic and genealogical patterns using homogeneity measures defined in the literature. The resulting model is both flexible and realistic, and it can be employed to answer a wide range of related questions.
Syntax-Aware Graph Attention Network for Aspect-Level Sentiment Classification
Authors: Lianzhe Huang, Xin Sun, Sujian Li, Linhao Zhang, Houfeng Wang
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Classification, Graph Attention, Language Modeling, Language Modelling, Sentence, Sentiment Analysis, Sentiment Classification

Aspect-level sentiment classification aims to distinguish the sentiment polarities over aspect terms in a sentence. Existing approaches mostly focus on modeling the relationship between the given aspect words and their contexts with attention, and ignore the use of more elaborate knowledge implicit in the context. In this paper, we exploit syntactic awareness to the model by the graph attention network on the dependency tree structure and external pre-training knowledge by BERT language model, which helps to model the interaction between the context and aspect words better. And the subwords of BERT are integrated into the dependency tree graphs, which can obtain more accurate representations of words by graph attention. Experiments demonstrate the effectiveness of our model.
Label Correction Model for Aspect-based Sentiment Analysis
Authors: Qianlong Wang, Jiangtao Ren
Date: 2020-12-01 00:00:00
Proceeding: COLING 2020 8
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Aspect Extraction, model, Sentence, Sentiment Analysis, Sentiment Classification

Aspect-based sentiment analysis includes opinion aspect extraction and aspect sentiment classification. Researchers have attempted to discover the relationship between these two sub-tasks and have proposed the joint model for solving aspect-based sentiment analysis. However, they ignore a phenomenon: aspect boundary label and sentiment label of the same word can correct each other. To exploit this phenomenon, we propose a novel deep learning model named the label correction model. Specifically, given an input sentence, our model first predicts the aspect boundary label sequence and sentiment label sequence, then re-predicts the aspect boundary (sentiment) label sequence using the embeddings of the previously predicted sentiment (aspect boundary) label. The goal of the re-prediction operation (can be repeated multiple times) is to use the information of the sentiment (aspect boundary) label to correct the wrong aspect boundary (sentiment) label. Moreover, we explore two ways of using label embeddings: add and gate mechanism. We evaluate our model on three benchmark datasets. Experimental results verify that our model achieves state-of-the-art performance compared with several baselines.