pwc-archive/papers-with-abstracts huggingface dataset:
Global entity alignment with Gated Latent Space Neighborhood Aggregation
Authors: Chen Wei, Chen Xiaoying, Xiong Shengwu
Date: 2021-08-01 00:00:00
Proceeding: CCL 2021 8
Tasks: Entity Alignment, Entity Embeddings

“Existing entity alignment models mainly use the topology structure of the original knowledge graph and have achieved promising performance. However they are still challenged by the heterogeneous topological neighborhood structures which could cause the models to produce different representations of counterpart entities. In the paper we propose a global entity alignment model with gated latent space neighborhood aggregation (LatsEA) to address this challenge. Latent space neighborhood is formed by calculating the similarity between the entity embeddings it can introduce long-range neighbors to expand the topological neighborhood and reconcile the heterogeneous neighborhood structures. Meanwhile it uses vanilla GCN to aggregate the topological neighborhood and latent space neighborhood respectively. Then it uses an average gating mechanism to aggregate topological neighborhood information and latent space neighborhood information of the central entity. In order to further consider the interdependence between entity alignment decisions we propose a global entity alignment strategy i.e. formulate entity alignment as the maximum bipartite matching problem which is effectively solved by Hungarian algorithm. Our experiments with ablation studies on three real-world entity alignment datasets prove the effectiveness of the proposed model. Latent space neighborhood informationand global entity alignment decisions both contributes to the entity alignment performance improvement.”
Few-Shot Charge Prediction with Multi-Grained Features and MutualInformation
Authors: Zhang Han, Zhu Yutao, Dou Zhicheng, Wen Ji-Rong
Date: 2021-08-01 00:00:00
Proceeding: CCL 2021 8
Tasks: Model Optimization, Prediction

“Charge prediction aims to predict the final charge for a case according to its fact descriptionand plays an important role in legal assistance systems. With deep learning based methods prediction on high-frequency charges has achieved promising results but that on few-shot chargesis still challenging. In this work we propose a framework with multi-grained features and mutual information for few-shot charge prediction. Specifically we extract coarse- and fine-grained features to enhance the model’s capability on representation based on which the few-shot chargescan be better distinguished. Furthermore we propose a loss function based on mutual information.This loss function leverages the prior distribution of the charges to tune their weights so the few-shot charges can contribute more on model optimization. Experimental results on several datasets demonstrate the effectiveness and robustness of our method. Besides our method can work wellon tiny datasets and has better efficiency in the training which provides better applicability in realscenarios.”
Sketchy Scene Captioning: Learning Multi-Level Semantic Information from Sparse Visual Scene Cues
Authors: Zhou Lian, Chen Yangdong, Zhang Yuejie
Date: 2021-08-01 00:00:00
Proceeding: CCL 2021 8
Tasks: Sentence

“To enrich the research about sketch modality a new task termed Sketchy Scene Captioning isproposed in this paper. This task aims to generate sentence-level and paragraph-level descrip-tions for a sketchy scene. The sentence-level description provides the salient semantics of asketchy scene while the paragraph-level description gives more details about the sketchy scene.Sketchy Scene Captioning can be viewed as an extension of sketch classification which can onlyprovide one class label for a sketch. To generate multi-level descriptions for a sketchy scene ischallenging because of the visual sparsity and ambiguity of the sketch modality. To achieve ourgoal we first contribute a sketchy scene captioning dataset to lay the foundation of this new task.The popular sequence learning scheme e.g. Long Short-Term Memory neural network with vi-sual attention mechanism is then adopted to recognize the objects in a sketchy scene and inferthe relations among the objects. In the experiments promising results have been achieved on the proposed dataset. We believe that this work will motivate further researches on the understanding of sketch modality and the numerous sketch-based applications in our daily life. The collected dataset is released at https://github.com/SketchysceneCaption/Dataset.”
BDCN: Semantic Embedding Self-explanatory Breast Diagnostic Capsules Network
Authors: Chen Dehua, Zhong Keting, He Jianrong
Date: 2021-08-01 00:00:00
Proceeding: CCL 2021 8
Tasks: Diagnostic, Text Categorization, text-classification, Text Classification

“Building an interpretable AI diagnosis system for breast cancer is an important embodiment ofAI assisted medicine. Traditional breast cancer diagnosis methods based on machine learning areeasy to explain but the accuracy is very low. Deep neural network greatly improves the accuracy of diagnosis but the black box model does not provide transparency and interpretation. In this work we propose a semantic embedding self-explanatory Breast Diagnostic Capsules Network(BDCN). This model is the first to combine the capsule network with semantic embedding for theAI diagnosis of breast tumors using capsules to simulate semantics. We pre-trained the extrac-tion word vector by embedding the semantic tree into the BERT and used the capsule network to improve the semantic representation of multiple heads of attention to construct the extraction feature the capsule network was extended from the computer vision classification task to the text classification task. Simultaneously both the back propagation principle and dynamic routing algorithm are used to realize the local interpretability of the diagnostic model. The experimental results show that this breast diagnosis model improves the model performance and has good interpretability which is more suitable for clinical situations.IntroductionBreast cancer is an important killer threatening women’s health because of rising incidence. Early detection and diagnosis are the key to reduce the mortality rate of breast cancer and improve the quality of life of patients. Mammary gland molybdenum target report contains rich semantic information whichcan directly reflect the results of breast cancer screening (CACA-CBCS 2019) and AI-assisted diagno-sis of breast cancer is an important means. Therefore various diagnostic models were born. Mengwan(2020) used support vector machine(SVM) and Naive Bayes to classify morphological features with anaccuracy of 91.11%. Wei (2009) proposed a classification method of breast cancer based on SVM andthe accuracy of the classifier experiment is 79.25%. These traditional AI diagnoses of breast tumors havelimited data volume and low accuracy. Deep Neural Networks (DNN) enters into the ranks of the diagno-sis of breast tumor. Wang (2019) put forward a kind of based on feature fusion with CNN deep features of breast computer-aided diagnosis methods the accuracy is 92.3%. Zhao (2018) investigated capsule networks with dynamic routing for text classification which proves the feasibility of text categorization. Existing models have poor predictive effect and lack of interpretation which can not meet the clinical needs.”
GCN with External Knowledge for Clinical Event Detection
Authors: Liu Dan, Zhang Zhichang, Peng Hui, Han Ruirui
Date: 2021-08-01 00:00:00
Proceeding: CCL 2021 8
Tasks: Decoder, Event Detection

“In recent years with the development of deep learning and the increasing demand for medical information acquisition in medical information technology applications such as clinical decision support Clinical Event Detection has been widely studied as its subtask. However directly applying advances in deep learning to Clinical Event Detection tasks often produces undesirable results. This paper proposes a multi-granularity information fusion encoder-decoder frameworkthat introduces external knowledge. First the word embedding generated by the pre-trained biomedical language representation model (BioBERT) and the character embedding generatedby the Convolutional Neural Network are spliced. And then perform Part-of-Speech attention coding for character-level embedding perform semantic Graph Convolutional Network codingfor the spliced character-word embedding. Finally the information of these three parts is fusedas Conditional Random Field input to generate the sequence label of the word. The experimental results on the 2012 i2b2 data set show that the model in this paper is superior to other existingmodels. In addition the model in this paper alleviates the problem that “occurrence” event typeseem more difficult to detect than other event types.”
A Prompt-independent and Interpretable Automated Essay Scoring Method for Chinese Second Language Writing
Authors: Wang Yupei, Hu Renfen
Date: 2021-08-01 00:00:00
Proceeding: CCL 2021 8
Tasks: Automated Essay Scoring

“With the increasing popularity of learning Chinese as a second language (L2) the development of an automatic essay scoring (AES) method specially for Chinese L2 essays has become animportant task. To build a robust model that could easily adapt to prompt changes we propose 90linguistic features with consideration of both language complexity and correctness and introducethe Ordinal Logistic Regression model that explicitly combines these linguistic features and low-level textual representations. Our model obtains a high QWK of 0.714 a low RMSE of 1.516 anda considerable Pearson correlation of 0.734. With a simple linear model we further analyze the contribution of the linguistic features to score prediction revealing the model’s interpretability and its potential to give writing feedback to users. This work provides insights and establishes asolid baseline for Chinese L2 AES studies.”
A Robustly Optimized BERT Pre-training Approach with Post-training
Authors: Liu Zhuang, Lin Wayne, Shi Ya, Zhao Jun
Date: 2021-08-01 00:00:00
Proceeding: CCL 2021 8
Tasks: Extractive Question-Answering, Question Answering

“In the paper we present a ‘pre-training’+‘post-training’+‘fine-tuning’ three-stage paradigm which is a supplementary framework for the standard ‘pre-training’+‘fine-tuning’ languagemodel approach. Furthermore based on three-stage paradigm we present a language modelnamed PPBERT. Compared with original BERT architecture that is based on the standard two-stage paradigm we do not fine-tune pre-trained model directly but rather post-train it on the domain or task related dataset first which helps to better incorporate task-awareness knowl-edge and domain-awareness knowledge within pre-trained model also from the training datasetreduce bias. Extensive experimental results indicate that proposed model improves the perfor-mance of the baselines on 24 NLP tasks which includes eight GLUE benchmarks eight Su-perGLUE benchmarks six extractive question answering benchmarks. More remarkably our proposed model is a more flexible and pluggable model where post-training approach is able to be plugged into other PLMs that are based on BERT. Extensive ablations further validate the effectiveness and its state-of-the-art (SOTA) performance. The open source code pre-trained models and post-trained models are available publicly.”
May I Ask Who’s Calling? Named Entity Recognition on Call Center Transcripts for Privacy Law Compliance
Authors: Micaela Kaplan
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (WNUT) 2020 11
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), speech-recognition, Speech Recognition

We investigate using Named Entity Recognition on a new type of user-generated text: a call center conversation. These conversations combine problems from spontaneous speech with problems novel to conversational Automated Speech Recognition, including incorrect recognition, alongside other common problems from noisy user-generated text. Using our own corpus with new annotations, training custom contextual string embeddings, and applying a BiLSTM-CRF, we match state-of- the-art results on our novel task.
“Did you really mean what you said?” : Sarcasm Detection in Hindi-English Code-Mixed Data using Bilingual Word Embeddings
Authors: Akshita Aggarwal, Anshul Wadhawan, Anshima Chaudhary, Kavita Maurya
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (WNUT) 2020 11
Tasks: Deep Learning, Sarcasm Detection, Word Embeddings

With the increased use of social media platforms by people across the world, many new interesting NLP problems have come into existence. One such being the detection of sarcasm in the social media texts. We present a corpus of tweets for training custom word embeddings and a Hinglish dataset labelled for sarcasm detection. We propose a deep learning based approach to address the issue of sarcasm detection in Hindi-English code mixed tweets using bilingual word embeddings derived from FastText and Word2Vec approaches. We experimented with various deep learning models, including CNNs, LSTMs, Bi-directional LSTMs (with and without attention). We were able to outperform all state-of-the-art performances with our deep learning models, with attention based Bi-directional LSTMs giving the best performance exhibiting an accuracy of 78.49%.
Noisy Text Data: Achilles’ Heel of BERT
Authors: Ankit Kumar, Piyush Makhija, Anuj Gupta
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (WNUT) 2020 11
Tasks: Sentiment Analysis, SST-2, STS, STS-B

Owing to the phenomenal success of BERT on various NLP tasks and benchmark datasets, industry practitioners are actively experimenting with fine-tuning BERT to build NLP applications for solving industry use cases. For most datasets that are used by practitioners to build industrial NLP applications, it is hard to guarantee absence of any noise in the data. While BERT has performed exceedingly well for transferring the learnings from one use case to another, it remains unclear how BERT performs when fine-tuned on noisy text. In this work, we explore the sensitivity of BERT to noise in the data. We work with most commonly occurring noise (spelling mistakes, typos) and show that this results in significant degradation in the performance of BERT. We present experimental results to show that BERT’s performance on fundamental NLP tasks like sentiment analysis and textual similarity drops significantly in the presence of (simulated) noise on benchmark datasets viz. IMDB Movie Review, STS-B, SST-2. Further, we identify shortcomings in the existing BERT pipeline that are responsible for this drop in performance. Our findings suggest that practitioners need to be vary of presence of noise in their datasets while fine-tuning BERT to solve industry use cases.
Combining BERT with Static Word Embeddings for Categorizing Social Media
Authors: Israa Alghanmi, Luis Espinosa Anke, Steven Schockaert
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (WNUT) 2020 11
Tasks: Word Embeddings

Pre-trained neural language models (LMs) have achieved impressive results in various natural language processing tasks, across different languages. Surprisingly, this extends to the social media genre, despite the fact that social media often has very different characteristics from the language that LMs have seen during training. A particularly striking example is the performance of AraBERT, an LM for the Arabic language, which is successful in categorizing social media posts in Arabic dialects, despite only having been trained on Modern Standard Arabic. Our hypothesis in this paper is that the performance of LMs for social media can nonetheless be improved by incorporating static word vectors that have been specifically trained on social media. We show that a simple method for incorporating such word vectors is indeed successful in several Arabic and English benchmarks. Curiously, however, we also find that similar improvements are possible with word vectors that have been trained on traditional text sources (e.g. Wikipedia).
Enhanced Sentence Alignment Network for Efficient Short Text Matching
Authors: Zhe Hu, Zuohui Fu, Cheng Peng, Weiwei Wang
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (WNUT) 2020 11
Tasks: Sentence, Text Matching

Cross-sentence attention has been widely applied in text matching, in which model learns the aligned information between two intermediate sequence representations to capture their semantic relationship. However, commonly the intermediate representations are generated solely based on the preceding layers and the models may suffer from error propagation and unstable matching, especially when multiple attention layers are used. In this paper, we pro-pose an enhanced sentence alignment network with simple gated feature augmentation, where the model is able to flexibly integrate both original word and contextual features to improve the cross-sentence attention. Moreover, our model is less complex with fewer parameters compared to many state-of-the-art structures.Experiments on three benchmark datasets validate our model capacity for text matching.
Cross-lingual sentiment classification in low-resource Bengali language
Authors: Salim Sazzed
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (WNUT) 2020 11
Tasks: Benchmarking, Classification, Cross-Lingual Sentiment Classification, Machine Translation, Sentiment Analysis, Sentiment Classification, Transfer Learning, Translation

Sentiment analysis research in low-resource languages such as Bengali is still unexplored due to the scarcity of annotated data and the lack of text processing tools. Therefore, in this work, we focus on generating resources and showing the applicability of the cross-lingual sentiment analysis approach in Bengali. For benchmarking, we created and annotated a comprehensive corpus of around 12000 Bengali reviews. To address the lack of standard text-processing tools in Bengali, we leverage resources from English utilizing machine translation. We determine the performance of supervised machine learning (ML) classifiers in machine-translated English corpus and compare it with the original Bengali corpus. Besides, we examine sentiment preservation in the machine-translated corpus utilizing Cohen’s Kappa and Gwet’s AC1. To circumvent the laborious data labeling process, we explore lexicon-based methods and study the applicability of utilizing cross-domain labeled data from the resource-rich language. We find that supervised ML classifiers show comparable performances in Bengali and machine-translated English corpus. By utilizing labeled data, they achieve 15%-20% higher F1 scores compared to both lexicon-based and transfer learning-based methods. Besides, we observe that machine translation does not alter the sentiment polarity of the review for most of the cases. Our experimental results demonstrate that the machine translation based cross-lingual approach can be an effective way for sentiment classification in Bengali.
The Non-native Speaker Aspect: Indian English in Social Media
Authors: Rupak Sarkar, Sayantan Mahinder, Ashiqur KhudaBukhsh
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (WNUT) 2020 11

As the largest institutionalized second language variety of English, Indian English has received a sustained focus from linguists for decades. However, to the best of our knowledge, no prior study has contrasted web-expressions of Indian English in noisy social media with English generated by a social media user base that are predominantly native speakers. In this paper, we address this gap in the literature through conducting a comprehensive analysis considering multiple structural and semantic aspects. In addition, we propose a novel application of language models to perform automatic linguistic quality assessment.
Sentence Boundary Detection on Line Breaks in Japanese
Authors: Yuta Hayashibe, Kensuke Mitsuzawa
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (WNUT) 2020 11
Tasks: Boundary Detection, Sentence

For NLP, sentence boundary detection (SBD) is an essential task to decompose a text into sentences. Most of the previous studies have used a simple rule that uses only typical characters as sentence boundaries. However, some characters may or may not be sentence boundaries depending on the context. We focused on line breaks in them. We newly constructed annotated corpora, implemented sentence boundary detectors, and analyzed performance of SBD in several settings.
Non-ingredient Detection in User-generated Recipes using the Sequence Tagging Approach
Authors: Yasuhiro Yamaguchi, Shintaro Inuzuka, Makoto Hiramatsu, Jun Harashima
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (WNUT) 2020 11

Recently, the number of user-generated recipes on the Internet has increased. In such recipes, users are generally supposed to write a title, an ingredient list, and steps to create a dish. However, some items in an ingredient list in a user-generated recipe are not actually edible ingredients. For example, headings, comments, and kitchenware sometimes appear in an ingredient list because users can freely write the list in their recipes. Such noise makes it difficult for computers to use recipes for a variety of tasks, such as calorie estimation. To address this issue, we propose a non-ingredient detection method inspired by a neural sequence tagging model. In our experiment, we annotated 6,675 ingredients in 600 user-generated recipes and showed that our proposed method achieved a 93.3 F1 score.
Intelligent Analyses on Storytelling for Impact Measurement
Authors: Koen Kicken, Tessa De Maesschalck, Bart Vanrumste, Tom De Keyser, Hee Reen Shim
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (WNUT) 2020 11
Tasks: Language Modelling, Sentiment Analysis

This paper explores how Dutch diary fragments, written by family coaches in the social sector, can be analysed automatically using machine learning techniques to quantitatively measure the impact of social coaching. The focus lays on two tasks: determining which sentiment a fragment contains (sentiment analysis) and investigating which fundamental social rights (education, employment, legal aid, etc.) are addressed in the fragment. To train and test the new algorithms, a dataset consisting of 1715 Dutch diary fragments is used. These fragments are manually labelled on sentiment and on the applicable fundamental social rights. The sentiment analysis models were trained to classify the fragments into three classes: negative, neutral or positive. Fine-tuning the Dutch pre-trained Bidirectional Encoder Representations from Transformers (BERTje) (de Vries et al., 2019) language model surpassed the more classic algorithms by correctly classifying 79.6% of the fragments on the sentiment analysis, which is considered as a good result. This technique also achieved the best results in the identification of the fundamental rights, where for every fragment the three most likely fundamental rights were given as output. In this way, 93% of the present fundamental rights were correctly recognised. To our knowledge, we are the first to try to extract social rights from written text with the help of Natural Language Processing techniques.
An Empirical Analysis of Human-Bot Interaction on Reddit
Authors: Ming-Cheng Ma, John P. Lalor
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (WNUT) 2020 11

Automated agents (“bots”) have emerged as an ubiquitous and influential presence on social media. Bots engage on social media platforms by posting content and replying to other users on the platform. In this work we conduct an empirical analysis of the activity of a single bot on Reddit. Our goal is to determine whether bot activity (in the form of posted comments on the website) has an effect on how humans engage on Reddit. We find that (1) the sentiment of a bot comment has a significant, positive effect on the subsequent human reply, and (2) human Reddit users modify their comment behaviors to overlap with the text of the bot, similar to how humans modify their text to mimic other humans in conversation. Understanding human-bot interactions on social media with relatively simple bots is important for preparing for more advanced bots in the future.