pwc-archive/papers-with-abstracts huggingface dataset:
ConceptBert: Concept-Aware Representation for Visual Question Answering
Authors: Fran{\c{c}}ois Gard{\`e}res, Maryam Ziaeefard, Baptiste Abeloos, Freddy Lecue
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Common Sense Reasoning, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Visual Question Answering (VQA) is a challenging task that has received increasing attention from both the computer vision and the natural language processing communities. A VQA model combines visual and textual features in order to answer questions grounded in an image. Current works in VQA focus on questions which are answerable by direct analysis of the question and image alone. We present a concept-aware algorithm, ConceptBert, for questions which require common sense, or basic factual knowledge from external structured content. Given an image and a question in natural language, ConceptBert requires visual elements of the image and a Knowledge Graph (KG) to infer the correct answer. We introduce a multi-modal representation which learns a joint Concept-Vision-Language embedding inspired by the popular BERT architecture. We exploit ConceptNet KG for encoding the common sense knowledge and evaluate our methodology on the Outside Knowledge-VQA (OK-VQA) and VQA datasets.
Revisiting Representation Degeneration Problem in Language Modeling
Authors: Zhong Zhang, Chongming Gao, Cong Xu, Rui Miao, Qinli Yang, Junming Shao
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Language Modeling, Language Modelling, Machine Translation, Text Generation, Translation, Word Embeddings

Weight tying is now a common setting in many language generation tasks such as language modeling and machine translation. However, a recent study reveals that there is a potential flaw in weight tying. They find that the learned word embeddings are likely to degenerate and lie in a narrow cone when training a language model. They call it the representation degeneration problem and propose a cosine regularization to solve it. Nevertheless, we prove that the cosine regularization is insufficient to solve the problem, as the degeneration is still likely to happen under certain conditions. In this paper, we revisit the representation degeneration problem and theoretically analyze the limitations of the previously proposed solution. Afterward, we propose an alternative regularization method called Laplacian regularization to tackle the problem. Experiments on language modeling demonstrate the effectiveness of the proposed Laplacian regularization.
Few-Shot Multi-Hop Relation Reasoning over Knowledge Bases
Authors: Chuxu Zhang, Lu Yu, Mandana Saebi, Meng Jiang, Nitesh Chawla
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Meta-Learning, reinforcement-learning, Reinforcement Learning (RL), Relation, Relation Prediction

Multi-hop relation reasoning over knowledge base is to generate effective and interpretable relation prediction through reasoning paths. The current methods usually require sufficient training data (fact triples) for each query relation, impairing their performances over few-shot relations (with limited triples) which are common in knowledge base. To this end, we propose FIRE, a novel few-shot multi-hop relation learning model. FIRE applies reinforcement learning to model the sequential steps of multi-hop reasoning, besides performs heterogeneous structure encoding and knowledge-aware search space pruning. The meta-learning technique is employed to optimize model parameters that could quickly adapt to few-shot relations. Empirical study on two datasets demonstrate that FIRE outperforms state-of-the-art methods.
A structure-enhanced graph convolutional network for sentiment analysis
Authors: Fanyu Meng, Junlan Feng, Danping Yin, Si Chen, Min Hu
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Dependency Parsing, Sentence, Sentiment Analysis

Syntactic information is essential for both sentiment analysis(SA) and aspect-based sentiment analysis(ABSA). Previous work has already achieved great progress utilizing Graph Convolutional Network(GCN) over dependency tree of a sentence. However, these models do not fully exploit the syntactic information obtained from dependency parsing such as the diversified types of dependency relations. The message passing process of GCN should be distinguished based on these syntactic information.To tackle this problem, we design a novel weighted graph convolutional network(WGCN) which can exploit rich syntactic information based on the feature combination. Furthermore, we utilize BERT instead of Bi-LSTM to generate contextualized representations as inputs for GCN and present an alignment method to keep word-level dependencies consistent with wordpiece unit of BERT. With our proposal, we are able to improve the state-of-the-art on four ABSA tasks out of six and two SA tasks out of three.
Empirical Studies of Institutional Federated Learning For Natural Language Processing
Authors: Xinghua Zhu, Jianzong Wang, Zhenhou Hong, Jing Xiao
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Federated Learning, GPU, intent-classification, Intent Classification, Sentence

Federated learning has sparkled new interests in the deep learning society to make use of isolated data sources from independent institutes. With the development of novel training tools, we have successfully deployed federated natural language processing networks on GPU-enabled server clusters. This paper demonstrates federated training of a popular NLP model, TextCNN, with applications in sentence intent classification. Furthermore, differential privacy is introduced to protect participants in the training process, in a manageable manner. Distinguished from previous client-level privacy protection schemes, the proposed differentially private federated learning procedure is defined in the dataset sample level, inherent with the applications among institutions instead of individual users. Optimal settings of hyper-parameters for the federated TextCNN model are studied through comprehensive experiments. We also evaluated the performance of federated TextCNN model under imbalanced data load configuration. Experiments show that, the sampling ratio has a large impact on the performance of the FL models, causing up to 38.4{\%} decrease in the test accuracy, while they are robust to different noise multiplier levels, with less than 3{\%} variance in the test accuracy. It is also found that the FL models are sensitive to data load balancedness among client datasets. When the data load is imbalanced, model performance dropped by up to 10{\%}.
NeuReduce: Reducing Mixed Boolean-Arithmetic Expressions by Recurrent Neural Network
Authors: Weijie Feng, Binbin Liu, Dongpeng Xu, Qilong Zheng, Yun Xu
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Form

Mixed Boolean-Arithmetic (MBA) expressions involve both arithmetic calculation (e.g.,plus, minus, multiply) and bitwise computation (e.g., and, or, negate, xor). MBA expressions have been widely applied in software obfuscation, transforming programs from a simple form to a complex form. MBA expressions are challenging to be simplified, because the interleaving bitwise and arithmetic operations causing mathematical reduction laws to be ineffective. Our goal is to recover the original, simple form from an obfuscated MBA expression. In this paper, we first propose NeuReduce, a string to string method based on neural networks to automatically learn and reduce complex MBA expressions. We develop a comprehensive MBA dataset, including one million diversified MBA expression samples and corresponding simplified forms. After training on the dataset, NeuReduce can reduce MBA rules to homelier but mathematically equivalent forms. By comparing with three state-of-the-art MBA reduction methods, our evaluation result shows that NeuReduce outperforms all other tools in terms of accuracy, solving time, and performance overhead.
Toward Recognizing More Entity Types in NER: An Efficient Implementation using Only Entity Lexicons
Authors: Minlong Peng, Ruotian Ma, Qi Zhang, Lujun Zhao, Mengxi Wei, Changlong Sun, Xuanjing Huang
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

In this work, we explore the way to quickly adjust an existing named entity recognition (NER) system to make it capable of recognizing entity types not defined in the system. As an illustrative example, consider the case that a NER system has been built to recognize person and organization names, and now it requires to additionally recognize job titles. Such a situation is common in the industrial areas, where the entity types required to recognize vary a lot in different products and keep changing. To avoid laborious data labeling and achieve fast adaptation, we propose to adjust the existing NER system using the previously labeled data and entity lexicons of the newly introduced entity types. We formulate such a task as a partially supervised learning problem and accordingly propose an effective algorithm to solve the problem. Comprehensive experimental studies on several public NER datasets validate the effectiveness of our method.
From Disjoint Sets to Parallel Data to Train Seq2Seq Models for Sentiment Transfer
Authors: Paulo Cavalin, Marisa Vasconcelos, Marcelo Grave, Claudio Pinhanez, Victor Henrique Alves Ribeiro
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Machine Translation, Semantic Similarity, Semantic Textual Similarity, Translation

We present a method for creating parallel data to train Seq2Seq neural networks for sentiment transfer. Most systems for this task, which can be viewed as monolingual machine translation (MT), have relied on unsupervised methods, such as Generative Adversarial Networks (GANs)-inspired approaches, for coping with the lack of parallel corpora. Given that the literature shows that Seq2Seq methods have been consistently outperforming unsupervised methods in MT-related tasks, in this work we exploit the use of semantic similarity computation for converting non-parallel data onto a parallel corpus. That allows us to train a transformer neural network for the sentiment transfer task, and compare its performance against unsupervised approaches. With experiments conducted on two well-known public datasets, i.e. Yelp and Amazon, we demonstrate that the proposed methodology outperforms existing unsupervised methods very consistently in fluency, and presents competitive results in terms of sentiment conversion and content preservation. We believe that this works opens up an opportunity for seq2seq neural networks to be better exploited in problems for which they have not been applied owing to the lack of parallel training data.
Event Extraction as Multi-turn Question Answering
Authors: Fayuan Li, Weihua Peng, Yuguang Chen, Quan Wang, Lu Pan, Yajuan Lyu, Yong Zhu
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Event Extraction, Question Answering, Reading Comprehension, Sentence

Event extraction, which aims to identify event triggers of pre-defined event types and their arguments of specific roles, is a challenging task in NLP. Most traditional approaches formulate this task as classification problems, with event types or argument roles taken as golden labels. Such approaches fail to model rich interactions among event types and arguments of different roles, and cannot generalize to new types or roles. This work proposes a new paradigm that formulates event extraction as multi-turn question answering. Our approach, MQAEE, casts the extraction task into a series of reading comprehension problems, by which it extracts triggers and arguments successively from a given sentence. A history answer embedding strategy is further adopted to model question answering history in the multi-turn process. By this new formulation, MQAEE makes full use of dependency among arguments and event types, and generalizes well to new types with new argument roles. Empirical results on ACE 2005 shows that MQAEE outperforms current state-of-the-art, pushing the final F1 of argument extraction to 53.4{\%} (+2.0{\%}). And it also has a good generalization ability, achieving competitive performance on 13 new event types even if trained only with a few samples of them.
Actor-Double-Critic: Incorporating Model-Based Critic for Task-Oriented Dialogue Systems
Authors: Yen-chen Wu, Bo-Hsiang Tseng, Milica Gasic
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Deep Reinforcement Learning, Spoken Dialogue Systems, Task-Oriented Dialogue Systems

In order to improve the sample-efficiency of deep reinforcement learning (DRL), we implemented imagination augmented agent (I2A) in spoken dialogue systems (SDS). Although I2A achieves a higher success rate than baselines by augmenting predicted future into a policy network, its complicated architecture introduces unwanted instability. In this work, we propose actor-double-critic (ADC) to improve the stability and overall performance of I2A. ADC simplifies the architecture of I2A to reduce excessive parameters and hyper-parameters. More importantly, a separate model-based critic shares parameters between actions and makes back-propagation explicit. In our experiments on Cambridge Restaurant Booking task, ADC enhances success rates considerably and shows robustness to imperfect environment models. In addition, ADC exhibits the stability and sample-efficiency as significantly reducing the baseline standard deviation of success rates and reaching the 80{\%} success rate with half training data.
Sequential Span Classification with Neural Semi-Markov CRFs for Biomedical Abstracts
Authors: Kosuke Yamada, Tsutomu Hirao, Ryohei Sasano, Koichi Takeda, Masaaki Nagata
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Sentence, Sentence Classification

Dividing biomedical abstracts into several segments with rhetorical roles is essential for supporting researchers{'} information access in the biomedical domain. Conventional methods have regarded the task as a sequence labeling task based on sequential sentence classification, i.e., they assign a rhetorical label to each sentence by considering the context in the abstract. However, these methods have a critical problem: they are prone to mislabel longer continuous sentences with the same rhetorical label. To tackle the problem, we propose sequential span classification that assigns a rhetorical label, not to a single sentence but to a span that consists of continuous sentences. Accordingly, we introduce Neural Semi-Markov Conditional Random Fields to assign the labels to such spans by considering all possible spans of various lengths. Experimental results obtained from PubMed 20k RCT and NICTA-PIBOSO datasets demonstrate that our proposed method achieved the best micro sentence-F1 score as well as the best micro span-F1 score.
Where to Submit? Helping Researchers to Choose the Right Venue
Authors: Konstantin Kobs, Tobias Koopmann, Albin Zehe, David Fernes, Philipp Krop, Andreas Hotho
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020

Whenever researchers write a paper, the same question occurs: {``}Where to submit?{''} In this work, we introduce WTS, an open and interpretable NLP system that recommends conferences and journals to researchers based on the title, abstract, and/or keywords of a given paper. We adapt the TextCNN architecture and automatically analyze its predictions using the Integrated Gradients method to highlight words and phrases that led to the recommendation of a scientific venue. We train and test our method on publications from the fields of artificial intelligence (AI) and medicine, both derived from the Semantic Scholar dataset. WTS achieves an Accuracy@5 of approximately 83{\%} for AI papers and 95{\%} in the field of medicine. It is open source and available for testing on https://wheretosubmit.ml.
Hybrid Emoji-Based Masked Language Models for Zero-Shot Abusive Language Detection
Authors: Michele Corazza, Stefano Menini, Elena Cabrio, Sara Tonelli, Serena Villata
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Abusive Language, Language Modeling, Language Modelling, Machine Translation, Natural Language Inference, Translation

Recent studies have demonstrated the effectiveness of cross-lingual language model pre-training on different NLP tasks, such as natural language inference and machine translation. In our work, we test this approach on social media data, which are particularly challenging to process within this framework, since the limited length of the textual messages and the irregularity of the language make it harder to learn meaningful encodings. More specifically, we propose a hybrid emoji-based Masked Language Model (MLM) to leverage the common information conveyed by emojis across different languages and improve the learned cross-lingual representation of short text messages, with the goal to perform zero- shot abusive language detection. We compare the results obtained with the original MLM to the ones obtained by our method, showing improved performance on German, Italian and Spanish.
SeNsER: Learning Cross-Building Sensor Metadata Tagger
Authors: Yang Jiao, Jiacheng Li, Jiaman Wu, Dezhi Hong, Rajesh Gupta, Jingbo Shang
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

Sensor metadata tagging, akin to the named entity recognition task, provides key contextual information (e.g., measurement type and location) about sensors for running smart building applications. Unfortunately, sensor metadata in different buildings often follows distinct naming conventions. Therefore, learning a tagger currently requires extensive annotations on a per building basis. In this work, we propose a novel framework, SeNsER, which learns a sensor metadata tagger for a new building based on its raw metadata and some existing fully annotated building. It leverages the commonality between different buildings: At the character level, it employs bidirectional neural language models to capture the shared underlying patterns between two buildings and thus regularizes the feature learning process; At the word level, it leverages as features the k-mers existing in the fully annotated building. During inference, we further incorporate the information obtained from sources such as Wikipedia as prior knowledge. As a result, SeNsER shows promising results in extensive experiments on multiple real-world buildings.
LiMiT: The Literal Motion in Text Dataset
Authors: Irene Manotas, Ngoc Phuoc An Vo, Vadim Sheinin
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Diversity

Motion recognition is one of the basic cognitive capabilities of many life forms, yet identifying motion of physical entities in natural language have not been explored extensively and empirically. We present the Literal-Motion-in-Text (LiMiT) dataset, a large human-annotated collection of English text sentences describing physical occurrence of motion, with annotated physical entities in motion. We describe the annotation process for the dataset, analyze its scale and diversity, and report results of several baseline models. We also present future research directions and applications of the LiMiT dataset and share it publicly as a new resource for the research community.
ESTeR: Combining Word Co-occurrences and Word Associations for Unsupervised Emotion Detection
Authors: Sujatha Das Gollapalli, Polina Rozenshtein, See-Kiong Ng
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Management, Word Embeddings

Accurate detection of emotions in user- generated text was shown to have several applications for e-commerce, public well-being, and disaster management. Currently, the state-of-the-art performance for emotion detection in text is obtained using complex, deep learning models trained on domain-specific, labeled data. In this paper, we propose ESTeR , an unsupervised model for identifying emotions using a novel similarity function based on random walks on graphs. Our model combines large-scale word co-occurrence information with word-associations from lexicons avoiding not only the dependence on labeled datasets, but also an explicit mapping of words to latent spaces used in emotion-enriched word embeddings. Our similarity function can also be computed efficiently. We study a range of datasets including recent tweets related to COVID-19 to illustrate the superior performance of our model and report insights on public emotions during the on-going pandemic.
Make Templates Smarter: A Template Based Data2Text System Powered by Text Stitch Model
Authors: Bingfeng Luo, Zuo Bai, Kunfeng Lai, Jianping Shen
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Diversity

Neural network (NN) based data2text models achieve state-of-the-art (SOTA) performance in most metrics, but they sometimes drop or modify the information in the input, and it is hard to control the generation contents. Moreover, it requires paired training data that are usually expensive to collect. Template-based methods have good fidelity and controllability but require heavy human involvement. We propose a novel template-based data2text system powered by a text stitch model. It ensures fidelity and controllability by using templates to produce the main contents. In addition, it reduces human involvement in template design by using a text stitch model to automatically stitch adjacent template units, which is a step that usually requires careful template design and limits template reusability. The text stitch model can be trained in self-supervised fashion, which only requires free texts. The experiments on a benchmark dataset show that our system outperforms SOTA NN-based systems in fidelity and surpasses template-based systems in diversity and human involvement.
GCDST: A Graph-based and Copy-augmented Multi-domain Dialogue State Tracking
Authors: Peng Wu, Bowei Zou, Ridong Jiang, AiTi Aw
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Dialogue State Tracking, Multi-domain Dialogue State Tracking, Task-Oriented Dialogue Systems

As an essential component of task-oriented dialogue systems, Dialogue State Tracking (DST) takes charge of estimating user intentions and requests in dialogue contexts and extracting substantial goals (states) from user utterances to help the downstream modules to determine the next actions of dialogue systems. For practical usages, a major challenge to constructing a robust DST model is to process a conversation with multi-domain states. However, most existing approaches trained DST on a single domain independently, ignoring the information across domains. To tackle the multi-domain DST task, we first construct a dialogue state graph to transfer structured features among related domain-slot pairs across domains. Then, we encode the graph information of dialogue states by graph convolutional networks and utilize a hard copy mechanism to directly copy historical states from the previous conversation. Experimental results show that our model improves the performances of the multi-domain DST baseline (TRADE) with the absolute joint accuracy of 2.0{\%} and 1.0{\%} on the MultiWOZ 2.0 and 2.1 dialogue datasets, respectively.