pwc-archive/papers-with-abstracts huggingface dataset:
RMM: A Recursive Mental Model for Dialogue Navigation
Authors: Homero Roman Roman, Yonatan Bisk, Jesse Thomason, Asli Celikyilmaz, Jianfeng Gao
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Answer Generation, Instruction Following, model

Language-guided robots must be able to both ask humans questions and understand answers. Much existing work focuses only on the latter. In this paper, we go beyond instruction following and introduce a two-agent task where one agent navigates and asks questions that a second, guiding agent answers. Inspired by theory of mind, we propose the Recursive Mental Model (RMM). The navigating agent models the guiding agent to simulate answers given candidate generated questions. The guiding agent in turn models the navigating agent to simulate navigation steps it would take to generate answers. We use the progress agents make towards the goal as a reinforcement learning reward signal to directly inform not only navigation actions, but also both question and answer generation. We demonstrate that RMM enables better generalization to novel environments. Interlocutor modelling may be a way forward for human-agent RMM where robots need to both ask and answer questions.
An Empirical Exploration of Local Ordering Pre-training for Structured Prediction
Authors: Zhisong Zhang, Xiang Kong, Lori Levin, Eduard Hovy
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Dependency Parsing, Language Modeling, Language Modelling, NER, POS, POS Tagging, Structured Prediction

Recently, pre-training contextualized encoders with language model (LM) objectives has been shown an effective semi-supervised method for structured prediction. In this work, we empirically explore an alternative pre-training method for contextualized encoders. Instead of predicting words in LMs, we {``}mask out{''} and predict word order information, with a local ordering strategy and word-selecting objectives. With evaluations on three typical structured prediction tasks (dependency parsing, POS tagging, and NER) over four languages (English, Finnish, Czech, and Italian), we show that our method is consistently beneficial. We further conduct detailed error analysis, including one that examines a specific type of parsing error where the head is misidentified. The results show that pre-trained contextual encoders can bring improvements in a structured way, suggesting that they may be able to capture higher-order patterns and feature combinations from unlabeled data.
Active Learning Approaches to Enhancing Neural Machine Translation
Authors: Yuekai Zhao, Haoran Zhang, Shuchang Zhou, Zhihua Zhang
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Active Learning, Machine Translation, NMT, Transfer Learning, Translation

Active learning is an efficient approach for mitigating data dependency when training neural machine translation (NMT) models. In this paper, we explore new training frameworks by incorporating active learning into various techniques such as transfer learning and iterative back-translation (IBT) under a limited human translation budget. We design a word frequency based acquisition function and combine it with a strong uncertainty based method. The combined method steadily outperforms all other acquisition functions in various scenarios. As far as we know, we are the first to do a large-scale study on actively training Transformer for NMT. Specifically, with a human translation budget of only 20{\%} of the original parallel corpus, we manage to surpass Transformer trained on the entire parallel corpus in three language pairs.
Continual Learning Long Short Term Memory
Authors: Xin Guo, Yu Tian, Qinghan Xue, Panos Lampropoulos, Steven Eliuk, Kenneth Barner, Xiaolong Wang
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Continual Learning, Deep Learning, Spoken Language Understanding

Catastrophic forgetting in neural networks indicates the performance decreasing of deep learning models on previous tasks while learning new tasks. To address this problem, we propose a novel Continual Learning Long Short Term Memory (CL-LSTM) cell in Recurrent Neural Network (RNN) in this paper. CL-LSTM considers not only the state of each individual task{'}s output gates but also the correlation of the states between tasks, so that the deep learning models can incrementally learn new tasks without catastrophically forgetting previously tasks. Experimental results demonstrate significant improvements of CL-LSTM over state-of-the-art approaches on spoken language understanding (SLU) tasks.
Improving End-to-End Bangla Speech Recognition with Semi-supervised Training
Authors: Nafis Sadeq, Nafis Tahmid Chowdhury, Farhan Tanvir Utshaw, Shafayat Ahmed, Muhammad Abdullah Adnan
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Automatic speech recognition systems usually require large annotated speech corpus for training. The manual annotation of a large corpus is very difficult. It can be very helpful to use unsupervised and semi-supervised learning methods in addition to supervised learning. In this work, we focus on using a semi-supervised training approach for Bangla Speech Recognition that can exploit large unpaired audio and text data. We encode speech and text data in an intermediate domain and propose a novel loss function based on the global encoding distance between encoded data to guide the semi-supervised training. Our proposed method reduces the Word Error Rate (WER) of the system from 37{\%} to 31.9{\%}.
SynET: Synonym Expansion using Transitivity
Authors: Jiale Yu, Yongliang Shen, Xinyin Ma, Chenghao Jia, Chen Chen, Weiming Lu
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020

In this paper, we study a new task of synonym expansion using transitivity, and propose a novel approach named SynET, which considers both the contexts of two given synonym pairs. It introduces an auxiliary task to reduce the impact of noisy sentences, and proposes a Multi-Perspective Entity Matching Network to match entities from multiple perspectives. Extensive experiments on a real-world dataset show the effectiveness of our approach.
Multi-Turn Dialogue Generation in E-Commerce Platform with the Context of Historical Dialogue
Authors: WeiSheng Zhang, Kaisong Song, Yangyang Kang, Zhongqing Wang, Changlong Sun, Xiaozhong Liu, Shoushan Li, Min Zhang, Luo Si
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Dialogue Generation, Representation Learning

As an important research topic, customer service dialogue generation tends to generate generic seller responses by leveraging current dialogue information. In this study, we propose a novel and extensible dialogue generation method by leveraging sellers{'} historical dialogue information, which can be both accessible and informative. By utilizing innovative historical dialogue representation learning and historical dialogue selection mechanism, the proposed model is capable of detecting most related responses from sellers{'} historical dialogues, which can further enhance the current dialogue generation quality. Unlike prior dialogue generation efforts, we treat each seller{'}s historical dialogues as a list of Customer-Seller utterance pairs and allow the model to measure their different importance, and copy words directly from most relevant pairs. Extensive experimental results show that the proposed approach can generate high-quality responses that cater to specific sellers{'} characteristics and exhibit consistent superiority over baselines on a real-world multi-turn customer service dialogue dataset.
Ruler: Data Programming by Demonstration for Document Labeling
Authors: Sara Evensen, Chang Ge, Cagatay Demiralp
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020

Data programming aims to reduce the cost of curating training data by encoding domain knowledge as labeling functions over source data. As such it not only requires domain expertise but also programming experience, a skill that many subject matter experts lack. Additionally, generating functions by enumerating rules is not only time consuming but also inherently difficult, even for people with programming experience. In this paper we introduce Ruler, an interactive system that synthesizes labeling rules using span-level interactive demonstrations over document examples. Ruler is a first-of-a-kind implementation of data programming by demonstration (DPBD). This new framework aims to relieve users from the burden of writing labeling functions, enabling them to focus on higher-level semantic analysis, such as identifying relevant signals for the labeling task. We compare Ruler with conventional data programming through a user study conducted with 10 data scientists who were asked to create labeling functions for sentiment and spam classification tasks. Results show Ruler is easier to learn and to use, and that it offers higher overall user-satisfaction while providing model performances comparable to those achieved by conventional data programming.
Chunk-based Chinese Spelling Check with Global Optimization
Authors: Zuyi Bao, Chen Li, Rui Wang
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: global-optimization, Optical Character Recognition, Optical Character Recognition (OCR), Sentence

Chinese spelling check is a challenging task due to the characteristics of the Chinese language, such as the large character set, no word boundary, and short word length. On the one hand, most of the previous works only consider corrections with similar character pronunciation or shape, failing to correct visually and phonologically irrelevant typos. On the other hand, pipeline-style architectures are widely adopted to deal with different types of spelling errors in individual modules, which is difficult to optimize. In order to handle these issues, in this work, 1) we extend the traditional confusion sets with semantical candidates to cover different types of errors; 2) we propose a chunk-based framework to correct single-character and multi-character word errors uniformly; and 3) we adopt a global optimization strategy to enable a sentence-level correction selection. The experimental results show that the proposed approach achieves a new state-of-the-art performance on three benchmark datasets, as well as an optical character recognition dataset.
Multi-pretraining for Large-scale Text Classification
Authors: Kang-Min Kim, Bumsu Hyeon, Yeachan Kim, Jun-Hyung Park, SangKeun Lee
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Classification, text-classification, Text Classification

Deep neural network-based pretraining methods have achieved impressive results in many natural language processing tasks including text classification. However, their applicability to large-scale text classification with numerous categories (e.g., several thousands) is yet to be well-studied, where the training data is insufficient and skewed in terms of categories. In addition, existing pretraining methods usually involve excessive computation and memory overheads. In this paper, we develop a novel multi-pretraining framework for large-scale text classification. This multi-pretraining framework includes both a self-supervised pretraining and a weakly supervised pretraining. We newly introduce an out-of-context words detection task on the unlabeled data as the self-supervised pretraining. It captures the topic-consistency of words used in sentences, which is proven to be useful for text classification. In addition, we propose a weakly supervised pretraining, where labels for text classification are obtained automatically from an existing approach. Experimental results clearly show that both pretraining approaches are effective for large-scale text classification task. The proposed scheme exhibits significant improvements as much as 3.8{\%} in terms of macro-averaging F1-score over strong pretraining methods, while being computationally efficient.
TSDG: Content-aware Neural Response Generation with Two-stage Decoding Process
Authors: Junsheng Kong, Zhicheng Zhong, Yi Cai, Xin Wu, Da Ren
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Decoder, Dialogue Generation, Response Generation, Vocal Bursts Valence Prediction

Neural response generative models have achieved remarkable progress in recent years but tend to yield irrelevant and uninformative responses. One of the reasons is that encoder-decoder based models always use a single decoder to generate a complete response at a stroke. This tends to generate high-frequency function words with less semantic information rather than low-frequency content words with more semantic information. To address this issue, we propose a content-aware model with two-stage decoding process named Two-stage Dialogue Generation (TSDG). We separate the decoding process of content words and function words so that content words can be generated independently without the interference of function words. Experimental results on two datasets indicate that our model significantly outperforms several competitive generative models in terms of automatic and human evaluation.
Unsupervised Cross-Lingual Adaptation of Dependency Parsers Using CRF Autoencoders
Authors: Zhao Li, Kewei Tu
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020

We consider the task of cross-lingual adaptation of dependency parsers without annotated target corpora and parallel corpora. Previous work either directly applies a discriminative source parser to the target language, ignoring unannotated target corpora, or employs an unsupervised generative parser that can leverage unannotated target data but has weaker representational power than discriminative parsers. In this paper, we propose to utilize unsupervised discriminative parsers based on the CRF autoencoder framework for this task. We train a source parser and use it to initialize and regularize a target parser that is trained on unannotated target data. We conduct experiments that transfer an English parser to 20 target languages. The results show that our method significantly outperforms previous methods.
Diversify Question Generation with Continuous Content Selectors and Question Type Modeling
Authors: Zhen Wang, Siwei Rao, Jie Zhang, Zhen Qin, Guangjian Tian, Jun Wang
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Diversity, Question Generation, Question-Generation, Vocal Bursts Type Prediction

Generating questions based on answers and relevant contexts is a challenging task. Recent work mainly pays attention to the quality of a single generated question. However, question generation is actually a one-to-many problem, as it is possible to raise questions with different focuses on contexts and various means of expression. In this paper, we explore the diversity of question generation and come up with methods from these two aspects. Specifically, we relate contextual focuses with content selectors, which are modeled by a continuous latent variable with the technique of conditional variational auto-encoder (CVAE). In the realization of CVAE, a multimodal prior distribution is adopted to allow for more diverse content selectors. To take into account various means of expression, question types are explicitly modeled and a diversity-promoting algorithm is proposed further. Experimental results on public datasets show that our proposed method can significantly improve the diversity of generated questions, especially from the perspective of using different question types. Overall, our proposed method achieves a better trade-off between generation quality and diversity compared with existing approaches.
Zero-Shot Rationalization by Multi-Task Transfer Learning from Question Answering
Authors: Po-Nien Kung, Tse-Hsuan Yang, Yi-Cheng Chen, Sheng-Siang Yin, Yun-Nung Chen
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Multi-Task Learning, Question Answering, Transfer Learning

Extracting rationales can help human understand which information the model utilizes and how it makes the prediction towards better interpretability. However, annotating rationales requires much effort and only few datasets contain such labeled rationales, making supervised learning for rationalization difficult. In this paper, we propose a novel approach that leverages the benefits of both multi-task learning and transfer learning for generating rationales through question answering in a zero-shot fashion. For two benchmark rationalization datasets, the proposed method achieves comparable or even better performance of rationalization without any supervised signal, demonstrating the great potential of zero-shot rationalization for better interpretability.
The Role of Reentrancies in Abstract Meaning Representation Parsing
Authors: Ida Szubert, Marco Damonte, Shay B. Cohen, Mark Steedman
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Abstract Meaning Representation, AMR Parsing

Abstract Meaning Representation (AMR) parsing aims at converting sentences into AMR representations. These are graphs and not trees because AMR supports reentrancies (nodes with more than one parent). Following previous findings on the importance of reen- trancies for AMR, we empirically find and discuss several linguistic phenomena respon- sible for reentrancies in AMR, some of which have not received attention before. We cate- gorize the types of errors AMR parsers make with respect to reentrancies. Furthermore, we find that correcting these errors provides an in- crease of up to 5{\%} Smatch in parsing perfor- mance and 20{\%} in reentrancy prediction
Cross-Lingual Suicidal-Oriented Word Embedding toward Suicide Prevention
Authors: Daeun Lee, Soyoung Park, Jiwon Kang, Daejin Choi, Jinyoung Han
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Word Embeddings

Early intervention for suicide risks with social media data has increasingly received great attention. Using a suicide dictionary created by mental health experts is one of the effective ways to detect suicidal ideation. However, little attention has been paid to validate whether and how the existing dictionaries for other languages (i.e., English and Chinese) can be used for predicting suicidal ideation for a low-resource language (i.e., Korean) where a knowledge-based suicide dictionary has not yet been developed. To this end, we propose a cross-lingual suicidal ideation detection model that can identify whether a given social media post includes suicidal ideation or not. To utilize the existing suicide dictionaries developed for other languages (i.e., English and Chinese) in word embedding, our model translates a post written in the target language (i.e., Korean) into English and Chinese, and then uses the separate suicidal-oriented word embeddings developed for English and Chinese, respectively. By applying an ensemble approach for different languages, the model achieves high accuracy, over 87{\%}. We believe our model is useful in accessing suicidal ideation using social media data for preventing potential suicide risk in an early stage.
Service-oriented Text-to-SQL Parsing
Authors: Wangsu Hu, Jilei Tian
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Information Retrieval, Natural Language Inference, Retrieval, SQL Parsing, Text Normalization, Text to SQL, Text-To-SQL

The information retrieval from relational database requires professionals who has an understanding of structural query language such as SQL. TEXT2SQL models apply natural language inference to enable user interacting the database via natural language utterance. Current TEXT2SQL models normally focus on generating complex SQL query in a precise and complete fashion while certain features of real-world application in the production environment is not fully addressed. This paper is aimed to develop a service-oriented Text-to-SQL parser that translates natural language utterance to structural and executable SQL query. We introduce a algorithmic framework named Semantic-Enriched SQL generator (SE-SQL) that enables flexibly access database than rigid API in the application while keeping the performance quality for the most commonly used cases. The qualitative result shows that the proposed model achieves 88.3{\%} execution accuracy on WikiSQL task, outperforming baseline by 13{\%} error reduction. Moreover, the framework considers several service-oriented needs including low-complexity inference, out-of-table rejection, and text normalization.
Reinforcement Learning with Imbalanced Dataset for Data-to-Text Medical Report Generation
Authors: Toru Nishino, Ryota Ozaki, Yohei Momoki, Tomoki Taniguchi, Ryuji Kano, Norihisa Nakano, Yuki Tagawa, Motoki Taniguchi, Tomoko Ohkuma, Keigo Nakamura
Date: 2020-11-01 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Data Augmentation, Medical Report Generation, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Automated generation of medical reports that describe the findings in the medical images helps radiologists by alleviating their workload. Medical report generation system should generate correct and concise reports. However, data imbalance makes it difficult to train models accurately. Medical datasets are commonly imbalanced in their finding labels because incidence rates differ among diseases; moreover, the ratios of abnormalities to normalities are significantly imbalanced. We propose a novel reinforcement learning method with a reconstructor to improve the clinical correctness of generated reports to train the data-to-text module with a highly imbalanced dataset. Moreover, we introduce a novel data augmentation strategy for reinforcement learning to additionally train the model on infrequent findings. From the perspective of a practical use, we employ a Two-Stage Medical Report Generator (TS-MRGen) for controllable report generation from input images. TS-MRGen consists of two separated stages: an image diagnosis module and a data-to-text module. Radiologists can modify the image diagnosis module results to control the reports that the data-to-text module generates. We conduct an experiment with two medical datasets to assess the data-to-text module and the entire two-stage model. Results demonstrate that the reports generated by our model describe the findings in the input image more correctly.