pwc-archive/papers-with-abstracts huggingface dataset:
When is Char Better Than Subword: A Systematic Study of Segmentation Algorithms for Neural Machine Translation
Authors: Jiahuan Li, Yutong Shen, ShuJian Huang, Xinyu Dai, Jiajun Chen
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Machine Translation, NMT, Segmentation, Translation

Subword segmentation algorithms have been a \textit{de facto} choice when building neural machine translation systems. However, most of them need to learn a segmentation model based on some heuristics, which may produce sub-optimal segmentation. This can be problematic in some scenarios when the target language has rich morphological changes or there is not enough data for learning compact composition rules. Translating at fully character level has the potential to alleviate the issue, but empirical performances of character-based models has not been fully explored. In this paper, we present an in-depth comparison between character-based and subword-based NMT systems under three settings: translating to typologically diverse languages, training with low resource, and adapting to unseen domains. Experiment results show strong competitiveness of character-based models. Further analyses show that compared to subword-based models, character-based models are better at handling morphological phenomena, generating rare and unknown words, and more suitable for transferring to unseen domains.
More than Text: Multi-modal Chinese Word Segmentation
Authors: Dong Zhang, Zheng Hu, Shoushan Li, Hanqian Wu, Qiaoming Zhu, Guodong Zhou
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Chinese Word Segmentation

Chinese word segmentation (CWS) is undoubtedly an important basic task in natural language processing. Previous works only focus on the textual modality, but there are often audio and video utterances (such as news broadcast and face-to-face dialogues), where textual, acoustic and visual modalities normally exist. To this end, we attempt to combine the multi-modality (mainly the converted text and actual voice information) to perform CWS. In this paper, we annotate a new dataset for CWS containing text and audio. Moreover, we propose a time-dependent multi-modal interactive model based on Transformer framework to integrate multi-modal information for word sequence labeling. The experimental results on three different training sets show the effectiveness of our approach with fusing text and audio.
A Mixture-of-Experts Model for Antonym-Synonym Discrimination
Authors: Zhipeng Xie, Nan Zeng
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Mixture-of-Experts

Discrimination between antonyms and synonyms is an important and challenging NLP task. Antonyms and synonyms often share the same or similar contexts and thus are hard to make a distinction. This paper proposes two underlying hypotheses and employs the mixture-of-experts framework as a solution. It works on the basis of a divide-and-conquer strategy, where a number of localized experts focus on their own domains (or subspaces) to learn their specialties, and a gating mechanism determines the space partitioning and the expert mixture. Experimental results have shown that our method achieves the state-of-the-art performance on the task.
Beyond Laurel/Yanny: An Autoencoder-Enabled Search for Polyperceivable Audio
Authors: Kartik Chandra, Chuma Kabaghe, Gregory Valiant
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5

The famous {``}laurel/yanny{''} phenomenon references an audio clip that elicits dramatically different responses from different listeners. For the original clip, roughly half the population hears the word {``}laurel,{''} while the other half hears {``}yanny.{''} How common are such {``}polyperceivable{''} audio clips? In this paper we apply ML techniques to study the prevalence of polyperceivability in spoken language. We devise a metric that correlates with polyperceivability of audio clips, use it to efficiently find new {``}laurel/yanny{''}-type examples, and validate these results with human experiments. Our results suggest that polyperceivable examples are surprisingly prevalent in natural language, existing for {\textgreater}2{\%} of English words.
Don't Let Discourse Confine Your Model: Sequence Perturbations for Improved Event Language Models
Authors: Mahnaz Koupaee, Greg Durrett, Nathanael Chambers, Niranjan Balasubramanian
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5

Event language models represent plausible sequences of events. Most existing approaches train autoregressive models on text, which successfully capture event co-occurrence but unfortunately constrain the model to follow the discourse order in which events are presented. Other domains may employ different discourse orders, and for many applications, we may care about different notions of ordering (e.g., temporal) or not care about ordering at all (e.g., when predicting related events in a schema). We propose a simple yet surprisingly effective strategy for improving event language models by perturbing event sequences so we can relax model dependence on text order. Despite generating completely synthetic event orderings, we show that this technique improves the performance of the event language models on both applications and out-of-domain events data.
Embedding Time Differences in Context-sensitive Neural Networks for Learning Time to Event
Authors: Nazanin Dehghani, Hassan Hajipoor, Hadi Amiri
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Multi-Task Learning

We propose an effective context-sensitive neural model for time to event (TTE) prediction task, which aims to predict the amount of time to/from the occurrence of given events in streaming content. We investigate this problem in the context of a multi-task learning framework, which we enrich with time difference embeddings. In addition, we develop a multi-genre dataset of English events about soccer competitions and academy awards ceremonies, and their relevant tweets obtained from Twitter. Our model is 1.4 and 3.3 hours more accurate than the current state-of-the-art model in estimating TTE on English and Dutch tweets respectively. We examine different aspects of our model to illustrate its source of improvement.
QA-Driven Zero-shot Slot Filling with Weak Supervision Pretraining
Authors: Xinya Du, Luheng He, Qi Li, Dian Yu, Panupong Pasupat, Yuan Zhang
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: slot-filling, Slot Filling, Zero-shot Slot Filling

Slot-filling is an essential component for building task-oriented dialog systems. In this work, we focus on the zero-shot slot-filling problem, where the model needs to predict slots and their values, given utterances from new domains without training on the target domain. Prior methods directly encode slot descriptions to generalize to unseen slot types. However, raw slot descriptions are often ambiguous and do not encode enough semantic information, limiting the models{'} zero-shot capability. To address this problem, we introduce QA-driven slot filling (QASF), which extracts slot-filler spans from utterances with a span-based QA model. We use a linguistically motivated questioning strategy to turn descriptions into questions, allowing the model to generalize to unseen slot types. Moreover, our QASF model can benefit from weak supervision signals from QA pairs synthetically generated from unlabeled conversations. Our full system substantially outperforms baselines by over 5{\%} on the SNIPS benchmark.
nmT5 - Is parallel data still relevant for pre-training massively multilingual language models?
Authors: Mihir Kale, Aditya Siddhant, Rami Al-Rfou, Linting Xue, Noah Constant, Melvin Johnson
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Language Modeling, Language Modelling, Machine Translation, Multilingual NLP, Translation

Recently, mT5 - a massively multilingual version of T5 - leveraged a unified text-to-text format to attain state-of-the-art results on a wide variety of multilingual NLP tasks. In this paper, we investigate the impact of incorporating parallel data into mT5 pre-training. We find that multi-tasking language modeling with objectives such as machine translation during pre-training is a straightforward way to improve performance on downstream multilingual and cross-lingual tasks. However, the gains start to diminish as the model capacity increases, suggesting that parallel data might not be as essential for larger models. At the same time, even at larger model sizes, we find that pre-training with parallel data still provides benefits in the limited labelled data regime
Towards Visual Question Answering on Pathology Images
Authors: Xuehai He, Zhuo Cai, Wenlan Wei, Yichen Zhang, Luntian Mou, Eric Xing, Pengtao Xie
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Decision Making, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Pathology imaging is broadly used for identifying the causes and effects of diseases or injuries. Given a pathology image, being able to answer questions about the clinical findings contained in the image is very important for medical decision making. In this paper, we aim to develop a pathological visual question answering framework to analyze pathology images and answer medical questions related to these images. To build such a framework, we create PathVQA, a VQA dataset with 32,795 questions asked from 4,998 pathology images. We also propose a three-level optimization framework which performs self-supervised pretraining and VQA finetuning end-to-end to learn powerful visual and textual representations jointly and automatically identifies and excludes noisy self-supervised examples from pretraining. We perform experiments on our created PathVQA dataset and the results demonstrate the effectiveness of our proposed methods. The datasets and code are available at https://github.com/UCSD-AI4H/PathVQA
Efficient Text-based Reinforcement Learning by Jointly Leveraging State and Commonsense Graph Representations
Authors: Keerthiram Murugesan, Mattia Atzeni, Pavan Kapanipathi, Kartik Talamadupula, Mrinmaya Sachan, Murray Campbell
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Graph Attention, Reinforcement Learning (RL), text-based games

Text-based games (TBGs) have emerged as useful benchmarks for evaluating progress at the intersection of grounded language understanding and reinforcement learning (RL). Recent work has proposed the use of external knowledge to improve the efficiency of RL agents for TBGs. In this paper, we posit that to act efficiently in TBGs, an agent must be able to track the state of the game while retrieving and using relevant commonsense knowledge. Thus, we propose an agent for TBGs that induces a graph representation of the game state and jointly grounds it with a graph of commonsense knowledge from ConceptNet. This combination is achieved through bidirectional knowledge graph attention between the two symbolic representations. We show that agents that incorporate commonsense into the game state graph outperform baseline agents.
Explicitly Capturing Relations between Entity Mentions via Graph Neural Networks for Domain-specific Named Entity Recognition
Authors: Pei Chen, Haibo Ding, Jun Araki, Ruihong Huang
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Named entity recognition (NER) is well studied for the general domain, and recent systems have achieved human-level performance for identifying common entity types. However, the NER performance is still moderate for specialized domains that tend to feature complicated contexts and jargonistic entity types. To address these challenges, we propose explicitly connecting entity mentions based on both global coreference relations and local dependency relations for building better entity mention representations. In our experiments, we incorporate entity mention relations by Graph Neural Networks and show that our system noticeably improves the NER performance on two datasets from different domains. We further show that the proposed lightweight system can effectively elevate the NER performance to a higher level even when only a tiny amount of labeled data is available, which is desirable for domain-specific NER.
Input Representations for Parsing Discourse Representation Structures: Comparing English with Chinese
Authors: Chunliu Wang, Rik van Noord, Arianna Bisazza, Johan Bos
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5

Neural semantic parsers have obtained acceptable results in the context of parsing DRSs (Discourse Representation Structures). In particular models with character sequences as input showed remarkable performance for English. But how does this approach perform on languages with a different writing system, like Chinese, a language with a large vocabulary of characters? Does rule-based tokenisation of the input help, and which granularity is preferred: characters, or words? The results are promising. Even with DRSs based on English, good results for Chinese are obtained. Tokenisation offers a small advantage for English, but not for Chinese. Overall, characters are preferred as input, both for English and Chinese.
Code Generation from Natural Language with Less Prior Knowledge and More Monolingual Data
Authors: Sajad Norouzi, Keyi Tang, Yanshuai Cao
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Code Generation, Inductive Bias, Semantic Parsing

Training datasets for semantic parsing are typically small due to the higher expertise required for annotation than most other NLP tasks. As a result, models for this application usually need additional prior knowledge to be built into the architecture or algorithm. The increased dependency on human experts hinders automation and raises the development and maintenance costs in practice. This work investigates whether a generic transformer-based seq2seq model can achieve competitive performance with minimal code-generation-specific inductive bias design. By exploiting a relatively sizeable monolingual corpus of the target programming language, which is cheap to mine from the web, we achieved 81.03{\%} exact match accuracy on Django and 32.57 BLEU score on CoNaLa. Both are SOTA to the best of our knowledge. This positive evidence highlights a potentially easier path toward building accurate semantic parsers in practice.
Issues with Entailment-based Zero-shot Text Classification
Authors: Tingting Ma, Jin-Ge Yao, Chin-Yew Lin, Tiejun Zhao
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Classification, Natural Language Inference, Sentence, text-classification, Text Classification, Zero-Shot Text Classification

The general format of natural language inference (NLI) makes it tempting to be used for zero-shot text classification by casting any target label into a sentence of hypothesis and verifying whether or not it could be entailed by the input, aiming at generic classification applicable on any specified label space. In this opinion piece, we point out a few overlooked issues that are yet to be discussed in this line of work. We observe huge variance across different classification datasets amongst standard BERT-based NLI models and surprisingly find that pre-trained BERT without any fine-tuning can yield competitive performance against BERT fine-tuned for NLI. With the concern that these models heavily rely on spurious lexical patterns for prediction, we also experiment with preliminary approaches for more robust NLI, but the results are in general negative. Our observations reveal implicit but challenging difficulties in entailment-based zero-shot text classification.
What Motivates You? Benchmarking Automatic Detection of Basic Needs from Short Posts
Authors: Sanja Stajner, Seren Yenikent, Bilal Ghanem, Marc Franco-Salvador
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Benchmarking, Binary Classification, Classification

According to the self-determination theory, the levels of satisfaction of three basic needs (competence, autonomy and relatedness) have implications on people{'}s everyday life and career. We benchmark the novel task of automatically detecting those needs on short posts in English, by modelling it as a ternary classification task, and as three binary classification tasks. A detailed manual analysis shows that the latter has advantages in the real-world scenario, and that our best models achieve similar performances as a trained human annotator.
Distinct Label Representations for Few-Shot Text Classification
Authors: Sora Ohashi, Junya Takayama, Tomoyuki Kajiwara, Yuki Arase
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Classification, Few-Shot Text Classification, text-classification, Text Classification

Few-shot text classification aims to classify inputs whose label has only a few examples. Previous studies overlooked the semantic relevance between label representations. Therefore, they are easily confused by labels that are relevant. To address this problem, we propose a method that generates distinct label representations that embed information specific to each label. Our method is applicable to conventional few-shot classification models. Experimental results show that our method significantly improved the performance of few-shot text classification across models and datasets.
Learning to Solve NLP Tasks in an Incremental Number of Languages
Authors: Giuseppe Castellucci, Simone Filice, Danilo Croce, Roberto Basili
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Continual Learning, Relational Reasoning, Sentence, Sentence Classification

In real scenarios, a multilingual model trained to solve NLP tasks on a set of languages can be required to support new languages over time. Unfortunately, the straightforward retraining on a dataset containing annotated examples for all the languages is both expensive and time-consuming, especially when the number of target languages grows. Moreover, the original annotated material may no longer be available due to storage or business constraints. Re-training only with the new language data will inevitably result in Catastrophic Forgetting of previously acquired knowledge. We propose a Continual Learning strategy that updates a model to support new languages over time, while maintaining consistent results on previously learned languages. We define a Teacher-Student framework where the existing model {``}teaches{''} to a student model its knowledge about the languages it supports, while the student is also trained on a new language. We report an experimental evaluation in several tasks including Sentence Classification, Relational Learning and Sequence Labeling.
Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialogue State Tracking
Authors: Yinpei Dai, Hangyu Li, Yongbin Li, Jian Sun, Fei Huang, Luo Si, Xiaodan Zhu
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: dialog state tracking, Dialogue State Tracking, Multi-domain Dialogue State Tracking

Existing dialog state tracking (DST) models are trained with dialog data in a random order, neglecting rich structural information in a dataset. In this paper, we propose to use curriculum learning (CL) to better leverage both the curriculum structure and schema structure for task-oriented dialogs. Specifically, we propose a model-agnostic framework called Schema-aware Curriculum Learning for Dialog State Tracking (SaCLog), which consists of a preview module that pre-trains a DST model with schema information, a curriculum module that optimizes the model with CL, and a review module that augments mispredicted data to reinforce the CL training. We show that our proposed approach improves DST performance over both a transformer-based and RNN-based DST model (TripPy and TRADE) and achieves new state-of-the-art results on WOZ2.0 and MultiWOZ2.1.
On the Generation of Medical Dialogs for COVID-19
Authors: Meng Zhou, Zechen Li, Bowen Tan, Guangtao Zeng, Wenmian Yang, Xuehai He, Zeqian Ju, Subrato Chakravorty, Shu Chen, Xingyi Yang, Yichen Zhang, Qingyang Wu, Zhou Yu, Kun Xu, Eric Xing, Pengtao Xie
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Multi-Task Learning

Under the pandemic of COVID-19, people experiencing COVID19-related symptoms have a pressing need to consult doctors. Because of the shortage of medical professionals, many people cannot receive online consultations timely. To address this problem, we aim to develop a medical dialog system that can provide COVID19-related consultations. We collected two dialog datasets {--} CovidDialog {--} (in English and Chinese respectively) containing conversations between doctors and patients about COVID-19. While the largest of their kind, these two datasets are still relatively small compared with general-domain dialog datasets. Training complex dialog generation models on small datasets bears high risk of overfitting. To alleviate overfitting, we develop a multi-task learning approach, which regularizes the data-deficient dialog generation task with a masked token prediction task. Experiments on the CovidDialog datasets demonstrate the effectiveness of our approach. We perform both human evaluation and automatic evaluation of dialogs generated by our method. Results show that the generated responses are promising in being doctor-like, relevant to conversation history, clinically informative and correct. The code and the data are available at https://github.com/UCSD-AI4H/COVID-Dialogue.
Exposing the limits of Zero-shot Cross-lingual Hate Speech Detection
Authors: Debora Nozza
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Cross-Lingual Transfer, Hate Speech Detection, Transfer Learning, Zero-Shot Cross-Lingual Transfer

Reducing and counter-acting hate speech on Social Media is a significant concern. Most of the proposed automatic methods are conducted exclusively on English and very few consistently labeled, non-English resources have been proposed. Learning to detect hate speech on English and transferring to unseen languages seems an immediate solution. This work is the first to shed light on the limits of this zero-shot, cross-lingual transfer learning framework for hate speech detection. We use benchmark data sets in English, Italian, and Spanish to detect hate speech towards immigrants and women. Investigating post-hoc explanations of the model, we discover that non-hateful, language-specific taboo interjections are misinterpreted as signals of hate speech. Our findings demonstrate that zero-shot, cross-lingual models cannot be used as they are, but need to be carefully designed.
Entity Enhancement for Implicit Discourse Relation Classification in the Biomedical Domain
Authors: Wei Shi, Vera Demberg
Date: 2021-08-01 00:00:00
Proceeding: ACL 2021 5
Tasks: Implicit Discourse Relation Classification, Relation, Relation Classification

Implicit discourse relation classification is a challenging task, in particular when the text domain is different from the standard Penn Discourse Treebank (PDTB; Prasad et al., 2008) training corpus domain (Wall Street Journal in 1990s). We here tackle the task of implicit discourse relation classification on the biomedical domain, for which the Biomedical Discourse Relation Bank (BioDRB; Prasad et al., 2011) is available. We show that entity information can be used to improve discourse relational argument representation. In a first step, we show that explicitly marked instances that are content-wise similar to the target relations can be used to achieve good performance in the cross-domain setting using a simple unsupervised voting pipeline. As a further step, we show that with the linked entity information from the first step, a transformer which is augmented with entity-related information (KBERT; Liu et al., 2020) sets the new state of the art performance on the dataset, outperforming the large pre-trained BioBERT (Lee et al., 2020) model by 2{\%} points.