pwc-archive/papers-with-abstracts huggingface dataset:
An Empirical Investigation of Bias in the Multimodal Analysis of Financial Earnings Calls
Authors: Ramit Sawhney, Arshiya Aggarwal, Rajiv Ratn Shah
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4

Volatility prediction is complex due to the stock market{'}s stochastic nature. Existing research focuses on the textual elements of financial disclosures like earnings calls transcripts to forecast stock volatility and risk, but ignores the rich acoustic features in the company executives{'} speech. Recently, new multimodal approaches that leverage the verbal and vocal cues of speakers in financial disclosures significantly outperform previous state-of-the-art approaches demonstrating the benefits of multimodality and speech. However, the financial realm is still plagued with a severe underrepresentation of various communities spanning diverse demographics, gender, and native speech. While multimodal models are better risk forecasters, it is imperative to also investigate the potential bias that these models may learn from the speech signals of company executives. In this work, we present the first study to discover the gender bias in multimodal volatility prediction due to gender-sensitive audio features and fewer female executives in earnings calls of one of the world{'}s biggest stock indexes, the S{\&}P 500 index. We quantitatively analyze bias as error disparity and investigate the sources of this bias. Our results suggest that multimodal neural financial models accentuate gender-based stereotypes.
Privacy Regularization: Joint Privacy-Utility Optimization in LanguageModels
Authors: FatemehSadat Mireshghallah, Huseyin Inan, Marcello Hasegawa, Victor R{\"u}hle, Taylor Berg-Kirkpatrick, Robert Sim
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Memorization, Privacy Preserving, Triplet

Neural language models are known to have a high capacity for memorization of training samples. This may have serious privacy im- plications when training models on user content such as email correspondence. Differential privacy (DP), a popular choice to train models with privacy guarantees, comes with significant costs in terms of utility degradation and disparate impact on subgroups of users. In this work, we introduce two privacy-preserving regularization methods for training language models that enable joint optimization of utility and privacy through (1) the use of a discriminator and (2) the inclusion of a novel triplet-loss term. We compare our methods with DP through extensive evaluation. We show the advantages of our regularizers with favorable utility-privacy trade-off, faster training with the ability to tap into existing optimization approaches, and ensuring uniform treatment of under-represented subgroups.
Topic Model or Topic Twaddle? Re-evaluating Semantic Interpretability Measures
Authors: Caitlin Doogan, Wray Buntine
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Topic Models

When developing topic models, a critical question that should be asked is: How well will this model work in an applied setting? Because standard performance evaluation of topic interpretability uses automated measures modeled on human evaluation tests that are dissimilar to applied usage, these models{'} generalizability remains in question. In this paper, we probe the issue of validity in topic model evaluation and assess how informative coherence measures are for specialized collections used in an applied setting. Informed by the literature, we propose four understandings of interpretability. We evaluate these using a novel experimental framework reflective of varied applied settings, including human evaluations using open labeling, typical of applied research. These evaluations show that for some specialized collections, standard coherence measures may not inform the most appropriate topic model or the optimal number of topics, and current interpretability performance validation methods are challenged as a means to confirm model quality in the absence of ground truth data.
tWT--WT: A Dataset to Assert the Role of Target Entities for Detecting Stance of Tweets
Authors: Ayush Kaushal, Avirup Saha, Niloy Ganguly
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Sentence, Stance Detection

The stance detection task aims at detecting the stance of a tweet or a text for a target. These targets can be named entities or free-form sentences (claims). Though the task involves reasoning of the tweet with respect to a target, we find that it is possible to achieve high accuracy on several publicly available Twitter stance detection datasets without looking at the target sentence. Specifically, a simple tweet classification model achieved human-level performance on the WT{--}WT dataset and more than two-third accuracy on various other datasets. We investigate the existence of biases in such datasets to find the potential spurious correlations of sentiment-stance relations and lexical choice associated with the stance category. Furthermore, we propose a new large dataset free of such biases and demonstrate its aptness on the existing stance detection systems. Our empirical findings show much scope for research on the stance detection task and proposes several considerations for creating future stance detection datasets.
Almost Free Semantic Draft for Neural Machine Translation
Authors: Xi Ai, Bin Fang
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Decoder, Machine Translation, Sentence, Translation

Translation quality can be improved by global information from the required target sentence because the decoder can understand both past and future information. However, the model needs additional cost to produce and consider such global information. In this work, to inject global information but also save cost, we present an efficient method to sample and consider a semantic draft as global information from semantic space for decoding with almost free of cost. Unlike other successful adaptations, we do not have to perform an EM-like process that repeatedly samples a possible semantic from the semantic space. Empirical experiments show that the presented method can achieve competitive performance in common language pairs with a clear advantage in inference efficiency. We will open all our source code on GitHub.
Multi-Hop Transformer for Document-Level Machine Translation
Authors: Long Zhang, Tong Zhang, Haibo Zhang, Baosong Yang, Wei Ye, Shikun Zhang
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Document Level Machine Translation, Document Translation, Machine Translation, NMT, Sentence, Translation

Document-level neural machine translation (NMT) has proven to be of profound value for its effectiveness on capturing contextual information. Nevertheless, existing approaches 1) simply introduce the representations of context sentences without explicitly characterizing the inter-sentence reasoning process; and 2) feed ground-truth target contexts as extra inputs at the training time, thus facing the problem of exposure bias. We approach these problems with an inspiration from human behavior {--} human translators ordinarily emerge a translation draft in their mind and progressively revise it according to the reasoning in discourse. To this end, we propose a novel Multi-Hop Transformer (MHT) which offers NMT abilities to explicitly model the human-like draft-editing and reasoning process. Specifically, our model serves the sentence-level translation as a draft and properly refines its representations by attending to multiple antecedent sentences iteratively. Experiments on four widely used document translation tasks demonstrate that our method can significantly improve document-level translation performance and can tackle discourse phenomena, such as coreference error and the problem of polysemy.
Continual Learning for Neural Machine Translation
Authors: Yue Cao, Hao-Ran Wei, Boxing Chen, Xiaojun Wan
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Continual Learning, Knowledge Distillation, Machine Translation, NMT, Translation

Neural machine translation (NMT) models are data-driven and require large-scale training corpus. In practical applications, NMT models are usually trained on a general domain corpus and then fine-tuned by continuing training on the in-domain corpus. However, this bears the risk of catastrophic forgetting that the performance on the general domain is decreased drastically. In this work, we propose a new continual learning framework for NMT models. We consider a scenario where the training is comprised of multiple stages and propose a dynamic knowledge distillation technique to alleviate the problem of catastrophic forgetting systematically. We also find that the bias exists in the output linear projection when fine-tuning on the in-domain corpus, and propose a bias-correction module to eliminate the bias. We conduct experiments on three representative settings of NMT application. Experimental results show that the proposed method achieves superior performance compared to baseline models in all settings.
Smart-Start Decoding for Neural Machine Translation
Authors: Jian Yang, Shuming Ma, Dongdong Zhang, Juncheng Wan, Zhoujun Li, Ming Zhou
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Machine Translation, Translation

Most current neural machine translation models adopt a monotonic decoding order of either left-to-right or right-to-left. In this work, we propose a novel method that breaks up the limitation of these decoding orders, called Smart-Start decoding. More specifically, our method first predicts a median word. It starts to decode the words on the right side of the median word and then generates words on the left. We evaluate the proposed Smart-Start decoding method on three datasets. Experimental results show that the proposed method can significantly outperform strong baseline models.
Quantitative Day Trading from Natural Language using Reinforcement Learning
Authors: Ramit Sawhney, Arnav Wadhwa, Shivam Agarwal, Rajiv Ratn Shah
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Stock Prediction

It is challenging to design profitable and practical trading strategies, as stock price movements are highly stochastic, and the market is heavily influenced by chaotic data across sources like news and social media. Existing NLP approaches largely treat stock prediction as a classification or regression problem and are not optimized to make profitable investment decisions. Further, they do not model the temporal dynamics of large volumes of diversely influential text to which the market responds quickly. Building on these shortcomings, we propose a deep reinforcement learning approach that makes time-aware decisions to trade stocks while optimizing profit using textual data. Our method outperforms state-of-the-art in terms of risk-adjusted returns in trading simulations on two benchmarks: Tweets (English) and financial news (Chinese) pertaining to two major indexes and four global stock markets. Through extensive experiments and studies, we build the case for our method as a tool for quantitative trading.
How low is too low? A monolingual take on lemmatisation in Indian languages
Authors: Kumar Saunack, Kumar Saurav, Pushpak Bhattacharyya
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Data Augmentation, Lemmatization, TAG

Lemmatization aims to reduce the sparse data problem by relating the inflected forms of a word to its dictionary form. Most prior work on ML based lemmatization has focused on high resource languages, where data sets (word forms) are readily available. For languages which have no linguistic work available, especially on morphology or in languages where the computational realization of linguistic rules is complex and cumbersome, machine learning based lemmatizers are the way togo. In this paper, we devote our attention to lemmatisation for low resource, morphologically rich scheduled Indian languages using neural methods. Here, low resource means only a small number of word forms are available. We perform tests to analyse the variance in monolingual models{'} performance on varying the corpus size and contextual morphological tag data for training. We show that monolingual approaches with data augmentation can give competitive accuracy even in the low resource setting, which augurs well for NLP in low resource setting.
Inductive Topic Variational Graph Auto-Encoder for Text Classification
Authors: Qianqian Xie, Jimin Huang, Pan Du, Min Peng, Jian-Yun Nie
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Classification, Inductive Learning, Representation Learning, Semi-Supervised Text Classification, text-classification, Text Classification

Graph convolutional networks (GCNs) have been applied recently to text classification and produced an excellent performance. However, existing GCN-based methods do not assume an explicit latent semantic structure of documents, making learned representations less effective and difficult to interpret. They are also transductive in nature, thus cannot handle out-of-graph documents. To address these issues, we propose a novel model named inductive Topic Variational Graph Auto-Encoder (T-VGAE), which incorporates a topic model into variational graph-auto-encoder (VGAE) to capture the hidden semantic information between documents and words. T-VGAE inherits the interpretability of the topic model and the efficient information propagation mechanism of VGAE. It learns probabilistic representations of words and documents by jointly encoding and reconstructing the global word-level graph and bipartite graphs of documents, where each document is considered individually and decoupled from the global correlation graph so as to enable inductive learning. Our experiments on several benchmark datasets show that our method outperforms the existing competitive models on supervised and semi-supervised text classification, as well as unsupervised text representation learning. In addition, it has higher interpretability and is able to deal with unseen documents.
TaxoClass: Hierarchical Multi-Label Text Classification Using Only Class Names
Authors: Jiaming Shen, Wenda Qiu, Yu Meng, Jingbo Shang, Xiang Ren, Jiawei Han
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Classification, Multi Label Text Classification, Multi-Label Text Classification, Natural Language Inference, TAG, text-classification, Text Classification

Hierarchical multi-label text classification (HMTC) aims to tag each document with a set of classes from a taxonomic class hierarchy. Most existing HMTC methods train classifiers using massive human-labeled documents, which are often too costly to obtain in real-world applications. In this paper, we explore to conduct HMTC based on only class surface names as supervision signals. We observe that to perform HMTC, human experts typically first pinpoint a few most essential classes for the document as its {``}core classes{''}, and then check core classes{'} ancestor classes to ensure the coverage. To mimic human experts, we propose a novel HMTC framework, named TaxoClass. Specifically, TaxoClass (1) calculates document-class similarities using a textual entailment model, (2) identifies a document{'}s core classes and utilizes confident core classes to train a taxonomy-enhanced classifier, and (3) generalizes the classifier via multi-label self-training. Our experiments on two challenging datasets show TaxoClass can achieve around 0.71 Example-F1 using only class names, outperforming the best previous method by 25{\%}.
``I'm Not Mad'': Commonsense Implications of Negation and Contradiction
Authors: Liwei Jiang, Antoine Bosselut, Chandra Bhagavatula, Yejin Choi
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Natural Language Inference, Negation

Natural language inference requires reasoning about contradictions, negations, and their commonsense implications. Given a simple premise (e.g., {``}I{'}m mad at you{''}), humans can reason about the varying shades of contradictory statements ranging from straightforward negations ({``}I{'}m not mad at you{''}) to commonsense contradictions ({``}I{'}m happy{''}). Moreover, these negated or contradictory statements shift the commonsense implications of the original premise in interesting and nontrivial ways. For example, while {``}I{'}m mad{''} implies {``}I{'}m unhappy about something,{''} negating the premise does not necessarily negate the corresponding commonsense implications. In this paper, we present the first comprehensive study focusing on commonsense implications of negated statements and contradictions. We introduce ANION, a new commonsense knowledge graph with 624K if-then rules focusing on negated and contradictory events. We then present joint generative and discriminative inference models for this new resource, providing novel empirical insights on how logical negations and commonsense contradictions reshape the commonsense implications of their original premises.
Identifying Medical Self-Disclosure in Online Communities
Authors: Mina Valizadeh, Pardis Ranjbar-Noiey, Cornelia Caragea, Natalie Parde
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4

Self-disclosure in online health conversations may offer a host of benefits, including earlier detection and treatment of medical issues that may have otherwise gone unaddressed. However, research analyzing medical self-disclosure in online communities is limited. We address this shortcoming by introducing a new dataset of health-related posts collected from online social platforms, categorized into three groups (No Self-Disclosure, Possible Self-Disclosure, and Clear Self-Disclosure) with high inter-annotator agreement ({\_}k{\_}=0.88). We make this data available to the research community. We also release a predictive model trained on this dataset that achieves an accuracy of 81.02{\%}, establishing a strong performance benchmark for this task.
Word Complexity is in the Eye of the Beholder
Authors: Sian Gooding, Ekaterina Kochmar, Seid Muhie Yimam, Chris Biemann
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Lexical Simplification

Lexical complexity is a highly subjective notion, yet this factor is often neglected in lexical simplification and readability systems which use a {''}one-size-fits-all{''} approach. In this paper, we investigate which aspects contribute to the notion of lexical complexity in various groups of readers, focusing on native and non-native speakers of English, and how the notion of complexity changes depending on the proficiency level of a non-native reader. To facilitate reproducibility of our approach and foster further research into these aspects, we release a dataset of complex words annotated by readers with different backgrounds.
Linguistic Complexity Loss in Text-Based Therapy
Authors: Jason Wei, Kelly Finn, Emma Templeton, Thalia Wheatley, Soroush Vosoughi
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Diversity

The complexity loss paradox, which posits that individuals suffering from disease exhibit surprisingly predictable behavioral dynamics, has been observed in a variety of both human and animal physiological systems. The recent advent of online text-based therapy presents a new opportunity to analyze the complexity loss paradox in a novel operationalization: linguistic complexity loss in text-based therapy conversations. In this paper, we analyze linguistic complexity correlates of mental health in the online therapy messages sent between therapists and 7,170 clients who provided 30,437 corresponding survey responses on their anxiety. We found that when clients reported more anxiety, they showed reduced lexical diversity as estimated by the moving average type-token ratio. Therapists, on the other hand, used language of higher reading difficulty, syntactic complexity, and age of acquisition when clients were more anxious. Finally, we found that clients, and to an even greater extent, therapists, exhibited consistent levels of many linguistic complexity measures. These results demonstrate how linguistic analysis of text-based communication can be leveraged as a marker for anxiety, an exciting prospect in a time of both increased online communication and increased mental health issues.
Constrained Multi-Task Learning for Event Coreference Resolution
Authors: Jing Lu, Vincent Ng
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: coreference-resolution, Coreference Resolution, Event Coreference Resolution, Multi-Task Learning

We propose a neural event coreference model in which event coreference is jointly trained with five tasks: trigger detection, entity coreference, anaphoricity determination, realis detection, and argument extraction. To guide the learning of this complex model, we incorporate cross-task consistency constraints into the learning process as soft constraints via designing penalty functions. In addition, we propose the novel idea of viewing entity coreference and event coreference as a single coreference task, which we believe is a step towards a unified model of coreference resolution. The resulting model achieves state-of-the-art results on the KBP 2017 event coreference dataset.
Data and Model Distillation as a Solution for Domain-transferable Fact Verification
Authors: Mitch Paul Mithun, Sandeep Suntwal, Mihai Surdeanu
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Fact Verification

While neural networks produce state-of-the-art performance in several NLP tasks, they generally depend heavily on lexicalized information, which transfer poorly between domains. We present a combination of two strategies to mitigate this dependence on lexicalized information in fact verification tasks. We present a data distillation technique for delexicalization, which we then combine with a model distillation method to prevent aggressive data distillation. We show that by using our solution, not only does the performance of an existing state-of-the-art model remain at par with that of the model trained on a fully lexicalized data, but it also performs better than it when tested out of domain. We show that the technique we present encourages models to extract transferable facts from a given fact verification dataset.
If You Want to Go Far Go Together: Unsupervised Joint Candidate Evidence Retrieval for Multi-hop Question Answering
Authors: Vikas Yadav, Steven Bethard, Mihai Surdeanu
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Answer Selection, Multi-hop Question Answering, Question Answering, Reranking, Retrieval

Multi-hop reasoning requires aggregation and inference from multiple facts. To retrieve such facts, we propose a simple approach that retrieves and reranks set of evidence facts jointly. Our approach first generates unsupervised clusters of sentences as candidate evidence by accounting links between sentences and coverage with the given query. Then, a RoBERTa-based reranker is trained to bring the most representative evidence cluster to the top. We specifically emphasize on the importance of retrieving evidence jointly by showing several comparative analyses to other methods that retrieve and rerank evidence sentences individually. First, we introduce several attention- and embedding-based analyses, which indicate that jointly retrieving and reranking approaches can learn compositional knowledge required for multi-hop reasoning. Second, our experiments show that jointly retrieving candidate evidence leads to substantially higher evidence retrieval performance when fed to the same supervised reranker. In particular, our joint retrieval and then reranking approach achieves new state-of-the-art evidence retrieval performance on two multi-hop question answering (QA) datasets: 30.5 Recall@2 on QASC, and 67.6{\%} F1 on MultiRC. When the evidence text from our joint retrieval approach is fed to a RoBERTa-based answer selection classifier, we achieve new state-of-the-art QA performance on MultiRC and second best result on QASC.