pwc-archive/papers-with-abstracts huggingface dataset:
Connecting the Dots between Audio and Text without Parallel Data through Visual Knowledge Transfer
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Audio Classification, Audio Tagging, Retrieval, Transfer Learning, Zero-shot Audio Classification

Machines that can represent and describe environmental soundscapes have practical potential, e.g., for audio tagging and captioning. Prevailing learning paradigms of audio-text connections have been relying on parallel audio-text data, which is, however, scarcely available on the web. We propose VIP-ANT that induces Audio-Text alignment without using any parallel audio-text data. Our key idea is to share the image modality between bi-modal image-text representations and bi-modal image-audio representations; the image modality functions as a pivot and connects audio and text in a tri-modal embedding space implicitly.In a difficult zero-shot setting with no paired audio-text data, our model demonstrates state-of-the-art zero-shot performance on the ESC50 and US8K audio classification tasks, and even surpasses the supervised state of the art for Clotho caption retrieval (with audio queries) by 2.2% R@1. We further investigate cases of minimal audio-text supervision, finding that, e.g., just a few hundred supervised audio-text pairs increase the zero-shot audio classification accuracy by 8% on US8K. However, to match human parity on some zero-shot tasks, our empirical scaling experiments suggest that we would need about $2^{21} \approx 2\text{M}$ supervised audio-caption pairs. Our work opens up new avenues for learning audio-text connections with little to no parallel audio-text data.
Simplifying Dataflow Dialogue Design
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

In \citep{andreas2020task-oriented}, a dataflow (DF) based dialogue system was introduced, showing clear advantages compared to many commonly used current systems. This was accompanied by the release of SMCalFlow, a practically relevant, manually annotated dataset, more detailed and much larger than any comparable dialogue dataset.Despite these remarkable contributions, the community has not shown further interest in this direction.What are the reasons for this lack of interest? And how can the community be encouraged to engage in research in this direction?One explanation may be the perception that this approach is too complex - both the the annotation and the system. This paper argues that this perception is wrong: 1) Suggestions for a simplified format for the annotation of the dataset are presented, 2) A basic implementation of the DF execution engine is released, which can serve as a sandbox allowing researchers to easily implement, and experiment with, new DF dialogue designs.The hope is that these contributions will help engage more practitioners in exploring new ideas and designs for DF based dialogue systems.
Lex2Sent: A bagging approach to unsupervised sentiment analysis
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Sentiment Analysis

Unsupervised sentiment analysis is traditionally performed by counting those words in a text that are stored in a sentiment lexicon and then assigning a label depending on the proportion of positive and negative words registered. While these "counting" methods are considered to be beneficial as they rate a text deterministically, their accuracy decreases when the analyzed texts are short or the vocabulary differs from what the lexicon considers default. The model proposed in this paper, called Lex2Sent, is an unsupervised sentiment analysis method to improve the classification of sentiment lexicon methods. For this purpose, a Doc2Vec-model is trained to determine the distances between document embeddings and the embeddings of the positive and negative part of a sentiment lexicon. These distances are then evaluated for multiple executions of Doc2Vec on resampled documents and are averaged to perform the classification task. For three benchmark datasets considered in this paper, the proposed Lex2Sent outperforms every evaluated lexicon, including state-of-the-art lexica like VADER or the Opinion Lexicon in terms of accuracy.
Biasly: a machine learning based platform for automatic racial discrimination detection in online texts
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: BIG-bench Machine Learning, Management

Detecting hateful, toxic, and otherwise racist or sexist language in user-generated online contents has become an increasingly important task in recent years. Indeed, the anonymity, transience, size of messages, and the difficulty of management, facilitate the diffusion of racist or hateful messages across the Internet. The critical influence of this cyber-racism is no longer limited to social media, but also has a significant effect on our society : corporate business operation, users' health, crimes, etc. Traditional racist speech reporting channels have proven inadequate due to the enormous explosion of information, so there is an urgent need for a method to automatically and promptly detect texts with racial discrimination. We propose in this work, a machine learning-based approach to enable automatic detection of racist text content over the internet. State-of-the-art machine learning models that are able to grasp language structures are adapted in this study. Our main contribution include 1) a large scale racial discrimination data set collected from three distinct sources and annotated according to a guideline developed by specialists, 2) a set of machine learning models with various architectures for racial discrimination detection, and 3) a web-browser-based software that assist users to debias their texts when using the internet. All these resources are made publicly available.
Magic Pyramid: Accelerating Inference with Early Exiting and Token Pruning
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: text-classification, Text Classification

Pre-training and then fine-tuning large language models is commonly used  to achieve state-of-the-art performance in natural language processing (NLP) tasks. However, most pre-trained models suffer from low inference speed. Deploying such large models to applications with latency constraints is challenging. In this work, we focus on accelerating the inference via conditional computations. To achieve this, we propose a novel idea, Magic Pyramid (MP), to reduce both width-wise and depth-wise computation via token pruning and early exiting for Transformer-based models, particularly BERT. The former manages to save the computation via removing non-salient tokens, while the latter can fulfill the computation reduction by terminating the inference early before reaching the final layer, if the exiting condition is met. Our empirical studies demonstrate that compared to previous state of arts, MP is not only able to achieve a speed-adjustable inference, but also to surpass token pruning and early exiting by reducing up to 70\% giga floating point operations (GFLOPs) with less than 0.5\% accuracy drop. Token pruning and early exiting express distinctive preferences to sequences with different lengths. However, MP is capable of achieving an average of 8.06x speedup on two popular text classification tasks, regardless of the sizes of the inputs.
GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Domain Adaptation, Retrieval, Unsupervised Domain Adaptation

Dense retrieval approaches can overcome the lexical gap and lead to significantly improved search results. However, they require large amounts of training data which is not available for most domains. As shown in previous work (Thakur et al., 2021b), the performance of dense retrievers severely degrades under a domain shift. This limits the usage of dense retrieval approaches to only a few domains with large training datasets.In this paper, we propose the novel unsupervised domain adaptation method Generative Pseudo Labeling (GPL), which combines a query generator with pseudo labeling from a cross-encoder. On six representative domain-specialized datasets, we find the proposed GPL can outperform an out-of-the-box state-of-the-art dense retrieval approach by up to 8.9 points nDCG@10. GPL requires less (unlabeled) data from the target domain and is more robust in its training than previous methods.We further investigate the role of six recent pre-training methods in the scenario of domain adaptation for retrieval tasks, where only three could yield improved results. The best approach, TSDAE (Wang et al., 2021) can be combined with GPL, yielding another average improvement of 1.0 points nDCG@10 across the six tasks.
Extractive Text Summarization with Latent Topics using Heterogeneous Graph Neural Network
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Extractive Text Summarization, Graph Neural Network, Sentence, Text Summarization

This paper presents a heterogeneous graph neural network (HeterGNN) model for extractive text summarization (ETS) by using latent topics to capture the important content of input documents. Specifically, topical information has been widely used as global information for sentence selection. However, most of the recent approaches use neural models, which lead the training models more complex and difficult for extensibility. In this regard, this study presents a novel graph-based ETS by adding a new node of latent topics into HeterGN for the summarization (TopicHeterGraphSum). Specifically, TopicHeterGraphSum includes three types of semantic nodes (i.e., topic-word-sentence) in order to enrich the cross-sentence relations. Furthermore, an extended version of TopicHeterGraphSum for multi documents extraction is also taken into account to emphasize the advantage of the proposed method. Experiments on benchmark datasets such as CNN/DailyMail and Multi-News show the promising results of our method compared with state-of-the-art models.
An Exploitation of Heterogeneous Graph Neural Network for Extractive Long Document Summarization
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Document Summarization, Extractive Document Summarization, Graph Attention, Graph Neural Network, Sentence

Heterogeneous Graph Neural Networks (HeterGNN) has been recently introduced as an emergent approach for many Natural Language Processing (NLP) tasks by enriching the complex information between word and sentence. In this paper, we try to improve the performance of Extractive Document Summarization (EDS) for long-form documents based on the concept of HeterGNN. Specifically, long documents (e.g., Scientific Papers) are truncated for most neural-based models, which leads to the challenge in terms of information loss of inter-sentence relations. In this regard, we present a new method by exploiting the capabilities of HeterGNN and pre-trained language models. Particularly, BERT is considered for improving the sentence information into the Heterogenous graph layer. Accordingly, two versions of the proposed method are presented which are: i) Multi Graph Neural Network (MTGNN-SUM), by combining both heterogeneous graph layer and graph attention layer; and ii) HeterGNN with BERT (HeterGNN-BERT-SUM), by integrating BERT directly into the heterogeneous graph structure. Experiments on two benchmark datasets of long documents such as PubMed and ArXiv show that our method outperforms state-of-the-art models in this research field
Global Entity Disambiguation with BERT
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Entity Disambiguation

We propose a global entity disambiguation (ED) model based on BERT. To capture global contextual information for ED, our model treats not only words but also entities as input tokens, and solves the task by sequentially resolving mentions to their referent entities and using resolved entities as inputs. We train the model using a large entity-annotated corpus obtained from Wikipedia. We achieve new state-of-the-art results on five standard ED datasets: AIDA-CoNLL, MSNBC, AQUAINT, ACE2004, and WNED-WIKI.
Fast and Accurate Span-based Semantic Role Labeling as Graph Parsing
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Semantic Role Labeling, Sentence

Currently, BIO-based and Tuple-based approaches perform quite well on the span-based semantic role labeling (SRL) task. However, the BIO-based approach usually needs to encode a sentence once for each predicate when predicting its arguments, and the Tuple-based approach has to deal with a huge search space of $O(n^3)$, greatly reducing the training and inference efficiency. Moreover, both BIO-based and Tuple-based approaches usually consider only local structural information when making predictions.This paper proposes to cast end-to-end span-based SRL as a graph parsing task. Based on a novel graph representation schema, we present a fast and accurate SRL parser on the shoulder of recent works on high-order semantic dependency graph parsing (SDGP).  Moreover, we propose a constrained Viterbi procedure to ensure the legality of the output graph. Experiments on CoNLL05, CoNLL12, and Chinese Proposition Bank 1.0 (CPB1.0) datasets show that our model achieves new state-of-the-art results and can parse over 600 sentences per second.
How to Fool Systems and Humans in Visually Grounded Interaction: A Case Study on Adversarial Attacks on Visual Dialog
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Visual Dialog

Adversarial attacks change predictions of deep neural network models, while aiming to remain unnoticed by the user.This is a challenge for textual attacks, which  target discrete text. In this study, we investigate the robustness of visually grounded dialog models towards textual attacks to understand how different input components can mitigate the attack. Our results show that dialog history is important for model robustness: models encoding history are more robust, and when launching an attack on history, model prediction becomes more uncertain. This is in contrast to prior work which finds that dialog history is negligible for model performance. We also evaluate how to generate adversarial examples which successfully attack the model but remain undetected by the user. We find that the textual, as well as the visual context is important to generate attacks which appear semantically coherent to humans.
Exploiting Topic Information for Joint Intent Detection and Slot Filling
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Intent Detection, Natural Language Understanding, Semantic Frame Parsing, slot-filling, Slot Filling

Intent detection and slot filling are two important basic tasks in natural language understanding. Actually, there are multiple intents in an utterance. How to map different intents to corresponding slot becomes a new challenge for recent research. Existing models solve this problem by using neural layers to adaptively capture related intent information for each slot, which the process of intent selection is not clear enough. It is observed that there is strong consistency between intents and topics of a sentence, thus we exploit topic information for joint intent detection and slot filling via a topic fusion mechanism, where token-level topic information take the place of intent information to guide slot prediction. In addition, sentence-level topic information is also utilized to enhance the intent detection. Experiment results show explicit improvements on two public datasets, where provide 4.8% improvement in sentence accuracy on MixATIS and 0.7% improvement in intent detection on MixSNIPS.
Transparent Human Evaluation for Image Captioning
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Image Captioning

We establish a rubric-based human evaluation protocol for image captioning models. Our scoring rubrics and their definitions are carefully developed based on machine- and human-generated captions on the MSCOCO dataset. Each caption is evaluated along two main dimensions in a tradeoff (precision and recall) as well as other aspects that measure the text quality (fluency, conciseness, and inclusive language). Our evaluations demonstrate several critical problems of the current evaluation practice. Human-generated captions show substantially higher quality than machine-generated ones, especially in coverage of salient information (i.e., recall), while most automatic metrics say the opposite. Our rubric-based results reveal that CLIPScore, a recent metric that uses image features, better correlates with human judgments than conventional text-only metrics because it is more sensitive to recall. We hope that this work will promote a more transparent evaluation protocol for image captioning and its automatic metrics.
IMPLI: Investigatng NLI Models' Performance on Figurative Language
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Natural Language Inference

    Natural language inference (NLI) has been widely used as a task to train and evaluate models for language understanding. However, the ability of NLI models to perform inferences requiring understanding of figurative language such as idioms and metaphors remains understudied. We introduce the IMPLI (Idiomatic and Metaphoric Paired Language Inference) dataset, an English dataset consisting of paired sentences spanning idioms and metaphors. We develop novel methods to generate 24k semi-automatic pairs as well as manually creating 1.8k gold pairs. We use IMPLI to evaluate NLI models based on RoBERTa fine-tuned on the MNLI dataset, and show that while they can reliably detect entailment relationship between figurative phrases with their literal counterparts, they perform poorly on examples where pairs are designed to be non-entailing. This suggests the limits of current NLI models with regard to understanding figurative language and this dataset serves as a benchmark for future improvements in this direction.
Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Image Captioning, Machine Translation, Text Generation

Natural language processing researchers have identified limitations of evaluation methodology for generation tasks, with new questions raised about the validity of automatic metrics and of crowdworker judgments. Meanwhile, efforts to improve generation models tend to focus on simple n-gram overlap metrics (e.g., BLEU, ROUGE). We argue that new advances on models and metrics should each more directly benefit and inform the other. We therefore propose a generalization of leaderboards, bidimensional leaderboards (Billboards), that simultaneously tracks progress in language generation tasks and metrics for their evaluation. Unlike conventional unidimensional leaderboards that sort submitted systems by predetermined metrics, a Billboard accepts both generators and evaluation metrics as competing entries. A Billboard automatically creates an ensemble metric that selects and linearly combines a few metrics based on a global analysis across generators. Further, metrics are ranked based on their correlation with human judgments. We release four Billboards for machine translation, summarization, and image captioning. We demonstrate that a linear ensemble of a few diverse metrics sometimes substantially outperforms existing metrics in isolation. Our mixed-effects model analysis shows that most automatic metrics, especially the reference-based ones, overrate machine over human generation, demonstrating the importance of updating metrics as generation models become stronger (and perhaps more similar to humans) in the future.
Mining Information from Event Structure Relation Graph for Event Argument Extraction
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Event Argument Extraction, Event Extraction, Question Answering, Relation

Event Argument Extraction is a vital subtask of Event Extraction. Despite the achievements in existing methods, they can not fully use the event structure information and the rich semantics of the labels, which can provide richer external knowledge for extracting event arguments. To this end, we propose an efficient and end-to-end event argument extraction model based on the Event Structure and Question Answering (ESQA-EAE): (1) we model a multi-relational graph of event ontologies to get the structure-aware node representations; (2) we encode the questions and event mentions separately to avoid premature fusion of the two features. Experiments on the ACE2005 show that ESQA-EAE surpasses the baseline models, which further show that ESQA-EAE can use the structural information to improve the accuracy of event argument extraction.
How Gender Debiasing Affects Internal Model Representations, and Why It Matters
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Common studies of gender bias in NLP focus either on extrinsic bias measured by model performance on a downstream task or on intrinsic bias found in models' internal representations. However, the relationship between extrinsic and intrinsic bias is relatively unknown. In this work, we illuminate this relationship by measuring both quantities together: we debias a model during downstream fine-tuning, which reduces extrinsic bias, and measure the effect on intrinsic bias, which is operationalized as bias extractability with information-theoretic probing. Through experiments on two tasks and multiple bias metrics, we show that our intrinsic bias metric is a better indicator of debiasing than (a contextual adaptation of) the standard WEAT metric, and can also expose cases of superficial debiasing. Our framework provides a comprehensive perspective on bias in NLP models, which can be applied to deploy NLP systems in a more informed manner. Our code will be made publicly available.
Minimally-Supervised Relation Induction from Pre-trained Language Model
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Language Modeling, Language Modelling, Relation, Word Embeddings

Relation Induction is a very practical task in Natural Language Processing (NLP) area. In practical application scenarios, people want to induce more entity pairs having the same relation from only a few seed entity pairs. Thus, instead of the laborious supervised setting, in this paper, we focus on the minimally-supervised setting where only a couple of seed entity pairs per relation are provided. Although the conventional relation induction methods have made some success, their performance depends heavily on the quality of word embeddings. The great success of Pre-trained Language Models, such as BERT, changes the NLP area a lot, and they are proven to be able to better capture relation knowledge. In this paper, we propose a novel method to induce relation with BERT under the minimally-supervised setting. Specifically, we firstly extract proper templates from the corpus by using the mask-prediction task in BERT to build pseudo-sentences as the context of entity pairs. Then we use BERT attention weights to better represent the pseudo-sentences. In addition, We also use the IntegratedGradient of entity pairs to iteratively select better templates further. Finally, with the high-quality pseudo-sentences, we can train a better classifier for relation induction. Experiments onGoogle Analogy Test Sets (GATS), Bigger Analogy TestSet (BATS) and DiffVec demonstrate that our proposed method achieves state-of-the-art performance.