pwc-archive/papers-with-abstracts huggingface dataset:
Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intents
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Community Question Answering, Information Retrieval, Question Answering, Retrieval, Sentence

Text semantic matching is a fundamental task that has been widely used in various scenarios, such as community question answering, information retrieval, and recommendation. Most state-of-the-art matching models, e.g., BERT, directly perform text comparison by processing each word uniformly. However, a query sentence generally comprises content that calls for different levels of matching granularity. Specifically, keywords represent factual information such as action, entity, and event that should be strictly matched, while intents convey abstract concepts and ideas that can be paraphrased into various expressions. In this work, we propose a simple yet effective training strategy for text semantic matching in a divide-and-conquer manner by disentangling keywords from intents. Our approach can be easily combined with pre-trained language models (PLM) without influencing their inference efficiency, achieving stable performance improvements against a wide range of PLMs on three benchmarks.
Defending against Backdoor Attacks in Natural Language Generation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Backdoor Attack, Dialogue Generation, Machine Translation, Text Generation, Translation

The frustratingly fragile nature of neural network models make current natural  language generation (NLG) systems prone to backdoor attacks and generate malicious sequences that could be sexist or offensive. Unfortunately, little effort has been invested to how backdoor attacks can affect current NLG models and how to defend against these attacks. In this work, we investigate this problem on two important NLG tasks, machine translation and dialogue generation. By giving a formal definition for backdoor attack and defense, and developing corresponding benchmarks, we design methods to attack NLG models, which achieve high attack success to ask NLG models to generate malicious sequences. To defend against these attacks, we propose to detect the attack trigger by examining the effect of deleting or replacing certain words on the generation outputs, which we find successful for certain types of attacks. We will discuss the limitation of this work, and hope this work can raise the awareness of backdoor risks concealed in deep NLG systems.
MANGO: Enhancing the Robustness of VQA Models via Adversarial Noise Generation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Logical Reasoning, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Large-scale pre-trained vision-and-language (V+L) transformers have propelled the state of the art (SOTA) on Visual Question Answering (VQA) task. Despite impressive performance on the standard VQA benchmark, it remains unclear how robust these models are. To investigate, we conduct a host of evaluations over 4 different types of robust VQA datasets: (i) Linguistic Variation; (ii) Logical Reasoning; (iii) Visual Content Manipulation; and (iv) Answer Distribution Shift. Experiments show that pre-trained V+L models already exhibit better robustness than many task-specific SOTA methods via standard model finetuning. To further enhance model robustness, we propose Mango, a generic and efficient approach that learns a Multimodal Adversarial Noise GeneratOr in the embedding space to fool V+L models. Differing from previous studies focused on one specific type of robustness, Mango is agnostic to robustness types, and enables universal performance lift for both task-specific and pre-trained  models over diverse robust VQA datasets designed to evaluate broad aspects of robustness. Comprehensive experiments demonstrate that Mango achieves new SOTA on 7 out of 9 robustness benchmarks.
Toward Automatic Misinformation Detection Utilizing Fact-checked Information
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Fact Checking, Misinformation, Semantic Role Labeling, Sentence, Sentence Embeddings

We proposed a new task FCCKB: Fact-checking by Claim Knowledge Base. The goal was to fact-check a sentence utilizing verified claims stored in the database. To retrieve relevant claims from the large database, we proposed applying Semantic Role Labeling(SRL) on the input sentence having rich semantics and then encoding the results to get fine-grained sentence embeddings. That improved semantic matching between the input sentence and the relevant claims. We used three sentence encoders for sentence encoding. In FEVER dataset, precision and recall was improved by more than 5 percent after SRL was applied.
Generating Scientific Claims for Zero-Shot Scientific Fact Checking
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Fact Checking, Negation

Automated scientific fact checking is difficult due to the complexity of scientific language and a lack of significant amounts of training data, as annotation requires domain expertise. To address this challenge, we propose  scientific claim generation, the task of generating one or more atomic and verifiable claims from scientific sentences, and demonstrate its usefulness in zero-shot fact checking for biomedical claims. We propose ClaimGen-BART, a new supervised method for generating claims supported by the literature, as well as KBIN, a novel method for generating claim negations. Additionally, we adapt an existing unsupervised entity-centric method of claim generation to biomedical claims, which we call ClaimGen-Entity. Experiments on zero-shot fact checking demonstrate that both ClaimGen-Entity and ClaimGen-BART, coupled with KBIN, achieve up to 90% performance of fully supervised models trained on manually annotated claims and evidence. A rigorous evaluation study demonstrates significant improvement in generated claim and negation quality over existing baselines
Do BERTs Learn to Use Browser User Interface? Exploring Multi-Step Tasks with Unified Vision-and-Language BERTs
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Unifying models by reducing task-specific structures have been studied to facilitate the transfer of learned knowledge.A text-to-text framework has pushed the unification of the model.However, the framework remains limited because it does not allow contents with a layout for input and has a basic assumption that the task can be solved in a single step.To address these limitations, in this paper, we explore a new framework in which a model performs a task by manipulating displayed web pages in multiple steps.We develop two types of task web pages with different levels of difficulty and propose a BERT extension for the framework.We trained the BERT extension with those task pages jointly, and the following observations were made.(1) The model maintains its performance greater than 80% of that of the original BERT separately fine-tuned in a single-step framework in five out of six tasks.(2) The model learned to solve both tasks of difficulty level. (3) The model did not generalize effectively on unseen tasks.These results suggest that although room for improvement exists, we can transfer BERTs to multi-step tasks, such as using graphical user interfaces.
Latent Group Dropout for Multilingual and Multidomain Machine Translation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Machine Translation, Translation

Multidomain and multilingual machine translation often rely on parameter sharing strategies, where large portions of the network are meant to capture the commonalities of the tasks at hand, while smaller parts are reserved to model the peculiarities of a language or a domain. In adapter-based approaches, these strategies are hardcoded in the network architecture, independent of the similarities between tasks. In this work, we propose a new method to better take advantage of these similarities, using a latent-variable model. We also develop new techniques to train this model end-to-end and report experimental results showing that the learned patterns are both meaningful and yield improved translation performance without any increase of the model size.
ITA: Image-Text Alignments for Multi-Modal Named Entity Recognition
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Multi-modal Named Entity Recognition, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

Recently, Multi-modal Named Entity Recognition (MNER) has attracted a lot of attention. Most of the work utilizes image information through region-level visual representations obtained from a pretrained object detector and  relies on an attention mechanism to model the interactions between image and text representations. However, it is difficult to model such interactions as image and text representations are trained separately on the data of their respective modality and are not aligned in the same space. As text representations take the most important role in MNER, in this paper, we propose {\bf I}mage-{\bf t}ext {\bf A}lignments (ITA) to  align image features into the textual space, so that the attention mechanism in transformer-based pretrained textual embeddings can be better utilized. ITA first aligns the image into regional object tags, image-level captions and optical characters as visual contexts, concatenates them with the input texts as a new cross-modal input, and then feeds it into a pretrained textual embedding model. This makes it easier for the attention module of a pretrained textual embedding model to model the interaction between the two modalities since they are both represented in the textual space. ITA further aligns the output distributions predicted from the cross-modal input and textual input views so that the MNER model can be more practical and robust to noises from images. In our experiments, we show that ITA models can achieve state-of-the-art accuracy on multi-modal Named Entity Recognition datasets, even without image information.
Detection and Mitigation of Political Bias in Natural Language Processing: A Literature Review
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

With the increasing importance of Natural Language Processing (NLP) tools, their implications on the propagation of societal biases become more and more relevant. In this context, the analysis of political bias in manually written and automatically generated text is a relatively understudied field. Political bias refers to the preference or prejudice towards one political ideology over another. To increase the discourse in this subject area, we analyze contemporary studies on detecting and mitigating political bias in this literature review. We further discuss the benefits and potential drawbacks of the considered methods and look at the ethical considerations involved with political bias in NLP, before we give suggestions for future studies.
CL-ReKD: Cross-lingual Knowledge Distillation for Multilingual Retrieval Question Answering
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Knowledge Distillation, Language Modeling, Language Modelling, Question Answering, Reading Comprehension, Retrieval, Sentence, Sentence Embedding, Sentence-Embedding

Cross-Lingual Retrieval Question Answering (CL-ReQA) is concerned with retrieving answer documents or passages to a question written in a different language. A common approach to CL-ReQA is to create a multilingual sentence embedding space such that question-answer pairs across different languages are close to each other. In this paper, we propose a novel CL-ReQA method utilizing the concept of knowledge distillation and a new cross-lingual consistency training technique to create a multilingual embedding space for ReQA. To assess the effectiveness of our work, we conducted comprehensive experiments on CL-ReQA and a downstream task, machine reading QA. We compared our proposed method with the current state-of-the-art solutions across three public CL-ReQA corpora. Our method outperforms competitors in 19 out of 21 settings of CL-ReQA. When used with a downstream machine reading QA task, our method outperforms the best existing language-model-based method by 10% in F1 while being 10 times faster in sentence embedding computation.
A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Translation

Recent advances in the pre-training for language models leverage large-scale datasets to create multilingual models. However, low-resource languages are  mostly left out in these datasets. This is primarily because many widely spoken languages that are not be well represented on the web and therefore excluded from the large-scale crawls for datasets. Furthermore, downstream users of these models are restricted to the selection of languages originally chosen for pre-training. This work investigates how to optimally leverage existing pre-trained models to create low-resource translation systems for 16 African languages. We focus on two questions: 1) How can pre-trained models be used for languages not included in the initial pretraining? and 2) How can the resulting translation models effectively transfer to new domains? To answer these questions, we create a novel African news corpus covering 16 languages, of which eight languages are not part of any existing evaluation dataset. We demonstrate that the most effective strategy for transferring both additional languages and additional domains is to leverage small quantities of high-quality translation data to fine-tune large pre-trained models.
Enhancing Robustness in Aspect-based Sentiment Analysis by Better Exploiting Data Augmentation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Data Augmentation, Sentiment Analysis

In this paper, we propose to leverage data augmentation to improve the robustness of aspect-based sentiment analysis models. Our method not only exploits augmented data but also makes models focus more on predictive features. We show in experiments that our method compares favorably against strong baselines on both robustness and standard datasets. In the contrary, the widely used adversarial training that only leverages the augmented data fails to improve performance due to the distribution shift caused by the augmented data.
Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Data Augmentation, Graph Attention, Position, Table-to-Text Generation, Text Generation

Controlled table-to-text generation seeks to generate natural language descriptions for highlighted subparts of a table. Previous SOTA systems still employ a sequence-to-sequence generation method, which merely captures the table as a linear structure and is brittle when table layouts change. We seek to go beyond this paradigm by (1) effectively expressing the relations of content pieces in the table, and (2) making our model robust to content-invariant structural transformations. Accordingly, we propose an equivariance learning framework, encoding tables with a structure-aware self-attention mechanism. This prunes the full self-attention structure into an order-invariant graph attention that captures the connected graph structure of cells belonging to the same row or column, and it differentiates between relevant cells and irrelevant cells from the structural perspective. Our framework also modifies the positional encoding mechanism to preserve the relative position of tokens in the same cell but enforce position invariance among different cells. Our technology is free to be plugged into existing table-to-text generation models, and has improved T5-based models to offer better performance on ToTTo and HiTab. Moreover, on a harder version of ToTTo, we preserve promising performance, while previous SOTA systems, even with transformation-based data augmentation, have seen significant performance drops.
It's What You Say and How You Say It: Exploring Textual and Audio Features for Podcast Data
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: TAG

Podcasts are relatively new media in the form of spoken documents or conversations with a wide range of topics, genres, and styles. With a massive increase in the number of podcasts and their listener base, it is beneficial to understand podcasts better, to derive insights into questions such as what makes certain podcasts more popular than others or which tags help in characterizing a podcast. In this work, we provide a comprehensive analysis of hand-crafted features from two modalities, i.e., text and audio. We explore multiple feature combinations considering podcast popularity prediction and multi-label tag assignment as proxy downstream tasks. In our experiments, we use document embeddings, affective features, named entities, tags, and topics as the textual features, while multi-band modulation and traditional speech processing features constitute the audio features. We find the audio feature prosody and textual affective features, sentiment, and emotions are significant for both the downstream tasks. We observe that the combination of textual and audio features helps in improving performance in the popularity prediction task.
Improving language models fine-tuning with representation consistency targets
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Fine-tuning contextualized representations learned by pre-trained language models has become a standard practice in the NLP field. However, pre-trained representations are prone to degradation (also known as representation collapse) during fine-tuning, which leads to instability, sub-optimal performance, and weak generalization. In this paper, we propose a novel fine-tuning method that avoids representation collapse during fine-tuning by discouraging undesirable changes of the representations. We show that our approach matches or exceeds the performance of the existing regularization-based fine-tuning methods across 13 language understanding tasks (GLUE benchmark and six additional datasets). We also demonstrate its effectiveness in low-data settings and robustness to label perturbation. Furthermore, we extend previous studies of representation collapse and propose several metrics to quantify it. Using these metrics and previously proposed experiments, we show that our approach obtains significant improvements in retaining the expressive power of representations.
Exploiting Coreference and Schema Structure for Document-level Event Extraction
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Document-level Event Extraction, Event Extraction

Document-level event extraction (DEE) extracts structured information of events from a document. Previous studies focus on improving the model architecture. We argue that exploiting data characteristics is also important. We propose to utilize coreference information to obtain better document-level entity representations, and propose the concept of core roles to adjust the schema structure to alleviate error propagation. Experiments demonstrate that our data exploitation methods significantly improve the performance of existing models on both the role-level and record-level metrics.
The Common Readability Formula & Five Adjusted Readability Formulas for Text Simplification, Medical Documents and Other General Uses
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Text Simplification

Traditional readability formulas, or equations, are inaccurate and measure highly limited linguistic properties. Despite the recent machine learning-based readability assessment models, many researchers insist on using the outdated formulas. To replace the linguistically-shallow inaccurate formulas, we: : 1. introduce Common Readability Formula (CoRF), 2. recalibrate outdated formulas (Flesch-Kincaid Grade Level, Fog Index, SMOG Index, Coleman-Liau Index, and Automated Readability Index), 3. evaluate the formulas, and 4. develop a Python library for the wide dispersal of our variations.
Sonnet Generation by Training on Non-poetic Texts with Discourse-level Coherence and Poetic Features
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Sonnet Generation, Text Generation

Poetry generation, and creative language generation in general, usually suffers from the lack of large training data. In this paper, we present a novel framework to generate sonnets that does not require training on poems. We design a hierarchical framework which plans the poem sketch before decoding. Specifically, a content planning module is trained on non-poetic texts to obtain discourse-level coherence; then a rhyme module generates rhyme words and a polishing module introduces imagery and similes for aesthetics purposes. Finally, we design a constrained decoding algorithm to impose the meter-and-rhyme constraint of the generated sonnets. Automatic and human evaluation show that our multi-stage approach without training on poem corpora generates more coherent, poetic, and creative sonnets than several strong baselines.
Learning from Mental Disorder Self-tests: Multi-head Siamese Network for Few-shot Knowledge Learning
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Social media is one of the most highly sought resources to analyze characteristics of the language by its users. In particular, many researchers utilized various linguistic features to identify users with mental disorders. However, generalizing linguistic features of such psychiatric patients is challenging since these features are apparently dependent on cultural or personal language habits. To address this challenge, we make use of the symptoms, which are shared properties of people with mental illness, concerning clinical contents rather than the ways of expressing them. In this paper, we aim to let our classification model identify informative features by training on knowledge about the symptoms. To this end, we propose a multi-head siamese network, which captures informative features based on the knowledge of mental illness symptoms and compares them to those of target text to be classified. The model is designed to learn the required knowledge by reading just a few questions from self-tests, and to identify similar stories from social media texts. Experimental results demonstrate that our model achieves improved performance as well as human-interpretable results for mental illness symptoms. A case study shows that our proposed model offers the possibility of automatic mental illness diagnosis, grounded on rational reasons.