pwc-archive/papers-with-abstracts huggingface dataset:
ReCo: Reliable Multi-hop Causal Reasoning via Structural Causal Recurrent Unit
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Decision Making

Multi-hop causal reasoning (MCR) is an essential ability for many decision-making AI systems, which requires the model to perform multi-step causal reasoning by connecting causal event pairs. However, MCR still suffers from two main transitive problems of threshold effect and scene drift. In this paper, we propose a novel reliable multi-hop causal reasoning framework (ReCo), which introduces a structural causal recurrent unit (SCRU) to model the causal reasoning chains. In SCRU, we devise two control mechanisms to solve the threshold effect and scene drift problems, and implement a logical constraint for better optimization of ReCo. Experiments show that ReCo achieves state-of-the-art (SOTA) results on both Chinese and English multi-hop causal reasoning datasets. Finally, BERT obtains the best results on four downstream causal-related tasks by injecting reliable causal knowledge distilled from ReCo.
Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Domain Adaptation, Vision-Language Navigation

Vision-language navigation (VLN) is a challenging task due to its large searching space in the environment. To address this problem, previous works have proposed some methods of fine-tuning a large model that pretrained on large-scale datasets. However, the conventional fine-tuning methods require extra human-labeled navigation data and lack self-exploration capabilities in environments, which hinders their generalization of unseen scenes.  To improve the ability of fast cross-domain adaptation, we propose Prompt-based Environmental Self-exploration (ProbES), which can self-explore the environments by sampling trajectories and automatically generates structured instructions via a large-scale cross-modal pretrained model (CLIP). Our method fully utilizes the knowledge learned from CLIP to build an in-domain dataset by self-exploration without human labeling. Unlike the conventional approach of fine-tuning, we introduce prompt tuning to achieve fast adaptation for language embeddings, which substantially improves the learning efficiency by leveraging prior knowledge. By automatically synthesizing trajectory-instruction pairs in any environment without human supervision and instruction prompt tuning, our model can adapt to diverse vision-language navigation tasks, including VLN and REVERIE. Both qualitative and quantitative results show that our ProbES significantly improves the generalization ability of the navigation model.
An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Prompt Engineering

Pre-trained language models derive substantial linguistic and factual knowledge from the massive corpora on which they are trained, and prompt engineering seeks to align these models to specific tasks. Unfortunately, existing prompt engineering methods require significant amounts of labeled data, access to model parameters, or both. We introduce a new method for selecting prompt templates \textit{without labeled examples} and \textit{without direct access to the model}. Specifically, over a set of candidate templates, we choose the template that maximizes the mutual information between the input and the corresponding model output. Across 8 datasets representing 7 distinct NLP tasks, we show that when a template has high mutual information, it also has high accuracy on the task. On the largest model, selecting prompts with our method gets 90\% of the way from the average prompt accuracy to the best prompt accuracy and requires no ground truth labels.
Textual Entailment with Dynamic Contrastive Learning for Zero-shot NER
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Contrastive Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Natural Language Inference, NER, Sentence

In this paper, we study the problem of zero-shot NER, which aims at building a Named Entity Recognition (NER) system from scratch. It needs to identify the entities in the given sentences when we have zero token-level annotations for training. Previous works usually use sequential labeling models to solve the NER task and obtain weakly labeled data from entity dictionaries in the zero-shot setting. However, these labeled data are quite noisy since we need the labels for each token and the entity coverage of the dictionaries is limited. Here we propose to formulate the NER task as a Textual Entailment problem and solve the task via Textual Entailment with Dynamic Contrastive Learning (TEDC). TEDC not only alleviates the noisy labeling issue, but also transfers the knowledge from pre-trained textual entailment models. Additionally, the dynamic contrastive learning framework contrasts the entities and non-entities in the same sentence and improves the model's discrimination ability. Experiments on two datasets show that TEDC can achieve state-of-the-art performance on the task of zero-shot NER.
Improving Compositional Generalization with Self-Training for Data-to-Text Generation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Data-to-Text Generation, Text Generation

Data-to-text generation focuses on generating fluent natural language responses from structured meaning representations (MRs). Such representations are compositional and it is costly to collect responses for all possible combinations of atomic meaning schemata, thereby necessitating few-shot generalization to novel MRs. In this work, we systematically study the compositional generalization of the state-of-the-art T5 models in few-shot data-to-text tasks. We show that T5 models fail to generalize to unseen MRs, and we propose a template-based input representation that considerably improves the model's generalization capability. To further improve the model's performance, we propose an approach based on self-training using fine-tuned BLEURT for pseudo-response selection. On the commonly-used SGD and Weather benchmarks, the proposed self-training approach improves tree accuracy by $46\%+$ and reduces the slot error rates by $73\%+$ over the strong T5 baselines in few-shot settings.
Towards Faithful Response Generation for Chinese Table Question Answering
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Question Answering, Response Generation

The response generation for TableQA aims to automatically generate a response to end-users from a SQL query and its corresponding execution result (in the form of table). It is an essential and practical task. However, there has been little work on it in recent years. We consider this may be blamed on the lack of large-scale and high-quality datasets in this area. In this paper, we present ResponseNLG, a large-scale and high-quality Chinese dataset for TableQA response generation, to advance the field in both academic and industrial communities. Further, to bridge the structural gap between the input SQL and table and establish better semantic alignments, we propose a Heterogeneous Graph Transformation approach. In this way, we establish a joint encoding space for the two heterogeneous input data and convert this task to a Graph-to-Text problem. We further introduce the Node Segment Embedding to better preserve the original graph structure upon PLMs based models.
QA Domain Adaptation using Data Augmentation and Contrastive Adaptation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Data Augmentation, Domain Adaptation, Question Answering, Question Generation, Question-Generation

Domain adaptation for question answering (QA) has recently shown impressive results for answering out-of-domain questions. Yet, a common challenge is to build approaches that are effective for niche domains with small text corpora. In this paper, we propose a novel framework called QADA for QA domain adaptation. QADA has two components: (1) A question generation model is used to generate synthetic question-answer samples from the target domain. Different from existing baselines, we enrich the samples via a novel pipeline for data augmentation: for questions, we introduce token-level augmentation (i.e., synonym replacement and token swapping), and, for contexts, we develop hidden-space augmentation which learns to drop context spans via a custom attentive sampling strategy. (2) The QA model is based on transformers. However, unlike existing approaches, we propose to train it via a novel attention-based contrastive adaptation. Here, we use the attention weights to sample informative tokens for discrepancy estimation that helps the QA model separate answers and generalize across source and target domain. To the best of our knowledge, our work is the first in QA domain adaptation to leverage data augmentation and attention-based contrastive adaptation. Our evaluation shows that QADA achieves considerable improvements over state-of-the-art baselines for QA domain adaptation.
RELiC: Retrieving Evidence for Literary Claims
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Information Retrieval, Retrieval, Semantic Similarity, Semantic Textual Similarity

Humanities scholars commonly provide evidence for claims that they make about a work of literature (e.g., a novel) in the form of quotations from the work. We collect a large-scale dataset (RELiC) of 90K literary quotations and surrounding critical analysis and use it to formulate the novel task of literary evidence retrieval, in which models are given an excerpt from a literary analysis surrounding a masked quotation and asked to retrieve the quoted passage from the set of all passages in the work. Solving this retrieval task requires a deep understanding of complex literary and linguistic phenomena, which proves challenging to methods that  overwhelmingly rely on lexical and semantic similarity matching. We implement a RoBERTa-based dense passage retriever for this task that outperforms existing pretrained information retrieval baselines; however, experiments and analysis by human domain experts indicate that there is substantial room for improvement.
Retrieval-guided Counterfactual Generation for QA
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: counterfactual, Data Augmentation, Diversity, Question Answering, Question Generation, Question-Generation, Reading Comprehension, Retrieval, World Knowledge

Deep NLP models have been shown to be brittle to input perturbations. Recent work has shown that data augmentation using counterfactuals --- i.e. minimally perturbed inputs --- can help ameliorate this weakness. We focus on the task of creating counterfactuals for question answering, which presents unique challenges related to world knowledge, semantic diversity, and answerability. To address these challenges, we develop a Retrieve-Generate-Filter(RGF) technique to create counterfactual evaluation and training data with minimal human supervision. Using an open-domain QA framework and question generation model trained on original task data, we create counterfactuals that are fluent, semantically diverse, and automatically labeled. Data augmentation with RGF counterfactuals improves performance on out-of-domain and challenging evaluation sets over and above existing methods, in both the reading comprehension and open-domain QA settings. Moreover, we find that RGF data leads to significant improvements in a model's robustness to local perturbations.
Extreme Multi-label Text Classification with Pseudo Label Descriptions
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Classification, Multi Label Text Classification, Multi-Label Text Classification, Pseudo Label, text-classification, Text Classification

Extreme multi-label text classification (XMTC) is the task of tagging each document with the relevant labels in a large predefined label space, where the label frequency distribution is often highly skewed.  That is, a large portion of labels (namely the tail labels) have very few positive instances, posing a hard optimization problem for training the classification models.  The severe data sparse issue with tail labels is more announced in recent neural classifiers, where the embeddings of both the input documents and the output labels need to be jointly learned, and the success of such learning relies on the availability of sufficient training instances. This paper addresses this tough challenge in XMTC by proposing a novel approach that combines the strengths of both traditional bag-of-words (BoW) classifiers and recent neural embedding based classifiers.  Specifically, we use a trained BoW model to generate a pseudo description for each label, and apply a neural model to establish the mapping between input documents and target labels in the latent embedding spaces. Our experiments show significant improvements of the proposed approach over other strong baseline methods on benchmark datasets, especially on tail label prediction. We also provide a theoretical analysis for relating BoW and neural models w.r.t. performance lower bound.
Pretraining over Interactions for Learning Grounded Object Representations
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Object

Large language models have been criticized for their limited ability to reason about \textit{affordances} - the actions that can be performed on an object. It has been argued that to accomplish this, models need some form of grounding, i.e., connection, to objects and how they interact in the physical world. Inspired by the way humans learn about the world through interaction, we develop an approach to learning physical properties directly. We introduce a dataset of 200k object interactions in a 3D virtual environment and a self-supervised pretraining objective for learning representations of these objects. We show with probing and clustering experiments that even in the zero-shot setting, derived models learn robust representations of objects and their affordances in an unsupervised manner. Our model outperforms pretrained language and vision models on an affordance prediction baseline, suggesting that pretraining on observed interactions encodes grounded information that is not readily learned in conventional text or vision models.
Automatic Song Translation for Tonal Languages
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Translation

This paper addresses automatic song translation (AST) for tonal languages and the unique challenge of aligning words' tones with the melody of a song in addition to conveying the original meaning. We propose three criteria for effective AST---preserving semantics, singability and intelligibility---and develop objectives for these criteria. We develop a new benchmark for English--Mandarin song translation and develop an unsupervised AST system, the Guided AliGnment for Automatic Song Translation (GagaST), which combines pre-training with three decoding constraints. Both automatic and human evaluations show GagaST successfully balances semantics and singability.
GauSE: Gaussian Enhanced Self-Attention for Event Extraction
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Event Extraction

Event Extraction (EE) has benefited from pre-trained language models (PLMs), in which the self-attention mechanism could pay attention to the global relationship between triggers/arguments and context words to enhance performance. However, existing PLM-based methods are not good enough at capturing local trigger/argument-specific knowledge. To this end, we propose a Gaussian enhanced Self-attention Event extraction framework (GauSE), which models the syntactic-related local information of trigger/argument as a Gaussian bias for the first time, to pay more attention to the syntactic scope of the local region. Furthermore, existing methods rarely consider multiple occurrences of the same triggers/arguments in EE. We explore the global interaction strategies among multiple localness of the same triggers/arguments to fuse the corresponding distributions and capture more latent information scopes. Compared to traditional GCN-based models, our methods could introduce syntactic relationships without over-smoothing problem in deep GCN layers. Experiments on EE datasets demonstrate the effectiveness and generalization of our proposed approach.
PeerSum: A Peer Review Dataset for Abstractive Multi-document Summarization
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Document Summarization, Multi-Document Summarization

We present PeerSum, a new MDS dataset using peer reviews of scientific publications. Our dataset differs from the existing MDS datasets in that our summaries (i.e., the meta-reviews) are highly abstractive and they are real summaries of the source documents (i.e., the reviews) and it also features disagreements among source documents. We found that current state-of-the-art MDS models struggle to generate high-quality summaries for PeerSum, offering new research opportunities.
A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Domain Adaptation

Pre-trained language models (PLMs) cannot well recall rich factual knowledge of entities exhibited in large-scale corpora, especially those rare entities. In this paper, we propose to build a simple but effective Pluggable Entity Lookup Table (PELT) on demand by aggregating the entity's output representations of multiple occurrences in the corpora.  PELT can be compatibly plugged as inputs  to infuse supplemental entity knowledge  into PLMs. Compared to previous knowledge-enhanced PLMs,  PELT only requires 0.2%~5% pre-computation with capability of acquiring knowledge from  out-of-domain corpora for domain adaptation scenario. The experiments on knowledge-related tasks demonstrate that our method, PELT, can flexibly and effectively transfer entity knowledge from related corpora into PLMs. We will make all the data and codes publicly available to facilitate future research.
Last to Learn Bias: Analyzing and Mitigating a Shortcut in Question Matching
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

Recent studies report that even if deep neural models make correct predictions, models may be relying on shortcut rather than understanding the semantics of the text. Previous studies indicate that the shortcut deriving from the biased data distribution in training set makes spurious correlations between features and labels. In this paper, we focus on analyzing and mitigating the biased data distribution in question matching by exploring the model behavior and performance. In particular, we define bias-word as the shortcut, and explore the following questions: (1) Will the bias affect the model? (2) How does the bias affect the model's decision? Our analysis reveals that bias-words make significantly higher contributions to model predictions than random words, and the models tend to assign labels that are highly correlated to the bias-words. To mitigate the effects of shortcut, we propose a simple approach that learns no-bias-examples first but bias-examples last. The experiments demonstrate the effectiveness of the proposed approach.
IndicBART: A Pre-trained Model for Indic Natural Language Generation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Extreme Summarization, Machine Translation, NMT, Text Generation, Transfer Learning, Translation

We study pre-trained sequence-to-sequence model for a specific-language family with a focus on Indic languages. We present IndicBART, a multilingual,  sequence-to-sequence pre-trained model focusing on 11 Indic languages and English. IndicBART utilizes the orthographic similarity between Indic scripts to improve transfer learning between similar Indic languages. We evaluate IndicBART on two NLG tasks: Neural Machine Translation (NMT) and extreme summarization. Our experiments on NMT and extreme summarization show that a language family-specific model like IndicBART is competitive with large pre-trained models like mBART50 despite being significantly smaller. It also performs well on very low-resource translation scenarios: languages not included in pre-training or fine-tuning. Script sharing, multilingual training and better utilization of limited model capacity contribute to the good performance of the compact IndicBART model.
Modeling Intensification for Signed Language Generation: A Computational Approach
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Text Generation

End-to-end sign language generation models do not accurately represent the prosody of the languages. This lack of temporal and spatial variation in generated signs leads to poor quality and lower human perception. In this paper, we seek to improve prosody in generated sign languages by modeling intensification in a data-driven manner with strategies grounded in the linguistics of sign language by enhancing the representation of intensifiers in the gloss annotations. To employ our strategies, we first annotate a subset of the benchmark PHOENIX14T dataset with different levels of intensification. We then use a supervised intensity tagger to extend the tagging to the whole dataset. This enhanced dataset is then used to train state-of-the-art transformer models for sign language generation. We find that our efforts in intensifier modeling yield better results evaluated with automated metrics. Human evaluation also indicates a significantly higher preference of the videos generated using our strategies in the presence of intensity modifiers.
Cross-Task Generalization via Natural Language Crowdsourcing Instructions
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Question Answering

Humans (e.g., crowdworkers) have a remarkable ability in solving different tasks, by simply reading textual instructions that define them and looking at a few examples. Despite the success of the conventional supervised learning on individual datasets, such models often struggle with generalization across tasks (e.g., a question-answering system cannot solve  classification tasks). A long-standing challenge in AI is to build a model that learns a new task by understanding the human-readable instructions that define it. To study this, we introduce NATURAL INSTRUCTIONS, a dataset of 61  distinct tasks, their human-authored instructions, and 193k task instances (input-output pairs). The instructions are obtained from crowdsourcing instructions used to create  existing NLP datasets and mapped to a unified schema. Using this meta-dataset, we measure cross-task generalization by training models on seen tasks and measuring generalization to the remaining unseen ones. We adopt generative pre-trained language models to encode task-specific instructions along with input and generate task output. Our results indicate that models benefit from instructions when evaluated in terms of generalization to unseen tasks (19% better for models utilizing instructions). These models, however, are far behind an estimated performance upperbound indicating significant room for more progress in this direction.