Reviewerly/exHarmony huggingface dataset:
CausalNLP Tutorial: An Introduction to Causality for Natural Language Processing

Causal inference is becoming an increasingly important topic in deep learning, with the potential to help with critical deep learning problems such as model robustness, interpretability, and fairness.In addition, causality is naturally widely used in various disciplines of science, to discover causal relationships among variables and estimate causal effects of interest.In this tutorial, we introduce the fundamentals of causal discovery and causal effect estimation to the natural language processing (NLP) audience, provide an overview of causal perspectives to NLP problems, and aim to inspire novel approaches to NLP further.This tutorial is inclusive to a variety of audiences and is expected to facilitate the community's developments in formulating and addressing new, important NLP problems in light of emerging causal principles and methodologies.
DropMix: A Textual Data Augmentation Combining Dropout with Mixup

Overfitting is a common problem when there is insufficient data to train deep neural networks in machine learning tasks.Data augmentation regularization methods such as Dropout, Mixup, and their enhanced variants, are effective and prevalent, and achieve promising performance to overcome overfitting.However, in text learning, most of the existing regularization approaches merely adopt ideas from computer vision without considering the importance of dimensionality in natural language processing.In this paper, we argue that the property is essential to overcome overfitting in text learning.Accordingly, we present a saliency map informed textual data augmentation and regularization framework, which combines Dropout and Mixup, namely DropMix, to mitigate the overfitting problem in text learning.In addition, we design a procedure that drops and patches fine grained shapes of the saliency map under the DropMix framework to enhance regularization.Empirical studies confirm the effectiveness of the proposed approach on 12 text classification tasks.
Precisely the Point: Adversarial Augmentations for Faithful and Informative Text Generation

Though model robustness has been extensively studied in language understanding, the robustness of Seq2Seq generation remains understudied.In this paper, we conduct the first quantitative analysis on the robustness of pre-trained Seq2Seq models.We find that even current SOTA pre-trained Seq2Seq model (BART) is still vulnerable, which leads to significant degeneration in faithfulness and informativeness for text generation tasks.This motivated us to further propose a novel adversarial augmentation framework, namely AdvSeq, for generally improving faithfulness and informativeness of Seq2Seq models via enhancing their robustness.AdvSeq automatically constructs two types of adversarial augmentations during training, including implicit adversarial samples by perturbing word representations and explicit adversarial samples by word swapping, both of which effectively improve Seq2Seq robustness.Extensive experiments on three popular text generation tasks demonstrate that AdvSeq significantly improves both the faithfulness and informativeness of Seq2Seq generation under both automatic and human evaluation settings.
Sequence Models for Document Structure Identification in an Undeciphered Script

This work describes the first thorough analysis of "header" signs in proto-Elamite, an undeciphered script from 3100-2900 BCE.Headers are a category of signs which have been provisionally identified through painstaking manual analysis of this script by domain experts.We use unsupervised neural and statistical sequence modeling techniques to provide new and independent evidence for the existence of headers, without supervision from domain experts.Having affirmed the existence of headers as a legitimate structural feature, we next arrive at a richer understanding of their possible meaning and purpose by (i) examining which features predict their presence; (ii) identifying correlations between these features and other document properties; and (iii) examining cases where these features predict the presence of a header in texts where domain experts do not expect one (or vice versa).We provide more concrete processes for labeling headers in this corpus and a clearer justification for existing intuitions about document structure in proto-Elamite.
Towards Generalized Open Information Extraction

Open Information Extraction (OpenIE) facilitates the open-domain discovery of textual facts.However, the prevailing solutions evaluate Ope-nIE models on in-domain test sets aside from the training corpus, which certainly violates the initial task principle of domain-independence.In this paper, we propose to advance OpenIE towards a more realistic scenario: generalizing over unseen target domains with different data distributions from the source training domains, termed Generalized OpenIE.For this purpose, we first introduce GLOBE, a largescale human-annotated multi-domain OpenIE benchmark, to examine the robustness of recent OpenIE models to domain shifts, and the relative performance degradation of up to 70% implies the challenges of generalized OpenIE.Then, we propose DragonIE, which explores a minimalist graph expression of textual fact: directed acyclic graph, to improve the OpenIE generalization.Extensive experiments demonstrate that DragonIE beats the previous methods in both in-domain and out-of-domain settings by as much as 6.0% in F1 score absolutely, but there is still ample room for improvement.
Weakly-Supervised Temporal Article Grounding

Given a long untrimmed video and natural language queries, video grounding (VG) aims to temporally localize the semantically-aligned video segments.Almost all existing VG work holds two simple but unrealistic assumptions: 1) All query sentences can be grounded in the corresponding video.2) All query sentences for the same video are always at the same semantic scale.Unfortunately, both assumptions make today's VG models fail to work in practice.For example, in real-world multimodal assets (e.g., news articles), most of the sentences in the article can not be grounded in their affiliated videos, and they typically have rich hierarchical relations (i.e., at different semantic scales).To this end, we propose a new challenging grounding task: Weakly-Supervised temporal Article Grounding (WSAG).Specifically, given an article and a relevant video, WSAG aims to localize all "groundable" sentences to the video, and these sentences are possibly at different semantic scales.Accordingly, we collect the first WSAG dataset to facilitate this task: Youwiki-How, which borrows the inherent multi-scale descriptions in wikiHow articles and plentiful YouTube videos.In addition, we propose a simple but effective method DualMIL for WSAG, which consists of a two-level MIL 1 loss and a single-/cross-sentence constraint loss.These training objectives are carefully designed for these relaxed assumptions.Extensive ablations have verified the effectiveness of DualMIL 2 .
Analyzing the Limits of Self-Supervision in Handling Bias in Language

Warning: This paper contains examples that may be offensive or upsetting.Prompting inputs with natural language task descriptions has emerged as a popular mechanism to elicit reasonably accurate outputs from large-scale generative language models with little to no in-context supervision.This also helps gain insight into how well language models capture the semantics of a wide range of downstream tasks purely from self-supervised pre-training on massive corpora of unlabeled text.Such models have naturally also been exposed to a lot of undesirable content like racist and sexist language and there is only some work on awareness of models along these dimensions.In this paper, we define and comprehensively evaluate how well such language models capture the semantics of four tasks for bias: diagnosis, identification, extraction and rephrasing.We define three broad classes of task descriptions for these tasks: statement, question, and completion, with numerous lexical variants within each class.We study the efficacy of prompting for each task using these classes and the null task description across several decoding methods and few-shot examples.Our analyses indicate that language models are capable of performing these tasks to widely varying degrees across different bias dimensions, such as gender and political affiliation.We believe our work is an important step towards unbiased language models by quantifying the limits of current self-supervision objectives at accomplishing such sociologically challenging tasks.* Work done as an intern at Amazon Alexa AI.
MCPG: A Flexible Multi-Level Controllable Framework for Unsupervised Paraphrase Generation

We present MCPG: a simple and effective approach for controllable unsupervised paraphrase generation, which is also flexible to adapt to specific domains without extra training.MCPG is controllable in different levels: local lexicons, global semantics, and universal styles.The unsupervised paradigm of MCPG combines factual keywords and diversified semantic embeddings as local lexical and global semantic constraints.The semantic embeddings are diversified by standard dropout, which is exploited for the first time to increase inference diversity by us.Moreover, MCPG is qualified with good domain adaptability by adding a transfer vector as a universal style constraint, which is refined from the exemplars retrieved from the corpus of the target domain in a training-free way.Extensive experiments show that MCPG outperforms state-of-the-art unsupervised baselines by a margin.Meanwhile, our domain-adapted MCPG also achieves competitive performance with strong supervised baselines even without training.
MIC: A Multi-task Interactive Curation Tool

This paper introduces MIC, a Multi-task Interactive Curation tool, a human-machine collaborative curation tool for multiple NLP tasks.The tool aims to borrow recent advances in literature to solve pain-points in real NLP tasks.Firstly, it supports multiple projects with multiple users which enables collaborative annotations.Secondly, MIC allows easy integration of pre-trained models, rules, and dictionaries to auto label the text and speed up the labeling process.Thirdly, MIC supports annotation at different scales (span of characters and words, tokens and lines, or document) and different types (free text, sentence labels, entity labels, and relationship triplets) with easy GUI operations.
Knowledge Stimulated Contrastive Prompting for Low-Resource Stance Detection

Stance Detection Task (SDT) aims at identifying the stance of the sentence towards a specific target and is usually modeled as a classification problem.Backgound knowledge is often necessary for stance detection with respect to a specific target, especially when there is no target explicitly mentioned in text.This paper focuses on the knowledge stimulation for low-resource stance detection tasks.We firstly explore to formalize stance detection as a prompt based contrastive learning task.At the same time, to make prompt learning suit to stance detection, we design a template mechanism to incorporate corresponding target into instance representation.Furthermore, we propose a masked language prompt joint contrastive learning approach to stimulate the knowledge inherit from the pre-trained model.The experimental results on three benchmarks show that knowledge stimulation is effective in stance detection accompanied with our proposed mechanism.
CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model

Manually designing cloze test consumes enormous time and efforts.The major challenge lies in wrong option (distractor) selection.Having carefully-design distractors improves the effectiveness of learner ability assessment.As a result, the idea of automatically generating cloze distractor is motivated.In this paper, we investigate cloze distractor generation by exploring the employment of pre-trained language models (PLMs) as an alternative for candidate distractor generation.Experiments show that the PLM-enhanced model brings a substantial performance improvement.Our best performing model advances the state-of-theart result from 14.94 to 34.17 (NDCG@10 score).Our code and dataset is available at https://github.com/AndyChiangSH/CDGP.
CISLR: Corpus for Indian Sign Language Recognition

Indian Sign Language, though used by a diverse community, still lacks well-annotated resources for developing systems that would enable sign language processing.In recent years researchers have actively worked for sign languages like American Sign Languages, however, Indian Sign language is still far from datadriven tasks like machine translation.To address this gap, in this paper, we introduce a new dataset CISLR (Corpus for Indian Sign Language Recognition) for word-level recognition in Indian Sign Language using videos.The corpus has a large vocabulary of around 4700 words covering different topics and domains.Further, we propose a baseline model for word recognition from sign language videos.To handle the low resource problem in the Indian Sign Language, the proposed model consists of a prototype-based one-shot learner that leverages resource-rich American Sign Language to learn generalized features for improving predictions in Indian Sign Language.Our experiments show that gesture features learned in another sign language can help perform one-shot predictions in CISLR.
Coreference Annotation of an Arabic Corpus using a Virtual World Game

Coreference resolution is a key aspect of text comprehension, but the size of the available coreference corpora for Arabic is limited in comparison to the size of the corpora for other languages.In this paper we present a Game-With-A-Purpose called Stroll with a Scroll created to collect from players coreference annotations for Arabic.The key contribution of this work is the embedding of the annotation task in a virtual world setting, as opposed to the puzzle-type games used in previously proposed Games-With-A-Purpose for coreference.
LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training

Language-based environment manipulation requires agents to manipulate the environment following natural language instructions, which is challenging due to the huge space of the environments.To address this challenge, various approaches have been proposed in recent work.Although these approaches work well for their intended environments, they are difficult to generalize across environments.In this work, we propose LEMON, a general framework for language-based environment manipulation tasks.Specifically, we first specify a task-agnostic approach for language-based environment manipulation tasks, which can deal with various environments using the same generative language model.Then we propose an execution-guided pre-training strategy to inject prior knowledge of environments to the language model with a pure synthetic pre-training corpus.Experimental results on tasks including ALCHEMY, SCENE, TANGRAMS, PROPARA and RECIPES demonstrate the effectiveness of LEMON: it achieves new state-of-the-art results on four of the tasks, and the executionguided pre-training strategy brings remarkable improvements on all experimental tasks 1 .
Detecting Unintended Social Bias in Toxic Language Datasets

Warning: This paper has contents which may be offensive, or upsetting however this cannot be avoided owing to the nature of the work.With the rise of online hate speech, automatic detection of Hate Speech, Offensive texts as a natural language processing task is getting popular.However, very little research has been done to detect unintended social bias from these toxic language datasets.This paper introduces a new dataset ToxicBias curated from the existing dataset of Kaggle competition named "Jigsaw Unintended Bias in Toxicity Classification".We aim to detect social biases, their categories, and targeted groups.The dataset contains instances annotated for five different bias categories, viz., gender, race/ethnicity, religion, political, and LGBTQ.We train transformerbased models using our curated datasets and report baseline performance for bias identification, target generation, and bias implications.Model biases and their mitigation are also discussed in detail.Our study motivates a systematic extraction of social bias data from toxic language datasets.All the codes and dataset used for experiments in this work are publicly available 1 .
Q-TOD: A Query-driven Task-oriented Dialogue System

Existing pipelined task-oriented dialogue systems usually have difficulties adapting to unseen domains, whereas end-to-end systems are plagued by large-scale knowledge bases in practice.In this paper, we introduce a novel querydriven task-oriented dialogue system, namely Q-TOD.The essential information from the dialogue context is extracted into a query, which is further employed to retrieve relevant knowledge records for response generation.Firstly, as the query is in the form of natural language and not confined to the schema of the knowledge base, the issue of domain adaption is alleviated remarkably in Q-TOD.Secondly, as the query enables the decoupling of knowledge retrieval from the generation, Q-TOD gets rid of the issue of knowledge base scalability.To evaluate the effectiveness of the proposed Q-TOD, we collect query annotations for three publicly available task-oriented dialogue datasets.Comprehensive experiments verify that Q-TOD outperforms strong baselines and establishes a new state-of-the-art performance on these datasets.
Exploring a POS-based Two-stage Approach for Improving Low-Resource AMR-to-Text Generation

This work presents a two-stage approach for tackling low-resource AMR-to-text generation for Brazilian Portuguese.Our approach consists of (1) generating a masked surface realization in which some tokens are masked according to its Part-of-Speech class and (2) infilling the masked tokens according to the AMR graph and the previous masked surface realization.Results show a slight improvement over the baseline, mainly in BLEU (1.63) and ME-TEOR (0.02) scores.Moreover, we evaluate the pipeline components separately, showing that the bottleneck of the pipeline is the masked surface realization.Finally, the human revision suggests that models still suffer from hallucinations, and some strategies to deal with the problems found are proposed.
Open Relation and Event Type Discovery with Type Abstraction

Conventional "closed-world" information extraction (IE) approaches rely on human ontologies to define the scope for extraction.As a result, such approaches fall short when applied to new domains.This calls for systems that can automatically infer new types from given corpora, a task which we refer to as type discovery.To tackle this problem, we introduce the idea of type abstraction, where the model is prompted to generalize and name the type.Then we use the similarity between inferred names to induce clusters.Observing that this abstraction-based representation is often complementary to the entity/trigger token representation, we set up these two representations as two views and design our model as a co-training framework.Our experiments on multiple relation extraction and event extraction datasets consistently show the advantage of our type abstraction approach.
Knowledge-Enhanced Self-Supervised Prototypical Network for Few-Shot Event Detection

Prototypical network based joint methods have attracted much attention in few-shot event detection, which carry out event detection in a unified sequence tagging framework.However, these methods suffer from the inaccurate prototype representation problem, due to two main reasons: the number of instances for calculating prototypes is limited; And, they do not well capture the relationships among event prototypes.To deal with this problem, we propose a Knowledge-Enhanced self-supervised Prototypical Network, called KE-PN, for few-shot event detection.KE-PN adopts hybrid rules, which can automatically align event types to an external knowledge base, i.e., FrameNet, to obtain more instances.It proposes a selfsupervised learning method to filter out noisy data from enhanced instances.KE-PN is further equipped with an auxiliary event type relationship classification module, which injects the relationship information into representations of event prototypes.Extensive experiments on three benchmark datasets, i.e., Few-Event, MAVEN, and ACE2005 demonstrate the state-of-the-art performance of KE-PN.
FRSUM: Towards Faithful Abstractive Summarization via Enhancing Factual Robustness

Despite being able to generate fluent and grammatical text, current Seq2Seq summarization models still suffering from the unfaithful generation problem.In this paper, we study the faithfulness of existing systems from a new perspective of factual robustness which is the ability to correctly generate factual information over adversarial unfaithful information.We first measure a model's factual robustness by its success rate to defend against adversarial attacks when generating factual information.The factual robustness analysis on a wide range of current systems shows its good consistency with human judgments on faithfulness.Inspired by these findings, we propose to improve the faithfulness of a model by enhancing its factual robustness.Specifically, we propose a novel training strategy, namely FRSUM, which teaches the model to defend against both explicit adversarial samples and implicit factual adversarial perturbations.Extensive automatic and human evaluation results show that FRSUM consistently improves the faithfulness of various Seq2Seq models, such as T5, BART.
Conditional Supervised Contrastive Learning for Fair Text Classification

Contrastive representation learning has gained much attention due to its superior performance in learning representations from both image and sequential data.However, the learned representations could potentially lead to performance disparities in downstream tasks, such as increased silencing of underrepresented groups in toxicity comment classification.In light of this challenge, in this work, we study learning fair representations that satisfy a notion of fairness known as equalized odds for text classification via contrastive learning.Specifically, we first theoretically analyze the connections between learning representations with a fairness constraint and conditional supervised contrastive objectives, and then propose to use conditional supervised contrastive objectives to learn fair representations for text classification.We conduct experiments on two text datasets to demonstrate the effectiveness of our approaches in balancing the trade-offs between task performance and bias mitigation among existing baselines for text classification.Furthermore, we also show that the proposed methods are stable in different hyperparameter settings. 1
Learning Invariant Representation Improves Robustness for MRC Models

The prosperity of Pretrained Language Models (PLM) has greatly promoted the development of Machine reading comprehension (MRC).However, these models are vulnerable and not robust to adversarial examples.In this paper, we propose Stable and Contrastive Question Answering (SCQA) to improve invariance of model representation to alleviate these robustness issues.Specifically, we first construct positive example pairs which have same answer through data augmentation.Then SCQA trains enhanced representations with better alignment between positive pairs by introducing stability and contrastive loss.Experimental results show that our approach can boost the robustness of QA models cross different MRC tasks and attack sets significantly and consistently. 1