pwc-archive/papers-with-abstracts huggingface dataset:
ULF: Cross-Validation for Weak Supervision
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Denoising, Weakly-supervised Learning

A way to overcome expensive and time-consuming manual data labeling is weak supervision - automatic annotation of data samples via a predefined set of labeling functions (LFs), rule-based mechanisms that generate potentially erroneous labels. In this work, we investigate noise reduction techniques for weak supervision based on the principle of k-fold cross-validation. In particular, we extend two frameworks for detecting the erroneous samples in manually annotated data to the weakly supervised setting. Our methods profit from leveraging the information about matching LFs and detect noisy samples more accurately. We also introduce a new algorithm for denoising the weakly annotated data called ULF, that refines the allocation of LFs to classes by estimating the reliable LFs-to-classes joint matrix. Evaluation on several datasets shows that ULF successfully improves weakly supervised learning without using any manually labeled data.
A Relation Semantic Information Attentive Stereoscopic Framework for Relational Triple Extraction
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Relation, Sentence

Extracting relational triples from unstructured text is crucial for information extraction.Recent methods extract relational triple from a stereoscopic perspective which can better capture the interaction between entity and relation. However, the stereoscopic models introduce redundant triples, which makes it difficult to  identify triples accurately. Since the relation is one of the elements of triples to be extracted, the introduction of its semantic information can make the triple information more complete, which is helpful to relational triple extraction. In this work, we propose a Relation Semantic Information Attentive Stereoscopic framework (RSIA) which can fully represent and use the semantic information of relations. Specifically, a fusion encoder from transformers on top of relation encoder and sentence encoder is designed to enrich the semantic information of relation. Then, the semantic representation of the relation is integrated into the stereoscopic 3D space as its relation dimension. Our model achieves state-of-the-art performance with F1 score up to 93.5\% and 94.3\% on two public datasets and delivers consistent performance gain on complex scenarios of overlapping triples.
Show, Don't Tell: Demonstrations Outperform Descriptions for Schema-Guided Task-Oriented Dialogue
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Dialogue State Tracking

Building universal dialogue systems that can seamlessly operate across multiple domains/APIs and can generalize to new ones with minimal supervision and low maintenance is a critical challenge. Recent works have leveraged natural language descriptions for schema elements to build such systems. However, descriptions only provide indirect supervision for downstream tasks, while still requiring effort to construct. In this work, we propose Show, Don't Tell, which uses a short labeled example dialogue to show the semantics of a schema rather than telling the model about the schema elements via descriptions. While requiring similar effort from service developers, we show that using short examples as schema representations with large language models results in stronger performance and better generalization on two popular dialogue state tracking benchmarks: the Schema-Guided Dialogue (SGD) dataset and the MultiWoZ leave-one-out benchmark.
ComSearch: Equation Searching with Combinatorial Mathematics for Solving Math Word Problems with Weak Supervision
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Math

Previous studies have introduced a weakly-supervised paradigm for solving math word problems requiring only the answer value annotation. While these methods search for correct value equation candidates as pseudo labels, they search among a narrow sub-space of the enormous equation space. To address this problem, we propose a novel search algorithm with combinatorial mathematics ComSearch, which can compress the search space by excluding mathematical equivalent equations. The compression allows the searching algorithm to enumerate all possible equations and obtain high-quality data. We investigate the noise in the pseudo labels that hold wrong mathematical logic, which we refer to as the false-matching problem, and propose a ranking model to denoise the pseudo labels. Our approach holds a flexible framework to utilize two existing supervised math word problem solvers to train pseudo labels, and both achieve state-of-the-art performance in the weak supervision task.
On the Anatomy of Latent-variable Generative Models for Conditional Text Generation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Anatomy, Conditional Text Generation, Text Generation

Conditional text generation is a non-trivial task, which is until now predominantly performed with latent-variable generative models. In this work, we intend to explore several choices that are shown to affect the two essential aspects of model performance: expressivity and controllability. We propose to experiment with a series of latent-variable models built around simple design changes under a general unified framework, with a particular focus on prior distributions based on Energy-Based Models instead of the usual standard Gaussian. Our experiments validate the claim that this richer prior allows for a better representational power, but it exhibits difficult training. We provide a comprehensive analysis of these difficulties and a close comparison with recent work on EBM-based priors for conditional text generation.
Unsupervised Common Sense Relation Extraction
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Common Sense Reasoning, Relation, Relation Extraction

Vast and diverse knowledge about the relations in the world help humans comprehend and argue about their environment. Equipping machines with this knowledge is challenging yet essential for general reasoning capabilities. Here, we propose to apply unsupervised relation extraction (URE), aiming to induce general relations between concepts from natural language. Previous work in URE has predominantly focused on relations between named entities in the encyclopedic domain. The more general, and more challenging, domain of common sense relation learning has not yet been addressed, partially due to a lack of datasets. We present a framework for common sense relation extraction from free-text, associated with two benchmark datasets. We present initial experiments using three state-of-the-art models developed for encyclopedic relation induction. Our results verify the utility of our benchmarks for common sense relation extraction, and suggest ample scope for future work on this important, yet challenging, task.
GraphDiffs: Graph Modeling with Differential Sequence for Document-Grounded Conversation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Knowledge grounded dialogue systems need to incorporate natural transitions between knowledge for dialogue to flow smoothly. Current systems not only lack good structured representations for knowledge that span multiple documents, but also effective algorithms that utilize such resources. We design a Co-Referential Multi-Document Graph(CoRM-DoG) that seamlessly captures inter-document correlations and intra-document co-referential knowledge relations. To best linearise this static graph into sequential dialogues, we contribute a Graph Modeling with Differential Sequence (GraphDiffs) method for knowledge transitions in dialogue. GraphDiffs performs knowledge selection by natively accounting for contextual graph structure and introducing differential sequence learning to effectively learn multi-turn knowledge transitions. Our analysis shows that GraphDiffs based on CoRM-DoG significantly outperforms the current state-of-the-art by 9.5\% and 7.4% on two public benchmarks, WoW and Holle-E, where the modeling of co-reference and differential sequence are critical factors for its success.
NeuS: Neutral Multi-News Summarization for Framing Bias Mitigation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Articles, News Summarization

Media framing bias can lead to increased political polarization, and thus, the need for automatic mitigation methods is growing. We propose a new task, a \textit{neutral} summary generation from multiple news articles of the varying political spectrum, to facilitate balanced and unbiased news reading.In this paper, we first collect a new dataset, obtain some insights about framing bias through a case study, and propose a new effective metric and models for the task. Lastly, we conduct experimental analyses to provide insights about remaining challenges and future directions. One of the most interesting observations is that generation models can hallucinate not only factually inaccurate or unverifiable content but also politically biased content.
Detecting Unintended Social Bias in Toxic Language Datasets
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Hate speech and offensive texts are examples of damaging online content that target or promote hatred towards a group or individual member based on their actual or perceived features of identification, such as race, religion, or sexual orientation. Sharing violent and offensive content has had a significant negative impact on society. These hate speech and offensive content generally contains societal biases in them. With the rise of online hate speech, automatic detection of such biases as a natural language processing task is getting popular. However, not much research has been done to detect unintended social bias from toxic language datasets. In this paper, we introduce a new dataset from an existing toxic language dataset, to detect social biases along with their categories and targeted groups. We then report baseline performances of both classification and generation tasks on our curated dataset using transformer-based models. Our study motivates a systematic extraction of social bias data from toxic language data.
Lexicon for multiword expression identification
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Following the idea that lexicons are needed in order for automatic identification of multiword expressions(MWE) to handle the unpredictable nature of MWEs, this paper proposes a lexicon formalism, itself declined in multitudes of possible sub-formalisms depending on the linguistic features considered , along with an evaluation method which could be used to compare lexicon formalisms to each other.An exploration of the powerset of features is done in order to find the bests of such subset of features to be used. The impact of the proposed lexicon formalism on MWE identification is investigated,  leading us to conjecture that lexicon indeed have the potential to help MWE identification.
LexiCon: Lexically Constrained Review Generation via Robust Insertion
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: CPU, Diversity, Informativeness, Review Generation

Existing review generators struggle to generate specific information correctly (e.g., Caesar salad, Snapdragon CPU), which prevents generated reviews from being more informative. In this paper, we propose to introduce lexical constraints into review generation which can be any key phrases to be contained in reviews. Compared to soft constraints (e.g., aspects) used in previous work, lexical constraints easily incorporate specific information which can largely improve the diversity and informativeness of generated reviews. To this end, we present LexiCon, a novel insertion-based review generation framework that can generate personalized reviews containing lexical constraints. Specifically, the proposed method progressively inserts new tokens between existing tokens in a parallel manner until a sequence is completed. Experimental results show that LexiCon outperforms the strongest review generation model by 20% BLEU-2 (coherence) and 68% Distinct-2 (diversity) on average. Human evaluation also shows that LexiCon is more robust to various lexical constraints than the state-of-the-art lexically-constrained model for general purpose.
Investigating the Benefits of Free-Form Rationales
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Form

Free-form rationales aim to aid model interpretability by supplying the background knowledge that can help understand model decisions. Crowdsourced rationales are provided for commonsense QA instances in popular datasets such as CoS-E and ECQA, but their utility remains under-investigated. We present human studies which show that ECQA rationales indeed provide additional information to understand a decision, while 70% of CoS-E rationales do not. Inspired by this finding, we ask: can the additional context provided by free-form rationales benefit models, similar to human users? We investigate the utility of rationales as an additional source of supervision, by varying the quantity and quality of rationales during training. After controlling for instances where rationales leak the correct answer, we find that incorporating only 5% of rationales during training can boost model performance by 16.89%. Moreover, we also show that rationale quality matters: compared to crowdsourced rationales, T5-generated rationales provide not only much weaker supervision to models, but are also not helpful for human users in aiding model interpretability.
Causal Language Model for Zero-shot Constrained Keyphrase Generation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Keyphrase Generation, Language Modeling, Language Modelling

Recently, most of the state-of-the-art keyphrase prediction models are based on a supervised generative model.Although it shows noticeable improvement over statistical methods, it still struggles with low performance on out of the domain and low-resource data. To overcome these limitations, unsupervised methods have also been critical and studied. But the unsupervised method also has a defect in the necessary process which extracting candidates before selecting keyphrases. As not including various forms of phrases, we note that the unsupervised method can't ensure oracle keyphrase.In this paper, we present zero-shot constrained keyphrase generation by leveraging a large-scale language model. To generate diverse keyphrases, we explore controlling a phrase during the generation. Finally, we evaluate benchmark datasets of the scholar domain. It results in better performances than unsupervised methods on several datasets without going through the candidate extraction stage. For domain robustness, we evaluate out-of-domain DUC compare with NUS. Since our method doesn't fine-tune to a corpus of a specific domain, it's better than supervised methods based on Sequence-to-Sequence.
Identifying and Measuring Token-Level Sentiment Bias in Pre-trained Language Models with Prompts
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Due to the superior performance, large-scale pre-trained language models (PLMs) have been widely adopted in many aspects of human society. However, we still lack effective tools to understand the potential bias embedded in the black-box models. Recent advances in prompt tuning show the possibility to explore the internal mechanism of the PLMs. In this work, we propose two token-level sentiment tests: Sentiment Association Test (SAT) and Sentiment Shift Test (SST) which utilize the prompt as a probe to detect the latent bias in the PLMs. Our experiments on the collection of sentiment datasets show that both SAT and SST can identify sentiment bias in PLMs and SST is able to quantify the bias. The results also prove that fine-tuning can augment existing bias in PLMs.
Consecutive Question Generation with Multitask Joint Reranking and Dynamic Rationale Search
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Diversity, Informativeness, Question Answering, Question Generation, Question-Generation, Reranking, Sentence

Automatic question generation (QG) aims to generate a set of questions for a given passage, and can be viewed as a dual task of question answering (QA). However, most current methods of QG tend to generate question by question independently, mainly based on specific extracted answer spans. In this paper, we propose to consecutively generate questions over a whole passage, with a comprehensive consideration of the aspects including accuracy, diversity, informativeness, and coverage. First we exam four key elements in QG, i.e., question, answer, rationale, and context history, and propose a novel multitask framework with one main task generating a question-answer pair, and four auxiliary tasks generating other elements alternately, improving model performance from all aspects through both joint training and reranking. Further, to learn the connection between questions and fully exploit the important information in every sentence, we propose a new consecutive generation strategy, which dynamically selects the rationales and searches for the best question series globally. Extensive experiments on different datasets show that our method can improve question generation significantly and benefit multiple related NLP tasks.
CQMrobust: A Chinese Dataset of Linguistically Perturbed Natural Questions for Evaluating the Robustness of Question Matching Models
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Natural Questions

In this paper, we focus on studying robustness evaluation of Chinese question matching. Most of the previous work on analyzing robustness issue focus on just one or a few types of artificial adversarial examples. Instead, we argue that it is necessary to formulate a comprehensive evaluation about the linguistic capabilities of models on natural texts. For this purpose, we create a Chinese dataset namely CQMrobust which contains natural questions with linguistic perturbations to evaluate the robustness of question matching models. CQMrobust contains 3 categories and 13 subcategories with 32 linguistic perturbations. The extensive experiments demonstrate that CQMrobust has a better ability to distinguish different models. Importantly, the detailed breakdown of evaluation by linguistic phenomenon in CQMrobust helps us easily diagnose the strength and weakness of different models. Additionally, our experiment results show that the effect of artificial adversarial examples does not work on the natural texts.The dataset and baseline codes will be publicly available in the open source community.
Gated Recursive and Sequential Deep Hierarchical Encoding for Detecting Incongruent News Articles
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Articles, Feature Engineering, Misinformation

With the increase in misinformation across digital platforms, incongruent news  detection is becoming an important research problem. Earlier, researchers have exploited various feature engineering approaches and deep learning models with embedding to capture incongruity between news headlines and the body. Recent studies have also shown the advantages of capturing structural properties of the body using hierarchical encoding. Hierarchical encoding decomposes the body of a news article into smaller segments such as sentences or paragraphs. However, the existing hierarchical methods have not considered two important aspects; (i) deeper hierarchical level, and (ii) importance of different paragraphs in generating document encoding. Motivated by this, in this paper, we propose a Gated RecursiveAnd Sequential Deep HierarchicalEncoding (GRASHE) method for detectingincongruent news articles by extends hierarchicalencoding upto word level and incorporatingincongruently weight of each paragraph. Experimental results show that the proposed models outperform the bag-of-word features, sequential and hierarchical encoding-based counterparts. We also perform various ablation analysis to support the proposed models.
Classification of Illegal Drug Sales Posts using Clustering-Based Topic Modeling.
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Clustering

Drugs illegally traded online are causing social problems around the world wide. One of the ways to solve this problem is to automatically delete sales posts quickly even if they are uploaded. We propose new data on illegal drug sales posts in Korean collected directly from Twitter. There are about 100K collected data, and labels were added directly to each data. Supervised learning-based models generally show high performance, but label information is essential. It is difficult to add labels to all texts in situations where a large amount of text occurs. In this work, we propose a topic modeling-based classification model that can perform higher with even a small number of labels. As a result of the experiment, higher classification performance is shown when Topic modeling is used as a small number of data.
Challenge for open-domain targeted sentiment analysis
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Sentence, Sentiment Analysis

Since previous studies on open-domain targeted sentiment analysis are limited in dataset domain variety and sentence level,  we propose a novel dataset consisting of 6,013 human-labeled data to extend the data domains in topics of interest and document level. Furthermore, we offer a nested target annotation schema to extract the complete sentiment information in documents, boosting the practicality and effectiveness of open-domain targeted sentiment analysis. Moreover, we leverage the pre-trained model BART in a sequence-to-sequence generation method for the task. Benchmark results show that there exists large room for improvement of open-domain targeted sentiment analysis. Meanwhile, experiments have shown that challenges remain in the effective use of open-domain data, long documents, the complexity of target structure, and domain variances.
A Study of Pre-trained Language Models for Analogy Generation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Text Generation

We propose a novel application of Pre-trained Language Models (PLMs) to generate analogies and study how to design effective prompts to prompt a PLM to generate a source concept analogous to a given target concept as well as to generate an explanation of the similarity between given pair of target concept and source concept.  We found that it is feasible to prompt a GPT-3 PLM to generate meaningful analogies and the best prompts tend to be precise imperative statements especially with low temperature setting.  We systematically analyzed the sensitivity of the GPT-3 model to prompt design and temperature and found that the model is particularly sensitive to certain variations (e.g., questions vs. imperative statements). We also investigated the suitability of using the existing reference-based metrics designed for evaluating natural language generation (NLG) to evaluate analogy generation and found that the recent BLEURT score is better than the others. We further propose a promising consensus measure based on diverse prompts and settings, which can be potentially used to both automatically evaluate the generated analogies in the absence of reference text (e.g., in novel domains) and rank a set of generated analogies to select analogies of different characteristics. Overall, our study shows that PLMs offer a promising new way to generate analogies in unrestricted domains, breaking the limitation of existing analogy generation methods in requiring structured representation.