pwc-archive/papers-with-abstracts huggingface dataset:
CPED: A Large-Scale Chinese Personalized and Emotional Dialogue Dataset for Open-Domain Conversation
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12

Recently, the personification and empathy capabilities of dialogue systems have received extensive attention from researchers. Although it is straightforward for humans to express themselves personally and empathically, this is highly difficult for dialogue systems since training data do not provide personalities or empathy knowledge. In this paper, we propose CPED, a large-scale Chinese personalized and emotional dialogue dataset, which consists of multisource knowledge related to empathy and personal characteristic. This knowledge covers 13 emotions, gender, Big Five personality traits, 19 dialogue acts and other knowledge. CPED contains more than 12K dialogues of 392 speakers from 40 TV shows. We also provide several strong baselines for open-domain conversation generation. The results show that explicitly infusing personalized knowledge and emotional information improves the personification level and empathy ability of dialogue systems, but the infusion method needs to be further studied. The dataset and baselines will be released on https://github.com/***/CPED.
A Two-Stream AMR-enhanced Model for Document-level Event Argument Extraction
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Document-level Event Extraction, Event Argument Extraction, Event Extraction, Sentence

Most previous studies aim at extracting events from a single sentence, while document-level event extraction still remains under-explored. In this paper, we focus on extracting event arguments from an entire document, which mainly faces two critical problems: a) the long-distance dependency between trigger and arguments over sentences; b) the distracting context towards an event in the document. To address these issues, we propose a \textbf{T}wo-\textbf{S}tream \textbf{A}bstract meaning \textbf{R}epresentation enhanced extraction model (TSAR). TSAR encodes the document from different perspectives by a two-stream encoding module, to utilize local and global information and lower the impact of distracting context. Besides, TSAR introduces an AMR-guided interaction module to capture both intra-sentential and inter-sentential features, based on the locally and globally constructed AMR semantic graphs. An auxiliary boundary loss is introduced to enhance the boundary information for text spans explicitly. Extensive experiments illustrate that TSAR outperforms previous state-of-the-art by a large margin, with $2.54$ F1 and $5.13$ F1 performance gain on the public RAMS and WikiEvents datasets respectively, showing the superiority in the cross-sentence arguments extraction. We will release our code upon acceptance.
Is syntax structure modeling worth? Leveraging pattern-driven modeling to enable affordable sentiment dependency learning
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Dependency Parsing, Sentiment Analysis, Sentiment Classification, Sentiment Dependency Learning

Is structure information modeling really worth in Aspect-based sentiment classification (ABSC)? Recent popular works tend to exploit syntactic information guiding sentiment dependency parsing, i.e., structure-based sentiment dependency learning. However, many works fall into the trap that confusing the concepts between syntax dependency and sentiment dependency. Besides, structure information (e.g., syntactic dependency tree) usually consumes expensive computational resources due to the extraction of the adjacent matrix. Instead, we believe the sentiment dependency mostly occurs between adjacent aspects. By proposing the sentiment patterns (SP) to boost the sentiment dependency learning, we introduce the Local dependency aggregating (Lena) to explore sentiment dependency in the text. Experiments show that Lena is more efficient than existing structure-based models without dependency matrix constructing and modeling expense. The performance on all five public ABSC datasets makes a big step compared to state-of-the-art models, and our work could inspire future research focusing on efficient local sentient dependency modeling.
Toward Preference-Aware Story Evaluation via Ranking, Rating and Reasoning
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12

Existing automatic story evaluation methods place a premium on story coherence, deviating from human preference. We go beyond such restrictions by presenting a more challenging task of \textbf{preference-aware story evaluation}. Given either a machine-generated or a human-written story, the task requires the machine to output a preference score that corresponds to human preference, along with specific ratings and comments for various aspects (e.g., opening, character-shaping). To support this novel task, we introduce a new dataset, namely \textbf{StoR3}, comprising (i) 100k ranked story pairs; and (ii) a set of 46k ratings and comments on various aspects of the story. To move towards preference-aware evaluation, we propose a model using the \textit{upvote count} as the criterion. The experiments show that the scores obtained by our model have a high correlation to human preference. Additionally, we discovered that the combination of aspect ratings and comments improves performance. Our dataset and benchmarks are publicly available to advance the research of story evaluation tasks.
Incorporate Dependency Relation Knowledge into Transformer Block for Multi-turn Dialogue Generation
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Dialogue Generation, Relation

Because of the compositionality of natural language, syntactic structure is one of the key factors for semantic understanding. However, the Transformer block, which is widely used for obtaining the distributed representations of sentences in dialogue generation tasks, views sentences as a sequence of words and does not effectively learn the syntactic structure.  In this work, we explore how to effectively incorporate dependency relation knowledge that contains syntactic structure information into Transformer block and propose Dependency Relation Attention(DRA). Experimental results demonstrate that DRA can further improve the performance of state-of-the-art models for multi-turn dialogue generation.
Anticipation-free Training for Simultaneous Translation
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Hallucination, Sentence, Translation

Simultaneous translation (SimulMT) speeds up the translation process by starting to translate before the source sentence is completely available. It is difficult due to limited context and word order difference between languages. Existing methods increase latency or introduce adaptive read-write policies for SimulMT models to handle local reordering and improve translation quality. However, the long-distance reordering would make the SimulMT models learn translation mistakenly. Specifically, the model may be forced to predict target tokens when the corresponding source tokens have not been read. This leads to aggressive anticipation during inference, resulting in the hallucination phenomenon. To mitigate this problem, we propose a new framework that decompose the translation process into the monotonic translation step and the reordering step, and we model the latter by the auxiliary sorting network (ASN). The ASN rearranges the hidden states to match the order in the target language, so that the SimulMT model could learn to translate more reasonably. The entire model is optimized end-to-end and does not rely on external aligners or data. During inference, ASN is removed to achieve streaming. Experiments show the proposed framework could outperform previous methods with less latency.
Explaining Ranking Models using Multiple Explainers
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12

Current approaches to interpreting complex ranking models are based on local approximations of the ranking model using a simple ranker in the locality of the query. Since rankings have multiple relevance factors and are aggregations of predictions,  existing approaches that use a single ranker might not be sufficient to approximate a complex model resulting in low local fidelity. In this paper, we overcome this problem by considering multiple simple rankers for better approximating the black box ranking model.  We pose the problem of local approximation as a Generalized Preference Coverage (GPC) problem that incorporates multiple simple rankers towards the post-hoc interpretability of ranking models. Our approach Multiplex uses a linear programming approach to judiciously extract the explanation terms. We conduct extensive experiments on a variety of ranking models and report fidelity improvements of $37\% - 54\%$ over existing baselines and competitors. We finally qualitatively compare modern neural ranking models in terms of their explanations to better understand the differences between them, showcasing our explainers' practical utility.
MOVER: Mask, Over-generate and Rank for Hyperbole Generation
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Sentence

Despite being a common figure of speech, hyperbole is under-researched in Figurative Language Processing. In this paper, we tackle the challenging task of hyperbole generation to transfer a literal sentence into its hyperbolic paraphrase. To address the lack of available hyperbolic sentences, we construct HYPO-XL, the first large-scale hyperbole corpus containing 17,862 hyperbolic sentences in a non-trivial way. Based on our corpus, we propose an unsupervised method for hyperbole generation that does not require parallel literal-hyperbole pairs. During training, we fine-tune BART to infill masked hyperbolic spans of sentences from HYPO-XL. During inference, we mask part of an input literal sentence and over-generate multiple possible hyperbolic versions. Then a BERT-based ranker selects the best candidate by hyperbolicity and paraphrase quality. Automatic and human evaluation results show that our model is effective at generating hyperbolic paraphrase sentences and outperforms several baseline systems.
A Survey on Stance Detection for Mis- and Disinformation Identification
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Fact Checking, Misinformation, Rumour Detection, Sentiment Analysis, Stance Detection

Understanding attitudes expressed in texts, also known as stance detection, plays an important role in systems for detecting false information online, be it misinformation (unintentionally false) or disinformation (intentionally false information). Stance detection has been framed in different ways, including (a) as a component of fact-checking, rumour detection, and detecting previously fact-checked claims, or (b) as a task in its own right. While there have been prior efforts to contrast stance detection with other related tasks such as argumentation mining and sentiment analysis, there is no existing survey on examining the relationship between stance detection and mis- and disinformation detection. Here, we aim to bridge this gap by reviewing and analysing existing work in this area, with mis- and disinformation in focus, and discussing lessons learnt and future challenges.
Joint Mitigation of Interactional Bias
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Word Embeddings

Machine learning algorithms have been found discriminative against groups of different social identities, e.g., gender and race. With the detrimental effects of these algorithmic biases, researchers proposed promising approaches for bias mitigation, typically designed for individual bias types. Due to the complex nature of social bias, we argue it is important to study how different biases interact with each other, i.e., how mitigating one bias type (e.g., gender) influences the bias results regarding other social identities (e.g., race and religion). We further question whether jointly debiasing multiple types of bias is desired in different contexts, e.g., when correlations between biases are different. To address these research questions, we examine bias mitigation in two NLP tasks -- toxicity detection and word embeddings -- on three social identities, i.e., race, gender, and religion. Empirical findings based on benchmark datasets suggest that different biases can be correlated and therefore, warranting attention for future research on joint bias mitigation.
LPC: A Logits and Parameter Calibration Framework on Continual Learning
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Continual Learning

Deep learning based pre-trained natural language processing (NLP) models typically pre-train on large unlabeled corpora first, then fine-tune on new tasks. When we execute such a paradigm on continuously sequential tasks, the model will suffer from the catastrophic forgetting problem (i.e., they forget the parameters learned in previous tasks when we train the model on newly emerged tasks). Inspired by the idea of how humans learn things, we aim to maintain the old knowledge when we transfer to novel contents and calibrate the old and new knowledge. We propose a Logits and Parameter Calibration (LPC) framework to reduce the catastrophic forgetting in the continual learning process. The proposed framework includes two important components, the Logits Calibration (LC) and Parameter Calibration (PC). The core idea is to reduce the difference between old knowledge and new knowledge by doing calibration on logits and parameters so that the model can maintain old knowledge while learning new tasks without preserving data in previous tasks. First, we preserve the parameters learned from the base tasks. Second, we train the existing model on novel tasks and estimate the difference between base logits and parameters and novel logits and parameters.  Third, we drift from the base tasks to novel tasks gradually. Furthermore, we integrate the logtis and parameter calibration into a brand-new optimization algorithm. Finally, we do experiments on 7 scenarios of the GLUE (the General Language Understanding Evaluation) benchmark. The experimental results show that our model achieves state-of-the-art performance on all 7 scenarios.
PaCo: Preconditions Attributed to Commonsense Knowledge
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12

Humans can seamlessly reason with circumstantial preconditions of commonsense knowledge. We understand that "a glass is used for drinking water", unless "the glass is broken" or "the water is toxic". Despite state-of-the-art(SOTA) language models’ (LMs) impressive performance on inferring commonsense knowledge, it is unclear whether they understand the circumstantial preconditions. To address this gap, we propose a novel challenge of reasoning with circumstantial preconditions. We collect a dataset, called PaCo, consisting of 12.4 thousand preconditions of commonsense statements expressed in natural language. Based on this dataset, we create three canonical evaluation tasks and use them to examine the capability of existing LMs to understand situational preconditions. Our results reveal a 10-30% gap between machine and human performance on our tasks, which shows that reasoning with preconditions is an open challenge. Upon acceptance, we will release the dataset and the code used to test models.
Understanding Attention for Vision-and-Language Tasks
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Image Generation, Image Retrieval, Question Answering, Retrieval, Sentence, Text to Image Generation, Text-to-Image Generation, Visual Question Answering, Visual Question Answering (VQA)

Attention mechanism has been used as an important component across Vision-and-Language(VL) tasks in order to bridge the semantic gap between visual and textual features. While attention has been widely used in VL tasks, it has not been examined the capability of different attention alignment calculation in bridging the semantic gap between visual and textual clues. In this research, we conduct a comprehensive analysis on understanding the role of attention alignment by looking into the attention score calculation methods and check how it actually represents the visual region's and textual token's significance for the global assessment. We also analyse the conditions which attention score calculation mechanism would be more (or less) interpretable, and which may impact the model performance on three different VL tasks, including visual question answering, text-to-image generation, text-and-image matching (both sentence and image retrieval). Our analysis is the first of its kind and provides useful insights of the importance of each attention alignment score calculation when applied at the training phase of VL tasks, commonly ignored in attention-based cross modal models, and/or pretrained models.
RoViST: Learning Robust Metrics for Visual Storytelling
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Sentence, Text Generation, Visual Grounding, Visual Storytelling

Visual storytelling (VST) is the task of generating a story paragraph that describes a given image sequence. Most existing storytelling approaches have evaluated their models using traditional natural language generation metrics like BLEU or CIDEr. However, such metrics based on $n$-gram matching tend to have poor correlation with human evaluation scores and do not explicitly consider other criteria necessary for storytelling such as sentence structure or topic coherence. Moreover, a single score is not enough to assess a story as it does not inform us about what specific errors were made by the model. In this paper, we propose 3 evaluation metrics sets that analyses which aspects we would look for in a good story: 1) visual grounding, 2) coherence, and 3) non-redundancy. We measure the reliability of our metric sets by analysing its correlation with human judgement scores on a sample of machine stories obtained from 4 state-of-the-arts models trained on the Visual Storytelling Dataset (VIST). Our metric sets outperforms other metrics on human correlation, and could be served as a learning based evaluation metric set that is complementary to existing rule-based metrics.
Your Answer is Incorrect... Would you like to know why? Introducing a Bilingual Short Answer Feedback Dataset
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: automatic short answer grading

Handing in a paper or exercise and merely receiving a "bad" or "incorrect" as feedback is not very helpful when the goal is to improve. Unfortunately, this is currently the kind of feedback given by many Automatic Short Answer Grading (ASAG) systems. One of the reasons for this is a lack of content-focused elaborated feedback datasets. To encourage research on explainable and understandable feedback systems, we present the Short Answer Feedback dataset (SAF). Similar to other ASAG datasets, SAF contains learner responses and reference answers to German and English questions. However, instead of only assigning a label or score to the learners' answers, SAF also contains elaborated feedback explaining the given score. Thus, SAF enables supervised training of models that grade answers and explain where and why mistakes were made. This paper discusses the need for enhanced feedback models in real-world pedagogical scenarios, describes the dataset annotation process, gives a comprehensive analysis of SAF, and demonstrates how SAF challenges T5 Transformer models.
Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Transfer Learning

Recent studies on the lottery ticket hypothesis (LTH) show that pre-trained language models (PLMs) like BERT contain matching subnetworks that have similar transfer learning performance as the original PLM. These subnetworks are found using magnitude-based pruning. In this paper, we find that the BERT subnetworks have even more potential than these studies have shown. Firstly, we discover that the success of magnitude pruning can be attributed to the preserved pre-training performance, which correlates with the downstream transferability. Inspired by this, we propose to directly optimize the subnetwork structure towards the pre-training objectives, which can better preserve the pre-training performance. Specifically, we train binary masks over model weights on the pre-training tasks, with the aim of preserving the universal transferability of the subnetwork, which is agnostic to any specific downstream tasks. We then fine-tune the subnetworks on the GLUE benchmark and the SQuAD dataset. The results show that, compared with magnitude pruning, mask training can effectively find BERT subnetworks with improved overall performance on downstream tasks. Moreover, our method is also more efficient in searching subnetworks and more advantageous when fine-tuning within a certain range of data scarcity. Our code will be released upon publication.
Few-Shot Self-Rationalization with Natural Language Prompts
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12

Self-rationalization models that predict task labels and generate free-text elaborations for their predictions could enable more intuitive interaction with NLP systems. These models are, however, currently trained with a large amount of human-written free-text explanations for each task which hinders their broader usage. We propose to study a more realistic setting of self-rationalization using few training examples. We present FEB---a standardized collection of four existing English-language datasets and associated metrics. We identify the right prompting approach by extensively exploring natural language prompts on FEB. Then, by using this prompt and scaling the model size, we demonstrate that making progress on few-shot self-rationalization is possible. We show there is still ample room for improvement in this task: the average plausibility of generated explanations assessed by human annotators is at most 51\%, while plausibility of human explanations is 76\%. We hope that FEB and our proposed approach will spur the community to take on the few-shot self-rationalization challenge.
Improving Tokenisation by Alternative Treatment of Spaces
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Natural Language Understanding

Tokenisation is the first step in almost all NLP tasks, and state-of-the-art transformer-based language models all use subword tokenisation algorithms to process input text. Existing algorithms have problems, often producing tokenisations of limited linguistic validity, and representing equivalent strings differently depending on their position within a word. We hypothesise that these problems hinder the ability of transformer-based models to handle complex words, and suggest that these problems are a result of allowing tokens to include spaces. We thus experiment with an alternative tokenisation approach where spaces are always treated as individual tokens, finding it alleviates existing problems, improving performance of models. Concretely, we apply a modification to the BPE and Unigram algorithms which implements this approach, and find it gives more morphologically correct tokenisations, in particular when handling prefixes. In addition, we show that the modified algorithms give improved performance on downstream NLP tasks that involve handling complex words, whilst having no detrimental effect on performance in general natural language understanding tasks. Given the results of our experiments, we advocate for always treating spaces as individual tokens as a superior tokenisation method.