pwc-archive/papers-with-abstracts huggingface dataset:
Towards Coherent and Captivating Topic Transitions in Knowledge-Grounded Conversations
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Knowledge-grounded conversations require skillful usage of knowledge to generate suitably diverse responses to keep user captivated while maintaining coherence to the dialogue context. However, current approaches that directly match knowledge with dialog context can result in capturing spurious correlations between knowledge and context, leading to either incoherent or mundane topic transitions in the generated dialogs that fail to engage.In this work, we introduce the Coherent and Captivating Topic Transition (C2T2) method to select the appropriate knowledge to be used in next response,  resulting in topic transitions that are coherent to the ongoing conversations while providing adequate topic development for an engaging dialog.Our C2T2 employs transition-aware features designed to consider both historical contextual coherence as well as sequential topic development under a knowledge shifting constraint to select the next knowledge, thereby generating the response for an engaging conversation.We also designed a pointer network-based knowledge inference module to take into consideration of the relations among knowledge candidates during knowledge inference. Extensive experiments on two public benchmarks demonstrated the superiority of C2T2 on knowledge selection. Analysis on fine-grained knowledge selection accuracy also showed that C2T2 could better balance the topic adhesion and knowledge diversity in dialogs than existing approaches.
CLoCE:Contrastive Learning Optimize Continous Prompt Embedding Space in Relation Extraction
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Contrastive Learning, Relation, Relation Extraction

Recent studies have proved that prompt tuning can improve the performance of pre-trained language models (PLMs) on downstream tasks. However, in the task of relation extraction (RE), there are still a large number of confusing samples that hinder prompt-tuning method from achieving higher accuracy. Inspired by previous works, we innovatively utilize contrastive learning to solve this problem. We propose a prompt-tuning-based framework and apply contrastive learning to optimize the representation of input sentences in embedding space. At the same time, we design a more general template for RE task, and further use knowledge injection to improve performance of the model. Through extensive experiments on public datasets, the micro F1-score of our model exceeds the existing SOTA on the Re-TACRED and TACREV datasets by 0.5 and 1.0, respectively. Meanwhile, in the few-shot scenario, our model also has a more robust performance than fine-tune methods.
DialogueScore: Evaluating Responses in Task-Oriented Dialogue
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Natural Language Inference, Task-Oriented Dialogue Systems

Task-Oriented Dialogue systems have been widely deployed in real-world applications in the last few years.Yet, evaluations of task-oriented dialogue systems are relatively limited.The informative and success score only consider the key entities in the generated responses to judge whether the user's goal is achieved.On the other hand, the fluency metric (BLEU score) cannot measure the quality of the short responses properly since the golden responses could be diversified. To better explore the behavior and evaluate the generation ability of task-oriented dialogue systems, we explore the relation between user utterances and system responses and their follow-up utterances. Therefore, we design a scorer named \textbf{DialogueScore} based on the natural language inference task and synthesize negative data to train the scorer.Via performances of \textbf{DialogueScore}, we observe that the dialogue system fails to generate high-quality responses compared with the reference responses. Therefore, our proposed scorer could provide a new perspective for future dialogue system evaluation and construction.
Low Resource Style Transfer via Domain Adaptive Meta Learning
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: General Knowledge, Language Modeling, Language Modelling, Meta-Learning, Style Transfer, Text Style Transfer, Unsupervised Text Style Transfer

Text style transfer (TST) without parallel data has achieved some practical success.  However, most of the existing unsupervised text style transfer methods suffer from (i) requiring massive amounts of nonparallel data to guide transferring different text styles. (ii) colossal performance degradation when fine-tuning the model in new domains.  In this work, we propose DAML-ATM(Domain Adaptive Meta-Learning with Adversarial Transfer Model), which consists of two parts, DAML and ATM. DAML is a domain adaptive meta-learning approach to refine general knowledge in multi-heterogeneous source domains, capable of adapting to new unseen domains with a small amount of data. Moreover, we propose a new unsupervised TST approach Adversarial Transfer Model (ATM), composed of a sequence-to-sequence pre-trained language model and uses adversarial style training for better content preservation and style transfer. Results on multi-domain datasets demonstrate that our approach generalizes well on unseen low-resource domains, achieving state-of-the-art results against ten-strong baselines.
Task Formulation Matters When Learning Continuously: A Case Study in Visual Question Answering
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Continual Learning, Incremental Learning, Question Answering, Semantic Similarity, Semantic Textual Similarity, Visual Question Answering, Visual Question Answering (VQA)

Continual learning is a promising alternative to the current pretrain-and-finetune paradigm: It aims to learn a model on a sequence of tasks without forgetting knowledge from preceding tasks. We investigate continual learning for Visual Question Answering and show that performance highly depends on task design, order, and similarity - where tasks may be formulated according to either modality. Our results suggest that incremental learning of language reasoning skills (such as questions about color, count etc.) is more difficult than incrementally learning visual categories. We show that this difficulty is related to task similarity, where heterogeneous tasks lead to more severe forgetting. We also demonstrate that naive finetuning of pretrained models is insufficient, and recent continual learning approaches can reduce forgetting by more than 20%. We propose a simple yet effective Pseudo-Replay algorithm, which improves results while using less memory compared to standard replay. Finally, to measure gradual forgetting we introduce a new metric that takes into account the semantic similarity of predicted answers.
Auto-regressive Text Generation with Pre-Trained Language Models: An Empirical Study on Question-type Short Text Generation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Math, Text Generation

We present a multi-way parallel math word problem dataset, which covers English, Tamil and Sinhala. We employ this dataset in an empirical analysis of GPT-2, BART, and T5, as well as mT5 and mBART in auto-regressive text generation. Our findings show that BART and T5 perform noticeably better than GPT-2 for the considered task, and text generation with mBART50 and mT5 provides very promising results even for languages under-represented in these pre-trained models.
Investigating Math Word Problems using Pretrained Multilingual Language Models
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Machine Translation, Math, Pretrained Multilingual Language Models, Translation

In this paper, we revisit math word problems~(MWPs) from the {\em cross-lingual} and {\em multilingual} perspective.We construct our MWP solvers over pretrained multilingual language models using the sequence-to-sequence model with copy mechanism.We compare how the MWP solvers perform in cross-lingual and multilingual scenarios.To facilitate the comparison of cross-lingual performance, we first adapt the large-scale English dataset MathQA as a counterpart of the Chinese dataset Math23K.Then we extend several English datasets to bilingual datasets through machine translation plus human annotation.Our experiments show that the MWP solvers may not be transferred to a different language even if the target expressions share the same numerical constants and operator set.However, it can be better generalized if problem types exist on both source language and target language.
An Empirical Study on Cross-Lingual and Cross-Domain Transfer for Legal Judgment Prediction
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Cross-Lingual Transfer, Transfer Learning

Cross-lingual transfer learning has proven useful in a variety of NLP tasks, but it is understudied in the context of legal NLP, and not at all on n Legal Judgment Prediction (LJP). We explore transfer learning techniques on LJP using the trilingual Swiss-Judgment-Prediction (SJP) dataset, including cases written in three languages (German, French, Italian). We find that Cross-Lingual Transfer (CLT) improves the overall results across languages, especially when we augment the dataset with machine-translated versions of the original documents, using a $3\times$ larger training corpus. Further on, we perform an analysis exploring the effect of cross-domain and cross-regional transfer, i.e., train a model across domains (legal areas), or regions. We find that in both settings (legal areas, origin regions), models trained across all groups perform overall better, while they also have improved results in the worst-case scenarios. Finally, we report improved results when we ambitiously apply cross-jurisdiction transfer, where we augment our dataset with Indian legal cases originally written in English.
When More is not Necessary Better: Multilingual Auxiliary Tasks for Zero-Shot Cross-Lingual Transfer of Hate Speech Detection Models
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Cross-Lingual Transfer, Hate Speech Detection, Language Modeling, Language Modelling, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Sentiment Analysis, Transfer Learning, Zero-Shot Cross-Lingual Transfer

Zero-shot cross-lingual transfer learning has been shown to be highly challenging for tasks involving a lot of linguistic specificities or when a cultural gap is present between languages, such as in hate speech detection. In this paper, we highlight this limitation on several datasets and investigate how training on multilingual auxiliary tasks -- sentiment analysis, named entity recognition, and tasks relying on syntactic information -- impacts the zero-shot transfer of hate speech detection models across languages. We show the positive impact of these tasks, particularly named entity recognition, for bridging the gap between languages. Then, we present cases where the language model training data prevents hate speech detection models from benefiting from a knowledge proxy brought by auxiliary tasks fine-tuning. Our results warrant further investigation on how to best address cultural gap issues in resource-scarce scenario.
Weakly Supervised Turn-level Engagingness Evaluator for Dialogues
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

The standard approach to evaluating dialogue engagingness is by measuring Conversation Turns Per Session (CTPS), which implies that the dialogue length is the main predictor of the user engagement with a dialogue system. The main limitation of CTPS is that it can only be measured at the session level, i.e., once the dialogue is over. But a dialogue system has to continuously monitor user engagement throughout the dialogue session as well. Existing approaches to measuring turn-level engagingness require human annotations for training. We pioneer an alternative approach, Weakly Supervised Engagingness Evaluator (WeSEE), which uses the remaining depth (RD) for each turn as a heuristic weak label for engagingness. WeSEE does not require human annotations and also relates closely to CTPS, thus serving as a good learning proxy for this metric. We show that WeSEE achieves the new state-of-the-art results on the Fine-grained Evaluation of Dialog (FED) dataset (0.38 Spearman) and the DailyDialog dataset (0.62 Spearman).
Investigating Zero- and Few-shot Generalization in Fact Verification
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Fact Verification

We explore zero- and few-shot generalization for fact verification (FV), which aims to generalize the FV model trained on well-resourced domains (e.g., Wikipedia) to low-resourced domains that lack human annotations.  To this end, we first construct a benchmark dataset collection which contains 11 FV datasets representing 6 domains. We conduct an empirical analysis of generalization across these FV datasets, finding that current models generalize poorly. Our analysis reveals that several factors affect generalization, including dataset size, length of evidence, and the type of claims. Finally, we show that two directions of work improve generalization: 1) incorporating domain knowledge via pretraining on specialized domains, and 2) automatically generating training data via claim generation.
Schema Encoding for Transferable Dialogue State Tracking
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Dialogue State Tracking

Dialogue state tracking (DST) is an essential sub-task for task-oriented dialogue systems.Recent work has focused on deep neural models for DST.However, the neural models require a large dataset for training.Furthermore, applying them to another domain needs a new dataset because the neural models are trained to imitate the given dataset.In this paper, we propose Schema Encoding for Transferable Dialogue State Tracking (SET-DST), which is a neural DST method for effective transfer to new domains.Transferable DST could assist developments of dialogue systems even with few dataset on target domains.We use a schema encoder not just to imitate the dataset but to comprehend the schema of the dataset.We aim to transfer the model to new domains by encoding new schemas and using them for DST.As a result, SET-DST improved the accuracy by 1.46 points on MultiWOZ 2.1.
Measuring and Improving Semantic Diversity of Dialogue Generation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Dialogue Generation, Diversity

Response diversity has become an important criterion for evaluating the quality of open-domain dialogue generation models. However, current evaluation metrics for response diversity do not capture semantic diversity of generated responses, as they only consider lexical aspects of the responses. In this paper, we introduce a new automatic evaluation metric to measure the semantic diversity of generated responses. Through human evaluation, we demonstrate that our proposed metric highly correlates to human judgments on response diversity than existing lexical-level diversity metrics. Furthermore, motivated by the analysis of an existing dialogue dataset, we propose a simple yet effective learning method that improves the semantic diversity of generated responses through response re-weighting based on the semantic distribution of the training dataset. Through automatic and human evaluation, we show that our proposed learning method better improves both response diversity and coherency compared to other baseline methods.
What do tokens know about their characters and how do they know it?
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Pre-trained language models (PLMs) that use subword tokenization schemes can succeed at a variety of language tasks that require character-level information, despite lacking explicit access to the character composition of tokens. Here, studying a range of models (e.g., GPT-J, BERT, RoBERTa, GloVe), we probe what word pieces encode about character-level information by training classifier to predict the presence or absence of a particular alphabetical character in an English-language token, based on its embedding (e.g., probing whether the model embedding for "cat" encodes that it contains the character "a"). We find that these models robustly encode character-level information and, in general, larger models perform better at the task. Through a series of experiments and analyses, we investigate the mechanisms through which PLMs acquire character information during training and argue that this knowledge is acquired through multiple phenomena, including a systematic relationship between particular characters and particular parts of speech, as well as natural variability in the tokenization of related strings.
Meta Learning for Natural Language Processing: A Survey
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Meta-Learning, Survey

Deep learning has been the mainstream technique in the natural language processing (NLP) area. However, deep learning requires many labeled data and is less generalizable across domains. Meta-learning is an arising field in machine learning. It studies approaches to learning better learning algorithms and aims to improve algorithms in various aspects, including data efficiency and generalizability. The efficacy of meta-learning has been shown in many NLP tasks, but there is no systematic survey of these approaches in NLP, which hinders more researchers from joining the field. Our goal with this survey paper is to offer researchers pointers to relevant meta-learning works in NLP and attract more attention from the NLP community to drive future innovation. This paper first introduces the general concepts of meta-learning and the common approaches. Then we summarize task construction settings, applications of meta-learning for various NLP problems and review the development of meta-learning in the NLP community.
Event Detection via Derangement Reading Comprehension
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Event Detection, Machine Reading Comprehension, Reading Comprehension

Event detection (ED), aiming to detect events from texts and categorize them, is vital to understanding the actual happenings in real life.  Recently, ED without triggers has been proposed and gained benefits since it relieves the tedious effort of data labeling.  However, it still suffers from several formidable challenges: multi-label, insufficient clues, and imbalanced event types.  We, therefore, propose a novel Derangement mechanism on a machine Reading Comprehension (DRC) framework to tackle the above challenges.  More specially, we treat the input text as {\em Context} and concatenate it with all event types that are deemed as {\em Answers} with an omitted default question.  Thus, by appending input text and event types simultaneously, we can facilitate the power of self-attention in pre-trained language models, e.g., BERT, to absorb the semantic relation among them.  Moreover, we design a simple yet effective {\em derangement} mechanism to relieve the imbalanced training.  By introducing such perturbation mainly on major events, we can prohibit major events from excessive learning or implicitly under-sample the instances of the major events.  This yields a more balanced training to resolve the imbalanced learning issue.  The empirical results show that: (1) our proposed framework attains state-of-the-art performance over previous competitive models, and (2) by-product, our model can signify the connection of triggers and arguments to events for further analysis.
CSD: A Chinese Dataset for Subtext Problem
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Subtext is a kind of deep semantics which can be acquired after one or more rounds of expression transformation. As a popular way of expressing one's intentions, it is well worth studying. In this paper, we propose two subtext-related tasks which are termed ``subtext recognition'' and ``subtext recovery'' and make a clear definition for their purposes. Moreover, we build a Chinese dataset whose source data comes from popular social media (e.g. Weibo, Netease Music, Zhihu, and Bilibili) and propose a new evaluation metric termed ``Two-stages Annotation Evaluation'' (TAE) for the validation of a multi-turn annotation process.
Empathic Machines: Using Intermediate Features as Levers to Emulate Emotions in Text-To-Speech Systems
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: text-to-speech, Text to Speech

We present a method to control the emotional prosody of Text to Speech (TTS) systems by using phoneme-level intermediate features (pitch, energy, and duration) as levers. As a key idea, we propose Differential Scaling (DS) to disentangle features relating to affective prosody from those arising due to acoustics conditions and speaker identity. With thorough experimental studies, we show that the proposed method improves over the prior art in accurately emulating the desired emotions while retaining the naturalness of speech. We extend the traditional evaluation of using individual sentences for a more complete evaluation of HCI systems. We present a novel experimental setup by replacing an actor with a TTS system in offline and live conversations. The emotion to be rendered is either predicted or manually assigned. The results show that the proposed method is strongly preferred over the state-of-the-art TTS system and adds the much-coveted "human touch" in machine dialogue. Audio samples from our experiments are available at: https://emtts.github.io/tts-demo/
A Two-Stage Approach towards Generalization in Knowledge Base Question Answering
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Knowledge Base Question Answering, Knowledge Graphs, Question Answering, Semantic Parsing, Transfer Learning, Vocal Bursts Valence Prediction

Most existing approaches for Knowledge Base Question Answering (KBQA) focus on a specific underlying knowledge base either because of inherent assumptions in the approach, or because evaluating it on a different knowledge base requires non-trivial changes. However, many popular knowledge bases share similarities in their underlying schemas that can be leveraged to facilitate generalization across knowledge bases. To achieve this generalization, we introduce a KBQA framework based on a  2-stage architecture that explicitly separates semantic parsing from the knowledge base interaction, facilitating transfer learning across datasets and knowledge graphs. We show that pretraining on datasets with a different underlying knowledge base can nevertheless provide significant performance gains and reduce sample complexity. Our approach achieves comparable or state-of-the-art performance for LC-QuAD (DBpedia), WebQSP (Freebase), SimpleQuestions (Wikidata) and MetaQA (Wikimovies-KG).