pwc-archive/papers-with-abstracts huggingface dataset:
An Encoder Attribution Analysis for Dense Passage Retriever in Open-Domain Question Answering
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Open-Domain Question Answering, Question Answering, Retrieval

The bi-encoder design of dense passage retriever (DPR) is a key factor to its success in open-domain question answering (QA).However, it is unclear how DPR's question encoder and passage encoder individually contributes to the overall performance, which we refer to as the encoder attribution problem.The problem is important as it helps us isolate responsible factors for individual encoders to further improve overall performance.In this paper, we formulate our analysis under a probabilistic framework called encoder marginalization, where we quantify the contribution of a single encoder by marginalizing over other variables.We find that the passage encoder contributes more than the question encoder to the in-domain retrieval accuracy.We further use an example to demonstrate how to find the affecting factors for each encoder, where we train multiple DPR models with different amounts of data and use encoder marginalization to analyze the results.We find that the positive passage overlap and corpus coverage of training data have big impacts on the passage encoder, while the question encoder is mainly affected by training sample complexity under this setting.Based on this framework, we can devise data-efficient training regimes: for example, we manage to train a  passage encoder on SQuAD using 60\% less training data without loss of accuracy.These results illustrate the utility of our encoder attribution analysis.
Translated Texts Under the Lens: From Machine Translation Detection to Source Language Identification
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Language Identification, Machine Translation, Translation

In this work, we tackle the problem of the detection of translated texts from different angles. On top of addressing the classic task of machine translation detection, we investigate and find the presence of common patterns across different machine translation systems as well as different source languages. Then, we show that it is possible to identify the translation systems used to produce a translated text (F1-score $88.5\%$) as well as the source language of the original text (F1-score $79\%$).We assess our tasks using Books, a new dataset we built from scratch based on excerpts of novels and the well-known Europarl dataset.
Match made by BERT? Towards Interpretable Paper-Reviewer Assignments in NLP
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Both scientific progress and individual researcher careers depend on the quality of peer review, which in turn depends on paper-reviewer matching. Surprisingly, this problem has been mostly approached simply as an automated recommendation problem, rather than as a matter where different stakeholders (authors, reviewers, area chairs) have accumulated experience worth taking into account. We present the results of the first survey of the NLP community, identifying common issues and perspectives on what factors should be considered in paper-reviewer matching. This study contributes actionable recommendations for improving future NLP conferences, and desiderata for interpretable peer review assignments.
Guiding Neural Story Generation with Reader Models
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Story Generation

Automated storytelling has long captured the attention of researchers for the ubiquity of narratives in everyday life. However, it is challenging to maintain coherence and stay on-topic toward a specific ending when generating narratives with neural language models. In this paper, we introduce Story generation with Reader Models (StoRM), a framework in which a reader model is used to reason about the story should progress.A reader model infers what a human reader believes about the concepts, entities, and relations about the fictional story world.We show how an explicit reader model represented as a knowledge graph affords story coherence and provides controllability in the form of achieving a given story world state goal.Experiments show that our model produces significantly more coherent and on-topic stories, outperforming baselines in dimensions including plot plausibility and staying on topic. Our system also outperforms outline-guided story generation baselines in composing given concepts without ordering.
Discriminative Models Can Still Outperform Generative Models in Aspect Based Sentiment Analysis
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Sentiment Analysis

Aspect-based Sentiment Analysis (ABSA) helps to explain customers' opinions towards products and services. In the past, ABSA models were discriminative, but more recently generative models have been used to generate aspects and polarities directly from text.  In contrast, discriminative models commonly first select aspects from the text, and then classify the aspect's polarity. Previous results showed that generative models outperform discriminative models on several English ABSA datasets. Here, we evaluate and contrast two state-of-the-art discriminative and generative models in several settings: cross-lingual, cross-domain, and cross-lingual and domain, to understand generalizability in settings other than English mono-lingual in-domain. Our more thorough evaluation shows that, contrary to previous studies, discriminative models can still outperform generative models in almost all settings.
Investigating the saliency of sentiment expressions in aspect-based sentiment analysis
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Sentiment Analysis

We examine the behaviour of an aspect-based sentiment classifier built by fine-tuning the English BERT base model on the SemEval 2016 English dataset. In a set of masking experiments, we examine the extent to which the tokens which express the sentiment towards the aspect are being used by the classifier. The enhanced performance of a classifier that only sees the relevant sentiment expressions  suggests that they are not being used to their full potential. Furthermore,  sentiment expressions which are not directly relevant to the aspect in focus also appear to be used. We then use a gradient-based  method to identify the most salient words. A comparison of these salient words, or rationales, with the  sentiment expressions reveals only a moderate level of agreement. Some disagreements are related to the fixed length of the rationales and the tendency of the rationales to contain content words related to the aspect itself.
Unsupervised Slot Schema Induction for Task-oriented Dialog
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Clustering, dialog state tracking, Response Generation

Carefully-designed schemas describing how to collect and annotate dialog corpora are a prerequisite towards building task-oriented dialog systems. In practical applications, manually designing schemas can be error-prone, laborious, iterative, and slow, especially when the schema is complicated. To alleviate this expensive and time consuming process, we propose an unsupervised approach for slot schema induction from unlabeled dialog corpora. Leveraging in-domain language models and unsupervised parsing structures, our data-driven approach extracts candidate slots without constraints, followed by coarse-to-fine clustering to induce slot types. We compare our method against several strong supervised baselines, and show significant performance improvement in slot schema induction on MultiWoz and SGD datasets. We also demonstrate the effectiveness of induced schemas on downstream applications including dialog state tracking and response generation.
Label-guided Data Augmentation for Prompt-based Few Shot Learners
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Data Augmentation, Few-Shot Learning, Few-Shot Text Classification, Natural Language Understanding, text-classification, Text Classification

Recent advances on large pre-trained language models (PLMs) lead impressive gains on many natural language understanding (NLU) tasks with task-specific fine-tuning. However, direct fine-tuning PLMs heavily rely on large amount of labeled instances, which are expensive and time-consuming to obtain. Prompt-based tuning on PLMs has proven valuable for few shot tasks. Existing works studying prompt-based tuning for few-shot NLU mainly focus on deriving proper label words with a verbalizer or generating prompt templates for eliciting semantics from PLMs. In addition, conventional data augmentation methods can enrich training data for improving few-shot learning, while ignoring the label semantics. It is promising to leverage the rich label semantics in label words for data augmentation to facilitate prompt-based tuning for the downstream NLU tasks. However, the work on this is rather limited. Therefore, we study a new problem of data augmentation for prompt-based few shot learners. We propose a novel label-guided data augmentation method PromptDA which exploits the enriched label semantic information for data augmentation. Experimental results on several few shot text classification tasks show that our proposed framework achieves superior performance by effectively leveraging label semantics and data augmentation in language understanding.
Context-Aware Query Rewriting for Improving Users' Search Experience on E-commerce Websites
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Graph Attention

E-commerce queries are often short and ambiguous. E-commerce query understanding often uses query rewriting to disambiguate user-input queries. While using e-commerce search tools, users tend to enter multiple searches, which we call context, before purchasing. These history searches contain contextual insights about users' true shopping intents. Therefore, modeling such contextual information is critical to a better query rewriting model.  However, existing query rewriting models ignore users' history behaviors and consider only the instant search query, which is often a short string offering limited information about the true shopping intent.We propose an end-to-end context-aware query rewriting model to bridge this gap, which takes the search context into account. Specifically, our model builds a session graph using the history search queries, their contained words, and auxiliary category information. We then employ a weighted graph attention mechanism that models cross-query relations and computes contextual information of the session. The model subsequently calculates session representations by combining the contextual information with the instant search query using an aggregation network. The session representations are then decoded to generate rewritten queries. Empirically, we demonstrate the superiority of our method to state-of-the-art approaches under various evaluation metrics. Our code and data will be publicly available.
Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Cross-Lingual Transfer, dialog state tracking, Retrieval

Research on (multi-domain) task-oriented dialog (TOD) has predominantly focused on the English language, primarily due to the shortage of robust TOD datasets in other languages, preventing the systematic investigation of cross-lingual transfer for this crucial NLP application area. In this work, we introduce Multi2WOZ, a new multilingual multi-domain TOD dataset, derived from the well-established English dataset MultiWOZ, that spans four typologically diverse languages: Chinese, German, Arabic, and Russian. In contrast to concurrent efforts, Multi2WOZ contains gold-standard dialogs in target languages that are directly comparable with development and test portions of the English dataset, enabling reliable and comparative estimates of cross-lingual transfer performance for TOD. This enables us to detect and explore crucial challenges for TOD cross-lingually. We then introduce a new framework for multilingual conversational specialization of pretrained language models (PrLMs) that aims to facilitate cross-lingual transfer for arbitrary downstream TOD tasks. Using such conversational PrLMs specialized for concrete target languages, we systematically benchmark a number of zero-shot and few-shot cross-lingual transfer approaches on two standard TOD tasks: Dialog State Tracking and Response Retrieval. Our experiments show that, in most setups, the best performance entails the combination of (i) conversational specialization in the target language and (ii) few-shot transfer for the concrete TOD task. Most importantly, we show that our conversational specialization in the target language allows for a much more sample-efficient few-shot transfer for downstream TOD tasks.
Controlling the Focus of Pretrained Language Generation Models
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Abstractive Text Summarization, Response Generation, Text Generation

The finetuning of pretrained transformer-based language generation models are typically conducted in an end-to-end manner, where the model learns to attend to relevant parts of the input by itself. However, there does not exist a mechanism to directly control the model's focus. This work aims to develop a control mechanism by which a user can select spans of context as "highlights'' for the model to focus on, and generate relevant output. To achieve this goal, we augment a pretrained model with trainable "focus vectors'' that are directly applied to the model's embeddings, while the model itself is kept fixed. These vectors, trained on automatic annotations derived from attribution methods, act as indicators for context importance. We test our approach on two core generation tasks: dialogue response generation and abstractive summarization. We also collect evaluation data where the highlight-generation pairs are annotated by humans. Our experiments show that the trained focus vectors are effective in steering the model to generate outputs that are relevant to user-selected highlights.
Investigating the Roots of Gender Bias in Machine Translation: Observations on Gender Transfer between French and English
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Machine Translation, Translation

This paper aims at identifying the inner mechanisms that make a  translation model choose a masculine rather than a feminine form, an  essential step to mitigate gender bias in MT. We conduct two series  of experiments using probing and comparing the predictions of  translation and a language models to show that i) gender information  is encoded in all decoder's and encoder's representations and ii)  the translation model does not need to use information from the  source to predict his.
Validated Image Caption Rating (VICR) Scale, Dataset, and Model
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Assessing the quality of an image caption is a complex task. We propose a new image caption rating system that consists of (1) a robust rating scale that is consistent, teachable, and externally validated, (2) an engaging and scalable data generation approach for the task, (3) a high-quality dataset, and (4) an effective image caption rating predictor. Using contemporary approaches from psychometrics we demonstrate that the proposed scale and rater training routine can support high quality annotation efforts for the task. We introduce two new datasets (one original and another derived) for the task. Our reference-free and multi-level rating predictor performance is on par with state-of-the-art approaches.
Towards Multi-Turn Empathetic Dialogs with Positive Emotion Elicitation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Emotional support is a crucial skill for many real-world scenarios, including caring for the elderly, mental health support, and customer service chats. This paper presents a novel task of empathetic dialog generation with positive emotion elicitation to promote users' positive emotion, similar to that of emotional support between humans. In this task, the agent conducts empathetic responses along with the target of eliciting the user's positive emotions in the multi-turn dialog. To facilitate the study of this task, we collect a large-scale emotional dialog dataset with positive emotion elicitation, called PosEmoDial (about 820k dialogs, 3M utterances). In these dialogs, the agent tries to guide the user from any possible initial emotional state, e.g., sadness, to a positive emotional state. Then we present a positive-emotion-guided dialog generation model with a novel loss function design. This loss function encourages the dialog model to not only elicit positive emotions from users but also ensure smooth emotional transitions along with the whole dialog. Finally, we establish benchmark results on PosEmoDial, and we will release this dataset and related source code to facilitate future studies.
Towards Policy-Guided Conversational Recommendation with Dialogue Acts
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Conversational Recommendation, Recommendation Systems, Response Generation

Conversation Recommender System (CRS) aims to recommend items through nature conversation. Existing works in open-ended CRS mainly focus on recommendation and generation, but lacks of control over dialogue policy. In addition, the system is unable to adapt user profile to the user's feedback. Thus, we present a new dataset named DA-ReDial (Recommendation through Dialogue guided by Dialogue Act). We  summarize 10 representative Dialog Acts and label dialogue with the DAs schema. To solve the problems above, we also propose a novel CRS called PGCR which stands for Policy-Guided Conversational Recommendation. It is able to formulate a DA-aware user profile, leverage Dialogue Acts to explicitly model the discourse structure of conversation and better guide the response generation. Extensive experiments on the new dataset show that our proposed model outperforms most baselines in dialog generation and recommendation. Also, the Policy Network fine-tuned by self-play can better control the dialogue policy and contribute a lot to recommendation strategy and user engagement in conversation.
A Study of the Attention Abnormality in Trojaned BERTs
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Trojan attacks raise serious security concerns. In this paper, we investigate the underlying mechanism of Trojaned BERT models. We observe the attention focus drifting behavior of Trojaned models, i.e., when encountering an poisoned input, the trigger token hijacks the attention focus regardless of the context. We provide a thorough qualitative and quantitative analysis of this phenomenon, revealing insights into the Trojan mechanism. Based on the observation, we propose an attention-based Trojan detector to distinguish Trojaned models from clean ones. To the best of our knowledge, we are the first to analyze the Trojan mechanism and develop a Trojan detector based on the transformer's attention.
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Knowledge Distillation, Mixture-of-Experts, Natural Language Understanding, Question Answering

Pre-trained language models have demonstrated superior performance in various natural language processing tasks. However, these models usually contain hundreds of millions of parameters, which limits their practicality because of latency requirements in real-world applications. Existing methods train small compressed models via knowledge distillation. However, performance of these small models drops significantly compared with the pre-trained models due to their reduced model capacity. We propose MoEBERT, which uses a Mixture-of-Experts structure to increase model capacity and inference speed. We initialize MoEBERT by adapting the feed-forward neural networks in a pre-trained model into multiple experts. As such, representation power of the pre-trained model is largely retained. During inference, only one of the experts is activated, such that speed can be improved. We also propose a layer-wise distillation method to train MoEBERT. We validate the efficiency and efficacy of MoEBERT on natural language understanding and question answering tasks. Results show that the proposed method outperforms existing task-specific distillation algorithms. For example, our method outperforms previous approaches by over $2\%$ on the MNLI (mismatched) dataset. Our code will be publicly available.
Embedding-Enhanced GIZA++: Improving Low-Resource Word Alignment Using Embeddings
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: de-en, Machine Translation, Translation, Word Alignment

Word alignment has been dominated until recently by GIZA++, a statistical method based on the 30-year-old IBM models. New methods primarily rely on large machine translation models, massively multilingual language models, or supervision.  We introduce Embedding-Enhanced GIZA++, and outperform GIZA++ without any of the aforementioned factors.  Taking advantage of monolingual embedding spaces of source and target language only, we exceed GIZA++'s performance in every tested scenario for three languages pairs.  In the lowest-resource setting, we outperform GIZA++ by 8.5, 10.9, and 12 AER for Ro-En, De-En, and En-Fr, respectively. We release our code at www.blind-review.code.
A Weak supervision with Syntactic Cues for Reference Resolution
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

In recipes, contextual understanding of instructions depends on temporal interpretation of the entities because of their spatio-temporal changes. Accordingly, we propose the use of reference resolution to find the origin action of entities, provided that the entity is an output from a previous action, instead of being a raw ingredient. Here, we introduce a weak supervision method that supports the syntactic features for producing latent links between entities and their origin actions. The results show that our weak supervision outperforms the previous unsupervised studies with \%8 F1. In particular, our approach indicates \%82 resolution performance for pronouns and \%85 for null entities.