pwc-archive/papers-with-abstracts huggingface dataset:
You Sound Like Someone Who Watches Drama Movies: Towards Predicting Movie Preferences from Conversational Interactions
Authors: Sergey Volokhin, Joyce Ho, Oleg Rokhlenko, Eugene Agichtein
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Collaborative Filtering, Conversational Recommendation, Domain Adaptation, Movie Recommendation

The increasing popularity of voice-based personal assistants provides new opportunities for conversational recommendation. One particularly interesting area is movie recommendation, which can benefit from an open-ended interaction with the user, through a natural conversation. We explore one promising direction for conversational recommendation: mapping a conversational user, for whom there is limited or no data available, to most similar external reviewers, whose preferences are known, by representing the conversation as a user{'}s interest vector, and adapting collaborative filtering techniques to estimate the current user{'}s preferences for new movies. We call our proposed method ConvExtr (Conversational Collaborative Filtering using External Data), which 1) infers a user{'}s sentiment towards an entity from the conversation context, and 2) transforms the ratings of {``}similar{''} external reviewers to predict the current user{'}s preferences. We implement these steps by adapting contextual sentiment prediction techniques, and domain adaptation, respectively. To evaluate our method, we develop and make available a finely annotated dataset of movie recommendation conversations, which we call MovieSent. Our results demonstrate that ConvExtr can improve the accuracy of predicting users{'} ratings for new movies by exploiting conversation content and external data.
Semi-Supervised Policy Initialization for Playing Games with Language Hints
Authors: Tsu-Jui Fu, William Yang Wang
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4

Using natural language as a hint can supply an additional reward for playing sparse-reward games. Achieving a goal should involve several different hints, while the given hints are usually incomplete. Those unmentioned latent hints still rely on the sparse reward signal, and make the learning process difficult. In this paper, we propose semi-supervised initialization (SSI) that allows the agent to learn from various possible hints before training under different tasks. Experiments show that SSI not only helps to learn faster (\textbf{1.2x}) but also has a higher success rate (\textbf{11{\%}} relative improvement) of the final policy.
Negative language transfer in learner English: A new dataset
Authors: Leticia Farias Wanderley, Nicole Zhao, Carrie Demmans Epp
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Grammatical Error Correction, Language Acquisition

Automatic personalized corrective feedback can help language learners from different backgrounds better acquire a new language. This paper introduces a learner English dataset in which learner errors are accompanied by information about possible error sources. This dataset contains manually annotated error causes for learner writing errors. These causes tie learner mistakes to structures from their first languages, when the rules in English and in the first language diverge. This new dataset will enable second language acquisition researchers to computationally analyze a large quantity of learner errors that are related to language transfer from the learners{'} first language. The dataset can also be applied in personalizing grammatical error correction systems according to the learners{'} first language and in providing feedback that is informed by the cause of an error.
SentSim: Crosslingual Semantic Evaluation of Machine Translation
Authors: Yurun Song, Junchen Zhao, Lucia Specia
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Machine Translation, Semantic Similarity, Semantic Textual Similarity, Sentence, Translation

Machine translation (MT) is currently evaluated in one of two ways: in a monolingual fashion, by comparison with the system output to one or more human reference translations, or in a trained crosslingual fashion, by building a supervised model to predict quality scores from human-labeled data. In this paper, we propose a more cost-effective, yet well performing unsupervised alternative SentSim: relying on strong pretrained multilingual word and sentence representations, we directly compare the source with the machine translated sentence, thus avoiding the need for both reference translations and labelled training data. The metric builds on state-of-the-art embedding-based approaches {--} namely BERTScore and Word Mover{'}s Distance {--} by incorporating a notion of sentence semantic similarity. By doing so, it achieves better correlation with human scores on different datasets. We show that it outperforms these and other metrics in the standard monolingual setting (MT-reference translation), a well as in the source-MT bilingual setting, where it performs on par with glass-box approaches to quality estimation that rely on MT model information.
Ol\'a, Bonjour, Salve! XFORMAL: A Benchmark for Multilingual Formality Style Transfer
Authors: Eleftheria Briakou, Di Lu, Ke Zhang, Joel Tetreault
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Formality Style Transfer, Style Transfer

We take the first step towards multilingual style transfer by creating and releasing XFORMAL, a benchmark of multiple formal reformulations of informal text in Brazilian Portuguese, French, and Italian. Results on XFORMAL suggest that state-of-the-art style transfer approaches perform close to simple baselines, indicating that style transfer is even more challenging when moving multilingual.
Grouping Words with Semantic Diversity
Authors: Karine Chubarian, Abdul Rafae Khan, Anastasios Sidiropoulos, Jia Xu
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Diversity, Machine Translation, POS, POS Tagging, Translation

Deep Learning-based NLP systems can be sensitive to unseen tokens and hard to learn with high-dimensional inputs, which critically hinder learning generalization. We introduce an approach by grouping input words based on their semantic diversity to simplify input language representation with low ambiguity. Since the semantically diverse words reside in different contexts, we are able to substitute words with their groups and still distinguish word meanings relying on their contexts. We design several algorithms that compute diverse groupings based on random sampling, geometric distances, and entropy maximization, and we prove formal guarantees for the entropy-based algorithms. Experimental results show that our methods generalize NLP models and demonstrate enhanced accuracy on POS tagging and LM tasks and significant improvements on medium-scale machine translation tasks, up to +6.5 BLEU points. Our source code is available at https://github.com/abdulrafae/dg.
FlowPrior: Learning Expressive Priors for Latent Variable Sentence Models
Authors: Xiaoan Ding, Kevin Gimpel
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Language Modeling, Language Modelling, Sentence

Variational autoencoders (VAEs) are widely used for latent variable modeling of text. We focus on variations that learn expressive prior distributions over the latent variable. We find that existing training strategies are not effective for learning rich priors, so we propose adding the importance-sampled log marginal likelihood as a second term to the standard VAE objective to help when learning the prior. Doing so improves results for all priors evaluated, including a novel choice for sentence VAEs based on normalizing flows (NF). Priors parameterized with NF are no longer constrained to a specific distribution family, allowing a more flexible way to encode the data distribution. Our model, which we call FlowPrior, shows a substantial improvement in language modeling tasks compared to strong baselines. We demonstrate that FlowPrior learns an expressive prior with analysis and several forms of evaluation involving generation.
Supervised Neural Clustering via Latent Structured Output Learning: Application to Question Intents
Authors: Iryna Haponchyk, Alessandro Moschitti
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Clustering, Prediction, Representation Learning, Structured Prediction

Previous pre-neural work on structured prediction has produced very effective supervised clustering algorithms using linear classifiers, e.g., structured SVM or perceptron. However, these cannot exploit the representation learning ability of neural networks, which would make supervised clustering even more powerful, i.e., general clustering patterns can be learned automatically. In this paper, we design neural networks based on latent structured prediction loss and Transformer models to approach supervised clustering. We tested our methods on the task of automatically recreating categories of intents from publicly available question intent corpora. The results show that our approach delivers 95.65{\%} of F1, outperforming the state of the art by 17.24{\%}.
Knowledge-Driven Slot Constraints for Goal-Oriented Dialogue Systems
Authors: Piyawat Lertvittayakumjorn, Daniele Bonadiman, Saab Mansour
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Benchmarking, Goal-Oriented Dialogue Systems

In goal-oriented dialogue systems, users provide information through slot values to achieve specific goals. Practically, some combinations of slot values can be invalid according to external knowledge. For example, a combination of {``}cheese pizza{''} (a menu item) and {``}oreo cookies{''} (a topping) from an input utterance {``}Can I order a cheese pizza with oreo cookies on top?{''} exemplifies such invalid combinations according to the menu of a restaurant business. Traditional dialogue systems allow execution of validation rules as a post-processing step after slots have been filled which can lead to error accumulation. In this paper, we formalize knowledge-driven slot constraints and present a new task of constraint violation detection accompanied with benchmarking data. Then, we propose methods to integrate the external knowledge into the system and model constraint violation detection as an end-to-end classification task and compare it to the traditional rule-based pipeline approach. Experiments on two domains of the MultiDoGO dataset reveal challenges of constraint violation detection and sets the stage for future work and improvements.
Clipping Loops for Sample-Efficient Dialogue Policy Optimisation
Authors: Yen-chen Wu, Carl Edward Rasmussen
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4

Training dialogue agents requires a large number of interactions with users: agents have no idea about which responses are bad among a lengthy dialogue. In this paper, we propose loop-clipping policy optimisation (LCPO) to eliminate useless responses. LCPO consists of two stages: loop clipping and advantage clipping. In loop clipping, we clip off useless responses (called loops) from dialogue history (called trajectories). The clipped trajectories are more succinct than the original ones, and the estimation of state-value is more accurate. Second, in advantage clipping, we estimate and clip the advantages of useless responses and normal ones separately. The clipped advantage distinguish useless actions from others and reduce the probabilities of useless actions efficiently. In experiments on Cambridge Restaurant Dialogue System, LCPO uses only 260 training dialogues to achieve 80{\%} success rate, while PPO baseline requires 2160 dialogues. Besides, LCPO receives 3.7/5 scores in human evaluation where the agent interactively collects 100 real-user dialogues in training phase.
Integrating Lexical Information into Entity Neighbourhood Representations for Relation Prediction
Authors: Ian Wood, Mark Johnson, Stephen Wan
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Knowledge Graph Completion, Relation, Relation Extraction, Relation Prediction

Relation prediction informed from a combination of text corpora and curated knowledge bases, combining knowledge graph completion with relation extraction, is a relatively little studied task. A system that can perform this task has the ability to extend an arbitrary set of relational database tables with information extracted from a document corpus. OpenKi[1] addresses this task through extraction of named entities and predicates via OpenIE tools then learning relation embeddings from the resulting entity-relation graph for relation prediction, outperforming previous approaches. We present an extension of OpenKi that incorporates embeddings of text-based representations of the entities and the relations. We demonstrate that this results in a substantial performance increase over a system without this information.
Graph Convolutional Networks for Event Causality Identification with Rich Document-level Structures
Authors: Minh Tran Phu, Thien Huu Nguyen
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Event Causality Identification, Sentence

We study the problem of Event Causality Identification (ECI) to detect causal relation between event mention pairs in text. Although deep learning models have recently shown state-of-the-art performance for ECI, they are limited to the intra-sentence setting where event mention pairs are presented in the same sentences. This work addresses this issue by developing a novel deep learning model for document-level ECI (DECI) to accept inter-sentence event mention pairs. As such, we propose a graph-based model that constructs interaction graphs to capture relevant connections between important objects for DECI in input documents. Such interaction graphs are then consumed by graph convolutional networks to learn document context-augmented representations for causality prediction between events. Various information sources are introduced to enrich the interaction graphs for DECI, featuring discourse, syntax, and semantic information. Our extensive experiments show that the proposed model achieves state-of-the-art performance on two benchmark datasets.
Choose Your Own Adventure: Paired Suggestions in Collaborative Writing for Evaluating Story Generation Models
Authors: Elizabeth Clark, Noah A. Smith
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Story Generation

Story generation is an open-ended and subjective task, which poses a challenge for evaluating story generation models. We present Choose Your Own Adventure, a collaborative writing setup for pairwise model evaluation. Two models generate suggestions to people as they write a short story; we ask writers to choose one of the two suggestions, and we observe which model{'}s suggestions they prefer. The setup also allows further analysis based on the revisions people make to the suggestions. We show that these measures, combined with automatic metrics, provide an informative picture of the models{'} performance, both in cases where the differences in generation methods are small (nucleus vs. top-k sampling) and large (GPT2 vs. Fusion models).
Context-Interactive Pre-Training for Document Machine Translation
Authors: Pengcheng Yang, Pei Zhang, Boxing Chen, Jun Xie, Weihua Luo
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Machine Translation, Sentence, Translation

Document machine translation aims to translate the source sentence into the target language in the presence of additional contextual information. However, it typically suffers from a lack of doc-level bilingual data. To remedy this, here we propose a simple yet effective context-interactive pre-training approach, which targets benefiting from external large-scale corpora. The proposed model performs inter sentence generation to capture the cross-sentence dependency within the target document, and cross sentence translation to make better use of valuable contextual information. Comprehensive experiments illustrate that our approach can achieve state-of-the-art performance on three benchmark datasets, which significantly outperforms a variety of baselines.
Cross-lingual Cross-modal Pretraining for Multimodal Retrieval
Authors: Hongliang Fei, Tan Yu, Ping Li
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Cross-Modal Retrieval, Machine Translation, Retrieval, Translation

Recent pretrained vision-language models have achieved impressive performance on cross-modal retrieval tasks in English. Their success, however, heavily depends on the availability of many annotated image-caption datasets for pretraining, where the texts are not necessarily in English. Although we can utilize machine translation (MT) tools to translate non-English text to English, the performance still largely relies on MT{'}s quality and may suffer from high latency problems in real-world applications. This paper proposes a new approach to learn cross-lingual cross-modal representations for matching images and their relevant captions in multiple languages. We seamlessly combine cross-lingual pretraining objectives and cross-modal pretraining objectives in a unified framework to learn image and text in a joint embedding space from available English image-caption data, monolingual and parallel corpus. We show that our approach achieves SOTA performance in retrieval tasks on two multimodal multilingual image caption benchmarks: Multi30k with German captions and MSCOCO with Japanese captions.
Wikipedia Entities as Rendezvous across Languages: Grounding Multilingual Language Models by Predicting Wikipedia Hyperlinks
Authors: Iacer Calixto, Alessandro Raganato, Tommaso Pasini
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Articles, Knowledge Graphs

Masked language models have quickly become the de facto standard when processing text. Recently, several approaches have been proposed to further enrich word representations with external knowledge sources such as knowledge graphs. However, these models are devised and evaluated in a monolingual setting only. In this work, we propose a language-independent entity prediction task as an intermediate training procedure to ground word representations on entity semantics and bridge the gap across different languages by means of a shared vocabulary of entities. We show that our approach effectively injects new lexical-semantic knowledge into neural models, improving their performance on different semantic tasks in the zero-shot crosslingual setting. As an additional advantage, our intermediate training does not require any supplementary input, allowing our models to be applied to new datasets right away. In our experiments, we use Wikipedia articles in up to 100 languages and already observe consistent gains compared to strong baselines when predicting entities using only the English Wikipedia. Further adding extra languages lead to improvements in most tasks up to a certain point, but overall we found it non-trivial to scale improvements in model transferability by training on ever increasing amounts of Wikipedia languages.
Adaptable and Interpretable Neural MemoryOver Symbolic Knowledge
Authors: Pat Verga, Haitian Sun, Livio Baldini Soares, William Cohen
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Question Answering

Past research has demonstrated that large neural language models (LMs) encode surprising amounts of factual information: however, augmenting or modifying this information requires modifying a corpus and retraining, which is computationally expensive. To address this problem, we develop a neural LM that includes an interpretable neuro-symbolic KB in the form of a {``}fact memory{''}. Each element of the fact memory is formed from a triple of vectors, where each vector corresponds to a KB entity or relation. Our LM improves performance on knowledge-intensive question-answering tasks, sometimes dramatically, including a 27 point increase in one setting of WebQuestionsSP over a state-of-the-art open-book model, despite using 5{\%} of the parameters. Most interestingly, we demonstrate that the model can be modified, without \textit{any} re-training, by updating the fact memory.
CLEVR\_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Images
Authors: Shailaja Keyur Sampat, Akshay Kumar, Yezhou Yang, Chitta Baral
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Most existing research on visual question answering (VQA) is limited to information explicitly present in an image or a video. In this paper, we take visual understanding to a higher level where systems are challenged to answer questions that involve mentally simulating the hypothetical consequences of performing specific actions in a given scenario. Towards that end, we formulate a vision-language question answering task based on the CLEVR (Johnson et. al., 2017) dataset. We then modify the best existing VQA methods and propose baseline solvers for this task. Finally, we motivate the development of better vision-language models by providing insights about the capability of diverse architectures to perform joint reasoning over image-text modality. Our dataset setup scripts and codes will be made publicly available at https://github.com/shailaja183/clevr{\_}hyp.
QuadrupletBERT: An Efficient Model For Embedding-Based Large-Scale Retrieval
Authors: Peiyang Liu, Sen Wang, Xi Wang, Wei Ye, Shikun Zhang
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Information Retrieval, Retrieval

The embedding-based large-scale query-document retrieval problem is a hot topic in the information retrieval (IR) field. Considering that pre-trained language models like BERT have achieved great success in a wide variety of NLP tasks, we present a QuadrupletBERT model for effective and efficient retrieval in this paper. Unlike most existing BERT-style retrieval models, which only focus on the ranking phase in retrieval systems, our model makes considerable improvements to the retrieval phase and leverages the distances between simple negative and hard negative instances to obtaining better embeddings. Experimental results demonstrate that our QuadrupletBERT achieves state-of-the-art results in embedding-based large-scale retrieval tasks.
Dynamically Disentangling Social Bias from Task-Oriented Representations with Adversarial Attack
Authors: LiWen Wang, Yuanmeng Yan, Keqing He, Yanan Wu, Weiran Xu
Date: 2021-06-01 00:00:00
Proceeding: NAACL 2021 4
Tasks: Adversarial Attack, Representation Learning

Representation learning is widely used in NLP for a vast range of tasks. However, representations derived from text corpora often reflect social biases. This phenomenon is pervasive and consistent across different neural models, causing serious concern. Previous methods mostly rely on a pre-specified, user-provided direction or suffer from unstable training. In this paper, we propose an adversarial disentangled debiasing model to dynamically decouple social bias attributes from the intermediate representations trained on the main task. We aim to denoise bias information while training on the downstream task, rather than completely remove social bias and pursue static unbiased representations. Experiments show the effectiveness of our method, both on the effect of debiasing and the main task performance.