pwc-archive/papers-with-abstracts huggingface dataset:
Lacuna Reconstruction: Self-supervised Pre-training for Low-Resource Historical Document Transcription
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Language Modeling, Language Modelling

We present a self-supervised pre-training approach for learning rich visual language representations for both handwritten and printed historical document transcription. After supervised fine-tuning of our pre-trained encoder representations for low-resource document transcription on two languages, (1) a heterogeneous set of handwritten Islamicate manuscript images and (2) early modern English printed documents, we show a meaningful improvement in recognition accuracy over the same supervised model trained from scratch with as few as 30 line image transcriptions for training. Our masked language model-style pre-training strategy, where the model is trained to be able to identify the true masked visual representation from distractors sampled from within the same line, encourages learning robust contextualized language representations invariant to scribal writing style and printing noise present across documents.
Using Natural Sentence Prompts for Understanding Biases in Language Models
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Sentence

Evaluation of biases in language models is often limited to synthetically generated datasets. This dependence traces back to the need of prompt-style dataset to trigger specific behaviors of language models. In this paper, we address this gap by creating a prompt dataset with respect to occupations collected from real-world natural sentences present in Wikipedia.We aim to understand the differences between using template-based prompts and natural sentence prompts when studying gender-occupation biases in language models. We find bias evaluations are very sensitiveto the design choices of template prompts, and we propose using natural sentence prompts as a way of more systematically using real-world sentences to move away from design decisions that may bias the results.
ViL-Sum: Enhancing Vision and Language Representations via Multi-task Learning for Multi-modal Summarization
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Decoder, Multi-Task Learning

With the advance of multimedia on the Internet, multi-modal summarization has drawn much attention. Most current methods follow a pipeline strategy, where an off-the-shelf object detector is used to extract visual features which are then fused with language representations for decoder to generate. However, these methods suffer two issues 1) separate vision and language representations fail to capture the interrelations within the two modalities; 2) from the local view, the semantic alignments between images and paragraphs are missing. In order to address these problems, in this paper, we propose a novel Vision-Language Summarization (ViL-Sum) model with a multi-task learning framework. Specifically, we train our model with two auxiliary tasks in a multi-task manner, that are images selection and images reordering. In this way, the interrelations within image and text are well captured. Besides, to further enhance the vision-language representation, we employ a unified transformer-based encoder-decoder structure. The encoder simultaneously takes image and text as input and jointly learns the representations of both. Then the representations are used by the decoder to generate the summary. Experimental results show that ViL-Sum significantly outperforms current state-of-the-art methods. In further analysis, we find that the enhanced representations via multi-task training and joint modeling learn reasonable relations between image and text.
A Text-Image Pair Is not Enough: Language-Vision Relation Inference with Auxiliary Modality Translation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Cross-Modal Retrieval, image-classification, Image Classification, Relation, Retrieval, Sentence, Sentiment Analysis, Translation

The semantic relations between language and vision modalities become more and more vital since they can effectively facilitate the downstream multi-modal tasks, such as cross-modal retrieval, multi-modal sentiment analysis and entity recognition. Although several approaches have been proposed to handle language-vision relation inference (LVRI), they normally rely on the limited information of the posted single sentence and single image. In this paper, to extend the information width of the original input, we introduce a concept of modality translation with two potential directions to generate additional modalities, and propose the auxiliary modality translation framework (AMT) for LVRI. This approach can not only generate the additional image by translating original text, but also the additional text by translating original image. Moreover, towards the potential three or four modalities as input, we employ a unified layer-wise transformer structure to perform multi-modal interactions. Systematic experiments and extensive analysis demonstrate that our approach with auxiliary modality translation significantly outperforms conventional approaches of LVRI and several competitive baselines for other text-image classification tasks.
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Continual Learning

Language Models (LMs) become outdated as the world changes; they often fail to perform tasks requiring recent factual information which was absent or different during training, a phenomenon called temporal misalignment. This is especially a challenging problem because the research community still lacks a coherent dataset for assessing the adaptability of LMs to frequently-updated knowledge corpus such as Wikipedia. To this end, we introduce TemporalWiki, a lifelong benchmark for ever-evolving LMs that utilizes the difference between the consecutive snapshots of Wikipedia and Wikidata for training and evaluation, respectively. The benchmark hence allows one to periodically track an LM's ability to retain previous knowledge and acquire new or updated knowledge at each point in time. We also find that training an LM on the diff data with an adapter achieves similar or better perplexity than on the entire snapshot in our benchmark with 12 times less computational cost, which verifies that factual knowledge in LMs can be safely updated with minimal training data via continual learning. The dataset and the code will be available at this link.
Improving Conversational Recommendation Systems’ Quality with Context-Aware Item Meta-Information
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Conversational Recommendation, Knowledge Graphs, Language Modelling, Recommendation Systems, Response Generation

A key challenge of Conversational Recommendation Systems (CRS) is to integrate the recommendation function and the dialog generation function smoothly. Previous works employ graph  neural  networks  with  external  knowledge graphs (KG) to model individual recommendation items and integrate KGs with language models through attention mechanism for response generation.   Although previous approaches prove effective, there is still room for improvement. For example, KG-based approaches only rely on entity relations and bag-of-words to recommend items and neglect the information in the conversational context. We propose to improve the usage of dialog context for  both  recommendation  and  response  generation using an encoding architecture along with the self-attention mechanism of transformers.   In this paper,  we propose a simple yet effective architecture comprising a pre-trained language model (PLM) and an item metadata encoder to integrate the recommendation and the dialog generation better. The proposed item encoder learns to map item metadata to embeddings reflecting the rich information of the item, which can be matched with dialog context. The  PLM  then  consumes  the  context-aware item embeddings and dialog context to generate high-quality recommendations and responses. Experimental results on the benchmark dataset ReDial show that our model obtains state-of-the-art results on both recommendation and response generation tasks.
A Holistic Framework for Analyzing the COVID-19 Vaccine Debate
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Sentiment Analysis

The Covid-19 pandemic has led to infodemic of low quality information leading to poor health decisions. Combating the outcomes of this infodemic is not only a question of identifying false claims, but also reasoning about the decisions individuals make.In this work we propose a holistic analysis framework connecting stance and reason analysis and fine-grained entity level moral sentiment analysis. We study how to model the dependencies between the different level of analysis and incorporate human insights into the learning process. Experiments show that our framework provides reliable predictions even in the low-supervision settings.
Fine-grained Location Extraction via Curriculum Learning
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Named Entity Recognition (NER) seeks to extract entity mentions from texts with predefined categories such as Person, Location. General domain NER datasets like CoNLL-2003 mostly annotate Location coarse-grained entities manner (e.g., a country or a city). However, many applications require to identify fine-grained locations from texts and map them precisely to geographic sites (e.g., a crossroad or a store). Therefore, we propose a new NER dataset HarveyNER with fine-grained locations annotated in tweets. This dataset presents unique challenges and characterizes many complex and long location mentions in informal descriptions. Considering Curriculum Learning can help a system better learn the hard samples, we adopt it and first design two heuristic curricula based on the characteristic difficulties of HarveyNER, and then propose a novel curriculum that takes the commonness of sample difficulty into consideration. Our curricula are simple yet effective and experimental results show that our methods can improve both the hard case and overall performance in HarveyNER over strong baselines without extra cost.
NewsEdits: A Dataset of News Article Revision Histories and a Novel Document-Level Reasoning Challenge
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Articles, Sentence

News article revision histories provide clues to narrative and factual evolution in news articles. To facilitate analysis of this evolution, we present the first publicly available dataset of news revision histories, NewsEdits. Our dataset is large-scale and multilingual; it contains 1.2 million articles with 4.6 million versions from over 22 English- and French-language newspaper sources based in three countries, spanning 15 years of coverage (2006-2021).We define article-level edit actions: Add, Delete, Edit and Move Sentence, and develop a high-accuracy extraction algorithm to identify these actions. To underscore the factual nature of many edit actions, we conduct analyses showing that added and deleted sentences are more likely to contain updating events, main content and quotes than unchanged sentences. Finally, to explore whether edit actions are predictable, we introduce three novel tasks aimed at predicting actions performed during version updates. We show that these tasks are challenging for large NLP models but are possible for expert humans. We hope this can spur research in narrative framing and help provide predictive tools for journalists chasing breaking news.
Database Search Results Disambiguation for Task-Oriented Dialog Systems
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Multi-Task Learning

As task-oriented dialog systems are becoming increasingly popular in our lives, more realistic tasks have been proposed and explored. However, new practical challenges arise. For instance, current dialog systems cannot effectively handle multiplesearch results when querying a database, due to the lack of such scenarios in existing public datasets. In this paper, we propose Database Search Result (DSR) Disambiguation, a novel task that focuses on disambiguating database search results, which enhances user experience by allowing them to choose from multiple options instead of just one. To study this task, we augment the popular task-oriented dialog datasets (MultiWOZ and SGD) with turns that resolve ambiguities by (a) synthetically generating turns through a pre-defined grammar, and (b) collecting human paraphrases for a subset. We find that training on our augmented dialog data improves the model's ability to deal with ambiguous scenarios, without sacrificing performance on unmodified turns. Furthermore, pre-fine tuning and multi-task learning help our model to improve performance on DSR-disambiguation even in the absence of in-domain data, suggesting that it can be learned as a universal dialog skill. Our data and code will be made publicly available.
Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Machine Translation, Style Transfer, Text Generation, Text Style Transfer, Unsupervised Text Style Transfer

Neural text generation models are typically trained by maximizing log-likelihood with the sequence cross entropy (CE) loss, which encourages an exact token-by-token match between a target sequence with a generated sequence. Such training objective is sub-optimal when the target sequence is not perfect, e.g., when the target sequence is corrupted with noises, or when only weak sequence supervision is available. To address the challenge, we propose a novel Edit-Invariant Sequence Loss (EISL), which computes the matching loss of a target $n$-gram with all $n$-grams in the generated sequence. EISL is designed to be robust to various noises and edits in the target sequences. Moreover, the EISL computation is essentially an approximate convolution operation with target $n$-grams as kernels, which is easy to implement and efficient to compute with existing libraries. To demonstrate the effectiveness of EISL, we conduct experiments on a wide range of tasks, including machine translation with noisy target sequences, unsupervised text style transfer with only weak training signals, and non-autoregressive generation with non-predefined generation order. Experimental results show our method significantly outperforms the common CE loss and other strong baselines on all the tasks. EISL has a simple API that can be used as a drop-in replacement of the CE loss: https://anonymous.4open.science/r/EISLLoss.
"my stance decides my language": Modeling of Framing and Political Stance in News Media
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Articles, Language Modeling, Language Modelling

Framing is a political strategy in which journalists and politicians highlight certain aspects of an issue or a problem to influence public opinion. Frameworks for detecting framing in news articles or social media posts are necessary in order to understand the spread of biased information in our society. Prior research efforts have shown that their framework for framing detection works well by predicting political affiliation afterward. In this paper, rather than predicting stance after detecting frames, we incorporate stance prediction into a framing detection model to jointly capture framing languages better. We take advantage of political stance data, which are more readily available than framing data that require manual annotation of professionals, and propose automatic framing detection models, which can detect previously unseen framing phrases. We compare two different methods of incorporation and show that leveraging stance prediction improves the separation of liberal and conservative biased frame language.
Entity Cloze By Date: Understanding what LMs know about unseen entities
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Articles, Date Understanding

Language models (LMs) are typically trained once on a large-scale corpus and used for years without being updated. Our world, however, is dynamic, and new entities constantly arise. We propose a framework to analyze what LMs can infer about new entities that did not exist when the LMs were pretrained. We derive a dataset of entities indexed by their origination date and paired with their English Wikipedia articles, from which we can find sentences about each entity. We evaluate LMs' perplexity on masked spans within these sentences. We show that models more informed about the entities, such as those with access to a textual definition of them, achieve lower perplexity on this benchmark. Our experimental results demonstrate that making inferences about new entities remains difficult for LMs. Given its wide coverage on entity knowledge and temporal indexing, our dataset can be used to evaluate LMs and techniques designed to modify or extend their knowledge. Our automatic data collection pipeline can be easily used to continually update our benchmark.
An Empirical Study of Representation, Training and Decoding for Span-based Named Entity Recognition
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Representation Learning

Named Entity Recognition (NER) is an important task in Natural Language Processing with application in many domains. While the dominant paradigm of NER is sequence labelling, span-based approaches have become very popular in recent times, but are less well understood. In this work, we study different aspects of span-based NER, namely the span representation, learning strategy, and decoding algorithms to avoid span overlap. We also propose an exact algorithm that efficiently finds the set of non-overlapping spans that maximize a global score, given a list of candidate spans. We perform our study on three benchmarks NER datasets from different domains. The code and supporting files for the experiments will be made publicly available.
Synopses of Movie Narratives: a Video-Language Dataset for Story Understanding
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Retrieval, Text Retrieval, Video-Text Retrieval

Despite recent advances of AI, story understanding remains an open and under-investigated problem. We collect, preprocess, and publicly release a video-language story dataset,  Synopses of Movie Narratives(SyMoN), containing 5,193 video summaries of popular movies and TV series. SyMoN captures naturalistic storytelling videos for human audience made by human creators, and has higher story coverage and more frequent mental-state references than similar video-language story datasets. Differing from most existing video-text datasets, SyMoN features large semantic gaps between the visual and the textual modalities due to the prevalence of reporting bias and mental state descriptions. We establish benchmarks on video-text retrieval and zero-shot alignment on movie summary videos. With SyMoN, we hope to lay the groundwork for progress in multimodal story understanding.
TVShowGuess: Character Comprehension in Stories as Speaker Guessing
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

We propose a new task for assessing machines' skills of understanding fictional characters in narrative stories. The task, TVShowGuess, builds on the scripts of TV series and takes the form of guessing the anonymous main characters based on the backgrounds of the scenes and the dialogues. Our human study supports that this form of task covers comprehension of multiple types of character persona, including understanding characters' personalities, facts and memories of personal experience, which are well aligned with the psychological and literary theories about the theory of mind (ToM) of human beings on understanding fictional characters during reading. We further propose new model architectures to support the contextualized encoding of long scene texts. Experiments show that our proposed approaches significantly outperform baselines, yet still largely lag behind the (nearly perfect) human performance.Our work serves as a first step toward the goal of narrative character comprehension.
That is a good looking car !: Visual Aspect based Sentiment Controlled Personalized Response Generation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Decoder, Response Generation

In a conversational system, generating utterances that communicate consistent and relevant preferences is vital for more personalized conversations. In this paper, we propose a task of generating utterances grounded on some assigned aspect-preferences-profile. These aspect-preference profiles consist of a list of aspect-sentiment tuples, denoting the preference of the speaker for some aspect in the form of sentiment ("positive" or "negative"). Since no prior dataset containing such profiles is available, we enhance Image-Chat data by assigning these profiles to each user in a conversation. The conversations in this dataset are based on an image, therefore the aspects are present in images as well as dialogue history. We build a BERT and ResNet-based encoder-decoder model with a memory network to store preference-profile. Through our experiments, we show that our model can generate responses that convey the sentiment of relevant aspects in accordance with the assigned profile. Both automatic and manual evaluations show the effectiveness of our model and dataset. Our proposed system when using these profiles achieves a BLEU-1 score of 15.93 on this new task, which is an improvement of 2.92 points from the baseline experiment that does not use aspect-preference profiles.
MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Language Modeling, Language Modelling, Math, Math Word Problem Solving, Program Synthesis, Representation Learning

Math word problem (MWP) solving faces a dilemma in number representation learning. In order to avoid the number representation issue and reduce the search space of feasible solutions, existing works striving for MWP solving usually replace real numbers with symbolic placeholders to focus on logic reasoning. However, different from common symbolic reasoning tasks like program synthesis and knowledge graph reasoning, MWP solving has extra requirements in numerical reasoning. In other words, instead of the number value itself, it is the reusable numerical property that matters more in numerical reasoning. Therefore, we argue that injecting numerical properties into symbolic placeholders with contextualized representation learning schema can provide a way out of the dilemma in the number representation issue here. In this work, we introduce this idea to the popular pre-training language model (PLM) techniques and build MWP-BERT, an effective contextual number representation PLM. We demonstrate the effectiveness of our MWP-BERT on MWP solving and several MWP-specific understanding tasks on both English and Chinese benchmarks.