pwc-archive/papers-with-abstracts huggingface dataset:
Augmenting Memory Networks for Rich and Efficient Retrieval in Grounded Dialogue
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Articles, Response Generation, Retrieval

Grounded dialogue consists of conditioning a conversation on additional latent inputs ("factoids") beyond the dialogue context, such as Wikipedia articles, IMDB reviews, persona, and images.  Due to a scarcity of <context, factoid> labels, it is common practice to jointly learn the knowledge-selection and grounded response generation tasks end-to-end. When conditioning the response on these factoids, previous work has either treated the factoids as a weighed average vector, or separately computed probabilities for each <context, factoid> pair. However, the former creates a bottleneck whilst the latter prevents factoids from being considered jointly. Our new method, PolyMemNet, learns a matrix representation of the context and factoids, allowing for multiple factoids to be jointly considered in response selection, without imposing a bottleneck. We show how this achieves up to a $17\%$ boost in knowledge-selection accuracy and $13\%$ in response-selection accuracy versus memory networks.
Generating Summaries for Scientific Paper Review
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

The review process is essential to ensure the quality of publications.  Recently, the increase of  submissions  for  top  venues  in  machine learning  and  NLP  has  caused  a  problem  of excessive  burden  on  reviewers  and  has  often caused  concerns  regarding  how  this  may  not only  overload  reviewers,  but  also  may  affect the quality of the reviews.  An automatic system  for  assisting  with  the  reviewing  process could be a solution for ameliorating the problem.   In  this  paper,  we  explore  automatic  review summary generation for scientific papers. We posit that neural language models have the potential to be valuable candidates for this task. In  order  to  test  this hypothesis,  we release  a new  dataset  of  scientific  papers  and  their  reviews, collected from papers published in the NeurIPS conference from 2013 to 2020.  We evaluate state of the art neural summarization models, present initial results on the feasibility of automatic review summary generation, and propose directions for the future
Strategies in subword tokenization: humans vs. algorithms
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Segmentation

The output of subword tokenization can be very different depending on what algorithm is used. It is typically judged as more or less plausible, depending on how much it corresponds to human intuition. A subword vocabulary overlap between manual and automatic segmentation is an indicator of plausibility, but it does not reveal much on how the process of segmentation compares with human analysis. In this study, we propose a new method to analyze subword segmentation strategies relying on a spatial analysis of the distribution of subwords' lengths. Our experiments on English, Finnish and Turkish show that humans tend to balance creativity and consistency, while algorithms tend to be either strongly biased or inconsistent. To imitate humans better, algorithms need to produce subword segments of moderately uneven length, which can be achieved by combining complementary strategies.
Local Structure Matters Most: Perturbation Study in NLU
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Natural Language Understanding, Position

Recent research analyzing the sensitivity of natural language understanding models to word-order perturbations has shown that neural models are surprisingly insensitive to the order of words.
In this paper, we investigate this phenomenon by developing order-altering perturbations on the order of words, subwords, and characters to analyze their effect on neural models' performance on language understanding tasks.
We experiment with measuring the impact of perturbations to the local neighborhood of characters and global position of characters in the perturbed texts and observe that perturbation functions found in prior literature only affect the global ordering while the local ordering remains relatively unperturbed.
We empirically show that neural models, invariant of their inductive biases, pretraining scheme, or the choice of tokenization, mostly rely on the local structure of text to build understanding and make limited use of the global structure.
Neural Dynamic Focused Topic Model
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: model, Topic Models, Variational Inference

Topic models and all their variants analyse text by learning meaningful representations through word co-occurrences. As pointed out by Williamson et al. (2010), such models implicitly assume that the probability of a topic to be active and its proportion within each document are positively correlated. This correlation can be strongly detrimental in the case of documents created over time, simply because recent documents are likely better described by new and hence rare topics.

In this work we leverage recent advances in neural variational inference and present an alternative neural approach to the Focused Topic Model and its dynamic extensions. Indeed, we develop a neural model for topic evolution which exploits a compound Bernoulli structure in order to track the appearances of topics, thereby decoupling their activities from their proportions.
On three different corpora namely, the UN general debates, the collection of NeurIPS papers, and the ACL Anthology dataset, our model outperforms  competing neural variational topic models.
Investigating the Use of BERT Anchors for Bilingual Lexicon Induction with Minimal Supervision
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Bilingual Lexicon Induction

This paper investigates the use of static anchors from transformer architectures for the task of Bilingual Lexicon Induction. We revisit an existing approach built around the ELMo architecture and explore the use of the methodology on the BERT family of language models. Experiments are performed and analysed for three language pairs, combining English with three target languages from very different language families, Hindi, Dutch, and Russian. Although the contextualised approach is not able to outperform the SOTA VecMap method, we find that it is easily adaptable to newer transformer models and can compete with the MUSE approach. An error analysis reveals interesting trends accross languages and shows how the method could be further improved by building on the basic hypothesis that transformer embeddings can indeed be decomposed into a static anchor and a dynamic context component. We make the code, the extracted anchors (before and after alignement) and the modified train and test sets available for use.
Neural Pipeline for Zero-Shot Data-to-Text Generation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Data-to-Text Generation, Text Generation

In data-to-text (D2T) generation, training on in-domain data leads to overfitting to the data representation and repeating training data noise. We examine how to avoid finetuning the pretrained language models (PLMs) on D2T generation datasets while still taking advantage of surface realization capabilities of PLMs. Inspired by pipeline approaches, we propose to generate text by rephrasing single-item templates using a sequence of modules trained on general-domain text-based operations—ordering, aggregation, and paragraph compression. We train PLMs for performing these operations on a synthetic corpus WikiFluent which we build from English Wikipedia. Our experiments on two major triple-to-text datasets—WebNLG and E2E—show that our approach enables D2T generation from RDF triples in zero-shot settings.
Controllable Natural Language Generation with Contrastive Prefixes
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Attribute, Language Modeling, Language Modelling, Text Generation

To guide the generation of large pretrained language models (LM), previous work has focused on directly fine-tuning the language model or utilizing an attribute discriminator. In this work, we propose a novel lightweight framework for controllable GPT2 generation, which utilizes a set of small attribute-specific vectors, called prefixes (Li and Liang, 2021), to steer natural language generation. Different from Li and Liang (2021), where each prefix is trained independently, we take the relationship among prefixes into consideration and train multiple prefixes simultaneously. We propose a novel supervised method and also an unsupervised method to train the prefixes for single-aspect control while the combination of these two methods can achieve multi-aspect control. Experimental results on both single-aspect and multi-aspect control show that our methods can guide generation towards the desired attributes while keeping high linguistic quality.
Learning to execute or ask clarification questions
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Learning to Execute, Minecraft

Collaborative tasks are ubiquitous activities where a form of communication is required in order to reach a joint goal. Collaborative building is one of such tasks. To this end, we wish to develop an intelligent builder agent in a simulated building environment (Minecraft) that can build whatever users wish to build by just talking to the agent. However, in order to achieve this goal, such agents need to be able to take the initiative by asking clarification questions when further information is needed. Existing work on Minecraft Corpus Dataset only learned to execute instructions neglecting the importance of asking for clarifications. In this paper, we extend the Minecraft Corpus Dataset by annotating all builder utterances into eight types, including clarification questions, and propose a new builder agent model capable of determining when to ask or execute instructions. Experimental results show that our model achieves state-of-the-art performance on the collaborative building task with a substantial improvement. We also provide baselines for the new tasks, learning to ask and the joint tasks, which consists in solving both collaborating building and learning to ask tasks jointly.
A Deep Generative XAI Framework for Natural Language Inference Explanations Generation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Decision Making, Explainable artificial intelligence, Explainable Artificial Intelligence (XAI), Explanation Generation, Natural Language Inference

Explainable artificial intelligence with natural language explanations (Natural-XAI) aims to produce human-readable explanations as evidence for AI decision-making. This evidence can enhance human trust and understanding of AI systems and contribute to AI explainability and transparency. However, the current approaches focus on single explanation generation only. In this paper, we conduct experiments with the state-of-the-art Transformer architecture and explore \textit{multiple explanations generation} using a public benchmark dataset, e-SNLI \cite{camburu2018snli}. We propose a novel deep generative Natural-XAI framework: \textbf{INITIATIVE}, standing for \textit{expla\underline{\textbf{I}}n a\underline{\textbf{N}}d pred\underline{\textbf{I}}c\underline{\textbf{T}} w\underline{\textbf{I}}th contextu\underline{\textbf{A}}l condi\underline{\textbf{TI}}onal \underline{\textbf{V}}ariational auto\underline{\textbf{E}}ncoder} for generating natural language explanations and making a prediction at the same time. Our method achieves competitive or better performance against the state-of-the-art baseline models on generation (4.7\% improvement in the BLEU score) and prediction (4.4\% improvement in accuracy) tasks. Our work can serve as a solid deep generative model baseline for future Natural-XAI research. Our code will be publicly available on GitHub upon paper acceptance.
Representation of Ambiguity in Pre-Trained Sentence Embeddings
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Sentence, Sentence Embeddings

Pre-trained language models have been shown to be very effective for various NLP tasks. All of these models are trained on different datasets and often have different architectures. Simultaneously, various approaches for analysis of what and how is encoded in the layers of these models is also being studied. In this work we focus on ambiguous sentences and examine how they are represented at various layers of BERT and GPT-2 in comparison to unambiguous sentences. Taking ambiguity detection as a probing task, we find that layers of BERT perform better than layers of GPT-2. Using Representational Similarity Analysis, we observe that differences corresponding to varying stimuli emerge in the deeper layers and that for ambiguous sentences the dissimilarity between BERT and GPT-2 representations decreases in deeper layers. We also find that for ambiguous sentences, representational dissimilarity across layers is greater for BERT than for GPT-2.
Revisiting the Compositional Generalization Abilities of Neural Sequence Models
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

Compositional generalization is a fundamental trait in humans, allowing us to effortlessly combine known phrases to form novel sentences. Recent works have claimed that standard seq-to-seq models severely lack the ability to compositionally generalize. In this paper, we focus on one-shot primitive generalization as introduced by the popular SCAN benchmark. We demonstrate that modifying the training distribution in simple and intuitive ways enables standard seq-to-seq models to achieve near-perfect generalization performance, thereby showing that their compositional generalization abilities were previously underestimated. We perform detailed empirical analysis of this phenomenon. Our results indicate that the generalization performance of models is highly sensitive to the characteristics of the training data which should be carefully considered while designing such benchmarks in future.
Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

Humans usually choose not to answer questions on which they are likely to be incorrect. In order to equip NLP systems with this selective answering capability, several task-specific approaches have been proposed. However, which approaches work best across tasks or even if they consistently outperform the simplest baseline `MaxProb' remains to be explored. To this end, we systematically study `selective prediction' in a large-scale setup of 17 datasets across several NLP tasks. Through comprehensive experiments under in-domain (IID), out-of-domain (OOD), and adversarial (ADV) settings, we show that despite leveraging additional resources (held-out data/computation), none of the existing approaches consistently and considerably outperforms MaxProb in all three settings. Furthermore, their performance does not translate well across tasks. For instance, 'Monte-Carlo Dropout' outperforms all other approaches on Duplicate Detection datasets but does not fare well on NLI datasets, especially in the OOD setting. Thus, we recommend that future selective prediction approaches should be evaluated across tasks and settings for reliable estimation of their capabilities.
WeaNF: Weak Supervision with Normalizing Flows
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

A popular approach to decrease the need for costly manual annotation of large data sets is weak supervision, which introduces problems of noisy labels, coverage and bias. Methods for overcoming these problems have either relied on discriminative models, trained with cost functions specific to weak supervision, and more recently, generative models, trying to  model the output of the automatic annotation process. In this work, we explore a novel direction of generative modeling for weak supervision: Instead of modeling the output of the annotation process (the labeling function matches), we generatively model the input-side data distributions (the feature space) covered by labeling functions. Specifically, we estimate a density for each weak labeling source, or labeling function, by using normalizing flows. An integral part of our method is the flow-based modeling of multiple simultaneously matching labeling functions, and therefore phenomena such as labeling function overlap and correlations are captured. We analyze the effectiveness and modeling capabilities on various commonly used weak supervision data sets, and show that weakly supervised normalizing flows compare favorably to standard weak supervision baselines.
Learning and Evaluating Character Representations in Novels
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Graph Neural Network, Word Embeddings

We address the problem of learning fixed-length vector representations of characters in novels. Recent advances in word embeddings have proven successful in learning entity representations from short texts, but fall short on longer documents because they do not capture full book-level information. To overcome the weakness of such text-based embeddings, we propose two novel methods for representing characters: (i) graph neural network-based embeddings from a full corpus-based character network; and (ii) low-dimensional embeddings constructed from the occurrence pattern of characters in each novel. We test the quality of these character embeddings using a new benchmark suite to evaluate character representations, encompassing 12 different tasks. We show that our representation techniques combined with text-based embeddings lead to the best character representations, outperforming text-based embeddings in four tasks. Our dataset and evaluation script will be made publicly available to stimulate additional work in this area.
Query and Extract: Refining Event Extraction as Type-oriented Binary Decoding
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Event Extraction, Multi-class Classification, Natural Language Queries, Vocal Bursts Type Prediction, Zero-shot Event Extraction

Event extraction is typically modeled as a multi-class classification problem where both event types and argument roles are treated as atomic symbols. These approaches are usually limited to a set of pre-defined types. We propose a novel event extraction framework that takes both event types and argument roles as natural language queries to extract candidate triggers and arguments from the input text. With the rich semantics in the queries, our framework benefits from the attention mechanisms to better capture the semantic correlation between the event types or argument roles and the input text. Furthermore, the query-and-extract formulation allows our approach to leverage all available event annotations from various ontologies as a unified model. Experiments on two public benchmark datasets, ACE and ERE, demonstrate that our approach achieves the state-of-the-art performance on each dataset and significantly outperforms existing methods on zero-shot event extraction. We will make all the programs publicly available once the paper is accepted.
RoMe: A Robust Metric for Evaluating Natural Language Generation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Natural Language Understanding, Semantic Similarity, Semantic Textual Similarity, Sentence, Text Generation

Evaluating Natural Language Generation (NLG) systems is a challenging task. Firstly, the metric should ensure that the generated hypothesis reflects the reference's semantics. Secondly, it should consider the grammatical quality of the generated sentence. Thirdly, it should be robust enough to handle various surface forms of the generated sentence. Thus, an effective evaluation metric has to be multifaceted. In this paper, we propose an automatic evaluation metric incorporating several core aspects of natural language understanding (language competence, syntactic and semantic variation). Our proposed metric, RoMe, is trained on language features such as semantic similarity combined with tree edit distance and grammatical acceptability, using a self-supervised neural network to assess the overall quality of the generated sentence. Moreover, we perform an extensive robustness analysis of the state-of-the-art methods and RoMe. Empirical results suggest that RoMe has a stronger correlation to human judgment over state-of-the-art metrics in evaluating system-generated sentences across several NLG tasks.
Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Language Modeling, Language Modelling, Machine Translation, Question Answering, Retrieval, Translation

Retrieval-based methods have been shown to be effective in NLP tasks via introducing external knowledge. However, the indexing and retrieving of large-scale corpora bring considerable computational cost. Surprisingly, we found that REtrieving from the traINing datA (REINA) only can lead to significant gains on multiple NLG and NLU tasks. We retrieve the labeled training instances most similar to the input text and then concatenate them with the input to feed into the model to generate the output. Experimental results show that this simple method can achieve significantly better performance on a variety of NLU and NLG tasks, including summarization, machine translation, language modeling, and question answering tasks. For instance, our proposed method achieved state-of-the-art results on XSum, BigPatent, and CommonsenseQA.
Multimodal Semi-supervised Learning for Disaster Tweet Classification
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Classification

During natural disasters, people often use social media platforms, such as Twitter, to post  information about casualties and damage produced by  disasters. This information can help relief authorities gain situational awareness in nearly real time, and enable them to quickly  distribute resources where most needed. However, annotating data for this purpose can be burdensome, subjective and expensive. In this paper, we investigate how to leverage the copious amounts of unlabeled data generated by disaster eyewitnesses and affected individuals during disaster events. To this end, we propose a semi-supervised learning approach to improve the performance of neural models on several multimodal disaster tweet classification tasks. Our approach shows significant improvements, obtaining up to $3.5\%$ F1 performance gain at no additional annotation cost.