pwc-archive/papers-with-abstracts huggingface dataset:
Emotion Flip Reasoning in Multiparty Conversations
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10

Emotion plays a major role in conversation analysis. To understand the emotional dynamics of the speakers in a conversation, it is imperative to reason out any change/flip of the emotion of a particular speaker. In this paper, we introduce a novel task -- Emotion Flip Reasoning (aka EFR) that aims to find a rationale or instigator behind the emotion flip of a speaker within a conversational dialog, e.g., joy --> anger. To this end, we introduce MELD-I, a new dataset with ground-truth EFR instigator labels. Following emotion psychology, we define 27 instigator labels in a hierarchical fashion - the coarse-grained set contains 14 instigators, while the fine-grained set comprises all 27 instigators. To benchmark the dataset, we propose TGIF, an efficient neural architecture that captures the dialog context, speaker dynamics, and emotion sequence in a conversation using Transformer encoders and stacked GRUs. Our evaluation yields state-of-the-art performance against five baselines adopted for our task. A detailed diagnosis of the competing models further illustrates the advantages and pitfalls of the neural architecture.
Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Abstractive Text Summarization

Despite recent progress in abstractive summarization, systems still suffer from faithfulness errors. While prior work has proposed models that improve faithfulness, it is unclear whether the improvement comes from an increased level of extractiveness of the model outputs as one naive way to improve faithfulness is to make summarization models more extractive. In this work, we present a framework for evaluating the effective faithfulness of summarization systems, by generating a faithfulness-abstractiveness trade-off curve that serves as a control at different operating points on the abstractiveness spectrum. We then show that the Maximum Likelihood Estimation (MLE) baseline as well as recently proposed methods for improving faithfulness, fail to consistently improve over the control at the same level of abstractiveness. Finally, we learn a selector to identify the most faithful and abstractive summary for a given document, and show that this system can attain higher faithfulness scores in human evaluations while being more abstractive than the baseline system on two datasets. Moreover, we show that our system is able to achieve a better faithfulness-abstractiveness trade-off than the control at the same level of abstractiveness.
A Unified Abstractive Model for Generating Question-Answer Pairs
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Answer Generation, Machine Reading Comprehension, Question Generation, Question-Generation, Reading Comprehension

Large-scale question-answer pairs (QAP) are valuable for many applications, such as knowledge bases construction and machine reading comprehension. Although its importance has been widely recognized, existing approaches are still faced with critical challenges. On the one hand, QAPs are obtained by selecting spans from original texts as their answers, while abstractive answer generation is more suitable and natural for complex QA applications. On the other hand, the interaction between the sub-tasks of answer generation and question generation should be well captured to enhance each other mutually. To this end, we propose a Unified Abstractive model for Question-Answer Pairs generation (UA-QAP). Specifically, we devise the joint model with a query-guided gate to collectively model the two sub-tasks simultaneously and capture the interaction information between them. Therefore, our model can generate semantically comprehensive question-answer pairs. We conduct extensive experiments on three large-scale datasets. The experimental results demonstrate that our model achieves state-of-the-art performance.
Towards Using Diachronic Distributed Word Representations as Models of Lexical Development
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Transfer Learning

Recent work has shown that distributed word representations can encode abstract information from child-directed speech. In this paper, we use diachronic distributed word representations to perform temporal modeling and analysis of lexical development in children. Unlike all previous work, we use temporally sliced corpus to learn distributed word representations of child-speech and child-directed speech under a curriculum-learning setting. In our experiments, we perform a lexical categorization task to plot the semantic and syntactic knowledge acquisition trajectories in children. Next, we perform linear mixed-effects modeling over the diachronic representational changes to study the role of input word frequencies in the rate of word acquisition in children. We also perform a fine-grained analysis of lexical knowledge transfer from adults to children using Representational Similarity Analysis. Finally, we perform a qualitative analysis of the diachronic representations from our model, which reveals the grounding and word associations in the mental lexicon of children. Our experiments demonstrate the ease of usage and effectiveness of diachronic distributed word representations in modeling lexical development.
Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Sentence, Text to SQL, Text-To-SQL

Recently, the challenge of compositional generalization in NLP has attracted more and more attention. Specifically, many prior works show that neural networks struggle with compositional generalization where training and testing distributions differ. However, most of these works are based on word-level synthetic data or a specific data split method to generate compositional biases. In this work, we propose a clause-level compositional example generation method, and we focus on text-to-SQL tasks. We start by splitting the sentences in the Spider text-to-SQL dataset into several sub-sentences, annotating each sub-sentence with its corresponding SQL clause, resulting in a new dataset, Spider-SS. Building upon Spider-SS, we further construct a new dataset named Spider-CG, by substituting and appending Spider-SS sub-sentences to test the ability of models to generalize compositionally. Experiments show that previous models suffer significant performance degradation when evaluated on Spider-CG, even though every sub-sentence has been seen during training. To deal with this problem, we modify the RATSQL+GAP model to fit the segmented data of Spider-SS, and results show that this method can improve generalization performance.
Can Language Models Take A Hint? Prompting for Controllable Contextualized Commonsense Inference
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Data Augmentation, Text Generation

Generating commonsense assertions, given a certain story context, is a tough challenge even for modern language models. One of the reasons for this may be that the model has to "guess" what topic or entity in a story to generate an assertion about.  Prior work has tackled part of the problem, by providing techniques to align commonsense inferences with stories and training language generation models on these.  However, none of the prior work provides means to control the parts of a generated assertion. In this work, we present "hinting", a data augmentation technique for improving inference of contextualized commonsense assertions. Hinting is a prefix prompting strategy that uses both hard and soft prompts.  We demonstrate the effectiveness of hinting by showcasing its effect on two contextual commonsense inference frameworks: ParaCOMET and  GLUCOSE, for both general and context-specific inference.
A Double-Graph Based Framework for Frame Semantic Parsing
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: graph construction, Semantic Parsing

Frame semantic parsing is a fundamental NLP task, which consists of three subtasks: frame identification, argument identification and role classification. Most previous studies tend to neglect relations between different subtasks and arguments and pay little attention to ontological frame knowledge defined in FrameNet. In this paper, we propose a Knowledge-guided Incremental semantic parser with Double-graph (KID). We first introduce Frame Knowledge Graph (FKG), a heterogeneous graph containing both frames and FEs (Frame Elements) built on the frame knowledge so that we can derive knowledge-enhanced representations for frames and FEs. Besides, we propose Frame Semantic Graph (FSG) to represent frame semantic structures extracted from the text with graph structures. In this way, we can transform frame semantic parsing into an incremental graph construction problem to strengthen interactions between subtasks and relations between arguments. Our experiments show that KID outperforms the previous state-of-the-art method by up to 1.7 F1-score on two FrameNet datasets.
A Copy-Augmented Generative Model for Open-Domain Question Answering
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Natural Questions, Open-Domain Question Answering, Question Answering, TriviaQA

Open-domain question answering is a challenging task with a wide variety of practical applications. Existing modern approaches mostly follow a standard two-stage paradigm: retriever then reader. In this article, we focus on improving the effectiveness of the reader module and propose a novel copy-augmented generative approach that integrates the merits of both extractive and generative readers. In particular, our model is built upon the powerful generative model FiD \citep{FiD}. We enhance the original generative reader by incorporating a pointer network to encourage the model to directly copy words from the retrieved passages. We conduct experiments on the two benchmark datasets, Natural Questions and TriviaQA, and the empirical results demonstrate the performance gains of our proposed approach.
Lexicon Creation for Interpretable NLP Models
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Feature Importance

Lexica--words and associated scores--are widely used as simple, interpretable, generalizable language features to predict sentiment, emotions, mental health, and personality traits.  Applying different feature importance methods to different predictive models yields lexica of varying quality. In this paper, we train diverse sequence classification models, including context-oblivious (SVMs, Feed-forward neural networks) and context-sensitive (RoBERTa, DistilBERT) models, and generate lexica based on different feature importance measurements, including attention, masking, and SHAP (SHapley Additive exPlanations) values. We evaluate the generated lexica on their predictive performance on test sets within the same corpus domain and on their generalization to different but similar domains. We find that simple context-oblivious models produce lexica of similar accuracy within domain and of better accuracy across domains to those from complex context-sensitive models. Based on human evaluator ratings of these lexica, we also find that context-oblivious models generate similar lexica that are more aligned with human judgments.
MTG: A Benchmarking Suite for Multilingual Text Generation
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Benchmarking, Question Generation, Question-Generation, Story Generation, Text Generation, Text Summarization, Transfer Learning

We introduce MTG, a new benchmark suite for training and evaluating multilingual text generation. It is the first and largest multilingual multiway text generation benchmark with 400k human-annotated data for four tasks (story generation, question generation, title generation and text summarization) across five languages (English, German, French, Spanish and Chinese). Its multiway characteristic makes it possible to create cross-lingual data between any of two languages and thus boosts the direct cross-lingual knowledge transfer. Based on it, we set various evaluation scenarios and make a deep analysis of several popular multilingual generation models from different aspects. Our benchmark suite will encourage multilingualism for the text generation community with more human-annotated parallel data and more diverse generation scenarios.
Turkish Named Entity Recognition: A Survey and Comparative Analysis
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Survey

Named entity recognition is a challenging task that has been widely studied in English. Although there are some efforts for named entity recognition in Turkish language, the reported results are limited to particular datasets and models. Moreover, there is a lack of comparative analysis for named entity recognition in Turkish. In this study, we contribute to the literature in three folds. First, we provide an up-to-date short survey on Turkish named entity recognition studies. Second, we compare state-of-the-art named entity recognition models on various Turkish datasets that we can access to. Lastly, we analyze a set of linguistic processing steps that would affect the performance of Turkish named entity recognition.
Decomposing Natural Logic Inferences in Neural NLI
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Decision Making, Negation, valid

In the interest of interpreting neural NLI models and their reasoning strategies, 
we carry out a systematic probing study which investigates whether these models
capture the crucial semantic features central to natural logic: \emph{monotonicity} and \emph{concept inclusion}.
Correctly identifying valid inferences in \emph{downward-monotone contexts} is a known stumbling block for NLI performance,
subsuming linguistic phenomena such as negation scope and generalized quantifiers.
To understand this difficulty, we emphasize monotonicity as a property of a \emph{context} and examine the extent to which 
models capture monotonicity information in the contextual embeddings which are 
intermediate to their decision making process.
Drawing on the recent advancement of the probing paradigm,
we compare the presence of monotonicity features across various models.
We find that monotonicity information is notably weak in the representations of popular
NLI models which achieve high scores on benchmarks, 
and observe that previous improvements to these models based on fine-tuning strategies 
have introduced stronger monotonicity features together with their improved performance on challenge sets.
Attention as Grounding: Exploring Textual and Cross-Modal Attention on Entities and Relations in Language-and-Vision Transformer
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Text Generation, Visual Grounding

We explore how a multi-modal transformer trained for generation of longer image descriptions learns syntactic and semantic representations about entities and relations grounded in objects at the level of masked self-attention (text generation) and cross-modal attention (information fusion). We observe that cross-attention learns the visual grounding of noun phrases into objects and high-level semantic information about spatial relations, while text-to-text attention captures low-level syntactic knowledge between words. This concludes that language models in a multi-modal task learn different semantic information about objects and relations cross-modally and uni-modally (text-only). Our code is available here: [the GitHub link placeholder].
Transformers Can Compose Skills To Solve Novel Problems Without Finetuning
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Question Answering, Semantic Similarity, Semantic Textual Similarity

It is possible to achieve improved prediction performance with Transformers on unseen datasets by adding disparate new training tasks to an existing multitask training regime. We demonstrate that this can be attributed to a compositional mechanism rather than memorisation. Performance on DROP, DROP-CS and ROPES datasets can be improved by over 26 percent without finetuning through application of numerical reasoning tasks, while performance on seven other question-answering datasets that would not be expected to be improved remains essentially unchanged. By filtering our evaluation datasets to only those samples that have no answer overlap to similar training samples, and then further restricting to those samples which have the least semantic similarity with the training set, we show that improved performance after adding numerical reasoning tasks was not attributable to direct lookup. Our code and filtered datasets are available at https://github.com/anonymised.
Widely Interpretable Semantic Representation: Frameless Meaning Representation for Broader Applicability
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Abstract Meaning Representation

This paper presents a semantic representation called WISeR that overcomes challenges for Abstract Meaning Representation (AMR). Despite its richness and exapandability, AMR is not easily applied to languages or domains without predefined semantic frames, and its use of numbered arguments results in semantic role labels which are not directly interpretable and are semantically overloaded for parsers. We examine the numbered arguments of predicates in AMR and convert them to thematic roles which do not require reference to semantic frames. We create a new corpus of 1K dialogue sentences annotated in both WISeR and AMR. WISeR shows stronger inter-annotator agreement for beginner and experienced annotators, with beginners becoming proficient in WISeR annotation sooner. Finally, we train two state-of-the-art parsers on the AMR 3.0 corpus and a WISeR corpus converted from AMR 3.0. The parsers are evaluated on these corpora and our dialogue corpus. WISeR models exhibit higher accuracy than their AMR counterparts across the board, demonstrating that WISeR is easier for parsers to learn.
Comprehension of Subtitles from Re-Translating Simultaneous Speech Translation
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Machine Translation, Translation

In simultaneous speech translation, one can vary the size of the output window,
system latency and sometimes the allowed level of rewriting. The effect of these
properties on readability and comprehensibility has not been tested with modern
neural translation systems.
In this work, we propose an evaluation method and investigate the effects on comprehension and user preferences. It is a pilot study with 14 users on 2 hours of German documentaries or speeches with online
translations into Czech. We collect continuous feedback and answers on factual
questions. Our results show that the subtitling layout or flicker have a little
effect on comprehension, in contrast to machine translation itself and
individual competence. Other results show that users with a limited knowledge of
the source language have different preferences to stability and latency than the
users with zero knowledge. The results are statistically insignificant, however, we show that our method works and can be reproduced in larger volume.
Gender Roles from Word Embeddings in a Century of Children’s Books
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Word Embeddings

When presenting content to children, educators and parents not only want to know whether characters of different backgrounds are represented; they also want to understand how these characters are depicted. In this paper, we measure the gender portrayal of central domains of social life as depicted in highly influential children's books using word co-occurrence and word embeddings. We find that females are more likely than males to be associated with words related both to family and appearance, while males are more associated with business-related words. The gender associations with appearance and business have endured over time, whereas family word associations have become more gender-neutral. We make two main contributions: one, we create a word embeddings data set, StoryWords 1.0, of 100 years of award-winning children's literature, and two, we show inequality in the portrayal of gender in this literature, which in turn may convey messages to children about differential roles in society. We include our code and models as supplemental data associated with this manuscript.
Towards Overcoming Practical Obstacles to Deploying Deep Active Learning
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Active Learning, text-classification, Text Classification

Active learning (AL) is a prominent technique for reducing the annotation effort required for training machine learning models. Deep learning offers a solution for several essential obstacles to deploying AL in practice but introduces many others. One of such problems is the excessive computational resources required to train an acquisition model and estimate its uncertainty on instances in the unlabeled pool. We propose two techniques that tackle this issue for text classification and tagging tasks, offering a substantial reduction of AL iteration duration and the computational overhead introduced by deep acquisition models in AL. We also demonstrate that our algorithm that leverages pseudolabeling and distilled models overcomes one of the obstacles revealed previously in the literature. Namely, it was shown that due to differences between an acquisition model used to select instances during AL and a successor model trained on the labeled data, the benefits of AL can diminish. We show that our algorithm, despite using a smaller and faster acquisition model, is capable of training a more expressive successor with higher performance.
FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Data Augmentation, Few-Shot Learning, Natural Language Understanding

Most previous methods for text data augmentation are limited to simple tasks and weak baselines. We explore data augmentation on hard tasks (i.e., few-shot natural language understanding) and strong baselines (i.e., pretrained models with over one billion parameters). Under this setting, we reproduced a large number of previous augmentation methods and found that these methods bring marginal gains at best and sometimes degrade the performance much. To address this challenge, we propose a novel data augmentation method FlipDA that jointly uses a generative model and a classifier to generate label-flipped data. Central to the idea of FlipDA is the discovery that generating label-flipped data is more crucial to the performance than generating label-preserved data. Experiments show that FlipDA achieves a good tradeoff between effectiveness and robustness---it substantially improves many tasks while not negatively affecting the others.