pwc-archive/papers-with-abstracts huggingface dataset:
Improving the Identification of the Discourse Function of News Article Paragraphs
Authors: Deya Banisakher, W. Victor Yarlott, Mohammed Aldawsari, Naphtali Rishe, Mark Finlayson
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Articles, Sentence

Identifying the discourse structure of documents is an important task in understanding written text. Building on prior work, we demonstrate an improved approach to automatically identifying the discourse function of paragraphs in news articles. We start with the hierarchical theory of news discourse developed by van Dijk (1988) which proposes how paragraphs function within news articles. This discourse information is a level intermediate between phrase- or sentence-sized discourse segments and document genre, characterizing how individual paragraphs convey information about the events in the storyline of the article. Specifically, the theory categorizes the relationships between narrated events and (1) the overall storyline (such as Main Events, Background, or Consequences) as well as (2) commentary (such as Verbal Reactions and Evaluations). We trained and tested a linear chain conditional random field (CRF) with new features to model van Dijk{'}s labels and compared it against several machine learning models presented in previous work. Our model significantly outperformed all baselines and prior approaches, achieving an average of 0.71 F1 score which represents a 31.5{\%} improvement over the previously best-performing support vector machine model.
Systematic Evaluation of a Framework for Unsupervised Emotion Recognition for Narrative Text
Authors: Samira Zad, Mark Finlayson
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Emotion Recognition

Identifying emotions as expressed in text (a.k.a. text emotion recognition) has received a lot of attention over the past decade. Narratives often involve a great deal of emotional expression, and so emotion recognition on narrative text is of great interest to computational approaches to narrative understanding. Prior work by Kim et al. 2010 was the work with the highest reported emotion detection performance, on a corpus of fairy tales texts. Close inspection of that work, however, revealed significant reproducibility problems, and we were unable to reimplement Kim{'}s approach as described. As a consequence, we implemented a framework inspired by Kim{'}s approach, where we carefully evaluated the major design choices. We identify the highest-performing combination, which outperforms Kim{'}s reported performance by 7.6 $F_1$ points on average. Close inspection of the annotated data revealed numerous missing and incorrect emotion terms in the relevant lexicon, WordNetAffect (WNA; Strapparava and Valitutti, 2004), which allowed us to augment it in a useful way. More generally, this showed that numerous clearly emotive words and phrases are missing from WNA, which suggests that effort invested in augmenting or refining emotion ontologies could be useful for improving the performance of emotion recognition systems. We release our code and data to definitely enable future reproducibility of this work.
Exploring the Effect of Author and Reader Identity in Online Story Writing: the STORIESINTHEWILD Corpus.
Authors: Tal August, Maarten Sap, Elizabeth Clark, Katharina Reinecke, Noah A. Smith
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7

Current story writing or story editing systems rely on human judgments of story quality for evaluating performance, often ignoring the subjectivity in ratings. We analyze the effect of author and reader characteristics and story writing setup on the quality of stories in a short storytelling task. To study this effect, we create and release STORIESINTHEWILD, containing 1,630 stories collected on a volunteer-based crowdsourcing platform. Each story is rated by three different readers, and comes paired with the author{'}s and reader{'}s age, gender, and personality. Our findings show significant effects of authors{'} and readers{'} identities, as well as writing setup, on story writing and ratings. Notably, compared to younger readers, readers age 45 and older consider stories significantly less creative and less entertaining. Readers also prefer stories written all at once, rather than in chunks, finding them more coherent and creative. We also observe linguistic differences associated with authors{'} demographics (e.g., older authors wrote more vivid and emotional stories). Our findings suggest that reader and writer demographics, as well as writing setup, should be accounted for in story writing evaluations.
Script Induction as Association Rule Mining
Authors: Anton Belyy, Benjamin Van Durme
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Cloze Test

We show that the count-based Script Induction models of Chambers and Jurafsky (2008) and Jans et al. (2012) can be unified in a general framework of narrative chain likelihood maximization. We provide efficient algorithms based on Association Rule Mining (ARM) and weighted set cover that can discover interesting patterns in the training data and combine them in a reliable and explainable way to predict the missing event. The proposed method, unlike the prior work, does not assume full conditional independence and makes use of higher-order count statistics. We perform the ablation study and conclude that the inductive biases introduced by ARM are conducive to better performance on the narrative cloze test.
Automatic extraction of personal events from dialogue
Authors: Joshua Eisenberg, Michael Sheriff
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Event Extraction

In this paper we introduce the problem of extracting events from dialogue. Previous work on event extraction focused on newswire, however we are interested in extracting events from spoken dialogue. To ground this study, we annotated dialogue transcripts from fourteen episodes of the podcast This American Life. This corpus contains 1,038 utterances, made up of 16,962 tokens, of which 3,664 represent events. The agreement for this corpus has a Cohen{'}s Kappa of 0.83. We have open-sourced this corpus for the NLP community. With this corpus in hand, we trained support vector machines (SVM) to correctly classify these phenomena with 0.68 F1, when using episode-fold cross-validation. This is nearly 100{\%} higher F1 than the baseline classifier. The SVM models achieved performance of over 0.75 F1 on some testing folds. We report the results for SVM classifiers trained with four different types of features (verb classes, part of speech tags, named entities, and semantic role labels), and different machine learning protocols (under-sampling and trigram context). This work is grounded in narratology and computational models of narrative. It is useful for extracting events, plot, and story content from spoken dialogue.
Annotating and quantifying narrative time disruptions in modernist and hypertext fiction
Authors: Edward Kearns
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7

This paper outlines work in progress on a new method of annotating and quantitatively discussing narrative techniques related to time in fiction. Specifically those techniques are analepsis, prolepsis, narrative level changes, and stream-of-consciousness and free-indirect-discourse narration. By counting the frequency and extent of the usage of these techniques, the narrative characteristics of different works from different time periods and genres can be compared. This project uses modernist fiction and hypertext fiction as its case studies.
Extracting Message Sequence Charts from Hindi Narrative Text
Authors: Swapnil Hingmire, Nitin Ramrakhiyani, Avinash Kumar Singh, Sangameshwar Patil, Girish Palshikar, Pushpak Bhattacharyya, Vasudeva Varma
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7

In this paper, we propose the use of Message Sequence Charts (MSC) as a representation for visualizing narrative text in Hindi. An MSC is a formal representation allowing the depiction of actors and interactions among these actors in a scenario, apart from supporting a rich framework for formal inference. We propose an approach to extract MSC actors and interactions from a Hindi narrative. As a part of the approach, we enrich an existing event annotation scheme where we provide guidelines for annotation of the mood of events (realis vs irrealis) and guidelines for annotation of event arguments. We report performance on multiple evaluation criteria by experimenting with Hindi narratives from Indian History. Though Hindi is the fourth most-spoken first language in the world, from the NLP perspective it has comparatively lesser resources than English. Moreover, there is relatively less work in the context of event processing in Hindi. Hence, we believe that this work is among the initial works for Hindi event processing.
Emotion Arcs of Student Narratives
Authors: Swapna Somasundaran, Xianyang Chen, Michael Flor
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: ARC

This paper studies emotion arcs in student narratives. We construct emotion arcs based on event affect and implied sentiments, which correspond to plot elements in the story. We show that student narratives can show elements of plot structure in their emotion arcs and that properties of these arcs can be useful indicators of narrative quality. We build a system and perform analysis to show that our arc-based features are complementary to previously studied sentiment features in this area.
Detecting and understanding moral biases in news
Authors: Usman Shahid, Barbara Di Eugenio, Andrew Rojecki, Elena Zheleva
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7

We describe work in progress on detecting and understanding the moral biases of news sources by combining framing theory with natural language processing. First we draw connections between issue-specific frames and moral frames that apply to all issues. Then we analyze the connection between moral frame presence and news source political leaning. We develop and test a simple classification model for detecting the presence of a moral frame, highlighting the need for more sophisticated models. We also discuss some of the annotation and frame detection challenges that can inform future research in this area.
Word Embeddings as Tuples of Feature Probabilities
Authors: Siddharth Bhat, Alok Debnath, Souvik Banerjee, Manish Shrivastava
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Word Embeddings

In this paper, we provide an alternate perspective on word representations, by reinterpreting the dimensions of the vector space of a word embedding as a collection of features. In this reinterpretation, every component of the word vector is normalized against all the word vectors in the vocabulary. This idea now allows us to view each vector as an $n$-tuple (akin to a fuzzy set), where $n$ is the dimensionality of the word representation and each element represents the probability of the word possessing a feature. Indeed, this representation enables the use fuzzy set theoretic operations, such as union, intersection and difference. Unlike previous attempts, we show that this representation of words provides a notion of similarity which is inherently asymmetric and hence closer to human similarity judgements. We compare the performance of this representation with various benchmarks, and explore some of the unique properties including function word detection, detection of polysemous words, and some insight into the interpretability provided by set theoretic operations.
Compositionality and Capacity in Emergent Languages
Authors: Abhinav Gupta, Cinjon Resnick, Jakob Foerster, Andrew Dai, Kyunghyun Cho
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Open-Ended Question Answering, Systematic Generalization

Recent works have discussed the extent to which emergent languages can exhibit properties of natural languages particularly learning compositionality. In this paper, we investigate the learning biases that affect the efficacy and compositionality in multi-agent communication in addition to the communicative bandwidth. Our foremost contribution is to explore how the capacity of a neural network impacts its ability to learn a compositional language. We additionally introduce a set of evaluation metrics with which we analyze the learned languages. Our hypothesis is that there should be a specific range of model capacity and channel bandwidth that induces compositional structure in the resulting language and consequently encourages systematic generalization. While we empirically see evidence for the bottom of this range, we curiously do not find evidence for the top part of the range and believe that this is an open question for the community.
Improving Bilingual Lexicon Induction with Unsupervised Post-Processing of Monolingual Word Vector Spaces
Authors: Ivan Vuli{\'c}, Anna Korhonen, Goran Glava{\v{s}}
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Bilingual Lexicon Induction

Work on projection-based induction of cross-lingual word embedding spaces (CLWEs) predominantly focuses on the improvement of the projection (i.e., mapping) mechanisms. In this work, in contrast, we show that a simple method for post-processing monolingual embedding spaces facilitates learning of the cross-lingual alignment and, in turn, substantially improves bilingual lexicon induction (BLI). The post-processing method we examine is grounded in the generalisation of first- and second-order monolingual similarities to the nth-order similarity. By post-processing monolingual spaces before the cross-lingual alignment, the method can be coupled with any projection-based method for inducing CLWE spaces. We demonstrate the effectiveness of this simple monolingual post-processing across a set of 15 typologically diverse languages (i.e., 15*14 BLI setups), and in combination with two different projection methods.
Evaluating Natural Alpha Embeddings on Intrinsic and Extrinsic Tasks
Authors: Riccardo Volpi, Luigi Malag{\`o}
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Word Embeddings

Skip-Gram is a simple, but effective, model to learn a word embedding mapping by estimating a conditional probability distribution for each word of the dictionary. In the context of Information Geometry, these distributions form a Riemannian statistical manifold, where word embeddings are interpreted as vectors in the tangent bundle of the manifold. In this paper we show how the choice of the geometry on the manifold allows impacts on the performances both on intrinsic and extrinsic tasks, in function of a deformation parameter alpha.
Exploring the Limits of Simple Learners in Knowledge Distillation for Document Classification with DocBERT
Authors: Ashutosh Adhikari, Achyudh Ram, Raphael Tang, William L. Hamilton, Jimmy Lin
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Document Classification, General Classification, Knowledge Distillation, Model Compression

Fine-tuned variants of BERT are able to achieve state-of-the-art accuracy on many natural language processing tasks, although at significant computational costs. In this paper, we verify BERT{'}s effectiveness for document classification and investigate the extent to which BERT-level effectiveness can be obtained by different baselines, combined with knowledge distillation{---}a popular model compression method. The results show that BERT-level effectiveness can be achieved by a single-layer LSTM with at least $40\times$ fewer FLOPS and only ${\sim}3\%$ parameters. More importantly, this study analyzes the limits of knowledge distillation as we distill BERT{'}s knowledge all the way down to linear models{---}a relevant baseline for the task. We report substantial improvement in effectiveness for even the simplest models, as they capture the knowledge learnt by BERT.
Joint Training with Semantic Role Labeling for Better Generalization in Natural Language Inference
Authors: Cemil Cengiz, Deniz Yuret
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Diagnostic, Natural Language Inference, Out-of-Distribution Generalization, Semantic Role Labeling, Sentence

End-to-end models trained on natural language inference (NLI) datasets show low generalization on out-of-distribution evaluation sets. The models tend to learn shallow heuristics due to dataset biases. The performance decreases dramatically on diagnostic sets measuring compositionality or robustness against simple heuristics. Existing solutions for this problem employ dataset augmentation which has the drawbacks of being applicable to only a limited set of adversaries and at worst hurting the model performance on other adversaries not included in the augmentation set. Instead, our proposed solution is to improve sentence understanding (hence out-of-distribution generalization) with joint learning of explicit semantics. We show that a BERT based model trained jointly on English semantic role labeling (SRL) and NLI achieves significantly higher performance on external evaluation sets measuring generalization performance.
A Metric Learning Approach to Misogyny Categorization
Authors: Juan Manuel Coria, Sahar Ghannay, Sophie Rosset, Herv{\'e} Bredin
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Metric Learning, Representation Learning, Sentence, Sentence Classification, Sentence Embeddings, Triplet

The task of automatic misogyny identification and categorization has not received as much attention as other natural language tasks have, even though it is crucial for identifying hate speech in social Internet interactions. In this work, we address this sentence classification task from a representation learning perspective, using both a bidirectional LSTM and BERT optimized with the following metric learning loss functions: contrastive loss, triplet loss, center loss, congenerous cosine loss and additive angular margin loss. We set new state-of-the-art for the task with our fine-tuned BERT, whose sentence embeddings can be compared with a simple cosine distance, and we release all our code as open source for easy reproducibility. Moreover, we find that almost every loss function performs equally well in this setting, matching the regular cross entropy loss.
Contextual and Non-Contextual Word Embeddings: an in-depth Linguistic Investigation
Authors: Alessio Miaschi, Felice Dell{'}Orletta
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Language Modeling, Language Modelling, Sentence, Word Embeddings

In this paper we present a comparison between the linguistic knowledge encoded in the internal representations of a contextual Language Model (BERT) and a contextual-independent one (Word2vec). We use a wide set of probing tasks, each of which corresponds to a distinct sentence-level feature extracted from different levels of linguistic annotation. We show that, although BERT is capable of understanding the full context of each word in an input sequence, the implicit knowledge encoded in its aggregated sentence representations is still comparable to that of a contextual-independent model. We also find that BERT is able to encode sentence-level properties even within single-word embeddings, obtaining comparable or even superior results than those obtained with sentence representations.
Enhancing Transformer with Sememe Knowledge
Authors: Yuhui Zhang, Chenghao Yang, Zhengping Zhou, Zhiyuan Liu
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Language Modeling, Language Modelling

While large-scale pretraining has achieved great success in many NLP tasks, it has not been fully studied whether external linguistic knowledge can improve data-driven models. In this work, we introduce sememe knowledge into Transformer and propose three sememe-enhanced Transformer models. Sememes, by linguistic definition, are the minimum semantic units of language, which can well represent implicit semantic meanings behind words. Our experiments demonstrate that introducing sememe knowledge into Transformer can consistently improve language modeling and downstream tasks. The adversarial test further demonstrates that sememe knowledge can substantially improve model robustness.
Evaluating Compositionality of Sentence Representation Models
Authors: Hanoz Bhathena, Angelica Willis, Nathan Dass
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Sensitivity, Sentence

We evaluate the compositionality of general-purpose sentence encoders by proposing two different metrics to quantify compositional understanding capability of sentence encoders. We introduce a novel metric, Polarity Sensitivity Scoring (PSS), which utilizes sentiment perturbations as a proxy for measuring compositionality. We then compare results from PSS with those obtained via our proposed extension of a metric called Tree Reconstruction Error (TRE) (CITATION) where compositionality is evaluated by measuring how well a true representation producing model can be approximated by a model that explicitly combines representations of its primitives.
Supertagging with CCG primitives
Authors: Aditya Bhargava, Gerald Penn
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: CCG Supertagging, Sentence

In CCG and other highly lexicalized grammars, supertagging a sentence{'}s words with their lexical categories is a critical step for efficient parsing. Because of the high degree of lexicalization in these grammars, the lexical categories can be very complex. Existing approaches to supervised CCG supertagging treat the categories as atomic units, even when the categories are not simple; when they encounter words with categories unseen during training, their guesses are accordingly unsophisticated. In this paper, we make use of the primitives and operators that constitute the lexical categories of categorial grammars. Instead of opaque labels, we treat lexical categories themselves as linear sequences. We present an LSTM-based model that replaces standard word-level classification with prediction of a sequence of primitives, similarly to LSTM decoders. Our model obtains state-of-the-art word accuracy for single-task English CCG supertagging, increases parser coverage and F1, and is able to produce novel categories. Analysis shows a synergistic effect between this decomposed view and incorporation of prediction history.
The SIGMORPHON 2020 Shared Task on Multilingual Grapheme-to-Phoneme Conversion
Authors: Kyle Gorman, Lucas F.E. Ashby, Aaron Goyzueta, Arya McCarthy, Shijie Wu, Daniel You
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Grapheme-to-Phoneme Conversion

We describe the design and findings of the SIGMORPHON 2020 shared task on multilingual grapheme-to-phoneme conversion. Participants were asked to submit systems which take in a sequence of graphemes in a given language as input, then output a sequence of phonemes representing the pronunciation of that grapheme sequence. Nine teams submitted a total of 23 systems, at best achieving a 18{\%} relative reduction in word error rate (macro-averaged over languages), versus strong neural sequence-to-sequence baselines. To facilitate error analysis, we publicly release the complete outputs for all systems{---}a first for the SIGMORPHON workshop.
One-Size-Fits-All Multilingual Models
Authors: Ben Peters, Andr{\'e} F. T. Martins
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: All, LEMMA

This paper presents DeepSPIN{'}s submissions to Tasks 0 and 1 of the SIGMORPHON 2020 Shared Task. For both tasks, we present multilingual models, training jointly on data in all languages. We perform no language-specific hyperparameter tuning {--} each of our submissions uses the same model for all languages. Our basic architecture is the sparse sequence-to-sequence model with entmax attention and loss, which allows our models to learn sparse, local alignments while still being trainable with gradient-based techniques. For Task 1, we achieve strong performance with both RNN- and transformer-based sparse models. For Task 0, we extend our RNN-based model to a multi-encoder set-up in which separate modules encode the lemma and inflection sequences. Despite our models{'} lack of language-specific tuning, they tie for first in Task 0 and place third in Task 1.