pwc-archive/papers-with-abstracts huggingface dataset:
Interpreting Emoji with Emoji
Authors: Jens Reelfs, Timon Mohaupt, Sandipan Sikdar, Markus Strohmaier, Oliver Hohlfeld
Date: 2022-07-01 00:00:00
Proceeding: NAACL (Emoji) 2022 7
Tasks: Word Embeddings

We study the extent to which emoji can be used to add interpretability to embeddings of text and emoji. To do so, we extend the POLAR-framework that transforms word embeddings to interpretable counterparts and apply it to word-emoji embeddings trained on four years of messaging data from the Jodel social network. We devise a crowdsourced human judgement experiment to study six usecases, evaluating against words only, what role emoji can play in adding interpretability to word embeddings. That is, we use a revised POLAR approach interpreting words and emoji with words, emoji or both according to human judgement. We find statistically significant trends demonstrating that emoji can be used to interpret other emoji very well.
Beyond emojis: an insight into the IKON language
Authors: Laura Meloni, Phimolporn Hitmeangsong, Bernhard Appelhaus, Edgar Walthert, Cesco Reale
Date: 2022-07-01 00:00:00
Proceeding: NAACL (Emoji) 2022 7

This paper presents a new iconic language, the IKON language, and its philosophical, linguistic, and graphical principles. We examine some case studies to highlight the semantic complexity of the visual representation of meanings. We also introduce the Iconometer test to validate our icons and their application to the medical domain, through the creation of iconic sentences.
Emoji semantics/pragmatics: investigating commitment and lying
Authors: Benjamin Weissman
Date: 2022-07-01 00:00:00
Proceeding: NAACL (Emoji) 2022 7

This paper presents the results of two experiments investigating the directness of emoji in constituting speaker meaning. This relationship is examined in two ways, with Experiment 1 testing whether speakers are committed to meanings they communicate via a single emoji and Experiment 2 testing whether that speaker is taken to have lied if that meaning is false and intended to deceive. Results indicate that emoji with high meaning agreement in general (i.e., pictorial representations of concrete objects or foods) reliably commit the speaker to that meaning and can constitute lying. Expressive emoji representing facial expressions and emotional states demonstrate a range of commitment and lie ratings: those with high meaning agreement constitute more commitment and more of a lie than those with less meaning agreement in the first place. Emoji can constitute speaker commitment and they can be lies, but this result does not apply uniformly to all emoji and is instead tied to agreement, conventionality, and lexicalization.
Understanding the Sarcastic Nature of Emojis with SarcOji
Authors: Vandita Grover, Hema Banati
Date: 2022-07-01 00:00:00
Proceeding: NAACL (Emoji) 2022 7
Tasks: Position, Sarcasm Detection

Identifying sarcasm is a challenging research problem owing to its highly contextual nature. Several researchers have attempted numerous mechanisms to incorporate context, linguistic aspects, and supervised and semi-supervised techniques to determine sarcasm. It has also been noted that emojis in a text may also hold key indicators of sarcasm. However, the availability of sarcasm datasets with emojis is scarce. This makes it challenging to effectively study the sarcastic nature of emojis. In this work, we present SarcOji which has been compiled from five publicly available sarcasm datasets. SarcOji contains labeled English texts which all have emojis. We also analyze SarcOji to determine if there is an incongruence in the polarity of text and emojis used therein. Further, emojis’ usage, occurrences, and positions in the context of sarcasm are also studied in this compiled dataset. With SarcOji we have been able to demonstrate that frequency of occurrence of an emoji and its position are strong indicators of sarcasm. SarcOji dataset is now publicly available with several derived features like sentiment scores of text and emojis, most frequent emoji, and its position in the text. Compilation of the SarcOji dataset is an initial step to enable the study of the role of emojis in communicating sarcasm. SarcOji dataset can also serve as a go-to dataset for various emoji-based sarcasm detection techniques.
Conducting Cross-Cultural Research on COVID-19 Memes
Authors: Jing Ge-Stadnyk, Lusha Sa
Date: 2022-07-01 00:00:00
Proceeding: NAACL (Emoji) 2022 7
Tasks: Cultural Vocal Bursts Intensity Prediction

A cross-linguistic study of COVID-19 memes should allow scholars and professionals to gain insight into how people engage in socially and politically important issues and how culture has influenced societal responses to the global pandemic. This preliminary study employs framing analysis to examine and compare issues, actors and stances conveyed by both English and Chinese memes. The overall findings point to divergence in the way individuals communicate pandemic-related issues in English-speaking countries versus China, although a few similarities were also identified. ‘Regulation’ is the most common issue addressed by both English and Chinese memes, though the latter does so at a comparatively higher rate. The ‘ordinary people’ image within these memes accounts for the largest percentage in both data sets. Although both Chinese and English memes primarily express negative emotions, the former often occurs on an interpersonal level, whereas the latter aims at criticizing society and certain group of people in general. Lastly, this study proposes explanations for these findings in terms of culture and political environment.
Investigating the Influence of Users Personality on the Ambiguous Emoji Perception
Authors: Olga Iarygina
Date: 2022-07-01 00:00:00
Proceeding: NAACL (Emoji) 2022 7

Emojis are an integral part of Internet communication nowadays. Even though, they are supposed to make the text clearer and less dubious, some emojis are ambiguous and can be interpreted in different ways. One of the factors that determine the perception of emojis is the user’s personality. In this work, I conducted an experimental study and investigated how personality traits, measured with a Big Five Inventory (BFI) questionnaire, affect reaction time when interpreting emoji. For a set of emoji, for which there are several possible interpretations, participants had to determine whether the emoji fits the presented context or not. Using regression analysis, I found that conscientiousness and neuroticism significantly predict the reaction time the person needs to decide about the emoji. More conscientious people take longer to resolve ambiguity, while more neurotic people make decisions about ambiguous emoji faster. The knowledge of the relationship between personality and emoji interpretation can lead to effective use of knowledge of people’s characters in personalizing interactive computer systems.
Semantic Congruency Facilitates Memory for Emojis
Authors: Andriana Ge-Christofalos, Laurie Feldman, Heather Sheridan
Date: 2022-07-01 00:00:00
Proceeding: NAACL (Emoji) 2022 7
Tasks: Sentence

Emojis can assume different relations with the sentence context in which they occur. While affective elaboration and emoji-word redundancy are frequently investigated in laboratory experiments, the role of emojis in inferential processes has received much less attention. Here, we used an online ratings task and a recognition memory task to investigate whether differences in emoji function within a sentence affect judgments of emoji-text coherence and subsequent recognition accuracy. Emojis that function as synonyms of a target word from the passages were rated as better fitting with the passage (more coherent) than emojis consistent with an inference from the passage, and both types of emojis were rated as more coherent than incongruent (unrelated) emojis. In a recognition test, emojis consistent with the semantic content of passages (synonym and inference emojis) were better recognized than incongruent emojis. Findings of the present study provide corroborating evidence that readers extract semantic information from emojis and then integrate it with surrounding passage content.
EmojiCloud: a Tool for Emoji Cloud Visualization
Authors: Yunhe Feng, Cheng Guo, Bingbing Wen, Peng Sun, Yufei Yue, Dingwen Tao
Date: 2022-07-01 00:00:00
Proceeding: NAACL (Emoji) 2022 7

This paper proposes EmojiCloud, an open-source Python-based emoji cloud visualization tool, to generate a quick and straightforward understanding of emojis from the perspective of frequency and importance. EmojiCloud is flexible enough to support diverse drawing shapes, such as rectangles, ellipses, and image masked canvases. We also follow inclusive and personalized design principles to cover the unique emoji designs from seven emoji vendors (e.g., Twitter, Apple, and Windows) and allow users to customize plotted emojis and background colors. We hope EmojiCloud can benefit the whole emoji community due to its flexibility, inclusiveness, and customizability.
Graphicon Evolution on the Chinese Social Media Platform BiliBili
Authors: Yiqiong Zhang, Susan Herring, Suifu Gan
Date: 2022-07-01 00:00:00
Proceeding: NAACL (Emoji) 2022 7
Tasks: Cultural Vocal Bursts Intensity Prediction

This study examines the evolutionary trajectory of graphicons in a 13-year corpus of comments from BiliBili, a popular Chinese video-sharing platform. Findings show that emoticons (kaomoji) rose and fell in frequency, while emojis and stickers are both presently on the rise. Graphicon distributions differ in comments and replies to comments. There is also a strong correlation between the types of graphicons used in comments and their corresponding replies, suggesting a priming effect. Finally, qualitative analysis of the 10 most-frequent kaomojis, emojis, and stickers reveals a trend for each successive graphicon type to become less about emotion expression and more integrated with platform-specific culture and the Chinese language. These findings lend partial support to claims in the literature about graphicon evolution.
Compositional generalization with a broad-coverage semantic parser
Authors: Pia Weißenhorn, Lucia Donatelli, Alexander Koller
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7

We show how the AM parser, a compositional semantic parser (Groschwitz et al., 2018) can solve compositional generalization on the COGS dataset. It is the first semantic parser that achieves high accuracy on both naturally occurring language and the synthetic COGS dataset. We discuss implications for corpus and model design for learning human-like generalization. Our results suggest that compositional generalization can be best achieved by building compositionality into semantic parsers.
AnaLog: Testing Analytical and Deductive Logic Learnability in Language Models
Authors: Samuel Ryb, Mario Giulianelli, Arabella Sinclair, Raquel Fernández
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7
Tasks: Language Modeling, Language Modelling, Logical Reasoning, Natural Language Inference

We investigate the extent to which pre-trained language models acquire analytical and deductive logical reasoning capabilities as a side effect of learning word prediction. We present AnaLog, a natural language inference task designed to probe models for these capabilities, controlling for different invalid heuristics the models may adopt instead of learning the desired generalisations. We test four languagemodels on AnaLog, finding that they have all learned, to a different extent, to encode information that is predictive of entailment beyond shallow heuristics such as lexical overlap and grammaticality. We closely analyse the best performing language model and show that while it performs more consistently than other language models across logical connectives and reasoning domains, it still is sensitive to lexical and syntactic variations in the realisation of logical statements.
A Simple Unsupervised Approach for Coreference Resolution using Rule-based Weak Supervision
Authors: Alessandro Stolfo, Chris Tanner, Vikram Gupta, Mrinmaya Sachan
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7
Tasks: coreference-resolution, Coreference Resolution

Labeled data for the task of Coreference Resolution is a scarce resource, requiring significant human effort. While state-of-the-art coreference models rely on such data, we propose an approach that leverages an end-to-end neural model in settings where labeled data is unavailable. Specifically, using weak supervision, we transfer the linguistic knowledge encoded by Stanford?s rule-based coreference system to the end-to-end model, which jointly learns rich, contextualized span representations and coreference chains. Our experiments on the English OntoNotes corpus demonstrate that our approach effectively benefits from the noisy coreference supervision, producing an improvement over Stanford?s rule-based system (+3.7 F1) and outperforming the previous best unsupervised model (+0.9 F1). Additionally, we validate the efficacy of our method on two other datasets: PreCo and Litbank (+2.5 and +5 F1 on Stanford’s system, respectively).
Dyna-bAbI: unlocking bAbI’s potential with dynamic synthetic benchmarking
Authors: Ronen Tamari, Kyle Richardson, Noam Kahlon, Aviad Sar-Shalom, Nelson F. Liu, Reut Tsarfaty, Dafna Shahaf
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7
Tasks: Benchmarking, Natural Language Understanding

While neural language models often perform surprisingly well on natural language understanding (NLU) tasks, their strengths and limitations remain poorly understood. Controlled synthetic tasks are thus an increasingly important resource for diagnosing model behavior. In this work we focus on story understanding, a core competency for NLU systems. However, the main synthetic resource for story understanding, the bAbI benchmark, lacks such a systematic mechanism for controllable task generation. We develop Dyna-bAbI, a dynamic framework providing fine-grained control over task generation in bAbI. We demonstrate our ideas by constructing three new tasks requiring compositional generalization, an important evaluation setting absent from the original benchmark. We tested both special-purpose models developed for bAbI as well as state-of-the-art pre-trained methods, and found that while both approaches solve the original tasks (99{% accuracy), neither approach succeeded in the compositional generalization setting, indicating the limitations of the original training data.We explored ways to augment the original data, and found that though diversifying training data was far more useful than simply increasing dataset size, it was still insufficient for driving robust compositional generalization (with 70{% accuracy for complex compositions). Our results underscore the importance of highly controllable task generators for creating robust NLU systems through a virtuous cycle of model and data development.
When Polysemy Matters: Modeling Semantic Categorization with Word Embeddings
Authors: Elizabeth Soper, Jean-Pierre Koenig
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7
Tasks: Word Embeddings

Recent work using word embeddings to model semantic categorization have indicated that static models outperform the more recent contextual class of models (Majewska et al, 2021). In this paper, we consider polysemy as a possible confounding factor, comparing sense-level embeddings with previously studied static embeddings on both coarse- and fine-grained categorization tasks. We find that the effect of polysemy depends on how one defines semantic categorization; while sense-level embeddings dramatically outperform static embeddings in predicting coarse-grained categories derived from a word sorting task, they perform approximately equally in predicting fine-grained categories derived from context-free similarity judgments. Our findings highlight the different processes underlying human behavior on different types of semantic tasks.
Word-Label Alignment for Event Detection: A New Perspective via Optimal Transport
Authors: Amir Pouran Ben Veyseh, Thien Nguyen
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7
Tasks: Event Detection, Sentence

Event Detection (ED) aims to identify mentions/triggers of real world events in text. In the literature, this task is modeled as a sequence-labeling or word-prediction problem. In this work, we present a novel formulation in which ED is modeled as a word-label alignment task. In particular, given the words in a sentence and possible event types, the objective is to infer an alignment matrix in which event trigger words are aligned with the most likely event types. Moreover, we show that this new perspective facilitates the incorporation of word-label alignment biases to improve alignment matrix for ED. Novel alignment biases and Optimal Transport are introduced to solve our alignment problem for ED. We conduct experiments on a benchmark dataset to demonstrate the effectiveness of the proposed model for ED.
A Dynamic, Interpreted CheckList for Meaning-oriented NLG Metric Evaluation – through the Lens of Semantic Similarity Rating
Authors: Laura Zeidler, Juri Opitz, Anette Frank
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7
Tasks: nlg evaluation, Semantic Similarity, Semantic Textual Similarity

Evaluating the quality of generated text is difficult, since traditional NLG evaluation metrics, focusing more on surface form than meaning, often fail to assign appropriate scores.This is especially problematic for AMR-to-text evaluation, given the abstract nature of AMR.Our work aims to support the development and improvement of NLG evaluation metrics that focus on meaning by developing a dynamic CheckList for NLG metrics that is interpreted by being organized around meaning-relevant linguistic phenomena. Each test instance consists of a pair of sentences with their AMR graphs and a human-produced textual semantic similarity or relatedness score. Our CheckList facilitates comparative evaluation of metrics and reveals strengths and weaknesses of novel and traditional metrics. We demonstrate the usefulness of CheckList by designing a new metric GraCo that computes lexical cohesion graphs over AMR concepts. Our analysis suggests that GraCo presents an interesting NLG metric worth future investigation and that meaning-oriented NLG metrics can profit from graph-based metric components using AMR.
Improved Induction of Narrative Chains via Cross-Document Relations
Authors: Andrew Blair-Stanek, Benjamin Van Durme
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7

The standard approach for inducing narrative chains considers statistics gathered per individual document. We consider whether statistics gathered using cross-document relations can lead to improved chain induction. Our study is motivated by legal narratives, where cases typically cite thematically similar cases. We consider four novel variations on pointwise mutual information (PMI), each accounting for cross-document relations in a different way. One proposed PMI variation performs 58% better relative to standard PMI on recall@50 and induces qualitatively better narrative chains.
DRS Parsing as Sequence Labeling
Authors: Minxing Shen, Kilian Evang
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7
Tasks: DRS Parsing

We present the first fully trainable semantic parser for English, German, Italian, and Dutch discourse representation structures (DRSs) that is competitive in accuracy with recent sequence-to-sequence models and at the same time {emph{compositional} in the sense that the output maps each token to one of a finite set of meaning {emph{fragments}, and the meaning of the utterance is a function of the meanings of its parts. We argue that this property makes the system more transparent and more useful for human-in-the-loop annotation. We achieve this simply by casting DRS parsing as a sequence labeling task, where tokens are labeled with both fragments (lists of abstracted clauses with relative referent indices indicating unification) and {emph{symbols} like word senses or names. We give a comprehensive error analysis that highlights areas for future work.
Leveraging Three Types of Embeddings from Masked Language Models in Idiom Token Classification
Authors: Ryosuke Takahashi, Ryohei Sasano, Koichi Takeda
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7
Tasks: Classification, token-classification, Token Classification, Word Embeddings

Many linguistic expressions have idiomatic and literal interpretations, and the automatic distinction of these two interpretations has been studied for decades. Recent research has shown that contextualized word embeddings derived from masked language models (MLMs) can give promising results for idiom token classification. This indicates that contextualized word embedding alone contains information about whether the word is being used in a literal sense or not. However, we believe that more types of information can be derived from MLMs and that leveraging such information can improve idiom token classification. In this paper, we leverage three types of embeddings from MLMs; uncontextualized token embeddings and masked token embeddings in addition to the standard contextualized word embeddings and show that the newly added embeddings significantly improve idiom token classification for both English and Japanese datasets.
“What makes a question inquisitive?” A Study on Type-Controlled Inquisitive Question Generation
Authors: Lingyu Gao, Debanjan Ghosh, Kevin Gimpel
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7
Tasks: Question Generation, Question-Generation, Question Selection, Vocal Bursts Type Prediction

We propose a type-controlled framework for inquisitive question generation. We annotate an inquisitive question dataset with question types, train question type classifiers, and finetune models for type-controlled question generation. Empirical results demonstrate that we can generate a variety of questions that adhere to specific types while drawing from the source texts. We also investigate strategies for selecting a single question from a generated set, considering both an informative vs. inquisitive question classifier and a pairwise ranker trained from a small set of expert annotations. Question selection using the pairwise ranker yields strong results in automatic and manual evaluation. Our human evaluation assesses multiple aspects of the generated questions, finding that the ranker chooses questions with the best syntax (4.59), semantics (4.37), and inquisitiveness (3.92) on a scale of 1-5, even rivaling the performance of human-written questions.
PropBank Comes of Age—Larger, Smarter, and more Diverse
Authors: Sameer Pradhan, Julia Bonn, Skatje Myers, Kathryn Conger, Tim O’Gorman, James Gung, Kristin Wright-Bettner, Martha Palmer
Date: 2022-07-01 00:00:00
Proceeding: *SEM (NAACL) 2022 7
Tasks: Semantic Role Labeling

This paper describes the evolution of the PropBank approach to semantic role labeling over the last two decades. During this time the PropBank frame files have been expanded to include non-verbal predicates such as adjectives, prepositions and multi-word expressions. The number of domains, genres and languages that have been PropBanked has also expanded greatly, creating an opportunity for much more challenging and robust testing of the generalization capabilities of PropBank semantic role labeling systems. We also describe the substantial effort that has gone into ensuring the consistency and reliability of the various annotated datasets and resources, to better support the training and evaluation of such systems