Assessing Emoji Use in Modern Text Processing Tools (2021)

Abstract: Emojis have become ubiquitous in digital communication, due to their visual appeal as well as their ability to vividly convey human emotion, among other factors. This also leads to an increased need for systems and tools to operate on text containing emojis. In this study, we assess this support by considering test sets of tweets with emojis, based on which we perform a series of experiments investigating the ability of prominent NLP and text processing tools to adequately process them. In particular, we consider tokenization, part-of-speech tagging, dependency parsing, as well as sentiment analysis. Our findings show that many systems still have notable shortcomings when operating on text containing emojis.

## introduction in our modern digital era, interpersonal communication often takes place via online channels such as instant messaging, email, social media, etc. this entails an increasing need for tools that operate on the resulting digital data. for instance, online conversations can be invaluable sources of insights that reveal fine-grained consumer preferences with regard to products, services, or businesses @xcite . however, the shifts in modality and medium also shape the way we express ourselves, making it increasingly natural for us to embed emojis, images, hashtags into our conversations. in this paper, we focus specifically on emojis, which have recently become fairly ubiquitous in digital communication, with a 2017 study reporting 5 billion emojis being sent daily just on facebook messenger @xcite . emojis are textual elements that are encoded as characters but rendered as small digital images or icons that can be used to express an idea or emotion. goals. due to their increasing prominence, there is a growing need to properly handle emojis when-ever one deals with text. we consider a set of popular nlp tools and empirically assess to what extent they support emojis across a set of standard tasks, encompassing tokenization, part-of-speech tagging, dependency parsing, and sentiment analysis. although emojis can be encoded as unicode characters, there are unique properties of emoji encoding that merit special consideration, such as skin tone modifiers and composite emoji incorporating multiple basic emojis. moreover, text harboring emojis may adhere to subtly different conventions than more traditional forms of text, e.g., with regard to token and sentence boundaries. emojis can take the place of words with different parts-of-speech and assume different grammatical roles. finally, emojis may of course also alter the semantics of the text, which in turn may, for instance, affect its sentiment polarity. ## related work while emoji characters have a long history, they have substantially grown in popularity since their incorporation into unicode 6.0 in 2010 followed by increasing support for them on mobile devices. accordingly, numerous studies have sought to explain how the broad availability of emojis has affected human communication, considering grammatical, semantic, as well as pragmatic aspects @xcite . only few studies have specifically considered some of the more advanced technical possibilities that the unicode standard affords, such as zero width joiners to express more complex concepts. for instance, with regard to emoji skin tone modifiers, @xcite study in depth how the use of such modifiers varies on social media, including cases of users modulating their skintone, i.e., using a different tone than the one they usually pick. given the widespread use of emojis in everyday communication, there is an increasing need for nlp tools that can handle them. prominent nlp toolkits such as stanford's stanza @xcite and nltk @xcite power a wide range of user-facing applications. a number of reports compare the pros and cons of popular nlp libraries @xcite @xcite , but these primarily consider the features and popularity of the tools, as well as their performance. there have not been studies assessing them with regard to their ability to cope with modern emoji-laden text. since emojis are becoming increasingly ubiquitous, it is crucial for developers and institutions deploying such software to know whether it can properly handle the kinds of text that nowadays may quite likely arrive as input data. in many real-world settings, applications and services are expected to operate on text containing emojis, and thus it is important to investigate these capabilities. many academic studies present new models for particular nlp tasks relating to emojis. for instance, @xcite developed an emoji prediction model for tweets. @xcite discussed how to extract essential keywords from a tweet using nlp tools. @xcite attempted to understand the use of emojis from a grammatical perspective, seeking to determine the parts-of-speech of emoji occurrences in a sentence or tweet. @xcite proposed an improved part-of-speech tagging model for online conversational text based on word clusters. proisl (2018) developed a part-of-speech tagger for german social media and @xcite developed a dependency parser for english tweets. however, such work mostly targets just one specific task and is typically not well-integrated with common open source toolkits, which we focus on in our study. ## experimental data as we wish to assess the support of emojis provided by different text processing tools, we first consider some of the different cases of emoji use that one may encounter, in order to compile relevant data. ## tokenization tokenization is the act of breaking up a sequence of strings into a sequence of basic pieces such as words, keywords, phrases, symbols, and other elements, referred to as tokens. in the process of tokenization, some characters such as punctuation marks may be discarded. it is important for a tokenizer to generate meaningful results, as the output of this step becomes the input for subsequent processing steps such as parsing and text mining in the pipeline. in our study, we expect a tokenizer to segment a text into tokens such as words, emojis, and other special characters. ## part-of-speech tagging part-of-speech (pos) tagging is the process of assigning each token a label that reflects its word class. this may be with respect to traditional parts of speech, such as noun, verb, adjective, etc., or using a more fine-grained inventory of classes. ## dependency parsing in dependency grammar, the syntactic structure of a sentence is described as a tree capturing relationships between head words and dependent words. given that emojis can have different grammatical roles within a sentence, we thus assessed to what extent popular dependency parsers are affected by the presence of emojis in the input. ## sentiment analysis although the word "emoji" is not etymologically related to the word "emotion", several studies show how emojis can help to express emotions (shoeb and de melo, 2020) and sentiment in textual communication @xcite . keeping this in mind, we further assessed how well nlp tools fare at the task of predicting the sentiment polarity of a text harboring emojis. ## discussion overall, based on for pos tagging, somewhat surprisingly, almost all tools did well with verbs, while they all struggled with punctuation emojis as well as adverbs. the results for adjectives were as well quite mixed. overall, nltk-tt and textblob achieved the highest success rate for pos tagging, although both still struggle with adverbs and punctuation, which can also lead to adverse effects in downstream tasks such as syntactic parsing. moreover, textblob requires the use of a modified tokenizer. for dependency parsing, we found stanford corenlp and stanza to be the most robust in correctly assessing emojis. spacy, in contrast, does not appear to generalize well enough to lexical items such as emojis that may be lacking in the training data. in general, there is a need for dependency parsers to be trained on more diverse data. thus, in practice one may wish to consider a mixand-match approach, using a tokenizer from one library, a tagger from another, and a dependency parser from yet another library. in our pos tagging and dependency parsing evaluations, we sought to study clear-cut cases to observe whether tools have basic support for emojis. further discussion is necessary on recommended annotation schemes for more diverse forms of emoji use for which the ground truth may not be as obvious. some researchers argue that the default tagging of emoji should be as adverbials, interjections, or punctuation @xcite . similarly, emojis are syntactically comparable to free adjuncts, which constrains the set of valid parse trees. hence, further work is necessary to devise broader-coverage benchmarks for the tasks considered in our study. ## conclusion emojis have become an integral part of modern interpersonal communication and text encountered in chat messages, social media, or emails is often laden with emojis. hence, it is important to endow nlp tools with emoji support not only to obtain a deeper understanding of this wealth of data but also to properly preserve and process them correctly. in this study, we assessed how well prominent nlp tools cope with text containing emoji characters. to this end, we evaluated a set of tools on three different tasks across a range of challenging test sets capturing particular phenomena and encodings. our study demonstrates that there are notable shortcomings in widely used nlp tools. although many tools are partially capable of operating on emojis, none of them proved fully equipped to tackle the full set of aspects considered in our study. hence, special care needs to be taken when developing applications that may encounter emojis.