Best Practices in the Creation and Use of Emotion Lexicons (2023)

Abstract: Words play a central role in how we express ourselves. Lexicons of word–emotion associations are widely used in research and real-world applications for sentiment analysis, tracking emotions associated with products and policies, studying health disorders, tracking emotional arcs of stories, and so on. However, inappropriate and incorrect use of these lexicons can lead to not just sub-optimal results, but also inferences that are directly harmful to people. This paper brings together ideas from Affective Computing and AI Ethics to present, some of the practical and ethical considerations involved in the creation and use of emotion lexicons – best practices. The goal is to provide a comprehensive set of relevant considerations, so that readers (especially those new to work with emotions) can find relevant information in one place. We hope this work will facilitate more thoughtfulness when one is deciding on what emotions to work on, how to create an emotion lexicon, how to use an emotion lexicon, how to draw meaningful inferences, and how to judge success.

## introduction words often convey affect (emotions, sentiment, feelings, and attitudes); either explicitly through their core meaning (denotation) or implicitly through connotation. for example, dejected denotes sadness. on the other hand, failure simply connotes sadness. either through denotation or connotation, both words are associated with sadness. a compilation of such associations is referred to as a word-affect association lexicon (aka emotion lexicon). 1 an entry in a lexicon usually includes a word, an emotion category or affect dimension (e.g., joy, fear, valence, arousal, etc.), and a score indicating association (or strength of association). examples of emotion lexicons include the general inquirer @xcite , anew @xcite , liwc @xcite , pittsburgh subjectivity lexicon @xcite , nrc emotion lexicon @xcite , and the nrc valence, arousal, and dominance (vad) lexicon @xcite . these were all created by manual annotation (either by experts or crowdsourced). there also exist lexicons that were generated automatically from large text corpora using statistical and/or machine learning algorithms; e.g., wordnet affect @xcite , senti-wordnet (swn) @xcite . emotion lexicons have a wide range of applications in commerce, @xcite . some notable examples include: tracking brand and product perception via social media posts, tracking support for controversial issues and policies, tracking buy-in for non-pharmaceutical health measures such as social distancing during a pandemic, literary analysis, and developing more natural dialogue systems. the lexicons can be used on their own or in support of neural machine learning (ml) algorithms for emotion recognition. lexicon-based emotion analyses are especially popular in real-world applications and research outside of computer science because they are interpretable, have a low carbon footprint, and do not require significant programming expertise. further, since outputs of ml models are highly dependent on training data, use of a model often requires retraining, and there may not exists labeled data from the target domain further, @xcite show that when determining broad trends (emotion arcs) and aggregating information from hundreds (if not more) instances for every time step, simple lexicon-based methods are extremely accurate (correlations above 0.95 with ground truth arcs). however, inappropriate and incorrect use of these lexicons, can lead to not just sub-optimal results, but also inferences that are directly harmful. for example, using lexicons to infer emotions from limited amount of data to make judgments about refugee applications, to make judgments about which groups of people are shown certain advertisements and which groups are not, marking businesses owned by some groups of people as less liked than that of others, etc. emotions are deeply personal, private, and complex. even the best natural language systems largely only employ pattern matching based on huge amounts of historical data, and thus often do not really understand what the user is trying to convey, let alone how they are feeling. in fact, some recent commercial and governmental uses of emotion recognition have garnered considerable criticism, including: infringing on one's privacy, exploiting vulnerable sub-populations, and even allegations of pseudo-science @xcite @xcite . this paper brings together ideas from affective computing and ai ethics to present, in one place, some of the practical and ethical considerations involved in the creation and use of emotion lexicons -best practices. 2 we hope this work will facilitate more thoughtfulness when one is deciding on what emotions to work on, how to create an emotion lexicon, how to use an emotion lexicon, and how to judge success. additional benefits of such a document include: ## best practices below we present various best practices (practical and ethical considerations) pertaining to 22 aspects of emotion lexicon creation and use. the 22 aspects are grouped under the coarser categories pertaining to a lexicon's life cycle: a. lexicon design, b. annotation, c. entries in the lexicon, and d. applying the lexicon. note that while many considerations are presented from the perspective of lexicon creation, they are also relevant to the users of a lexicon -knowing what decisions were made during the creation of a lexicon help one to assess appropriateness of using the lexicon. the goal is to provide a comprehensive set of relevant considerations, so that readers (especially those new to research or new to work with emotions) can find the information in one place. thus, we include both the considerations that are especially specific to emotions, as well as others that apply more broadly (even if they are somewhat well known). also, the points listed below are not meant to be the final word, but rather jumping off points for further thought and discussion. ## limitations this paper does not present a new nlp model or dataset. thus, there are no corresponding limitations to discuss. however, the paper itself can be viewed as a document discussing limitations of existing approaches to do sentiment and emotion analysis using emotion lexica. the 22 best practises presented in the paper discuss approaches to engage with and counter these limitations. while this document was a result of engaging a larger community through blog posts, talks, and discussions, we had relatively low access to developers of commercial sentiment analysis systems. thus the list presented here may have missed some important considerations. we encourage readers and impacted stakeholders to challenge the assumptions latent in the document, and identify new ethical considerations not included here or not gaining adequate attention in the research community. ## concluding remarks emotion lexicons are simple yet powerful tools to analyze text. however, use of the lexicons (even for tasks that it is suited for) can lead to inappropriate bias. applying a lexicon to any new data should only be done after first investigating its suitability, and requires careful analysis to minimize unintentional harm. in this paper, we presented 22 best practises that include considerations that can help mitigate such unwanted outcomes, as well as strategies to make the best use of emotion lexicons towards drawing meaningful and accurate inferences. the best practises are organized as per a lexicon's life cycle: a. lexicon design, b. annotation, c. entries in the lexicon, and d. applying the lexicon. we also provide pointers to relevant literature to explore the best practises in more detail. it should be noted that these practises are not meant to be the final word, but rather jumping off points for further thought, discussion, and additional measures towards the responsible use of emotion lexicons.