Using Word Embeddings for Identifying Emotions Relating to the Body in a N eo- A ssyrian Corpus (2023)

Abstract: Research into emotions is a developing field within Assyriology, and NLP tools for Akkadian texts offers a new perspective on the data. In this submission, we use PMI-based word embeddings to explore the relationship between parts of the body and emotions. Using data downloaded from Oracc, we ask which parts of the body were semantically linked to emotions. We do this through examining which of the top 10 results for a body part could be used to express emotions. After identifying two words for the body that have the most emotion words in their results list ( libbu and kabattu ), we then examine whether the emotion words in their results lists were indeed used in this manner in the Neo-Assyrian textual corpus. The results indicate that of the two body parts, kabattu was semantically linked to happiness and joy, and had a secondary emotional field of anger.

## introduction the study of emotions in ancient near eastern cultures has grown in the past five years, with two edited volumes covering the topic (hsu and llop raduà, 2021; sonik and steinert, 2022). a key question has been how ancient languages used the body to express emotions -or the 'embodiment of emotions'. for akkadian, this question has been addressed by ulrike steinert (steinert, 2022, 2021). her work involved the traditional approach of close reading akkadian texts and close analysis of the most important akkadian dictionaries to identify words relating to emotions, and how the body was used to express them. she identified 22 akkadian words that refer to parts of the body that were used to express emotions (steinert, 2022). as steinert's results were reflective of akkadian language as a whole, we were interested in whether her results could be replicated for a textual corpus from a more narrow timespan -namely the neo-assyrian period (c. 934-612 bce). 1 1 there is growing research into emotions in neo-assyrian material (valk, 2016; battini, 2022; bach, 2022; nadali, 2022; neo-assyria is the best represented time period currently available on the open richly annotated cuneiform corpus (oracc). we can therefore use word embeddings to gather quantitative data of whether the body words identified by steinert to express emotions were used similarly to emotion words during the neo-assyrian period. in addition, word embeddings will allow us to determine whether any of these body words can be considered as part of a semantic field of emotions. ## data the data for this project consists of two sections: the corpus of neo-assyrian texts, and the list of akkadian words that relate to parts of the body. ## word embeddings word embeddings represent words as real-valued vectors in a multi-dimensional vector space. in simple terms, this vector space can be understood as a matrix, where the rows represent each word in the corpus and the columns represent abstractions of their co-occurrences with other words. this property makes word embeddings useful for measuring lexical similarity: if two words a and b are similar to each other in meaning, they likely occur in similar contexts, and thus their vectors should show higher similarity to each other. the most closely related words (often called nearest neighbors) can @xcite as akkadian is a highly inflected language, and the research was not focused on syntax or morphology, we worked from the lemmatised version of the dataset. this follows previous assyriological research projects based on oracc data (svärd et al., 2020; sahala and svärd, 2021; bennett, 2023) 4 this is how the akkadian words will appear in the tables of this contribution for the ease of non-assyriological readers, but in the main body we will be using assyriological standards of italicising the lemma. @xcite a full list of these can be found in the accompanying zenodo repository, the url of which can be found in appendix a. @xcite an unlemmatised word in oracc can be due to many factors, such as a broken text (indicated by 'x' in transliterations), or a word with uncertain meanings. @xcite the dataset is part of the supplementary material, which is described in appendix a. the corpus can be broken down according to genre in the following manner: letters (33%), royal inscriptions (17%), transactions (15%), scholarly texts (14%), and the remaining 21% was a mix of administrative, religious, legal, political, literaturary, and untagged texts. @xcite we did not include bodily emissions in this research, but could be an interesting future field of emotions research (sonik, 2022a). in word embedding calculations, this metric has a theoretical upper bound of 1 in case the words are perfect synonyms, and a lower bound of 0 in case the semantic relationship between the words is independent. @xcite some of the better known tools for building word embeddings are word2vec (mikolov et al., 2013) and fasttext (bojanowski et al., 2017), which both use shallow neural networks to learn the word embedding space from the corpus. while the neural network based methods are powerful in large datasets, these methods are known to have issues in smaller datasets comprising only a million words (jungmaier et al., 2020). for low-resource languages and small corpora, word embeddings can be successfully built by using count-based methods, such as word association measures combined with matrix factorization (levy et al., 2015; jungmaier et al., 2020). the count-based method used in this paper produces vectors only for words that are documented in the data set. therefore the similarity between out-of-vocabulary words cannot be measured like in fasttext. pmi-embeddings is a python script for building count-based word embeddings for low-resource languages. @xcite it combines several features from various research papers on count-based word embeddings that utilize pointwise mutual information (pmi) (church and hanks, 1990) and singular value decomposition (svd). briefly described, pmi-embeddings builds a sparse matrix of the vocabulary that encodes statistical significance of cooccurrences in terms of pmi within a symmetric context window of arbitrary size. this sparse matrix is then truncated into desired dimensionality (typically between 60 and 300) using svd, yielding the final vector space that can be saved into the standard word2vec format to be used in various nlp applications. the script allows tuning several hyperparameters to find optimal settings for the given dataset. these include, but are not limited to, dirichlet smoothing (turney and pantel, 2010; jungmaier et al., 2020), several variants of pmi such as shifted and context distribution smoothed pmi (levy et al., 2015) with various different shifting mechanisms, dynamic context window that gives less significance for co-occurrences that are further apart (sahlgren, 2006), and context similarity weighting (sahala and lindén, 2020) that downsamples noise caused by duplication and repetitiveness in the dataset, which is a problem especially in formulaic mespototamian royal inscriptions. if a human-evaluated gold standard for semantic similarity is available, well performing parameters can be searched by using the hypertune.py script distributed with pmi-embeddings. currently the default settings of pmi-embeddings have been defined for the first millennium akkadian texts using a work-in-progress gold standard based on independent word similarity rankings done by five assyriologists. this gold standard is distributed as a part of pmi-embeddings. foot_3 ## analysis the only body words that had more than three results which could be used to express emotions were libbu and kabattu. previous scholarship regarding neo-assyrian embodied emotions has identified these were often used in phrases to express emotions (bach, 2022; luukko, 2021; morello, 2022; sonik, 2022b). the results appeared to therefore align with current assyriological research. thus far, we have identified words in results lists that could be used to express emotions. many of these words can be translated in different ways and have alternative usages. this section will assess whether the words in the results lists for libbu and kabattu were indeed used to convey emotions in the neo-assyrian corpus. this will indicate whether 23 cad p: 517 purīdu; cad m2: 286 muštarhu; cad p: 338 petû. ## conclusions pmi-embeddings has therefore proven to be an important tool to not only identify which areas of the body were most associated with emotions, but to identify which body parts were semantically connected to specific emotional fields in neo-assyrian texts. we highlighted two words that were most similar in usage to words that could be used to express emotions: libbu and kabattu. our much more limited list of akkadian words for body parts that were semantically linked with emotions than steinert's list suggests the neo-assyrian corpus has temporallyspecific methods of embodying emotions. future research could compare our results to datasets built from texts of a different period, such as middle assyrian. libbu and kabattu were the two akkadian words relating to the body that were most likely to be part of the semantic field of emotions, aligning with the findings of previous assyriological research (sonik, 2022b; steinert, 2021; luukko, 2021; wende, 2022). we have been able to corroborate results with close readings of texts for kabattu, and solidify that in the neo-assyrian texts a primary semantic field of the liver was not just emotions generally, but more specifically the semantic fields of happiness and anger. we have also demonstrated how the results from pmi-embeddings can complement and add to close reading approaches in order to provide a more nuanced reading of neo-assyrian embodied emotions. we demonstrated that the emotion words most similar to the usage of libbu were in the emotional field of fear and anger, which does not align with traditional close reading approaches. furthermore, our results demonstrate that emotions were only one facet of the many usages of libbu in neo-assyrian texts. pmi-embeddings is therefore a tool that can both corroborate results from close-readings, but more importantly has provided new perspectives on how neo-assyrian texts embodied emotions. on the basis of this single case study, pmi-embeddings will become vital in the suite of tools for digital assyriology.