Understanding Slang with LLM s: Modelling Cross-Cultural Nuances through Paraphrasing (2024)

Abstract: In the realm of social media discourse, the integration of slang enriches communication, reflecting the sociocultural identities of users. This study investigates the capability of large language models (LLMs) to paraphrase slang within climate-related tweets from Nigeria and the UK, with a focus on identifying emotional nuances. Using DistilRoBERTa as the base-line model, we observe its limited comprehension of slang. To improve cross-cultural understanding, we gauge the effectiveness of leading LLMs ChatGPT 4, Gemini, and LLaMA3 in slang paraphrasing. While ChatGPT 4 and Gemini demonstrate comparable effectiveness in slang paraphrasing, LLaMA3 shows less coverage, with all LLMs exhibiting limitations in coverage, especially of Nigerian slang. Our findings underscore the necessity for culturally sensitive LLM development in emotion classification, particularly in non-anglocentric regions.

## introduction in the age of social media, platforms like x (formerly twitter) have become a vital medium for public discourse, where users express a wide array of views and emotions on various topics @xcite @xcite . however, sociocultural identities including regional background, gender, age, and sub-cultural affiliations have a big impact on communication styles. people often blend formal and informal language, incorporating specific dialects or slang into their discourse. discourse from non-anglocentric countries may thus contain cultural references and idioms that are not easily understood by outsiders. for instance, nigerian tweets may utilize pidgin english to convey emotions, such as the phrase "i dey happy no be small" meaning "i am very happy". emotion classification is a key task in sentiment analysis. despite llms' impressive capabilities in various linguistic tasks, they often encounter challenges in accurately capturing cultural nuances like emotions, resulting in inaccuracies, particularly in diverse settings @xcite . in this paper we focus on llms' knowledge of slang and how stateof-the-art models might misinterpret or overlook emotions in tweets containing slang across different varieties of english. we propose a novel approach to integrating detailed slang representations into llms. leveraging generative models such as openai's chatgpt 4 (openai, 2024), google's gemini @xcite , and meta's llama3 (ai@meta, 2024), we systematically investigate how paraphrased slang influences emotional expressions in tweets from diverse cultures, focusing specifically on nigeria and the uk. we make the following contributions: our study highlights the need to model slang in reducing biases within llms, especially in regions with diverse linguistic backgrounds. our research demonstrates the cultural insensitivity of llms for emotion classification in tweets from nigeria and the united kingdom (uk). by incorporating nigerian perspectives, we address a critical gap in understanding cultural nuances and linguistic expressions in underrepresented groups. modelling slang with llms in this paper, we focus on the effect of slang on the task of emotion classification, specifically comparing british and nigerian english. this links with previous studies that have explored cross-cultural context in slang analysis. @xcite introduce socvec, which aims to compute cross-cultural differences in understanding slang terms across languages. the method is evaluated on two tasks focused on mining crosscultural differences in named entities and slang. similarly, @xcite use llms to detect slang and attribute regional and historical context. despite gpt-4's high performance in zero-shot settings, the study reveals that smaller, fine-tuned bert models achieve comparable results. both studies underscore the significance of regional and cultural contexts in understanding slang. in a similar vein, @xcite introduced a computational framework for slang generation that incorporates syntactic and contextual knowledge. the framework leverages probabilistic inference and neural contrastive learning and outperforms existing language models in accurately predicting historical slang emergence from the 1960s to 2000s. @xcite compare classifiers for slang detection and highlight the syntactic shift of words as a key feature of slang. @xcite enhanced llms for chinese slang comprehension contrasting llm performance with a custom punchline entity recognition (per) system, integrating phonetic matching. also, sultan (2023) classify emotions in tweets containing slang based on wordnet for synonymous phrase generation and a cnn for classification. they show a significant improvement in emotion classification for slang-filled social media texts. furthermore, @xcite detect internet slang based on a hierarchical multi-task bert model, using two-layer annotation and word embeddings. the model excelled in identifying subcategories of internet slang, demonstrating the effectiveness of two-layer annotation. ## results and discussion in this section, we aim to determine the effect that slang, and understanding its correct meaning, has on emotion classification in tweets. to this end, we present two sets of results: (1) the emotion distribution in uk and nigerian english in the original climate tweets in section 3.1, and (2) the percentage changes in emotion distribution with each set of llm-generated paraphrases in section 3.2. figure 1 illustrates percentage changes between original and paraphrased tweets for each of the llms. it highlights distinct changes in emotion expression between nigerian and uk tweets when paraphrased with different models. nigerian tweets exhibit increased fear with chatgpt (10.36%), while uk tweets show decreased fear with llama3 (-11.31%) and gemini (-13.1%). anger decreases across both countries, notably in uk tweets paraphrased with chatgpt (-61.54%). paraphrased tweets often show heightened joy, especially in nigerian tweets paraphrased with llama3 (148.48%). additionally, both countries experience reduced neutral emotions postparaphrasing, indicating a shift towards more polarised language, or just highlighting that llms find it harder to discern emotions from slang. overall, the effects of paraphrasing slang show significant variation between countries and mod-els. nigerian tweets typically demonstrate more pronounced emotional shifts across all models, with chatgpt often amplifying emotions. in contrast, uk tweets exhibit more subtle changes, with llama3, gemini, and chatgpt each impacting emotions differently. this indicates the significant influence of both cultural context and modelspecific behaviour on emotion extraction. these variations may stem from inherent biases in llms towards underrepresented dialects, as evidenced by previous studies @xcite @xcite . ## conclusion in summary, our research evaluates the efficacy of llms in modelling slang, particularly in the context of climate-related tweets, though we speculate that our findings transfer to other topics. we observed significant emotional shifts when integrating slang paraphrases, in uk tweets and especially in nigerian tweets. the shifts vary across llama3, gemini, and chatgpt used for slang paraphrasing. furthermore, factors like extensive training data and commercial nature likely contribute to chatgpt's and gemini's observed superiority on the task in comparison with llama3. our study highlights potential biases in llms towards non-anglocentric regions and emphasizes the need for culturally-sensitive llm development. in future work, we plan to explore additional llms to facilitate a more comprehensive compari