How do Language Models Generate Slang: A Systematic Comparison between Human and Machine-Generated Slang Usages (2025)

Abstract: Slang is a commonly used type of informal language that poses a daunting challenge to NLP systems. Recent advances in large language models (LLMs), however, have made the problem more approachable. While LLM agents are becoming more widely applied to intermediary tasks such as slang detection and slang interpretation, their generalizability and reliability are heavily dependent on whether these models have captured structural knowledge about slang that align well with human attested slang usages. To answer this question, we contribute a systematic comparison between human and machine-generated slang usages. Our evaluative framework focuses on three core aspects: 1) Characteristics of the usages that reflect systematic biases in how machines perceive slang, 2) Creativity reflected by both lexical coinages and word reuses employed by the slang usages, and 3) Informativeness of the slang usages when used as gold-standard examples for model distillation. By comparing human-attested slang usages from the Online Slang Dictionary (OSD) and slang generated by GPT-4o and Llama-3, we find significant biases in how LLMs perceive slang. Our results suggest that while LLMs have captured significant knowledge about the creative aspects of slang, such knowledge does not align with humans sufficiently to enable LLMs for extrapolative tasks such as linguistic analyses.

## introduction slang is a type of informal language that is commonly used in colloquial speech @xcite . the use of slang is both creative @xcite and ephemeral @xcite , meaning that slang is not only more difficult to comprehend compared to conventional language, but it is also necessary to handle an ever-evolving repertoire of novel slang usages. such characteristics of slang necessitate natural language processing (nlp) systems that can adapt to unseen slang usages without expansive retraining. while earlier work on nlp for slang often resorts to retrieval-based systems that can hardly generalize (e.g., pal and saha, 2013), recent work has been successful in developing systems for the automatic detection @xcite @xcite , generation @xcite , and interpretation @xcite @xcite of slang that can generalize well toward novel slang usages that have never been seen during training. in particular, large language models (llms) have been very effective in many tasks involving slang under both zero-shot and few-shot settings, suggesting that the llms have, to some extent, captured structural knowledge about slang that enables generalization. it is not well understood, however, what underlying structures about slang the llms have captured and whether they are comparable to human knowledge. such insights are valuable in gauging the reliability of downstream applications that require a precise characterization of slang. for example, a model with misaligned knowledge would be sys-tematically biased toward detecting certain types of slang usages. slang usages generated by such models can also misinform downstream agents that consume the content (e.g., model distillation), as well as linguistic analyses that rely on llms for automatic annotation. we make a first step toward the interpretation of llms' internal knowledge about slang by collecting a dataset of machine-generated slang usages from @xcite and llama-3 (ai@meta, 2024) that span a diverse set of controlled conditions. specifically, we prompt the llms to generate novel slang usages that each encompasses 1) a slang term, 2) a sense definition sentence, and 3) a usage context. we perform both controlled generation where the model is provided with existing senses from a slang dictionary and an uncontrolled setting where the model is allowed to generate slang usages attached to any senses. under each condition, we further constrain the type of word choices that can be made. this includes 1) lexical coinage where the the model is prompted to create a new term, 2) word reuse where the model chooses an existing term in the lexicon, and 3) free-form generation without any restrictions. we collect at least 1,000 valid generations in each setting for a total of 58,197 machinegenerated slang usages foot_0 . using this dataset, we make a systematic comparison between human-generated slang usages attested by slang dictionaries and machine-generated slang usages. illustrated in figure 1 , we propose an evaluative framework that makes comparisons along three core aspects: 1) our framework compares the aggregate usage characteristics of the generated slang usages to discern any systematic biases in how machines perceive slang; 2) it measures and compares the creativity of slang usages, examining morphological complexity and semantic coherence of coined terms as well as semantic novelty and contextual surprisal for cases of reuse; 3) model distillation is performed using both human and machine-generated slang usages as examples to measure their informativeness toward a diverse set of nlp tasks for slang. our results show that while llms have captured sufficient creative knowledge to generate plausible slang usages, the generated usages still deviate significantly from human-generated usages in certain aspects. we make the following contributions in this paper: 1) a dataset of slang usages generated by gpt-4o and llama-3 that enables studies of machinegenerated informal language; 2) an evaluative framework that assesses knowledge alignment between human and machine-generated language use; 3) the first systematic comparison between human and machine-generated slang usages. ## data we first collect sets of slang usages generated by both humans and machines. we use attested slang usages from the online slang dictionary (osd) foot_1 as the set of human-generated slang. in osd, each slang usage consists of: ## conclusion we have presented the first systematic comparison between human and machine-generated slang usages. our results suggest that while llms such as gpt-4o achieve strong results on a wide range of evaluative tasks involving slang, structural knowledge about slang encoded in these models show notable distinctions when compared to real usages from humans. specifically, llms show strong preferences toward certain characteristics and creative qualities, and such preferences can affect how the generated usages inform their users. our findings suggest that although llms are capable of processing slang in ways that reflect many aspects of human knowledge, they have not yet fully captured nuanced structures in human slang usage.