Are U a Joke Master? Pun Generation via Multi-Stage Curriculum Learning towards a Humor LLM (2024)

Abstract: Although large language models (LLMs) acquire extensive world knowledge and some reasoning abilities, their proficiency in generating humorous sentences remains a challenge. Previous research has demonstrated that the humor generation capabilities of ChatGPT are confined to producing merely 25 unique jokes. In this work, we concentrate on endowing LLMs with the ability of generating puns, a particular category of humor by preference learning method. We propose a multi-stage curriculum preference learning framework to optimize both pun structure preferences and humor preferences. Specifically, we improve the Direct Preference Optimization (DPO) algorithm to address the challenge of multi-objective alignment problem. Besides, to facilitate further advancement in this field, we collect a Chinese Pun (ChinesePun) dataset, containing 2.1k puns and corresponding annotations. Experimental results on both Chinese and English benchmark datasets demonstrate that our method significantly outperforms all the baseline models.

## introduction humor serves various purposes and offers numerous benefits for humans, including the relief of anxiety, avoidance of painful emotions, and facilitation of learning @xcite . however, it remains a challenge to let machines generate humorous sentences as humans @xcite . empowering llms with a sense of humor has become an increasingly important topic in the field of natural language generation. as a particular category of humor, the creative utilization of puns, wordplay, and ambiguity plays a significant role in humor generation @xcite . puns, utilizing identical or phonetically similar words, are categorized into homographic and homophonic types * corresponding author. @xcite . homographic puns play on different meanings of the same word; for instance, 'all right' can signify 'satisfactory' or 'not left'. this word creates humor through contrasting interpretations within context. on the other hand, homophonic puns involve words with similar sounds but distinct meanings, like 'weak @xcite as pun generation aims to generate a pun to create humorous effects given a pair of pun word and alternative word as shown in to address this problem, we propose a multistage curriculum learning framework to generate a sentence satisfying both the provided pun structure and high humorous level. inspired by the paradigm of curriculum learning which learns samples from easy to hard, our model learns the two preference targets separately from easy to hard, i.e., first learns pun structure preference in stage 1, and then learns humor preference in stage 2. specifically, to address the inherent catastrophic forgetting problem type dataset pun pun word, alternative word homophonic chinesepun 篮球队教练在开会时抱怨道："我们球队的配置太差了，现在需要一个投篮的。"过了一会， 助理教练领着机器猫走了进来："教练，你看他的头蓝吗？" the coach of the basketball team was in a meeting and complained, "our team is so poorly set up, we need a shooter now." a little while later, the assistant coach walks in with robot cat: "coach, do you see his head in blue?" ## problem definition the input to our system consists of a pun word pair, which includes a pun word (p w , e.g., weak) and a corresponding alternative word (a w , e.g., week). p w are defined as words that evoke humor by possessing conflicting and ambiguous meanings within jokes. in the case of homophonic puns, the pun words have a homonym a w with the same or similar pronunciation but different meanings. for homographic puns where the pun word carries dual meanings that are logically coherent within the context, we adopt a same representation, setting p @xmath0, following @xcite . the objective is typically to generate a pun that contains p w with high humorous level, e.g., "i lift weights only on saturday and sunday because monday to friday are weak days.". thus, the generated pun should comply with two conditions: pun structure (e.g., containing pun word) and humorous. ## experiments 6.1 dataset to prove the effectiveness of our method, we conduct experiments on chinesepun and semeval 2017 task 7 dataset. ## conclusion in this work, we propose a multi-stage curriculum learning approach with the direct preference optimization technique to effectively endow llms with the capability of generating humorous puns. specifically, to alleviate the effects of catastrophic forgetting in the multi-objective preference alignment, we present a novel triplets preference learning schema, which is quite different from standard dpo. to verify the broad applicability of our method in different languages, we construct a new benchmark dataset, called chinesepun, which is the very first dataset on the chinese pun generation task. the evaluation shows that our method significantly outperforms all the existing methods in chinese and english, even for chatgpt.