Cards Against AI : Predicting Humor in a Fill-in-the-blank Party Game (2022)

Abstract: Humor is an inherently social phenomenon, with humorous utterances shaped by what is socially and culturally accepted. Understanding humor is an important NLP challenge, with many applications to human-computer interactions. In this work we explore humor in the context of Cards Against Humanity – a party game where players complete fill-in-the-blank statements using cards that can be offensive or politically incorrect.We introduce a novel dataset of 300,000 online games of Cards Against Humanity, including 785K unique jokes, analyze it and provide insights. We trained machine learning models to predict the winning joke per game, achieving performance twice as good (20%) as random, even without any user information.On the more difficult task of judging novel cards, we see the models’ ability to generalize is moderate. Interestingly, we find that our models are primarily focused on punchline card, with the context having little impact.Analyzing feature importance, we observe that short, crude, juvenile punchlines tend to win.

## introduction humor is a universal phenomenon, fulfilling important social roles: approaching social taboos, expressing criticism against individuals and institutions, and consolidating a sense of belonging to a group @xcite . humorous utterances are shaped by what is socially and culturally accepted. humor underpins many social interactions @xcite . it increases likeability and trust @xcite . thus, humor is also a crucial component in developing personable human-computer interactions. specifically, we focus on the task of humor recognition -determining whether a sentence in a given context is funny. this task is difficult, as humor is a diverse, amorphous and complex phenomenon. it requires creativity and common sense, and is very challenging to model @xcite , considered by some researchers to be ai-complete @xcite . thus, designing a general humor recognition algorithm currently seems beyond our reach, and works on computational humor tend to focus on narrow, specific types of humor, such as knock-knock jokes, one-liners, or even that's-what-she-said jokes @xcite @xcite in this work we explore humor in the context of the immensely popular card game "cards against humanity" (cah). the game mechanics are simple: players are dealt ten cards ("punchlines"). the judge of the round draws a "prompt" card posing a question or a "fill-in-the-blank" statement. each player submits an answer from their hand, and the judge picks the winner. an example prompt is "tsa guidelines now prohibit ___ on airplanes". candidate punchlines are "goblins", "batman!!!", "poor people", and "the right amount of cocaine". importantly, many cards are offensive or politically incorrect. we introduce a novel dataset of 300k online cah games. while most current humor datasets are lacking in size @xcite , or have weak labels (e.g., upvotes without total views), our dataset is large and strongly labeled. we train machine learning models foot_0 to predict the winning joke per round and show models can somewhat generalize to novel (unseen) punchline cards. surprisingly, we find that our models primarily focus on the punchline card alone, and the impact of the prompt is limited. we also identify potential behavioral biases in the data. our main goal here is to explore humor through a data-driven lens, and we believe cah provides a unique perspective to this end. most existing studies on humor recognition formulate the problem as a binary classification task and try to recognize jokes via a set of linguistic features @xcite @xcite . one of the common problems those works face is the construction of negative instances, which are often sampled from a different domain (e.g., news). in contrast, the cah task does not suffer from this problem. perhaps the closest setting to ours is humorous fill-in-the-blank @xcite , where users complete a joke however they see fit. however, our setting is a lot more restricted: players choose (rank) an answer from a small set of options, enabling comparisons that would be hard to test on other corpora. from a humor-theory point of view, we believe cah serves as an interesting example of frame blends and frame shifts @xcite , where a speaker's mental model suddenly shifts to new situations, or two distinct situations create a hybrid. cah provides a relatively clean setting to explore this phenomenon, as the jokes are short, with simple syntax and narrative structure. to the best of our knowledge, cah has only been explored in the literature through pedagogical, ethical or sociological lenses (e.g., @xcite ), not computational or linguistic ones. we note the data contains offensive humor, and should be very carefully used as training data. however, we believe it is important to study offensive humor too and understand its role in generating and reinforcing social boundaries and inequalities. ## data the dataset consists of games played on the online cah labs website, https://lab. cardsagainsthumanity.com . the players played the game voluntarily, for fun; they are not our annotators or workers. in each round a user is presented with a random prompt card, 10 potential punchlines cards, and chooses the funniest punchline. the raw data had 298,955 past games (i.e., we did not perform any additional experimentation ourselves).there are 581 unique black prompt cards and 2,128 white punchline cards, including cards from the official cah game and expansions, resulting in 1,236,368 possible unique jokes (where a joke is the result of filling in the blank of the prompt card with a punchline). each round is effectively unique due to the large number of combinations. the data we received from cah did not include any demographic or geographic characteristics, user identifiers or personally identifiable information. 5% of games were skipped by users and were excluded, as were a minority of prompts that required picking more than one punchline. data is available upon request to cah at mail@cardsagainsthumanity.com. ## predicting winning jokes given a prompt and 10 punchlines (i.e., a round), our goal in this section is to predict the punchline most likely to be picked as funniest. we note that unlike in traditional user-item ranking or recommendation tasks, we lack any user information. ## novel punchlines given the finding in section 3, we set out to evaluate if our models merely memorized funny cards, or whether they could generalize to novel punchlines. ## discussion and conclusions in this work we explore humor in the context of the popular card game cards against humanity. we introduce a novel dataset of 300,000 online games and 785k unique jokes, analyze it and provide insights. we trained state-of-the-art machine learning models to predict the winning joke per game. interestingly, we find that past performance of the punchline card is a very strong indicator (unrelated to the prompt), and that short and crude punchlines tend to win. we show our models primarily focus on punchline, and observe potential behavioral biases in the data. on the more difficult task of judging novel cards, we see the models' ability to generalize is moderate, leaving room for future work. we believe humor is a crucial component in developing personable human-computer interactions, and the cah dataset has several characteristics rendering it particularly attractive for nlp research. we hope it will promote further work in this area.