# Poetic or Humorous Text Generation: Jam Event at PFIA2022

Anne-Gwenn Bosser, Liana Ermakova, Florence Dupin de Saint-Cyr, Pierre de Loor, Victor Charpenay, Nicolas Pépin-Hermann, Benoît Alcaraz, Jean-Victor Autran, Alexandre Devillers, Juliette Grosset, et al.

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Anne-Gwenn Bosser, Liana Ermakova, Florence Dupin de Saint-Cyr, Pierre de Loor, Victor Charpenay, et al.. Poetic or Humorous Text Generation: Jam Event at PFIA2022. 13th Conference and Labs of the Evaluation Forum (CLEF 2022), Sep 2022, Bologna, Italy. pp.1719-1726. [hal-03795272](https://hal.science/hal-03795272)

HAL Id: hal-03795272

https://hal.science/hal-03795272

Submitted on 4 Oct 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Poetic or Humorous Text Generation: Jam Event at PFIA2022

Anne-Gwenn Bosser<sup>1,2</sup>, Liana Ermakova<sup>3,4</sup>, Florence Dupin de Saint-Cyr<sup>5</sup>, Pierre De Loor<sup>1,2</sup>, Victor Charpennay<sup>6</sup>, Nicolas Pepin-Hermann, Benoît Alcaraz, Jean-Victor Autran<sup>8</sup>, Alexandre Devillers, Juliette Grosset, Aymeric Hénard<sup>10,2</sup> and Florian Marchal-Bornert<sup>11</sup>

## Abstract

In this paper, we describe the organization and the results of a Jam event on computational creativity at a major French national conference in AI PFIA2022. During this Jam event, participants were challenged by the task of humor or poetry generation using a number of resources and corpus.

## Keywords

Computational creativity, Computational Humour, Humour generation, Wordplay, Artificial Intelligence, Scientific mediation, Large pre-trained models, Few-shot learning

## 1. Introduction

The first edition of the Poetic and/or Humorous Text-generation Jam organized by the French Association for Artificial Intelligence (AFIA<fnref n="1" />) took place in Saint-Étienne, France, from June 27 to July 1, 2022 during the platform of AI conferences PFIA2022 <fnref n="2" />. This annual French-speaking event in AI gathers nine conferences, five workshops, five thematic days for working groups, and ten tutorials (450 registered participants in total).

Inspired by the Game Jam subculture, at this Generation Jam, we proposed to participants to play together to generate texts by experimenting with tools related to the practice of AI. A jam is a playful event promoting creativity. It is an opportunity for experimentation, where skill-sharing and learning are encouraged.

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<footnote n="1"><sup>1</sup>ENIB, Brest</footnote>

<footnote n="2"><sup>2</sup>Lab-STIC CNRS UMR6285</footnote>

<footnote n="3"><sup>3</sup>Université de Bretagne Occidentale, HCTI, Brest, France</footnote>

<footnote n="4"><sup>4</sup>Maison des sciences de l'homme en Bretagne, Rennes, France</footnote>

<footnote n="5"><sup>5</sup>IRIT CNRS UMR5505</footnote>

<footnote n="6"><sup>6</sup>EMSE</footnote>

<footnote n="8"><sup>8</sup>IRL CROSSING, ArianeGroup</footnote>

<footnote n="9"><sup>9</sup>ECAM Rennes</footnote>

<footnote n="10"><sup>10</sup>Université de Bretagne Occidentale, Brest, France</footnote>

<footnote n="11"><sup>11</sup>LORIA</footnote>

<footnote n="1">¹https://afia.asso.fr/</footnote>

<footnote n="2">²https://ci.mines-stetienne.fr/pfia2022/</footnote>
In this paper, we describe the organization of this event, which made extensive use of the resources provided by the CLEF workshop JOKER [1] as well as other resources. We also discuss the outputs and the feedback on the participants’ experience. We believe that such resources are a key for easily reproducible scientific mediation events, providing a relatively uncontroversial context for discussing the capabilities and limitations of AI with a general audience.

## 2. Event preparation

Before the event, a sample of resources including research papers with code were gathered by the organizers to provide inspiration with early or recent work.

* A system by He, Peng, and Liang [2] for generating puns using a language model and semantic distance to create a humorous effect *Pun Generation with Surprise* <fnref n="3" />;

* A system by Weller, Fulda, and Seppi [3] for creating humorous headlines from non-humorous headlines: *Humorous Headline Generation via Style Transfer (a.k.a. Humor Translation)* <fnref n="4" />;

* Tim Van de Cruys’ poetry generation system [4], *Automatic Poetry Generation from Prosaic Text* <fnref n="5" />;

* How to generate puns and make people laugh with taboo, from a corpus of SMS, by Valitutti, Toivonen, Doucet, and Toivanen [5]. *"Let Everything Turn Well in Your Wife": Generation of Adult Humor Using Lexical Constraints"* <fnref n="6" />;

* *The Oulipo constraints* <fnref n="7" />;

* The page of Darius Kazemi <fnref n="8" />, artist in procedural generation;

* *La Machine à Ecrire* <fnref n="9" /> from Jean Baudot (1964).

We also provided tools and data suggestions:

* Lexicon 3 <fnref n="10" />, and the openlexicon project (Boris New and Christophe Pallier). This very rich lexicon for the French language allows for example to obtain the phonetic form, the frequency of use, and many other morphological and linguistic information to process words. As many NLP libraries for the French language are much more limited when compared to their English language counterparts, this lexicon proved to be an essential building block for most of the participants’ creation. Besides, we provided a custom-made lexicon of paraphones for French nouns and adjectives. We considered as candidate paraphones words whose phonetic transcription were Levenshtein-distant by less than 2.

<footnote n="3">³https://github.com/hhexiy/pungen</footnote>

<footnote n="4">⁴https://github.com/orionw/humorTranslate</footnote>

<footnote n="5">⁵https://github.com/timvdc/poetry</footnote>

<footnote n="6">⁶https://aclanthology.org/P13-2044/</footnote>

<footnote n="7">⁷https://www.oulipo.net/fr/contraintes</footnote>

<footnote n="8">⁸https://tinysubversions.com/</footnote>

<footnote n="9">⁹https://archive.org/details/xfoml0001/page/n13/mode/2up</footnote>

<footnote n="10">¹⁰http://www.lexique.org/</footnote>
* Semantic networks for helping to compute the semantic distance of words: for example ConceptNet <fnref n="11" /> which uses word embeddings <fnref n="12" />, or Fasttext [6] <fnref n="13" />.

* the corpus of puns collected during the JOKER workshop at CLEF 2022 <fnref n="14" /> [7, 1].

* A GPT-3 playground <fnref n="15" />: after registration, many prompt tools are available to experiment with this powerful generator.

These resources were previously tested for reproducibility, and tried by several students interning for small-scaled projects. These projects allowed us to verify that the Jam was correctly dimensioned for the audience and the timespan of the conference (level of challenge, time to complete a workable solution). For instance, these 2 projects tried out Large Language Models:

* Interactive poetic generation in French with GPT-2 <fnref n="16" />

* Generation of wordplay in French with the Jurassic language model and Fasttext <fnref n="17" />.

Specific challenges were proposed by the event coordinators or scientific organizations. These took the form of themes or constraints on the generated text. They were unveiled during the launching session:

1. Theme: **Haikuweather**. Constraints: the texts must be about the weather.

2. Theme: **Together**. Constraints: the texts must emphasize the human-machine complementarity in creation.

3. Theme: **Guybrush Threepwood**. Constraints: the generated texts must be in the style of a famous video game character. (challenge proposed by the working group AI and Games of the CNRS Research Group Formal Aspects of AI.)

4. Theme: **Source code**. Constraints: the texts are generated from source code as input to the system (challenge proposed by the IRILL ¹⁸¹⁹).

## 3. Participation

The jam was intended to occupy the quiet times of the conference and to provide a physical space for sociability.

Conference participants could join the jam at any time. Registration gave access to a Discord group where resources were shared. All that was needed was a laptop or even a phone (one of the participants was able to download word2vec on his smartphone), equipped with the tools of their choice. The theme of the event was chosen for the range of AI techniques that could be used, as well as its potential for inclusiveness. Everyone could of course come with the objective of illustrating the creative power of their AI specialty, armed with their model

<footnote n="11">¹¹https://conceptnet.io/</footnote>

<footnote n="12">¹²https://github.com/commonsense/conceptnet-numberbatch</footnote>

<footnote n="13">¹³https://fasttext.cc/</footnote>

<footnote n="14">¹⁴https://www.joker-project.com/clef-2022/EN/project.html</footnote>

<footnote n="15">¹⁵https://beta.openai.com/playground</footnote>

<footnote n="16">¹⁶https://git.enib.fr/deloor/poesygeneration/-/tree/pytorch</footnote>

<footnote n="17">¹⁷https://gitlab.com/loicgle/computational-humor-pun-generation</footnote>

<footnote n="18">¹⁸https://www.irill.org/</footnote>

<footnote n="19">¹⁹Initiative for Research and Innovation on Free Software</footnote>
previously trained for the occasion. But participants could also come with the intention of
simply experimenting with previously unknown techniques or libraries.

After the first full meeting where the challenges were unveiled, daily gathering allowed to put together the texts generated to discuss the most promising approaches. These sometimes involved guests specialized in computational creativity. A selection of the texts generated were selected and displayed daily for the benefit of the conference attendees. The closing session took place during the last coffee break of the conference and allowed to reflect about the event and gather ideas for future developments.

4. Jam Progress

The first session of the first day presented the jam principle, challenges and resources available. It ended with a round table discussion where everyone introduced themselves, and started to sketch out ideas to develop. Most of the participants had a computer science or theoretical computer science background, except for a doctoral student in art history member of the CulturIA project<fnref n="20" />.

Several ideas were progressively sketched-out:

* combining two poems with very contrasting styles such as in the mash-up Pride and Prejudice and Zombies [8]. In the same vein, transforming an existing text using a very different style (e.g. from classic French Molière vs rap music);

* generating portmanteaux-words by joining two words from Lexique 3 with an important semantic distance (in order to generate a maximum contrast). This evolved towards the generation of riddles based on the template "Mr and Mme X have a child, what is its name?";

* prompting the large language model "Jurassic" with several examples of puns from the JOKER-2022 Corpus, all following a strong template, to obtain new ones. This later evolved towards using GPT-3.

* Various applications of Oulipo constraints (eg. Haikuisation)

During one of the sessions, the participants benefited from Alessandro Valitutti’s presentation of his recent research on computational humor [9, 10, 11] accompanied by a demonstration of the protocol described in [12].

Although we did not evaluate the results formally, the audience of the conference provided feedback indicating that generated text improved during the course of the conference. Here are some of the outputs that were produced and displayed in the conference hall (in French):

Day 1 : *Le ballon a rencontré son petit-ami au bal*. (obtained with Jurassic prompts)

Day 2 : Portmanteaux word: *Narguiléviathan*; Pun: *Les vieux jardiniers ne meurent jamais, ils sèment juste leurs dernières années*.

Day 3 : "Gouache wesh ! Maintenant chevalet faire un peu de peinture" dit Tom d'un air pincé.

<footnote n="20">²⁰CulturIA project examines the cultural history of artificial intelligence through the combined perspectives of the history of science, the history of ideas, the sociology of science and technology, and fieldwork.</footnote>
Day 4 : Movie Summary generated from the previoulsy generated portmanteaux word: Film title = Narguiléviathan, Summary=La petite fille fleur bleue et son narguilé en forme de leviatan souffrent d'une même maladie : l'insouciance.

Day 5 : Movie summaries gained silliness for real movie titles and made-up ones.

- Antman: Un gars petit comme une fourmi devient gros comme un éléphant et ça le sauve d'une vie de merde.

- Coco veut un gâteau: Un petit garçon mexicain veut un gâteau pour son anniversaire, les araignées s'en chargent.

- Une jambe, un œil et une main, la révolte des faux: Une jambe, un œil et une main se révoltent contre leur propriétaire pour être traités comme des êtres humains.

Genetic algorithms transformed the style of sentences, e.g. from "Les oiseaux ça vole et ça fait des cercles dans le ciel." to "les oiseaux qui volent, stupides, qui semblent démanteler craintivement le ciel." A Haiku was also generated using GPT-3 prompted with weather forecast:

Contre vents et marées
Il y a des gens qui restent,
Pas de risque de pluie !

## 5. AI methods used by the participants

A number of AI techniques were either used or considered by the participants.

The generation of portmanteaux-words was done by combinatorial combination of words from a lexicon including information for syllable decomposition and by using pre-trained neural networks for word embeddings.

For the generation of poems, an approach was developed based on genetic programming, starting from an input sentence and performing mutations on the sentences.

Pun generation with Large Language Models relied on prompts made up of labeled puns that were part of the CLEF JOKER-2022 data. Other prompts were handcrafted (such as the title and spoof summaries of movies).

Finally, although this was not finalized during the jam, we considered using Multi-valued Decision Diagrams such as in [13] which were presented at the same conference.

One of the insights from this experiment was that the curation of the generated results is very important. For the portmanteaux-words, it was initially done by humans, but later automated based on score assignment depending on the semantic distances. We assumed that the semantic distance is related to the level of incongruity.

Although we did not conduct any experiment to confirm this, we couldn't help but notice that participants laughed more when they were more involved (in terms of efforts) in the process of joke creation. Jokes and poetry generated by GPT-3 with simple prompts, even when successful, seemed to leave people dubious and suspicious, checking if they were not just already present on the internet for instance before recognizing the creativity of the latest Large Language Models (see Figure 1). This is similar to the "IKEA effect" that has been noticed in the context of music generation [14].
<figure><img src="image_5.png" /><figcaption>**Figure 1:** Prompting the model with a code input and an instruction asking for a joke, it generates two of them.</figcaption></figure>

## 6. Conclusion

The event attracted about forty participants. Although the involvement of the participants varied, we consider it a success. The Jam raised several questions that we plan to investigate further:

* How much would an isolated person enjoy an artificially created joke? In other words, without the jam collective, would people have laughed as much?

* How to evaluate the creativity of the Large Language Models? The difficulty to appreciate jokes which are produced with minimum human involvement may come either from an "Ikea effect" or a lack of trust towards the actual creativity of the AI that we were after.

For more information, you can visit the AFIA web page dedicated to this competition: www.cecilia-afia.fr/.

Due to the interest of the public and the participants in the jam and its themes, we plan to organize similar events in the future, including for a general audience.
## Acknowledgements

The entire organizing committee would like to warmly thank the local organizing committee of PFIA'2022 and the volunteer staff. We also thank the participants for investing time and energy, even when they did not end up sharing a generated text.

A number of people helped to set up this event: Rym Bousrih, Erwan Claquin, Sarah Clauzade-Robert, Kevin Cornut, Loic Glémarec, Oumaima Oumaadi, and Maxime Yonnet.

## References

[1] L. Ermakova, T. Miller, F. Regattin, A.-G. Bosser, E. Mathurin, G. L. Corre, S. Araújo, J. Boccou, A. Digue, A. Damoy, B. Jeanjean, Overview of JOKER@CLEF 2022: Automatic Wordplay and Humour Translation workshop, Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Thirteenth International Conference of the CLEF Association (CLEF 2022) 13390 (2022).

[2] H. He, N. Peng, P. Liang, Pun Generation with Surprise, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, 2019, pp. 1734–1744. URL: https://aclanthology.org/N19-1172. doi:10.18653/v1/N19-1172.

[3] O. Weller, N. Fulda, K. Seppi, Can humor prediction datasets be used for humor generation? humorous headline generation via style transfer, in: Proceedings of the Second Workshop on Figurative Language Processing, Association for Computational Linguistics, Online, 2020, pp. 186–191. URL: https://aclanthology.org/2020 figlang-1.25. doi:10.18653/v1/2020 figlang-1.25.

[4] T. Van de Cruys, Automatic poetry generation from prosaic text, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, 2020, pp. 2471–2480. URL: https://aclanthology.org/2020.acl-main.223. doi:10.18653/v1/2020.acl-main.223.

[5] A. Valitutti, H. Toivonen, A. Doucet, J. M. Toivanen, “Let Everything Turn Well in Your Wife”: Generation of Adult Humor Using Lexical Constraints, in: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Association for Computational Linguistics, Sofia, Bulgaria, 2013, pp. 243–248. URL: https://aclanthology.org/P13-2044.

[6] P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, Enriching word vectors with subword information, Transactions of the Association for Computational Linguistics 5 (2016). doi:10.1162/tac1_a_00051.

[7] L. Ermakova, T. Miller, O. Puchalski, F. Regattin, E. Mathurin, S. Araújo, A.-G. Bosser, C. Borg, M. Bokiniec, G. L. Corre, B. Jeanjean, R. Hannachi, G. Mallia, G. Matas, M. Saki, CLEF Workshop JOKER: Automatic Wordplay and Humour Translation, in: M. Hagen, S. Verberne, C. Macdonald, C. Seifert, K. Balog, K. Nørvåg, V. Setty (Eds.), Advances in Information Retrieval, Lecture Notes in Computer Science, Springer International Publishing, Cham, 2022, pp. 355–363. doi:10.1007/978-3-030-99739-7_45.
[8] J. Austen, S. Grahame-Smith, Pride and Prejudice and Zombies: The Deluxe Heirloom Edition, Chronicle Books, 2009.

[9] A. Valitutti, Humor facilitation of polarized events, in: International Conference on Distributed, Ambient, and Pervasive Interactions, Springer, 2018, pp. 337–347.

[10] A. Valitutti, Making fun of failures computationally, in: International Conference on Distributed, Ambient, and Pervasive Interactions, Springer, 2017, pp. 684–695.

[11] A. Valitutti, N. Novielli, Polarity contrast for the detection of verbal irony., in: NL4AI@AI* IA, 2017, pp. 51–56.

[12] A. Valitutti, T. Veale, Infusing humor in unexpected events, in: International Conference on Distributed, Ambient, and Pervasive Interactions, Springer, 2016, pp. 370–379.

[13] A. Bonlarron, A. Calabrèse, P. Kornprobst, J.-C. Régin, Génération de texte sous contraintes pour mesurer des performances de lecture: Une nouvelle approche basée sur les diagrammes de décisions multivalués, in: Journées Francophones de Programmation par Contraintes (JFPC), 2022.

[14] F. Pachet, P. Roy, B. Carr'e, Assisted music creation with flow machines: towards new categories of new, in: Handbook of Artificial Intelligence for Music, Springer, 2021, pp. 485–520.