T wist L ist: Resources and Baselines for Tongue Twister Generation (2023)

Abstract: Previous work in phonetically-grounded language generation has mainly focused on domains such as lyrics and poetry. In this paper, we present work on the generation of tongue twisters - a form of language that is required to be phonetically conditioned to maximise sound overlap, whilst maintaining semantic consistency with an input topic, and still being grammatically correct. We present TwistList, a large annotated dataset of tongue twisters, consisting of 2.1K+ human-authored examples. We additionally present several benchmark systems (referred to as TwisterMisters) for the proposed task of tongue twister generation, including models that both do and do not require training on in-domain data. We present the results of automatic and human evaluation to demonstrate the performance ofexisting mainstream pre-trained models in this task with limited (or no) task specific training and data, and no explicit phonetic knowledge. We find that the task of tongue twister generation is challenging for models under these conditions, yet some models are still capable of generating acceptable examples of this language type.

## introduction phonetically constrained language generation is a primary subarea of computational creativity in natural language generation (nlg), primarily encompassing lyric and poetry generation @xcite @xcite @xcite , as well as pun generation @xcite @xcite , and continues to prove challenging for myriad reasons. primarily, such works require the inclusion of phonetic factors such as metre and rhyme, which involves careful consideration of candidate vocabulary on the syllable level, leading to a reduced pool of allowable vocabulary once these constraints are in place. in this paper, we present work on the generation of tongue twisters, a type of phonetically constrained language that is rarely explored in the nlg community. as a form of creative generation, tongue twisters can facilitate numerous useful applications, including: (1) being used as a pedagogical tool @xcite @xcite ; (2) as a source of humorous entertainment stemming from unintentional mispronunciations; tongue twister generation posits unique challenges compared to other generation tasks. one of the most pertinent features of tongue twisters is the presence of high levels of phonetic overlap across tokens @xcite . consequently, whilst other types of creative generation may require only some output tokens to consider phonetics (such as rhyme or syllable counts), tongue twisters present an extreme version of this problem where the phonetics of almost all generated tokens must be considered. this leads to a very small vocabulary from which to choose semantically relevant words, and presents further challenges with maintaining grammatical validity. the only work that we are aware of on tongue twister generation at the time of conducting this research is by @xcite , who present models that train on graphemes and phonemes, and take either a starting prompt to be continued, or keywords around which to theme an output. they release tt-corp, a dataset of 644 tongue twisters with parallel non-twister equivalents. we differentiate our work through the release of a dataset that is over 3x larger and which has undergone substantial human quality control. furthermore, we assess the results of a wider range of popular pre-trained models on this task, including chatgpt, without explicit injection of phonetic knowledge due to the difficulty in encoding phonetics and the expertise required to utilise phonetic characteristics appropriately. our experimental results show that most popular pretrained language models (plms) rely on pure word repetition to generate tongue twisters, whilst some (i.e. bart) are able to generate more sophisticated examples. additionally, very large zero-shot models (i.e. chatgpt) are able to generate convincing tongue twisters almost on-par with human equivalents. 1 to summarise our contributions, we present: • twistlist, a large annotated dataset of humanauthored tongue twisters, containing 2.1k+ examples with human evaluation of their quality. ## case study within the example in ## conclusion we present work on the topic of tongue twister generation, a form of phonetically-constrained language generation that aims to maximise phonetic overlap, whilst conveying meaningful semantics. we motivate the potential application domains for such generated language, and provide a large annotated dataset of tongue twisters, twistlist, to encourage further work. finally, we present a series of benchmark models alongside automatic/human evaluation to assess generation quality.