A fri KI : Machine-in-the-Loop A frikaans Poetry Generation (2021)

Abstract: This paper proposes a generative language model called AfriKI. Our approach is based on an LSTM architecture trained on a small corpus of contemporary fiction. With the aim of promoting human creativity, we use the model as an authoring tool to explore machine-in-the-loop Afrikaans poetry generation. To our knowledge, this is the first study to attempt creative text generation in Afrikaans.

## introduction afrikaans foot_0 is a language spoken largely in south africa, namibia, botswana and zimbabwe. @xcite ) draws important attention to the current disproportion of nlp research and resources with respect to african languages. in fact, in the entire acl anthology, 2 of the thirteen studies that mention "afrikaans" in their titles, only four @xcite @xcite appeared in the last five years. by no means do we ignore studies with inclusive @xcite and multilingual approaches @xcite or those published via other platforms @xcite . this is simply an indication that nlp research in afrikaans is limited, especially in comparison to resource-rich languages, i.e. the so-called "winners" in the taxonomy of @xcite . in this paper, we present a generative language model called afriki, an abbreviation for "afrikaanse kunsmatige intelligensie" (afrikaans artificial intelligence). we use this model as an authoring tool to explore machine-in-the-loop poetry generation in afrikaans. machine-in-theloop frameworks promote human creativity through computational assistance, as opposed to human-inthe-loop pipelines, which aim to strengthen machine learning models @xcite . we treat poetry generation as a hybrid system, an experimental approach that enables the generation of high-quality poetic text with very limited data. to our knowledge, this is the first study in creative text generation as well as an initial step towards automatic poetry generation in afrikaans. whereas nlg in its quest for full automation may frown upon human involvement, our humancentred framework does the opposite. according to @xcite , one criticism of artificial intelligence programs that claim to be creative is exactly that a human plays a role at some point, which reduces the autonomy of the machine. from the hci perspective [...] these "failed" ai creativity programs are examples of successful human-computer interactions to facilitate creativity. this study demonstrates that human-machine collaboration could enhance human creativity. we agree with @xcite that support tools "make more people more creative more often". ## related work several computational models focus on automatic poetry generation. first approaches follow rulebased, template-based systems @xcite . @xcite and @xcite apply genetic algorithms while @xcite and @xcite use statistical machine translation, with @xcite utilising text summarisation to generate poetry. @xcite provides a clear overview of early systems and presents a comparable method (2012). starting with zhang and lapata (2014), we have seen great advancements in poetry generation using neural networks. wang et al. (2016a) extend this using the attention mechanism @xcite . there are many attempts to improve the quality of learning-based generated poetry, by using planning models @xcite , finitestate machinery @xcite , reinforcement learning @xcite as well as variational autoencoders @xcite . conventional recurrent neural networks (rnn) are not suitable for learning long range dependencies @xcite due to the vanishing gradient problem @xcite . long short-term memory (lstm) networks @xcite the remarkable quality and results of these studies are indisputable. however, they all concentrate on data-rich languages such as english, chinese, italian and russian. for example, the character language model of @xcite uses a poetry corpus consisting of 7.56 million words and 34.34 million characters. likewise, a recent study by @xcite trained on over 200 thousand poems and 3 million ancient chinese prose texts. we trained an lstm network for poetic text generation as well. however, our approach differs in significant ways. first, whereas these studies generate verse in a fully automatic manner, we emphasise human creativity, introducing a strong computational component to the creative writing process. second, the aforementioned studies either trained on comprehensive poetry datasets or model poetic qualities. to illustrate the latter, the recent work of van de cruys (2020) focuses on specifically nonpoetic text in english and french, however, is able to model the rhyme constraint using phonetic representation of words from wiktionary. since there is no publicly available large-scale poetry dataset in afrikaans, we follow an alternative approach, constructing our model as a text generator that produces individual sentences and phrases instead of stanzas of verse. in other words, the model outputs a set of lines, which we arrange vertically into short poems without modification. ## model in this section, we explain the dataset, model architecture as well as the co-creative poetry generation process. corpus: afriki trained on a lengthy (208,616word) literary novel titled die biblioteek aan die einde van die wêreld (the library at the end of the world) (van heerden, 2019) by the south african novelist etienne van heerden. in 2020, the book was awarded the university of johannesburg prize for literature @xcite . this work of new journalism combines fictional techniques with documentary language, and is particularly suitable given its use of rich imagery, figurative language as well as different afrikaans varieties like kaaps (or cape afrikaans) and standard afrikaans. figure 1 shows a word cloud of its most commonly used words. model architecture: experimenting with several architectures, including lstm, multi-layer lstm and bi-lstm, we obtain best results with the following two-layer lstm architecture. we use a vanilla lstm structure @xcite and, to avoid repetitiveness, omit to describe the network diagram and equations, similar to @xcite . we start with 100-dimensional word embeddings with a vocabulary size of 23,317 words, where weights are randomly initialised from a normal distribution with zero mean and standard deviation 0.01. next, we stack two lstm layers with 50 units in each layer followed by dropout layers with the ## results trained on prose, the text is generated as free verse (i.e. free from the restrictions of rhythm and rhyme) which we associate with contemporary poetry. in the lines, various poetic devices can be identified, such as alliteration (e.g. "golwe van gister") and assonance (e.g. "maak 'n vraag waarbinne"). the generated lines abound with figurative language as well. as an instance of an extended metaphor, the first stanza of the second poem suggests sensitivity to the country's turbulent history. personification is particularly prevalent, lending a visceral quality to the text: africa drinks, the landscape tilts its back, the sea breathes, and table mountain poses a question. the imagery is vivid, portraying sight (tonight's curtains / glitter at sliding windows / in the city lights), smell (the introspective sea / lifts the scents and utters / a breath) and sound (roar of a wave). the language can be described as minimalist, evocative and abstract, and therefore open to interpretation, resembling imagist and surrealist poetry. afrikaans has a rich poetic tradition (brink and opperman, 2000), and we believe that creative text generation has the potential to enrich poetic lan-guage. alongside afrikaans varieties, the corpus contains some english as well, which influenced the generated text in interesting ways. as one example, it is grammatically incorrect in standard afrikaans to use "sun" as both noun and verb, e.g. "to sun in the garden". the model, however, adopted this and other patterns from the english, generating novel phrases (that do not sound anglicised) such as "sonlig son die promenade" -sunlight suns the promenade. ## conclusion in this study, we present afrikaans poetry generation in a machine-in-the-loop setting. each and every line of poetry is automatically generated by the proposed lstm network. in order to clearly identify the machine's contribution to the process, the human writer's interaction is limited to the selection and vertical arrangement of the lines -without any modification. we believe this is the first creative text generation study in the afrikaans language. more broadly, the work encourages humancentred design in low-resource languages. creative industries would benefit from co-creative tools and methods @xcite , perhaps more than fully automatic approaches.