Breaking Writer’s Block: Low-cost Fine-tuning of Natural Language Generation Models (2021)

Abstract: It is standard procedure these days to solve Information Extraction task by fine-tuning large pre-trained language models. This is not the case for generation task, which relies on a variety of techniques for controlled language generation. In this paper, we describe a system that fine-tunes a natural language generation model for the problem of solving writer’s block. The fine-tuning changes the conditioning to also include the right context in addition to the left context, as well as an optional list of entities, the size, the genre and a summary of the paragraph that the human author wishes to generate. Our proposed fine-tuning obtains excellent results, even with a small number of epochs and a total cost of USD 150. The system can be accessed as a web-service and all the code is released. A video showcasing the interface and the model is also available.

## introduction thanks to the powerful capacity of large neural networks based on the attention mechanism @xcite , the current practice in nlp is to start from pre-trained models, which were trained to predict words in context @xcite or to perform various other tasks @xcite . these pre-trained models are then fine-tuned to solve the task at hand: all top entries of the superglue benchmark foot_3 for instance follow this trend. ## related work recent progress in transfer-learning has shown that large pre-trained models are powerful enough to be quickly fine-tuned to solve natural language understanding tasks. diverging from this, current approaches to adapt generation models are generally based on picking carefully the prompt on which the text is to be generated. this was popularized by @xcite and has since then seen steady growth by different proposals aiming to find good prompts @xcite @xcite . a related approach is to adapt the model so that it presents desired biases. this can be done by training with control tokens @xcite or by adapting an existing model with additional layers @xcite or sampling techniques @xcite . those methods allow generating style variations of the same form. however, they are less well suited to changes in the conditioned text, such as providing not only a prefix but also a continuation of the text to be generated. we are particularly interested in conditioning on categorical variables, like grover @xcite , but without retraining. our experiments show that fine-tuning is surprisingly effective for this. several research directions have explored the use of languages models for creative writing and interactive story-telling @xcite . this also includes online tools such as plot generator foot_6 or talk to transformers, foot_7 where the control that can be exerted over the text remains quite rudimentary. of special inspiration for this work was ai dungeon. foot_8 our approach is separated into three main steps: (i) data preparation (ii) transformation of the data and (iii) fine-tuning: ## web service architecture the model was enriched with a user interface, and opened to a small targeted public (online community of authors), to gather relevant feedback on both model generation and user-friendliness of the interface. to gain in flexibility in the choice of instances, to perform the heavy computations and to allow load balancing on several instances, we uncoupled the master instance -serving the javascript frontend and general data -from the computational instances, performing ner and text generation on demand. it is also possible for the client to run the servers locally to avoid delays and server overloads. fig. 4 shows the general architecture of our service. the interface allows users to write some text in a simple editor. named entities of the four types (characters, locations, organisations and others) are detected on the fly by the ner backend and displayed on the left panel. they can be manually edited. users have the possibility to select several options: length of the desired paragraph, genre of their work and list of entities they want to see appear in the generation. they can also highlight a small part of the text that will act as a summary (or a list of keywords). a snapshot of the interface is shown in fig. 5 . ## generation and evaluation at inference time we provide the prefix x and generate until reaching the end-of-sentence symbol, using nucleus sampling @xcite with @xmath0.9. ## conclusion in this paper, we present an end-to-end pipeline allowing authors to break writer's block. the objective is to allow users -at any point during the creative writing process -to generate new paragraphs that are consistent with the rest of the writing, especially previous and following paragraphs. the presented tool gives the possibility to select entities (characters, locations, etc.) that have been previously introduced in the novel and that should appear in the target paragraph. similarly, the author can specify the size of the desired text, its content via a small summary or keywords and even the genre of the book. in the end, the tool proposes several suggestions that users can choose from and edit. the aim is to produce creative outputs that give new ideas to the writers. the underlying model is obtained by fine-tuning a gpt-2 model on a carefully designed dataset, obtained through a selection and cleaning of books from the project gutenberg library. our experiments show that the generated text is significantly more similar to the gold paragraphs on a variety of metrics and is able to successfully take into consideration the context specified by the user. fine-tuning is often discarded for natural language generation in favour of other cheaper methods, such as prompt engineering or adapter layers. this work shows a use-case where a pre-trained neural language generation model can be fine-tuned with a reduced economic and ecological cost: the complete training (including preliminary experiments as well as the final mode) was done with a budget of usd 150.