Complex language modelstrained on huge text corpora have shown unparalleled text generation capabilities, and thanks to transfer learning, are accessible to a greater number. However, despite recent developments, users are not yet able to fully control particular aspects of the text produced. This is why we propose a finetuned OpenAI GPT-2 model for controllable and contextualised text generation specific to novels. By integrating it into a web-service , we would like to enable authors to write and ask for automatic text generation which is consistent with both previous and next paragraphs. They can specify the genre of their book, the length of the desired text, the entities it should mention and its content via keywords or a short summary . We explore the technical possibilities and limitations around these objectives .
It is standard procedure these days to solve Information Extraction task by fine-tuning large pre-trained language models. This is not the case for generation task, which relies on a variety of techniques for controlled language generation. In this paper, we describe a system that fine-tunes a natural language generation model for the problem of solving Writer's Block. The fine-tuning changes the conditioning to also include the right context in addition to the left context, as well as an optional list of entities, the size, the genre and a summary of the paragraph that the human author wishes to generate. Our proposed fine-tuning obtains excellent results, even with a small number of epochs and a total cost of USD 150. The system can be accessed as a web-service , and all the code is released. A video showcasing the interface and the model is also available .
R
Complex language modelstrained on huge text corpora have shown unparalleled text generation capabilities, and thanks to transfer learning, are accessible to a greater number. However, despite recent developments, users are not yet able to fully control particular aspects of the text produced. This is why we propose a finetuned OpenAI GPT-2 model for controllable and contextualised text generation specific to novels. By integrating it into a
It is standard procedure these days to solve Information Extraction task by fine-tuning large pre-trained language models. This is not the case for generation task, which relies on a variety of techniques for controlled language generation. In this paper, we describe a system that fine-tunes a natural language generation model for the problem of solving Writer's Block. The fine-tuning changes the conditioning to also include the right context in addition to the left context, as well as an optional list of entities, the size, the genre and a summary of the paragraph that the human author wishes to generate. Our proposed fine-tuning obtains excellent results, even with a small number of epochs and a total cost of USD 150. The system can be accessed as a
R
, we would like to enable authors to write and ask for automatic text generation which is consistent with both previous and next paragraphs. They can specify the genre of their book, the length of the desired text, the entities it should mention and its content via keywords or a short summary . We explore the technical possibilities and limitations around these objectives
, and all the code is released. A video showcasing the interface and the model is also available