Tiago Sousa Garcia and Kathryn Garside: The Creativity Engine
Channel: Society of Research Software Engineering
Date: 2022-12-01

Full title: The Creativity Engine — from messy notebooks to real-time inference

Details: https://virtual.oxfordabstracts.com/#/event/3101/submission/49
Slides: https://docs.google.com/presentation/d/1EdARSIhedXOAVii2Q3DXqpzAIrDKCW516LFENHJaFOc/edit?usp=sharing

=====

Additional Q&A

Q: Do you think the use of this engine could feedback suggested reading material to users? With collaboration to libraries this could be beneficial.
A: Yes, we hope to use the Creativity Engine to direct users to information and further resources about the source material authors and Seven Stories archive. An example might be to look for the occurrence of particular characters appearing in the the model output texts and refer the user to their literary origin.

Q: You mentioned making the model available to GLAMs. Is the trained model itself interesting to them or would they want to add their own input data?
A: The model we trained will be available to one specific GLAM, Seven Stories, but it is unlikely to be of use to other institutions. More generally though, we would like to offer the Creativity Engine —and the data wrangling and model fine-tuning in particular—as a case study of what can be done with other GLAM collections

Q: Can a user keep writing for a sentence after the initial inferred text and will the model then suggest some further text?
A: Yes! There is a back and forth response between the user and the model to allow a collaborative approach to creating new texts.

Q: Can the model be adapted for more specific narratives, e.g. specific genres/age groups?
A: In the case of the Creativity Engine the style of the model output is dependent on the source materials from Seven Stories which were used to finetune the baseline model. The size and content of this specific dataset was not sufficient to create more than a fiction and poetry version of the model. More generally however, the baseline model can be finetuned with various other sources of data to influence the style or genre of generated texts.

Q: How do you ‘detect’ toxicity in the training data and output?
A: For the model output we use Perspective API (https://perspectiveapi.com) to filter potentially toxic output from the model. The API uses a machine learning model to identify the perceived impact the text may have in a conversation and provide a score of this likelihood. A threshold can be applied to filter texts with a suitable toxicity probability score. Due to the data processing pipeline and size of the dataset all training data was manually approved by the project team.

Q: Can copyright issues occur? For example copying character names.
A: We have explicit permission for the data used to fine tune the model from Seven Stories so there are no potential copyright issues.

=====

Abstract: The Creativity Engine project leverages large, transformer-based, language models and the SevenStories archive to help children and young people kickstart their creative writing aspirations. Its end-result is a webapp that allows young writers to interact with a text-generating AI, trained on the works of some of the UK’s leading children and YA authors.

In this talk, we will share the difficulties experienced and the solutions we found that, we believe, would be of use to the RSE community. We will describe two main challenges the team dealt with: the nature of the source data and the infrastructure currently available to deploy these models. 

Source data was provided as digital facsimiles of archival material at SevenStories and composed of notebooks, drafts, notes, and proofs related to the work of six contemporary authors in their collection. The path from source to training data proved far from trivial. Given its characteristics, and the intended use-case of the data, a completely automated data preparation pipeline was not viable, highlighting the difficulty of working with humanities’ datasets in the age of big data.

Deploying large fine-tuned language models into a consumable app requires careful balancing of performance and cost. Size and complexity of these models precluded standard affordable options.

While these issues are shared by many projects, it raises important questions of accessibility not only to individual researchers, but institutions that are perennially underfunded and often estranged from the latest developments in digital methods, such as the GLAM (Galleries, Libraries, Archives & Museums) sector.

=====

0:00 Speaker introduction
0:19 Talk
20:25 Q&A