VS cript: Controllable Script Generation with Visual Presentation (2022)

Abstract: In order to offer a customized script tool and inspire professional scriptwriters, we present VScript. It is a controllable pipeline that generates complete scripts, including dialogues and scene descriptions, as well as presents visually using video retrieval. With an interactive interface, our system allows users to select genres and input starting words that control the theme and development of the generated script. We adopt a hierarchical structure, which first generates the plot, then the script and its visual presentation. A novel approach is also introduced to plot-guided dialogue generation by treating it as an inverse dialogue summarization. The experiment results show that our approach outperforms the baselines on both automatic and human evaluations, especially in genre control.

## introduction artificial intelligence (ai) introduces significant changes in the creation of artworks, such as stylistic painting @xcite , poem writing @xcite , music composition @xcite , and converting the perception of creativity. in particular, ai can assist in streamlining the art-making process and give humans fresh inspirations @xcite , and one plausible application is scriptwriting. as a specific literary form, the script is indispensable in cinematography and theater @xcite . to enable the collaboration between scriptwriter and ai, an automatic script generation system must equip with three aspects. first, the system must generate a complete script consisting of chronological scene descriptions and dialogues. second, the system should provide controllability, e.g., customize script genre or storyline @xcite . third, as a creative work, the generated script is required to have rich and diverse content. while @xcite propose a narrativeguided script generation task, they only focus on given the inputs, i.e., genre and starting words, a plot is generated, which guides the generation of a script consisting of a scene description and a dialogue. the words highlighted in pink show the belongingness to the given genre (sci-fi). additionally, the video vividly presents the script. in this paper, we present vscript, a controllable script generation system that includes all essential components of a script. we adopt a hierarchical structure for our framework by implementing highlevel planning @xcite and following the guidelines of video-making processes @xcite . as shown in figure 2 , vscript is composed by script generation and visual presentation modules. the examples of corresponding components are in figure 1 . script generation module consists of three sub-modules: 1) plot generation, 2) dialogue generation, and 3) scene description generation. firstly, the system generates a genre-specific plot as an outline of the script. to avoid generating aimless scripts, the system allows users to provide some brief initial information, i.e., genre and the beginning of the plot, for a high degree of control over the script on the genre and story trend. secondly, we conduct a zero-shot plotguided dialogue generation by framing the problem as an inverse task of abstractive dialogue summarization. finally, we generate scene descriptions based on the dialogues to produce complete scripts. visual presentation module vividly demonstrates the script by retrieving videos from an automatically constructed video database. it can serve as a rough visual draft to improve users' engagement and help scriptwriters rapidly iterate ideas. to our best knowledge, we are the first to tackle the controllable script generation task, which controls a script's genre and storyline. we also introduce a practical approach for plot-guided dialogue generation by treating the task as inverse dialogue summarization, which improves the diversity of the generated dialogue while maintaining relevancy to the plot. in addition, we explore effective methods to produce an eloquent real-time visual presentation from the generated script. according to the evaluation results, our scripts are controllable and preferred by humans. thus, vscript can serve as a tool for users to produce scripts with their preferences and for scriptwriters to optimize the writing process. we think vscript has the potential to promote ai-human collaboration in script generation. limitations and ethical considerations are discussed in appendix a and b. controllable text generation model conditional deep generative models are effective in improving the controllability of the models. ctrl @xcite is a class-conditional language model (cc-lm) pre-trained on 50 domains with different control codes. plug and play language models (pplm) @xcite combine pre-trained language models and attribute classifiers to steer generation. gedi @xcite incorporates cc-lm as a discriminator to control generation towards the desired attribute. ## methodology as shown in figure 2 , our framework can be decomposed into two modules: script generation and visual presentation. we will describe these two modules in detail in the following sections. ## interactive user interface an example of the interaction between users and vscript is illustrated in figure 3 . the user in-terface comprises three parts: the script area (top left), the video area (top right), and the interaction area (bottom centre). first, users select the script genre among crime, sci-fi, war, romance, and genre-free. second, users type the starting words into the input box and submit. finally, the generated script will be displayed in the script area and its visual presentation in the video area. users can interrupt at any time and choose the genre or input some words to steer the script's development. ## conclusion we propose the first controllable script generation framework vscript that can generate scripts of specific genres and follow the plots. our framework adopts a hierarchical structure, which generates the plot, then the script and its visual presentation. we adopt inversed abstractive summarization for dialogue generation. based on our experiments, vscript outperforms the baselines, and its effectiveness in genre control is proven. 7 please refer to appendix g for the results of each aspect. 8 the result is statistically significant with @xmath0.05.