Automatic Summarization for Creative Writing: BART based Pipeline Method for Generating Summary of Movie Scripts (2022)

Abstract: This paper documents our approach for the Creative-Summ 2022 shared task for Automatic Summarization of Creative Writing. For this purpose, we develop an automatic summarization pipeline where we leverage a denoising autoencoder for pretraining sequence-to-sequence models and fine-tune it on a large-scale abstractive screenplay summarization dataset to summarize TV transcripts from primetime shows. Our pipeline divides the input transcript into smaller conversational blocks, removes redundant text, summarises the conversational blocks, obtains the block-wise summaries, cleans, structures, and then integrates the summaries to create the meeting minutes. Our proposed system achieves some of the best scores across multiple metrics(lexical, semantical) in the Creative-Summ shared task.

## introduction text summarization captures salient information by condensing long documents into short paragraphs. with the surge of online records, automatic text summarization makes it convenient for people to extract relevant information to their interests. there are several challenges with the current state-of-art summarization methods (i) processing long sequences spanning hundreds of pages of text, (ii) analyzing complex discourse structures such as narrative and multi-party dialogs (iii) interpreting figurative languages to understand and convey the salient points in the input. most current works focus on the news, text, and scientific domain with limited input length, literal and technical language, hallucinations, positional biases, and constrained discourse structure. one under-explored text summarization domain is creative writing, which includes documents such as books, stories, scripts from plays, tv shows, and movies. the task of automatically summarizing the creative content is not straight forward. it involves, long length document, non-trival temporal dependency (parallel plot threads and non-linear plot development), complex structures with frequent context drifts combining narative creations and multi-party dialogs with variety of styles. in this paper we summarize these creative content of movie scripts with literary interpretations, conveying implicit information and heavily paraphrasing the input using state-of-art text summarization models. ## related work text summarization has been a topic of research since the mid 20th century. most earlier methods relied on statistical analysis to score the importance of sentences and then extracting the sentences with the most importance. @xcite proposed using tf-idf (term frequency-inverse document frequency) to score sentences. they found tf-idf to be effective at extractive summarization and found it to outperform other state-of-art system available. @xcite proposed bayesian modeling as an approach to summarize text data. kamal proposed using extractive key concept summarization and found it to match the performance and even outperform some of the existing models at the time. @xcite proposed a novel pattern based summarization technique and found it to perform much better than standard text based methods. following statistical methods, deep learning methods were utilized to attempt to find a solution to the problem of text summarization. @xcite proposed a novel sequence-to-sequence (seq2seq) model to perform abstractive summarization rather than extractive summarization. @xcite proposed a novel graph based attention mechanism in a hierarchical encoder-decoder framework, and propose a hierarchical beam search algorithm to generate multi-sentence summary. this architecture provided significant improvements over previously existing models. @xcite introduced a feature enhanced seq2seq model. this model improved the encode and decoder performance using a 2-feature capture network which improves the models capability of storing long term features, this makes the generated summaries far more concise and accurate. following the introduction of the transformer model by @xcite most work in nlp shifted to models based on the transformer architecture. @xcite proposed the pegasus (pre-training with extracted gap-sentences for abstractive summarization) model which demonstrated the effects of the pre-training corpora, gapsentences ratios, vocabulary sizes and scaled up the best configuration to achieve state-of-the-art results on 12 diverse downstream datasets considered. @xcite introduced gpt unidirectional transformer encoder model, which improved the score of downstream nlp tasks by combining pre-training and fine tuning. then, based on the two pre-training models, there are many fusion algorithm models to deal with the nlp task of automatic summarization. @xcite introduced the novel mass model, which allows the encoder and decoder to learn at the same time in the pre-training stage. it is the first time to realize the unification of the bert plus generation model, and the rouge score is improved compared with the bert and other models. ## proposed methodology as shown in figure 1 , our suggested system is divided into three main modules, which include pre-processing input transcripts, a conventional sequence-to-sequence model for generating summaries, and post-processing, which further unifies summaries and eliminates redundant information. ## experimental setup in this section, we describe dataset details in subsection 4.1, hyper-parameter setting in section 4.2 and evaluation strategy in section 4.3. ## results and analysis our summarization model is evaluated on 16 unique metrics. these metrics tries to access model's performance both lexically and semantically against the human annotated summaries. table ?? shows the evaluation results obtained on the test dataset provided under the creativesumm 2022 task. the test data comprises of 679 different scripts/transcripts extracted from various discrete episodes from summscreen tms dataset. as depicted in the table our system performs better overall in comparisons to other submission. we outperfrom their second best performing system by a margin of 0.05 across metrics like rouge-n, litepyramid and summaczs. our results are more adequate and fluent when referred against the original human generated annotations. figure 3 illustrates an actual example extracted from the summscreen tms dataset. it also portrays the outputs generated by passing the tv transcripts via our proposed summarization pipeline. as it is clearly depicted our proposed system is able to extract and relate small details in the tv transcripts. these generated summaries are also grammatically tuned and adequate. as described earlier we adopt a segmentation procedure to avoid the max-token-length conflicts posed by the current summarization models. however, a major drawback of this is that there is a loss of information which get ignored when we segement the input data. this can be solved by adopting an appropriate co-reference resolution procedure to 5 shiyue/roberta-large-tac08 seamlessly connect various integrative parts of the text such as noun-pronoun pairs verbs, associating adverbs which sometime due to segmentation gets disentangled. even though, during pre-processing we try to precisely segment the text by using a floating max_threshold ranging between {512, 1024}, however sometimes these references get lost in during processing and thus might not be reflected precisely into the output text. ## conclusion in this paper, we propose a unique architecture comprising of various segement working together seamlessly to produce good quality summaries of tv transcripts. we utilise a bart model initially fine-tuned on human text conversations and then on scripts specifically derived from the summ-screen tms dataset. our system submissions outperformed every other contribution across various evaluation metrics under the creativesumm 2022 task. however, the proposed pipeline still need some refinement in terms of the language model and the inclusion of various pre-processing techniques. this could result in a significant improvement in the model's generation quality, making it more reliable and adequate.