Enhancing Presentation Slide Generation by LLM s with a Multi-Staged End-to-End Approach (2024)

Abstract: Generating presentation slides from a long document with multimodal elements such as text and images is an important task. This is time consuming and needs domain expertise if done manually. Existing approaches for generating a rich presentation from a document are often semi-automatic or only put a flat summary into the slides ignoring the importance of a good narrative. In this paper, we address this research gap by proposing a multi-staged end-to-end model which uses a combination of LLM and VLM. We have experimentally shown that compared to applying LLMs directly with state-of-the-art prompting, our proposed multi-staged solution is better in terms of automated metrics and human evaluation.

## introduction presentations are a visually effective way to convey an idea to a broad audience @xcite . they are heavily used in academia, marketing and sales. a presentation often needs to be generated from a long multimodal document which contains both text and images. a narrative @xcite in a presentation generated from a document means (i) the sequence of slide tiles (topics) and (ii) the source content (sections / subsections) from the document for individual slides. making such a presentation from a document is very time consuming and needs domain expertise. there are rule-based approaches to generate a presentation from a document @xcite . automatically generating a presentation from a given multimodal document is challenging because of several reasons. compared to a flat document summary, the slide narrative should convey a story to its audience and is often non-linear with respect to the flow of information in the document @xcite . the content of a slide should be concise, easy to follow and visually appealing. so, it needs reasoning over both text and images, and their interrelationship. assuming the slide titles to be the same as the document sections, there are works which use a query specific summarizer @xcite , learn sentence importance @xcite and extract hierarchical relations between phrases @xcite to generate the presentation. @xcite takes the outline from the user and uses that to extract multimodal content and summarize that to slides. @xcite proposes a sequence-to-sequence architecture and a trainable policy to determine when to proceed to the next section/slide. but, it needs large amount of document-to-slides parallel training data which makes it difficult to generalize and scale. recent developments in large language models (llms) and vision language models (vlms) have been successfully applied in several multimodal generation tasks. these methods are also easy to use since they can generate content based on simple text prompts and can be generalized to multiple domains. however, compared to open domain generative task, generating a presentation from a specific document is much more challenging because of the following reasons: (i) it is difficult to feed an entire long document to an llm because of its upper limit on the context length (the number of tokens it can process at a shot) @xcite . (ii) the performance of llms drops with the length of the context within a prompt. in fact the performance degrades significantly when llms need to access relevant information in the middle of long contexts @xcite , which is a must requirement for our task. (iii) finally, llms are poor in attributing the exact source of the generated content (i.e., mapping a generated slide to some subsections of the document). both vlms and llms are prone to hallucinations and this tendency increases with the longer and incomplete context @xcite . thus, directly using llms for generating slides from a long document is not a good strategy. with this motivation, we try to divide the task of generating presentation from a long document into multiple simpler sub-tasks. the choice of individual smaller tasks is made from three perspectives: (i) how do humans create a presentation from a document? (ii) how to provide only a small amount of context in each call to the llm? and (iii) how to naturally satisfy properties such as coverage, nonlinearity, and source attribution? following are the novel contributions made in this paper: (i) we have proposed an unsupervised multi-staged hierarchical approach to generate slides from a long document, referred as docpres (document to presentation). our approach is multimodal-in and multimodal out in nature. (ii) we conduct thorough experimentation involving a state-of-the-art llm. we are able to show the merit of our multi-staged approach through automated and human evaluations. ## proposed solution approach let the input document be represented as @xmath0, (f j ) n @xmath1, where s i is the ith section (or a subsection) and f j is the jth figure in the document. both sections and images are associated with their positions (bounding box coordinates and page numbers) in the document. given the document, we aim to generate a set of slides @xmath2, • • • , l k } where each slide has some text and optional images coming from the document. as the first step, we extract the text and images from the input document (pdf) using a publicly available api 1 which gives the content of the document in a hierarchical fashion with section titles and the corresponding text within them with po-1 https://developer.adobe.com/ document-services/apis/pdf-extract/ sitions. images present in the document are also extracted with their locations. ## discussions and conclusion this work presented a novel multi-staged framework for generating presentations from documents. by breaking down the task into five sub-tasks, our approach achieved significant improvements compared to baselines including single-shot prompting to llms. comprehensive evaluations, both automatic and human, confirmed the merit of our multi-stage approach. the presentations from our approach demonstrated better coverage, readability, consistency, diversity, flow, and overall usability. the success of our multi-stage approach highlights 3 we could not use d2s here since we were not able to run its available code on any other dataset except sciduet. the benefits of decomposing complex tasks into smaller and well-defined subtasks, with limited context, for llms.