VISIAR : Empower MLLM for Visual Story Ideation (2025)

Abstract: Ideation, the process of forming ideas from concepts, is a big part of the content creation process. However, the noble goal of helping visual content creators by suggesting meaningful sequences of visual assets from a limited collection is challenging. It requires a nuanced understanding of visual assets and the integration of open-world knowledge to support creative exploration. Despite its importance, this task has yet to be explored fully in existing literature. To fill this gap, we propose Visual Story Ideation, a novel and underexplored task focused on the automated selection and arrangement of visual assets into coherent sequences that convey expressive storylines.We also present VISIAR, Visual Ideation through Sequence Integration and Asset Rearrangement, a robust framework leveraging Multimodal Large Language Models (MLLMs), and a novel Story Graph mechanism. Our framework operates in three key stages: visual content understanding, candidate asset selection, and asset rearrangement via MLLMs. In addition, we curated a new benchmark dataset, called VTravel, to evaluate our methods both qualitatively and quantitatively.User studies and GPT-as-the-judge evaluation show that our approach surpasses GPT-4o based baseline by an average of 33.5% and 18.5% across three different metrics, demonstrating the effectiveness of our framework for generating compelling visual stories.

## introduction visual story @xcite plays a pivotal role across diverse domains, including digital marketing, recreational content creation, and personal content generation for social media, due to its ability to captivate audiences. however, content creators often face significant challenges in crafting expressive and coherent visual stories from their personalized assets, as the asset collection can contain several hours of content, making it overwhelming to sift through manually. more importantly, ideating compelling storylines from hundreds or thousands of assets is time-consuming and laborintensive, requiring professional expertise in interpreting visual content, selecting relevant elements, and arranging them into an expressive storyline. automating the ideation process can not only accelerate visual storytelling but also inspire creators, broaden access to novice users, and expand creative possibilities across industries. while research on visual storytelling @xcite is of great interest, this automatic ideation process remains largely underexplored. to bridge this gap, we introduce a novel task, visual story ideation, aimed at automatically deriving underlying visual stories from a collection of visual assets. developing an automatic visual story ideation system presents significant challenges. such a system must demonstrate a deep understanding of visual assets, leverage open-world knowledge for creative story ideation, and efficiently navigate large collections of assets. the lack of established benchmarks or a well-defined framework further complicates this task. current visual storytelling methods @xcite often rely on predefined sequences of visual assets or accompanying text scripts, which are not available in real-world scenarios (such as creating short reels with a coherent story from hours of randomly taken vlogs), leaving the ideation process untouched. to address these challenges, we propose visiar, visual ideation through sequence integration and asset rearrangement, a framework that empowers mllm for visual story ideation. it encompasses three key stages: visual content understanding, candidate asset selection, and asset rearrangement for expressive storyline creation. in the first stage, mllm is used to analyze visual content because of their remarkable visual understanding ability @xcite . however, mllm's limitations of context window and longcontext understanding make it unsuitable for the designated task. therefore, in the second stage, we explore and implement multiple methods for candidate asset selection to empower mllm to navigate through hundreds of visual assets. we develop feature-based clustering and graph-based clustering approaches. specifically, we develop a group of story-graph enhanced methods, which construct a story graph using mllms to propose candidate visual assets for each storyline. in particular, we invent the llm-ideation graph method: leveraging mllm's open-world knowledge for story graph construction. finally, in the third stage, mllms are employed to rearrange the selected assets and generate expressive storylines. to establish a benchmark for this novel task, we curate a new dataset, vtravel, comprising short video clips to assemble realistic scenarios. user studies and the gpt-as-the-judge evaluation are conducted to assess our methods, demonstrating that our framework with the llm-ideation graph method surpasses the performance of the state-ofthe-art mllm baseline in generating coherent and expressive visual stories. in summary, our contributions are as follows: ## related work to the best of our knowledge, we are the first to investigate the challenging visual story ideation task. our task is related to the visual storytelling task, ideation based on large language models, and tangentially related to multi-modal large language models. ## conclusion in this paper, we introduce visual story ideation, a novel task focused on selecting and combining visual asset sequences to create compelling storylines from limited collections. unlike prior work relying on predefined sequences or scripts, our approach automates the ideation process, addressing a key gap in visual storytelling. for this task, we also propose visiar, visual ideation through sequence integration and asset rearrangement, a framework leveraging stateof-the-art multimodal large language models (mllms), and a story-graph-enhanced selection process. evaluations on a new dataset, vtravel, demonstrate its ability to outperform baselines by generating coherent and expressive asset sequences. our work establishes a foundation for future research in automated visual story ideation, with potential applications in advertising, media production, and creative content generation. we believe this framework opens new avenues for innovation in both academic research and practical implementations of visual storytelling.