ViStoryBench: Comprehensive Benchmark Suite for Story Visualization
Authors: name: Cailin Zhuang, name: Ailin Huang, name: Wei Cheng, name: Jingwei Wu, name: Yaoqi Hu, name: Jiaqi Liao, name: Zhewei Huang, name: Hongyuan Wang, name: Xinyao Liao, name: Weiwei Cai, name: Hengyuan Xu, name: Xuanyang Zhang, name: Xianfang Zeng, name: Gang Yu, name: Chi Zhang
Organization: StepFun

Story visualization, which aims to generate a sequence of visually coherent
images aligning with a given narrative and reference images, has seen
significant progress with recent advancements in generative models. To further
enhance the performance of story visualization frameworks in real-world
scenarios, we introduce a comprehensive evaluation benchmark, ViStoryBench. We
collect a diverse dataset encompassing various story types and artistic styles,
ensuring models are evaluated across multiple dimensions such as different
plots (e.g., comedy, horror) and visual aesthetics (e.g., anime, 3D
renderings). ViStoryBench is carefully curated to balance narrative structures
and visual elements, featuring stories with single and multiple protagonists to
test models' ability to maintain character consistency. Additionally, it
includes complex plots and intricate world-building to challenge models in
generating accurate visuals. To ensure comprehensive comparisons, our benchmark
incorporates a wide range of evaluation metrics assessing critical aspects.
This structured and multifaceted framework enables researchers to thoroughly
identify both the strengths and weaknesses of different models, fostering
targeted improvements.