Pay More Attention to Images: Numerous Images-Oriented Multimodal Summarization (2025)

Abstract: Existing multimodal summarization approaches struggle with scenarios involving numerous images as input, leading to a heavy load for readers. Summarizing both the input text and numerous images helps readers quickly grasp the key points of multimodal input. This paper introduces a novel task, Numerous Images-Oriented Multimodal Summarization (NIMMS). To benchmark this task, we first construct the dataset based on a public multimodal summarization dataset. Considering that most existing metrics evaluate summaries from a unimodal perspective, we propose a new Multimodal Information evaluation (M-info) method, measuring the differences between the generated summary and the multimodal input. Finally, we compare various summarization methods on NIMMS and analyze associated challenges. Experimental results have shown that M-info correlates more closely with human judgments than five widely used metrics. Meanwhile, existing models struggle with summarizing numerous images. We hope that this research will shed light on the development of multimodal summarization. Furthermore, our code and dataset will be released to the public.

## introduction multimodal summarization is an emerging research area driven by advancements in multimodal learning. images serve to either spotlight the essential content or to provide rich information. most existing multimodal summarization studies focus on identifying the most informative visual keyframes and distilling the text content into pivotal points. while tasks involving both multimodal input and multimodal output have been extensively explored, existing studies struggle to account for scenarios input document: the lavish lifestyle of an 18-year-old "rich kid on instagram" who drives a maserati, splurges on designer clothes, and vacations on a superyacht -all financed by her "generous father". clarisse lafleur is "ashamed to name" the amount of money she spends on designer clothing and accessories...... ## conclusion in this paper, we introduce the nimms task, aiming at generating summaries by integrating both textual and numerous image inputs. besides, we propose m-info, a new evaluation method proven effective for nimms. through comprehensive analysis and comparison of various summarization methods applied to nimms, we identify significant challenges, particularly the limitations of current multimodal models in deeply understanding longsequence asynchronous images and generating cohesive summaries that unify both textual and visual information. we hope nimms will serve as a valuable benchmark to enhance multimodal large language models.