Document-level Simplification and Illustration Generation Multimodal Coherence (2025)

Abstract: We present a novel method for document-level text simplification and automatic illustration generation aimed at enhancing information accessibility for individuals with cognitive impairments. While prior research has primarily focused on sentence- or paragraph-level simplification and text-to-image generation for narrative contexts this work addresses the unique challenges of simplifying long-form documents and generating semantically aligned visuals. The pipeline consists of three stages (1) discourse-aware segmentation using large language models (2) visually grounded description generation via abstraction and (3) controlled image synthesis using state-of-the-art diffusion models including DALLE 3 and FLUX1-dev. We further incorporate stylistic constraints to ensure visual coherence and we conduct a human evaluation measuring comprehension semantic alignment and visual clarity. Experimental results demonstrate that our method effectively combines simplified text and visual content with generated illustrations enhancing textual accessibility.

## introduction simplified language aims to enhance information accessibility for individuals with cognitive impairments, learning disabilities, and others who may have difficulty comprehending standard-language written texts @xcite . existing research primarily focuses on transforming complex sentences or paragraphs into more comprehensible variants, particularly for readers with low literacy skills and non-native speakers @xcite . while automatic text simplification technologies have made significant progress in recent years, the role of visual aids in supporting textual comprehension is comparatively underresearched. studies have shown that incorporating illustrations into simplified texts can further enhance understanding among individuals with cognitive disabilities @xcite @xcite . most existing work has focused on sentence-or paragraph-level simplification and accompanying image generation @xcite @xcite . with the growing contextual understanding capabilities of large language models (llms), document-level simplification and illustration generation for longer texts become possible. our goal is to leverage these capabilities to build a pipeline for document-level text simplification and corresponding illustration generation, aiming to improve information comprehension for persons with cognitive impairments. ## related work in recent years, advances in natural language processing have significantly accelerated the development of automatic text simplification technologies. concurrently, researchers have begun to explore multimodal approaches to further enhance the comprehensibility of simplified texts, especially through image generation. illustrating text with images is an effective strategy to support comprehension. visual elements not only help capture the reader's attention but also concretize abstract concepts, thereby reducing cognitive load @xcite @xcite . the recent progress in text-to-image generation (t2i) has made the automated realization of this idea increasingly feasible. diffusionbased models @xcite have emerged as the dominant paradigm in t2i and have achieved remarkable breakthroughs. state-of-the-art models such as openai's dall•e 3 @xcite , google's imagen 3 @xcite , and stability ai's stable diffusion 3 @xcite can generate high-resolution, semantically relevant, and visually creative images from complex textual descriptions. existing research has demonstrated the use of dall•e 3 and stable diffusion 3, to generate high-quality visual content that is semantically well-aligned with easy-to-read (e2r) textual materials @xcite . these advances are largely attributed to the models' pretraining on massive text-image paired datasets, which enable them to learn nuanced mappings between textual semantics and visual representations. our work builds upon these powerful diffusion models as the foundation for visual generation. while most existing systems are still confined to sentence-or paragraph-level t2i generation, generating contextually appropriate illustrations from long-form documents requires document-level understanding and scene planning capabilities. with the rapid evolution of llms, new frameworks are emerging that allow for these models to manage long-document processing and orchestrate image generation. in such settings, llms serve as directors or scriptwriters that structure the narrative and guide visual synthesis @xcite . however, these systems are primarily tailored for narrative storytelling. in contrast, generating illustrations for informational documents demands greater factual accuracy and lower semantic ambiguity. our research aims to bridge this gap by developing a multimodal system that integrates an llm with a diffusion-based image generation model, enabling more accurate document-level text simplification and illustration. the proposed system features a structure-aware text simplification module and a semantically aligned image generator. through semantic optimization and cross-modal feedback mechanisms, our method enhances the coherence between text and images and improves cognitive accessibility. ## method to generate visual content for complex documents, we propose a multi-stage generative pipeline. this pipeline first decomposes a document into semantically coherent units, then translates these units into visually grounded descriptions, and finally renders them into images. our approach integrates the advanced capabilities of llms for complex text processing with state-of-the-art t2i models for visual synthesis. the entire framework consists of four key stages: (1) document-level text simplification, (2) semantic document segmentation, (3) visually grounded description generation, and (4) controlled image generation. ## discussion and conclusion our study indicates that automatic simplification produced texts that readers judged as fluent and largely faithful to source meaning, while perceived ease-of-reading is more sensitive to individual evaluator standards. on the visual side, both image generation models were generally effective in providing semantically supportive illustrations, though their perceived utility varied across evaluators and dimensions. ratings tended to cluster around the mid-scale (3-4), suggesting that while ai-generated visuals achieved a baseline adequacy, they rarely reached the level of high-quality humancreated illustrations. this implies that the models captured textual meaning with reasonable reliability but seldom produced outputs regarded as exemplary in terms of clarity, stylistic appropriateness, or overall communicative effectiveness. divergent preferences among evaluators, some favoring flux.1-dev and others dall•e 3 highlight that judgments of quality are not solely determined by semantic accuracy, but are also shaped by individual aesthetic expectations and tolerance for stylistic variation. particularly noteworthy was the tendency of participants to misclassify outputs from flux.1-dev as manually created, suggesting that its visual naturalness may enhance perceived authenticity. while such naturalness is promising for accessibility and engagement, it also raises potential concerns about transparency and user trust in contexts where it is important to distinguish humanfrom machine-generated content. qualitative comments about images containing too much text point to avoidable extraneous cognitive load. images that incorporated excessive textual content or visually dense layouts were often judged as distracting, thereby diminishing rather than enhancing comprehension. this observation underscores the importance of maintaining visual simplicity, especially when designing for audiences with cognitive impairments. for such users, the trade-off between semantic fidelity and visual clarity becomes particularly critical; cluttered or overly detailed images may undermine the intended benefits of simplified language materials. these findings point to the necessity of introducing explicit design constraints in text-to-image workflows. mechanisms such as filtering strategies, prompt engineering techniques that enforce minimalism, or post-processing methods to eliminate superfluous elements could help align outputs more effectively with accessibility goals. the pattern of high fluency and adequate semantics in text, combined with mid-scale visual ratings, points to a practical synthesis: when textual simplification reliably preserves meaning and flow, illustrations function best as lightweight scaffolds rather than dense carriers of information. simplified text can shoulder the primary communicative load, while images should reinforce key entities, relations, or processes without introducing visual clutter. this aligns with our finding that evaluators penalize visually dense layouts: if text is already fluent and semantically adequate, adding heavy captioning or intricate scene details may yield diminishing returns or even harm comprehension. therefore, downstream design should prioritize (i) simplicity-first visual layouts, (ii) restrained use of textual overlays inside images, and (iii) explicit alignment between each image and a small set of core propositions in the simplified text. future research should extend beyond expertbased evaluations to incorporate direct feedback from end users, particularly individuals with cognitive impairments, in order to ensure that the generated visuals truly enhance comprehension and accessibility.