D oc F usion: A Unified Framework for Document Parsing Tasks (2025)

Abstract: Document parsing involves layout element detection and recognition, essential for extracting information. However, existing methods often employ multiple models for these tasks, leading to increased system complexity and maintenance overhead. While some models attempt to unify detection and recognition, they often fail to address the intrinsic differences in data representations, thereby limiting performance in document processing. Our research reveals that recognition relies on discrete tokens, whereas detection relies on continuous coordinates, leading to challenges in gradient updates and optimization. To bridge this gap, we propose the Gaussian-Kernel Cross-Entropy Loss (GK-CEL), enabling generative frameworks to handle both tasks simultaneously. Building upon GK-CEL, we propose DocFusion, a unified document parsing model with only 0.28B parameters. Additionally, we construct the DocLatex-1.6M dataset to provide high-quality training support. Experimental results show that DocFusion, equipped with GK-CEL, performs competitively across four core document parsing tasks, validating the effectiveness of our unified approach.

## introduction document parsing plays a significant role in extracting structured data from documents, making it foundational for various downstream applications. for example, in retrieval-augmented generation (rag) workflows @xcite , extracting well-organized and contextually rich information from documents can improve the performance of large language models (llms) @xcite . however, real-world documents often embed information in several studies have attempted to apply generative frameworks to integrate object detection and content recognition, achieving promising results on natural images @xcite . however, extending such frameworks to document images presents significant challenges due to the inherent structural and representational differences between these domains. through experiments, we identify the primary issue as the fundamental conflict between the continuous nature of coordinate data and the discrete nature of token generation, which disrupts gradient updates during multi-task training (discussed in section 3.2). in natural scene images, small deviations in coordinates and text are generally tolerable. however, in document parsing, even minor errors in latex code can critically impact compilation success rates. this imposes stricter accuracy requirements when applying such frameworks to document understanding tasks. to address these challenges, we propose gaussian-kernel cross-entropy loss (gk-cel), an improved objective function designed to mitigate the inconsistencies between discrete and continuous data representations, enhancing the performance of generative frameworks in document parsing. mer and tr are essential for latex-based document processing, but existing datasets suffer from inconsistent formatting and redundant characters, where different writing styles generate identical compiled outputs, introducing noise that hinders model training (details are provided in appendix a.3). to address this, we propose doclatex-1.6m, a large-scale, high-quality dataset that enhances annotation consistency and improves model training efficiency. experiments demonstrate that docfusion trained on this dataset outperforms taskspecific models with fewer parameters. our contributions are summarized as follows: page-level end-to-end ocr models like nougat @xcite and got @xcite simplify workflows by integrating multi recognition tasks. ## method we introduce the model architecture (3.1) and explain how detection tasks are represented into the generative framework. then, we discuss the challenges (3.2) of detection tasks within this framework. next, we explain the gaussian-kernel cross-entropy loss(3.3) ## conclusion in this work, we introduced docfusion, the first approach to integrate the four modules of a document parsing pipeline into a unified model by designing gaussian-kernel cross-entropy loss tailored to handle diverse data types across tasks. our method achieved sota performance on multiple benchmarks. to enable downstream applications, we re-annotated the widely used doclaynet dataset and constructed a large-scale formula-to-latex dataset, applying a unified standardization process. through detailed analysis, we observed that docfusion, as a lightweight model, effectively integrates multiple tasks into a single framework, demonstrating both efficiency and versatility in handling complex document parsing challenges. in the future, we aim to extend docfusion to larger models and further improve dataset standardization to enhance its performance and applicability across broader tasks and domains.