Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting (2025)

Abstract: Document image parsing is challenging due to its complexly intertwined elements such as text paragraphs, figures, formulas, and tables. Current approaches either assemble specialized expert models or directly generate page-level content autoregressively, facing integration overhead, efficiency bottlenecks, and layout structure degradation despite their decent performance. To address these limitations, we present Dolphin ( ), a novel multimodal document image parsing model following an analyze-then-parse paradigm. In the first stage, Dolphin generates a sequence of layout elements in reading order. These heterogeneous elements, serving as anchors and coupled with task-specific prompts, are fed back to Dolphin for parallel content parsing in the second stage. To train Dolphin, we construct a large-scale dataset of over 30 million samples, covering multi-granularity parsing tasks. Through comprehensive evaluations on both prevalent benchmarks and self-constructed ones, Dolphin achieves state-of-the-art performance across diverse page-level and element-level settings, while ensuring superior efficiency through its lightweight architecture and parallel parsing mechanism. The code and pre-trained models are publicly available at https://github.com/ByteDance/Dolphin

## introduction document image parsing @xcite aims to extract structured content from images containing intertwined elements such as text paragraphs, figures, tables, and formulas. as a foundational capability for downstream content analysis @xcite , it bridges the gap between visual content and machine-readable formats. with the formula-spe (cdm) table-pubtabnet (teds) table-pubtab1m (teds) fps 0.0114 0.0131 0.1028 0.0029 0.0121 0.0136 0.9850 0.9685 0.8739 0.9515 0.9625 0.1729 qwen2-vl-7b got gpt-4o claude3.5-sonnet dolphin figure 1: comparison of dolphin with advanced vlms across benchmarks: page-level parsing (plain and complex documents), element-level parsing (text paragraph, table, and formula), and running efficiency (fps). the outer area represents better performance. dolphin exhibits the best performance in most evaluations. to synergize the advantages of both approaches while addressing their limitations, we present dolphin (document image parsing via heterogeneous anchor prompting), a novel vision-language model following an analyze-then-parse paradigm. rather than relying on multiple expert models or purely autoregressive generation, dolphin decomposes document parsing into two strategic stages. in the first stage, dolphin performs comprehensive page-level layout analysis by generating an element sequence in natural reading order, while preserving rich structural relationships (e.g., figure-caption pairs, tablecaption associations, and section title-paragraph hierarchies). these analyzed elements then serve as anchors for the second stage, where elementspecific prompts enable efficient parallel parsing of multiple elements. the focused context within each element allows the vision-language model to effectively recognize the document contents. to train dolphin on different granularities of tasks, we construct a large-scale dataset of 30 million samples containing both page-level documents and element-level blocks. notably, dolphin's element-decoupled parsing strategy offers unique advantages in data collection, as acquiring isolated element images (e.g., tables, formulas) and their annotations is more feasible than collecting full document pages with diverse elements. comprehensive evaluations are conducted on prevalent benchmarks and self-constructed ones. the results show that dolphin achieves state-ofthe-art performance across diverse page-level and element-level parsing tasks (figure 1 ). moreover, benefiting from its lightweight architecture and parallel element parsing mechanism, dolphin exhibits considerable advantages in running efficiency. ## related work document image parsing enables robust content extraction from rendered document images without relying on source file formats or parsing libraries (e.g., pymupdf). existing solutions can be categorized into two streams: integration-based methods that assemble multiple expert models in a pipeline, and end-to-end approaches that leverage vision-language models to directly generate structured results via autoregressive decoding. ## dataset to enable comprehensive training and evaluation, we construct large-scale datasets spanning multiple document granularities and parsing tasks. ## conclusion we present dolphin, a novel document image parsing model that leverages an analyze-then-parse paradigm to address the challenges in document parsing. our approach first performs page-level layout analysis to generate structured layout elements in reading order, then enables parallel element parsing through heterogeneous anchor prompting. this two-stage design effectively balances efficiency and accuracy, while maintaining a lightweight architecture. through extensive experiments, we demonstrate dolphin's strong performance in both page-level and element-level parsing tasks, particularly excelling in handling complex documents with interleaved tables, formulas, and rich formatting in both chinese and english.