## 2 Related Works  
Extracting text information from images, a type of VDU task, has always been an important research topic in the field of computer science. As an interdisciplinary task, it aims to achieve the transformation from visual text to structured text information. Current studies typically divide this task into two stages: OCR and text parsing. In this case, researchers utilize methods from computer vision (CV) and natural language processing (NLP) to enhance the accurate extraction and understanding of text in images. Recent OCR studies [18; 19; 20] have focused on two main tasks: text detection and text recognition. Specific neural network models are constructed independently for each task to complete the entire OCR process. The text detection model is responsible for acquiring the position of text in images, while the text recognition model focuses on transforming the detected visual text areas into real text. [PERSON] et al. [21] proposed an efficient and highly scalable parallel architecture for segmenting input images containing both bordered and borderless table data into cells and reconstructing tables, thereby improving the recognition accuracy of OCR systems in tabular text. OCR performance is constrained by its sensitivity to text in various languages and scenes, leading to limitations in its quality. Some researchers enhance the quality of OCR by incorporating additional OCR post-correction modules into their system [19; 22; 23]. However, this method introduces additional complexity to the entire system and reduces its generalization. In the domain of text parsing, there is a growing trend in the research community towards utilizing pre-trained transformers as a universal solution. Commencing with LayoutLM [7], these models are composed of various transformer-based structures, and we refer to them as the LayoutLM series. With the development of these models, VDU tasks are largely dominated by transformer architectures. Researchers aim to transfer the success of models like BERT [24] from the natural language to the visual document domain. LayoutLM is pre-trained in a manner very similar to BERT, but incorporates 2D spatial position information. By fine-tuning the model for specific tasks, it can perform various functions, achieving significant success in VDU tasks. However, these methods still rely on OCR. In the past, the backbone networks of visual modules were mainly based on various CNN structures, including AlexNet [25], VGG [26], ResNet [27], and EfficientNet [28]. Recently, with the significant success of pre-trained models and transformers in NLP, researchers have introduced transformers to the CV domain. Swin Transformer [29] is a notable example that has successfully introduced a shift-window-based self-attention mechanism, addressing the issue of high time complexity in global self-attention methods. It has been used as a visual backbone network and achieved excellent results in various VDU tasks. Researchers have explored the development of end-to-end VDU models. [PERSON] et al. [30] proposed Trie++, a model that simultaneously learns text reading and information extraction tasks by connecting multimodal contextual blocks for visual and natural language processing tasks. [PERSON] et al. [31] introduced VIES, which performs text detection, recognition, and information extraction simultaneously. However, both Trie++ and VIES require complete document annotations for training. [PERSON] et al. [10] proposed donut, an end-to-end document understanding model employing an encoder-decoder architecture. They explored all visual backbone pre-trained networks on ImageNet and highlighted the superiority of Swin Transformer over other CNN-based backbone networks. [PERSON] et al. [11] proposed dessurt, a model that processes three token streams representing visual tokens, query tokens, and responses. Cross-attention is applied between different streams, enabling them to share information and propagate it to the responses. [PERSON] et al.