## 1 Introduction  
With the advancement of digital transformation, the push for digitizing paper-based documents and reports is ever-increasing, especially in sectors such as healthcare, insurance, and finance [1; 2; 3]. In this context, the medical examination report (MER) stands out. MER, serving as an important type of medical document, encapsulates crucial information such as patient demographics, doctor's advice, test items, and results. This is important for guiding medical decision-making. As such, the transition of the MER to an electronic format has great practical value. The purpose of this study is to explore and enable the effective conversion of MER document images into digital formats, which can improve the accessibility of healthcare information, facilitate information exchange,support healthcare decision-making, and facilitate the modernization process of healthcare and related industries. As shown in Figure 1a, the MER image contains two tables, delineated by red boxes, termed \"Header\" and \"Body\". The \"Header\" includes the testing organization and the patient's basic information, while each row in the \"Body\" section means a specific test item, including item name, result, unit, reference range, anomalies indication, etc. At the early stage, optical character recognition (OCR) technology is used to recognize image textual content [4]. However, MER document images not only have text information but also contain table-structured information, such as the layout position and layout structure of the text. The structured output of text information cannot be satisfied by relying only on OCR technology. Convolutional neural networks (CNNs) and sequence recognition-based methods describe the structural sequences of tables in structured languages such as HTML and LaTeX [5]. However, these methods could be more accurate for table structure reconstruction recognition in practical scenes. In general, the prevalent approach to digitizing paper-based MER involves recovering the structure and content of the \"Body\" of the form and converting them into computer-readable structured data, as shown in Figure 1b. However, the current methods need to pay more attention to some unstructured information in the table header. This omission results in the loss of key information and fails to establish the correlation among key forms of information across the entire MER, hindering optimal medical decision-making and analysis. The development of information extraction (IE) technology provides new ideas for document image digitization. Open information extraction (OIE) can automatically extract structured information from unstructured data [6]. Key information extraction (KIE) is more suitable for scenarios where key information in text needs to be extracted accurately, where the semantic entity recognition (SER) task categorizes blocks of text, and the relationship extraction (RE) task extracts the relationships that exist between different entities [7]. The early KIE-based methods are limited to a single modality, ignoring the location of key information and image layout information [8]. Nowadays, mainstream methods focus on combining multimodal features in document images to improve the system's performance. The grid, graph neural network (GNN), Transformer [9], and end-to-end-based approaches have progressed in this direction [10; 11; 12; 13]. However, exploring different modalities by these models is still limited, as it is difficult to balance the different contribution levels of multimodal features in different scenarios. For documents such as MER, which are rich in text and layout structures and have less visual information, the effect of visual modalities is less effective. Therefore, further exploration of the extent of multimodal information utilization in different scenes and enhancement of the interaction between different modalities are directions for further study.