## 4 Experiments and Analyses
### Result and Discussion  
For key information extraction, in this paper, we perform 5-fold cross-validation. In each fold, images are randomly selected from the CMERD dataset according to a 7:3 ratio for the training and test sets. The final results are shown in Figure 12. Table 2 lists the averages of the results from the 5-fold cross-validation of the different KIE models on the CMERD dataset. For the SER task, our method achieves the highest F1 score (97.38%). Also, it reveals that the P and R of RVI-LayoutXLM are relatively high. In the RE task, our approach achieves the highest F1 score (95.77%), further demonstrating its consistent performance across diverse tasks. To verify whether the attention refinement module significantly improves accuracy in both SER and RE tasks, two-sample \(t\)-tests are performed. Specifically, five sets of results based on five-fold cross-validation are compared between the RVI-LayoutXLM model, incorporating refined attention and the other models, respectively. The test results are shown in Figure 13. The \(t\)-test results show that RVI-LayoutXLM significantly outperforms the baseline model, on average, in the SER task and the RE task at the 95% confidence level, demonstrating the effectiveness of the attention refinement module for entity classification and table header relationship extraction. It is pertinent to highlight that, due to the limitation of dataset annotation, all the comparison methods used in this paper serialize the document images and fuse the multimodal features using the Transformer-based network. This ensures a balanced evaluation of the capability of RVI-LayoutXLM within the context of the MER image KIE. To evaluate the visual modality in CMERD, this paper attaches image modality to the RVI-LayoutXLM. As depicted in Figure 9, the sample count for the \"other\" category surpasses that of the \"header\" by a factor of 25. This significant category imbalance hinders \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Task** & **Model** & **Modality** & **F1 (\%)** & **P (\%)** & **R (\%)** \\ \hline \multirow{4}{*}{SER} & LayoutLM [12] & Text + Layout & 95.41 & 95.61 & 95.22 \\  & LayoutLMv2 [22] & Text + Layout + Image & 96.73 & 96.64 & 96.84 \\  & LayoutXLM [23] & Text + Layout + Image & 96.74 & 96.31 & 97.18 \\  & VI-LayoutXLM [29] & Text+ + Layout & 97.08 & 96.75 & 97.41 \\  & RVI-LayoutXLM (Ours) & Text+ + Layout & 97.38 & 97.01 & 97.76 \\ \hline \multirow{4}{*}{RE} & LayoutLMv2 [22] & Text + Layout + Image & 89.02 & 85.73 & 92.58 \\  & LayoutXLM [23] & Text+ Layout + Image & 93.22 & 93.97 & 92.49 \\ \cline{1-1}  & VI-LayoutXLM [29] & Text+ + Layout & 95.22 & 95.40 & 95.04 \\ \cline{1-1}  & RVI-LayoutXLM (Ours) & Text+ + Layout & 95.77 & 95.75 & 95.80 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Evaluation results of multimodal models with key information extraction subtasks for the SER and RE tasks. Figure 12: Comparison of F1 results obtained through 5-fold cross-validation on CMERD for different KIE models; (**a**) is the comparison of F1 results obtained through 5-fold cross-validation for the SER task; (**b**) is the comparison of F1 results obtained through 5-fold cross-validation for the RE task. Figure 13: Results of the two-sample \(t\)-test for F1 scores obtained through 5-fold cross-validation on the CMERD dataset for different KIE models; (**a**) results of the two-sample \(t\)-test for the SER task; (**b**) results of the two-sample \(t\)-test for the RE task. the recognition accuracy of less prevalent categories. To solve this problem, we replace the cross-entropy loss [47] of the SER with focal loss [48]. Specifically, focal loss introduces a modulation factor that builds upon the standard cross-entropy loss function, expressed through the following formula:  
\[\text{Focal Loss}(p_{t})=-\alpha_{t}(1-p_{t})^{\gamma}\log(p_{t}) \tag{17}\]  
where \(p_{t}\) represents the probability predicted by the model.