# Investigating Attention Mechanism for Page Object Detection in Document Images
## 5 Results and Discussion
### DocBank  
Table 2 shows the experimental results obtained from the baseline model and the attention-based models. All the results shown are calculated for bounding boxes to fairly compare with the baseline model. To the best of my knowledge, this work is the first at computing the metrics on DocBank. Therefore, a comparison with the prior literature is not available. ConvNeXt and DyHead with the standard Faster-RCNN gives the best performance of all our experiments on the DocBank dataset. The Faster-RCNN+ConvNeXt+DyHead model is trained from scratch, as ConvNeXt is a relatively new backbone compared to Swin Transformers or ResNet-50, containing publicly available pre-trained weights trained on ImageNet [44]. Furthermore, Table 3 shows the F1-score, Precision, and Recall for IoU levels of 0.5 to 0.9, using the best model, i.e., Faster-RCNN+ConvNeXt+DyHead. The model performs the best at the IoU level of 0.5, and the performance gradually declines as the IoU increases due to a reduction in False Positives. Figure 5 shows the visual improvements in the detections carried out by the baseline model, Faster-RCNN+ResNet-50+DyHead, and Faster-RCNN+ConvNeXt+DyHead variants. \begin{table}
\begin{tabular}{c c c c c c} \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{\
\begin{tabular}{c} **Faster-RCNN** \\ **(Baseline)** \\ \end{tabular} } & \multicolumn{2}{c}{**Faster-RCNN**} & \multicolumn{2}{c}{**Sparse-RCNN**} \\  & & & **+DyHead** & & **+DyHead** \\ \hline \multicolumn{2}{c}{**Backbone**} & **ResNet-50** & **ResNet-50** & **ConvNeXt (tiny)** & **Swin (tiny)** & **ResNet-50** \\ \hline \multirow{8}{*}{
\begin{tabular}{c} **Average** \\ **Precision** \\ **(AP)** \\ \end{tabular} } & paragraph & 74.35 & 80.01 & 87.51 & 40.46 & 84.52 \\  & equation & 65.17 & 74.22 & 83.45 & 61.03 & 78.08 \\  & abstract & 83.22 & 86.40 & 92.07 & 90.40 & 91.61 \\  & table & 71.14 & 77.68 & 86.73 & 76.02 & 86.31 \\  & list & 54.88 & 66.82 & 81.31 & 68.71 & 76.97 \\  & section & 65.10 & 71.93 & 80.02 & 63.37 & 75.15 \\  & figure & 62.62 & 64.99 & 71.39 & 60.61 & 71.92 \\  & author & 44.79 & 55.72 & 76.36 & 72.4 & 74.18 \\  & footer & 64.51 & 70.61 & 81.84 & 77.03 & 78.66 \\  & caption & 71.12 & 75.65 & 83.32 & 75.75 & 80.78 \\  & date & 7.14 & 0 & 44.17 & 0 & 36.85 \\  & title & 77.27 & 79.80 & 85.96 & 85.45 & 82.44 \\  & reference & 77.31 & 82.18 & 88.13 & 81.91 & 87.03 \\ \hline \multicolumn{2}{c}{**mAP**} & 62.97 & 68.15 & 80.18 & 65.63 & 77.27 \\ \hline \end{tabular}
\end{table} 
Table 2: Performance evaluations of DocBank using Faster-RCNN with different backbones. Results with Sparse-RCNN using ResNet-50 backbone are also presented. For Faster-RCNN+ResNet-50, the inclusion of the attention mechanism improves the mAP by 5.18. The Faster-RCNN+DyHead+ConvNeXt (tiny) combination outperforms baseline model and the other model variants containing DyHead by a large margin. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**IoU** & **AP** & **Precision** & **Recall** & **F1-Score** \\ \hline
0.5 & 90.95 & 0.94 & 0.91 & 0.86 \\
0.6 & 89.99 & 0.93 & 0.90 & 0.85 \\
0.7 & 87.64 & 0.91 & 0.88 & 0.84 \\
0.8 & 82.85 & 0.87 & 0.83 & 0.81 \\
0.9 & 66.79 & 0.72 & 0.67 & 0.72 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The model performance results for Faster-RCNN+ConvNeXt+DyHead (bbox) on DocBank. The DynamicHead module clearly improves the detection of graphical objects when compared with the baseline. Furthermore, DynamicHead coupled with ConvNeXt proves to perform the best while also dealing with the nested detections found in the Faster-RCNN+ResNet-50+DyHead variant. Although the detections performed are adequate, certain discrepancies in the DocBank dataset annotations slightly impact the performance of the model. Some of the examples are shown in Figure 6, e.g., figures are annotated as paragraphs, and annotations contain nested annotations within them. Hence, these discrepancies hinder the model's representation learning, leading to misclassifications.