# Investigating Attention Mechanism for Page Object Detection in Document Images
## 5 Results and Discussion
### PublayNet  
In the case of PublayNet, the proposed architecture beats the baseline results. Table 4 shows the test results using different variants of the architecture. The inclusion of Dynamic-Head improves the baseline mAP scores by 1.65. Furthermore, ConvNeXt as the backbone further improves the scores by 4.22.  
The combination of ConvNeXt and DyHead with the standard Faster-RCNN surpasses the baseline scores. Table 5 shows the F1-score, Precision, and Recall values for IoU levels of 0.5 to 0.9 for this model variant. Figure 7 shows detections obtained on PublayNet samples using different variants of the model. At first glance, it is clear that the use of DynamicHead improves the localization of the graphical objects. Similar to the results in Section 5.1, Faster-RCNN+ConvNeXt+DyHead performs the best out of all the variants.  
The model performs fairly better on PublayNet compared to DocBank. The dataset contains distinct classes which allow the model to discreetly interpret their distinct features. The F1-score at different IoU levels, as shown in Table 5, also conveys the robustness of the model to detect objects even at the highest IoU level, i.e., 0.9. Additionally, with the existence of high-quality annotations present in both train and test sets, the model does not suffer from confusion.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Faster-RCNN**} & \multicolumn{2}{c}{**Faster-RCNN**} & \multicolumn{2}{c}{**Sparse-RCNN**} \\  & \multicolumn{2}{c}{**(baseline)**} & \multicolumn{2}{c}{**+DyHead**} & \multicolumn{2}{c}{**+DyHead**} \\ \hline \multicolumn{2}{c}{**Bacbone**} & \multicolumn{2}{c}{**ResNet-50**} & \multicolumn{2}{c}{**ResNet-50**} & \multicolumn{2}{c}{**ConvNeXt**} & \multicolumn{2}{c}{**Swint-tiny**} & **ResNet-50** \\ \hline \multirow{4}{*}{**Average Precision**} & text & 91.3 & 92.72 & 94.29 & 92.23 & 92.71 \\  & title & 81.2 & 85.94 & 88.72 & 85.31 & 82.75 \\ \multirow{2}{*}{**Precision (AP)**} & list & 88.5 & 89.04 & 94.34 & 87.81 & 87.84 \\  & table & 94.3 & 96.57 & 97.62 & 96.73 & 95.32 \\  & figure & 94.5 & 94.01 & 96.12 & 94.23 & 92.05 \\ \hline \hline mAP & 90 & 91.65 & 94.22 & 91.27 & 90.14 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Performance evaluations of PublayNet using Faster-RCNN with different backbones. Results with Sparse-RCNN using ResNet-50 backbone are also presented. For Faster-RCNN+ResNet-50, the attention mechanism contributes a slight improvement. However, the model benefits largely from the ConvNeXt backbone complemented with the attention mechanism to improve the feature saliency.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**IoU** & **AP** & **Precision** & **Recall** & **F1-Score** \\ \hline
0.5 & 97.17 & 0.98 & 0.97 & 0.95 \\
0.6 & 96.71 & 0.97 & 0.97 & 0.95 \\
0.7 & 95.88 & 0.97 & 0.96 & 0.95 \\
0.8 & 93.49 & 0.94 & 0.93 & 0.93 \\
0.9 & 84.43 & 0.86 & 0.84 & 0.86 \\ \hline \hline \end{tabular}
\end{table}
Table 5: The model performance results for ConvNeXt+DyHead backbone with Faster-RCNN (bbox) on PublayNet.