# Investigating Attention Mechanism for Page Object Detection in Document Images
## 6 Conclusions and Future Work  
In this paper, we propose an object detection pipeline by combining standard modules present in the object-detection world to demonstrate the effectiveness of the attention mechanism on page object detection. The model is trained using three datasets having high-quality annotations and a huge corpus of images for the model to learn adequately without over-fitting. A new backbone, ConvNeXt, was incorporated to improve the quality of model learning, showing that the combination of the right backbone and attention mechanism can improve object detection by a good margin. DocBank and PublayNet share similar visual representations, whereas the IIIT-AR-13K images contain different visual embeddings. Hence, the model proposed in this paper, trained on either DocBank or PublayNet, never encounters graphical objects found in IIIT-AR-13K and fails to classify and segment them appropriately. For the DocBank dataset, the model performance shows an adequate understanding of the data distribution. In some areas, the model suffers from outliers while distinguishing between identical graphical objects, such as paragraphs and captions. Overall, the proposed attention-based model performs the best with the PublayNet dataset due to distinct class labels and clean annotations. The model also benefited from the huge training set. Finally, with IIIT-AR-13K, the model performance takes a big leap compared to the baseline model, although the visual embeddings are extremely ambiguous compared to the other two datasets. A cross-validation experiment is also conducted between the datasets. This experiment evaluates the generalization of models based on a reduced dataset containing the common class labels in all three datasets. The model trained on the reduced set of DocBank generalises extremely well, which is indicated in Table 8. Overall, the DynamicHead module proves to be extremely useful in providing saliency over the important regions of the documents that can help improve the models' performance. To better understand the importance of attention, the future aspects are to evaluate the activations on the input images by extracting the gradients and assessing them. Furthermore, due to the specific visual properties of documents, compared to complex natural images, the model can over-fit easily. To avoid that, increasing the dataset further will help overcome sensitivity issues towards the outliers and help the model achieve robustness. An additional future task would include filtering the anomalies from annotations of the dataset to avoid introducing information ambiguity to the model. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Backbone**} & \multirow{2}{*}{**Runtime (FPS)**} &
\begin{tabular}{c} **No. of Parameters** \\ **(in Millions)** \\ \end{tabular} & \multirow{2}{*}{**GPU**} & \multirow{2}{*}{**Batch Size**} \\  & & & & & \\ \hline Faster-RCNN (Baseline) & ResNet-50 & 9.25 & 41.3 & RTX3090 & 1 \\ \hline Faster-RCNN + DyHead & ResNet-50 & 4.84 & 53.2 & RTX3090 & 1 \\ \hline Faster-RCNN + DyHead & Swin (tiny) & 4.24 & 59.3 & RTX3090 & 1 \\ \hline Faster-RCNN + DyHead & ConvNeXt (tiny) & 4.41 & 56.6 & RTX3090 & 1 \\ \hline Sparse-RCNN + DyHead & ResNet-50 & 6.41 & 111.93 & RTX3090 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: The computational complexity of our algorithm with various configurations. The baseline configuration consists of a ResNet-50 backbone and Faster-RCNN as the head. All the rest use DyHead as the attention mechanism along with a backbone and the head. The addition of DyHead module adds an overhead to the runtime of the overall model, due to additional parameters. An interesting observation is the runtime of Sparse-RCNN with DyHead. Due to learned proposals, the model runs faster than its counterpart with a similar configuration, i.e., Faster-RCNN + ResNet-50 + DyHead. Author Contributions:Writing--original draft preparation, S.N., K.A.H. and M.Z.A. ; writing--review and editing, K.A.H., M.Z.A. and M.L.