# Toward Semi-Supervised Graphical Object Detection in Document Images
## 4 Experiments
### Results and Discussion  
#### 4.4.1 Publisher  
In this section, the results of multiple models trained on the PubLayNet dataset with the proposed framework are compared with the models trained using supervised techniques. The partially annotated PubLayNet models are first compared with the corresponding supervised models. Table 5 shows the comparison of the Faster-RCNN supervised models on the ResNet-50 backbone with different versions of Soft Teacher that are trained using Faster-RCNN and Swin-T transformers. For the detailed comparison, the Faster-RCNN is further trained using two backbones, i.e., ResNet50 and ResNet101. The semi-supervised models showed around 4.9 points, 5.8 points and 5.5 points improvement in the bounding box mean Average Precision when compared under \(1\%,5\%\) and \(10\%\) of annotated data, respectively.  
Further, a Soft Teacher (Faster-RCNN + ResNet101) model which is trained on \(10\%\) of labeled data for 180,000 iterations with a batch size of 8 is compared with the existing baseline model by [26]. The baseline models are trained using fully labeled PubLayNet data on Faster-RCNN for 180,000 iterations. The results in Table 6 show that the Soft Teacher model outperforms the supervised model's average precision at (IoU = 0.5) by 2.7% and exhibits a similar mean Average Precision of around 90.0%. Finally, the inference times in terms of FPS of our models are shown in Table 5. Although there is no reference FPS from the earlier methods, we tried to compare inference time with our models and found that our model equipped with Faster R-CNN and ResNet50 outperformed ResNet101 and Swin-T.  
In addition to the total mAP, the label-wise AP comparison is shown in Table 7. The graphical objects Table, Figure and List outperform the baseline model by 5.4%, 1.2% and 3.2%, respectively. Moreover, Table 8 as well as Figures 6 and 7 explain and show the detailed metrics of Precision, Recall and F-1 score at various IoU thresholds. The results for the Soft Teacher model trained on 10% labeled PubLayNet data are shown in Figure 1. Additionally, Figure 8 portrays the qualitative results, and Figure 8 (right) shows the model figure and table resulting in False Positives.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Technique** & **Detector** & **[EMAIL_ADDRESS]** & **[EMAIL_ADDRESS]** & **mAP** & **FPS** \\ \hline \multirow{2}{*}{[PERSON] et al. [26]} & Supervised + Faster & \multirow{2}{*}{93.7} & \multirow{2}{*}{91.1} & \multirow{2}{*}{90.2} & \multirow{2}{*}{-} \\  & R-CNN & & & & \\ \hline \multirow{2}{*}{Ours} & Faster R-CNN & \multirow{2}{*}{96.4} & \multirow{2}{*}{93.8} & \multirow{2}{*}{90.0} & \multirow{2}{*}{15.8} \\  & (10\% labeled data) & & & & \\ \hline \hline \end{tabular}
\end{table}
Table 6: Performance comparison of Faster R-CNN baseline model trained on fully labeled data with our semi-supervised model which is trained on 10% labeled PubLayNet dataset. The model uses Faster R-CNN and ResNet101.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Technique** & **Detector + Backbone** & **1\%** & **5\%** & **10\%** & **FPS** \\ \hline \multirow{2}{*}{Supervised} & Faster R-CNN + ResNet50 & 82.5 & 83.3 & 83.4 & 12.3 \\ \hline \multirow{3}{*}{Ours} & Faster R-CNN + ResNet50 & 84.9 & 87.2 & 87.3 & 16.7 \\ \cline{2-6}  & Faster R-CNN + ResNet101 & 87.4 & 88.2 & 88.9 & 15.8 \\ \cline{2-6}  & Swin-T & 88.3 & 89.1 & 88.6 & 14.9 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Performance mAP (0.5:0.95) comparison of supervised and ours models trained on 1%,5% and 10% of PubLayNet dataset.  
Figure 6: A visual representation of Precision, Recall and F1-Score at multiple IoUs. Our model is trained on 10% PubLayNet labeled data with Faster R-CNN and ResNet101.  
Figure 7: A label-wise visual representation of Precision at multiple IoUs. Our model is trained on 10% PubLayNet labeled data with Faster R-CNN and ResNet101.