# Cascade Network with Deformable Composite Backbone for Formula Detection in Scanned Document Images
## 4 Experimental Results
### Result and Discussion
#### 4.3.1 ICDAR-17  
We follow the evaluation protocol as elaborated in ICDAR-2017 POD [27]. We first calculate the number of true positives, false positives, and false negatives from the complete test set. We then compute the precision, recall, and F1-Score as calculated in the prior methods [20; 41]. Moreover, we also report the mAP score by evaluating the performance of our method on the test set. Following the criteria of the competition, we present results on the IoU threshold of 0.6 and 0.8. It is essential to emphasize that we have employed the recently published corrected version of the dataset [20]. Therefore, only the methods that have reported results on the corrected version of the dataset are directly comparable with our approach. Table 3 presents the results that are achieved by our proposed end-to-end method with and without incorporating the pre-processing technique. After setting an IoU threshold of 0.6, we achieve a precision of 0.95, recall of 0.948, f1-score of 0.949, and mAP of 0.97 without the inclusion of the pre-processing method. The results further improve with an average of almost 0.04 after employing the proposed pre-processing. Upon increasing the IoU threshold value to 0.8, our network reaches a precision of 0.914, recall of 0.912, f1-score of 0.913, and mAP score of 0.949 in the absence of pre-processing method, and the presence of pre-processing advances the results with an average difference of 0.04. For the completeness of the paper, we compute the f1-score of the proposed method on different IoU thresholds ranging from 0.5 to 1.0. Figure 7 illustrates the performance of our approach in terms of f1-score. Figures 8 and 9 depict the qualitative performance of our proposed system. Out of 1929 isolate formulas present in the test set, our cascade formula detection network correctly predicted the region for 1836 formulas at an IoU threshold of 0.6. Moreover, it is vital to mention that even at a higher IoU threshold of 0.8, the system identified correct boundaries for 1767 formulas present in the test set. We also observe some rare cases of false positive and false negative samples, which are exhibited in Figure 9.  
Comparison with State-of-the-Art Methods  
By looking at Table 3, it is evident that our hybrid method of cascade network leveraging deformable composite backbone with lightweight pre-processing has outperformed the prior state-of-the-art method [20] on a higher IoU threshold of 0.8 with an average f1-score of 0.917, thus reducing the relative error by 7.8%. Furthermore, we achieve an almost identical f1-score at an IoU threshold of 0.6. It is essential to emphasize that the previous state-of-the-art work [20] depends on the heavy pre-processing pipeline consisting of distance transform and connected components analysis (CCA) applied on grayscale images. However, our generic data-driven method operates on the lightweight dilation technique to produce better results. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{6}{c}{**ICDAR-2017 POD**} \\ \hline \multirow{2}{*}{**Method**} & \multicolumn{6}{c}{**IoU = 0.6**} & \multicolumn{6}{c}{**IoU = 0.8**} \\ \cline{2-9}  & **Precision** & **Recall** & **F1-Score** & **AP** & **Precision** & **Recall** & **F1-Score** & **AP** \\ \hline NLPR-PAL [27] \(\dagger\) & 0.901 & 0.929 & 0.915 & 0.839 & 0.888 & 0.916 & 0.902 & 0.816 \\ \hline [PERSON] et al. [41] \(\dagger\) & 0.935 & 0.331 & 0.489 & 0.312 & 0.877 & 0.310 & 0.459 & 0.274 \\ \hline Fi-Fo Detector & 0.910 & 0.927 & 0.918 & 0.953 & 0.860 & 0.877 & 0.868 & 0.928 \\ Non Deformable [20] & & & & & & & & \\ \hline Fi-Fo Detector & 0.957 & 0.952 & 0.954 & 0.949 & 0.913 & 0.908 & 0.910 & 0.898 \\ Deformable [20] & & & & & & & & \\ \hline Ours (Without & 0.950 & 0.948 & 0.949 & 0.97 & 0.914 & 0.912 & 0.913 & 0.949 \\ Pre-Processing) & & & & & & & & \\ \hline Ours (Complete Method) & 0.954 & 0.952 & 0.953 & 0.97.5 & 0.918 & 0.916 & 0.917 & 0.954 \\ \hline \hline \end{tabular}
\end{table}