# Fi-Fo Detector: Figure and Formula Detection Using Deformable Networks
## 4 Experimental Results
### Results and Discussions
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & & & \multicolumn{6}{c}{**ICDAR-2017 POD**} \\ \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Class**} & \multicolumn{6}{c}{_IoU = 0.6_} & \multicolumn{6}{c}{_IoU = 0.8_} \\ \cline{3-10}  & & **Precision** & **Recall** & **F1-Score** & **AP** & **Precision** & **Recall** & **F1-Score** & **AP** \\ \hline \multirow{2}{*}{NLPR-PAL [19]} & Formula & 0.901 & 0.929 & 0.915 & 0.839 & 0.888 & 0.916 & 0.902 & 0.816 \\  & Figure & 0.920 & 0.933 & 0.927 & 0.849 & 0.892 & 0.904 & 0.898 & 0.805 \\ \hline \multirow{2}{*}{[PERSON] et al. [9]} & Formula & 0.93 & 0.953 & 0.942 & 0.878 & 0.921 & 0.944 & 0.932 & 0.863 \\ ICDAR-2017 POD & Figure & 0.948 & 0.940 & 0.944 & 0.896 & 0.921 & 0.913 & 0.917 & 0.85 \\ \hline \multirow{2}{*}{Deformable Faster R-CNN} & Formula & 0.882 & 0.738 & 0.803 & 0.660 & 0.638 & 0.534 & 0.582 & 0.337 \\  & Figure & 0.929 & 0.872 & 0.899 & 0.660 & 0.855 & 0.802 & 0.828 & 0.720 \\ \hline \multirow{2}{*}{Deformable R-FCN} & Formula & 0.914 & 0.918 & 0.916 & 0.915 & 0.832 & 0.836 & 0.834 & 0.826 \\  & Figure & 0.904 & 0.920 & 0.912 & 0.903 & 0.86 & 0.875 & 0.867 & 0.864 \\ \hline \multirow{2}{*}{Fi-Fo Detector} & Formula & 0.909 & 0.927 & 0.918 & 0.911 & 0.856 & 0.878 & 0.867 & 0.854 \\  & Figure & 0.918 & 0.883 & 0.90 & 0.894 & 0.871 & 0.838 & 0.854 & 0.861 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of Fi-Fo with existing state-of-the-art methods using ICDAR-2017 POD annotations both for training and testing. to establish a fair comparison with the existing state-of-the-art system. We trained the Fi-Fo detector using the ICDAR-2017 POD dataset, while evaluation was performed on ICDAR-2017 POD (corrected) for both the Fi-Fo detector and [PERSON] et al. [9], and the results are furnished in Table 3.  
Results produced by the Fi-Fo detector on ICDAR-2017 POD (corrected) outperformed existing state-of-the-art system by large margins. On an IoU threshold of 0.6, Fi-Fo achieved the F1-score of 0.947 with the average precision (AP) of 0.967 in comparison to [PERSON] et al. 's [9] system with an F1-score of 0.489 and AP of 0.312, for formula detection. Similarly, there is an enormous difference in the results of figure detection produced by the Fi-Fo detector and [PERSON] et al. [9] on ICDAR-2017 POD (corrected). Fi-Fo detected the figures with an F1-score of 0.931, whereas [PERSON] et al. 's F1-score is 0.271, as shown in Table 3. On IoU threshold of 0.8, AP for [PERSON] et al. was computed to be 0.253 and 0.274 in comparison to Fi-Fo's AP score of 0.941 and 0.952 for figure and formula detection, respectively. One of the potential reasons for dramatic decline in performance of [PERSON] et al. 's method on updated annotations might be its inability as an end-to-end system. Since the dataset itself had inconsistencies, state-of-the-art methods had to leverage hand-defined heuristics, which catered for these inconsistencies. Upon removal of these inconsistencies from the dataset, the heuristics themselves had to be adapted, which is a major short-coming of heuristics-based systems. The given results establish the weakness of the existing state-of-the-art system both on the ICDAR-2017 POD as well as the ICDAR-2017 POD (corrected) dataset. Using ICDAR-2017 POD, their system failed to detect and highlight the missing or wrong labels, and it failed to capitalize on the ICDAR-2017 POD (corrected). It is worth emphasizing again that all the reported results on the original and updated annotations in this paper are generated using the official evaluation code released by organizers of the ICDAR-2017 competition on page object detection [18]. Despite reporting significantly high metric-scores, [PERSON] et al. 's [9] method was unable to find any inconsistencies in the ICDAR-2017 POD dataset. As their approach relies heavily on heuristics along with pre/post-processing, which were specifically tuned to the inconsistencies present in the dataset.