# Fi-Fo Detector: Figure and Formula Detection Using Deformable Networks
## 4 Experimental Results
### Results and Discussions
\begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline  & & & & \multicolumn{6}{c}{**ICDAR-2017 POD**} \\ \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Image Representation**} & \multirow{2}{*}{**Class**} & \multicolumn{6}{c}{_IoU = 0.6_} & \multicolumn{6}{c}{_IoU = 0.8_} \\ \cline{3-11}  & & & **Precision** & **Recall** & **F1-Score** & **AP** & **Precision** & **Recall** & **F1-Score** & **AP** \\ \hline \multirow{2}{*}{\begin{tabular}{c} Fi-Fo Detector \\ Deformable \\ \end{tabular} } & \multirow{2}{*}{Raw} & Formula & 0.867 & 0.918 & 0.892 & 0.893 & 0.780 & 0.826 & 0.802 & 0.780 \\ \cline{3-11}  & & Figure & 0.860 & 0.869 & 0.864 & 0.847 & 0.818 & 0.827 & 0.822 & 0.799 \\ \hline \multirow{2}{*}{\begin{tabular}{c} Fi-Fo Detector \\ Non Deformable \\ \end{tabular} } & \multirow{2}{*}{Fi-Fo} & Formula & 0.867 & 0.874 & 0.871 & 0.917 & 0.712 & 0.694 & 0.703 & 0.837 \\ \cline{3-11}  & & Figure & 0.856 & 0.821 & 0.838 & 0.929 & 0.801 & 0.739 & 0.769 & 0.889 \\ \hline \multirow{2}{*}{\begin{tabular}{c} Fi-Fo Detector \\ Deformable \\ \end{tabular} } & \multirow{2}{*}{Fi-Fo} & Formula & 0.909 & 0.927 & 0.918 & 0.911 & 0.856 & 0.878 & 0.867 & 0.854 \\ \cline{3-11}  & & Figure & 0.918 & 0.883 & 0.90 & 0.894 & 0.871 & 0.838 & 0.854 & 0.861 \\ \hline \multicolumn{11}{c}{\multirow{2}{*}{\begin{tabular}{c} Fi-Fo Detector \\ Deformable \\ \end{tabular} }} & \multicolumn{6}{c}{**ICDAR-2017 POD (Corrected)**} \\ \hline \multirow{2}{*}{\begin{tabular}{c} Fi-Fo Detector \\ Deformable \\ \end{tabular} } & \multirow{2}{*}{Raw} & Formula & 0.949 & 0.945 & 0.947 & 0.973 & 0.897 & 0.893 & 0.895 & 0.967 \\ \cline{3-11}  & & Figure & 0.930 & 0.932 & 0.931 & 0.971 & 0.897 & 0.90 & 0.899 & 0.959 \\ \hline \multirow{2}{*}{\
\begin{tabular}{c} Fi-Fo Detector \\ Non Deformable \\ \end{tabular} } & \multirow{2}{*}{Fi-Fo} & Formula & 0.910 & 0.927 & 0.918 & 0.953 & 0.860 & 0.877 & 0.868 & 0.928 \\ \cline{3-11}  & & Figure & 0.879 & 0.822 & 0.850 & 0.948 & 0.847 & 0.792 & 0.819 & 0.958 \\ \hline \multirow{2}{*}{
\begin{tabular}{c} Fi-Fo Detector \\ Deformable \\ \end{tabular} } & \multirow{2}{*}{Fi-Fo} & Formula & 0.957 & 0.952 & 0.954 & 0.949 & 0.913 & 0.908 & 0.910 & 0.898 \\ \cline{3-11}  & & Figure & 0.931 & 0.913 & 0.922 & 0.905 & 0.901 & 0.885 & 0.893 & 0.870 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: An ablation study on the performance of Fi-Fo detector using raw image representation, Fi-Fo image representation, using non deformable FPN, and deformable FPN, both for ICDAR-2017 POD and ICDAR-2017 POD (corrected). Figure 6: Analysis of results generated by Fi-Fo detector using the ICDAR-2017 POD (corrected) dataset, green color highlights true positives (TPs), blue color signifies false positives (FPs) for formulas and magenta color flags FPs for figures, and red color annotates FNs for both classes. Moreover, it is also demonstrated that the Fi-Fo detector shows progressive performance moving from ICDAR-2017 POD to ICDAR-2017 POD (corrected) at every subsequent step, whereas existing state-of-the-art failed to do so, as shown in Table 4. Since Fi-Fo is a data-driven approach, significant improvements in performance could be achieved by increasing the amount of training data. Results can further be boosted by post-processing particularly for the detected figure regions using computer-vision approaches, considering the case of Figure 6c into account. We did fine-tune the annotations of ICDAR-2017 POD and make them publicly available as ICDAR-2017 POD (corrected), but the problem requires much more effort in terms of relabeling the document page objects, potentially requiring an introduction of a new and cleaned POD dataset based on hard defined conventions, which is out of the scope of current work. Here, we aim to demonstrate the potential of a hybrid approach, a combination of Fi-Fo image representation and deformable neural networks to detect page objects from document images.