# A Structured Recognition Method for Invoices Based on StrucTexT Model
## 3 Experiment and Analysis
### Comparison of Experimental Results  
The experiments were first conducted on the SROIE public dataset, and the optimized model StrucTexT_slim's performance in entity labeling was compared with that of LayoutLM, LayoutLMv2, YOLOv4, LeNet-5, and Tesseract-OCR. The results are shown in Table 1.  
The time in the table indicates the average time required to recognize an image. It is obvious from the table that the improved model StrucTexT_slim has an accuracy of 94.7% on the SROIE dataset, which is 2% lower than that of the original model but second only to LayoutLMv2_Large in terms of recognition time. It also has the smallest model size. The comparison results on the FUNSD dataset are shown in Table 2.  
In the FUNSD dataset, entities belonging to other categories were ignored, and the average performance of the three categories (title, question, and answer) was used as the final result. Although the StrucTexT_silm model is not optimal in all metrics, the best-performing LayoutLMv2_LARGE model is only 2.6% higher in F1 score than the StrucTexT_silm model. Meanwhile, the LayoutLMv2_LARGE model is larger, consisting of a 24-layer and 16-head transformer that contains 426 M parameters. In addition, the LayoutLMv2_LARGE model uses 11M documents for pre-training. During the product's actual application and deployment, StrucTexT_silm performs significantly better.  
The experiments also used the self-built dataset mentioned in Section 3.1 to recognize some keywords in VAT invoices and airline trip tickets in order to test the model's matching of keyword segments with the recognized information. The recognition results are shown in Table 3, and the model performs better in recognizing tickets such as VAT invoices and airline trip tickets.  
As can be seen from the table, the recognition rate of the model for each key area of the VAT invoices is over 95%, which meets the requirements of invoice recognition.  
From the above experiments, it can be concluded that the optimized model StrucTexT_slim has a higher accuracy; its recognition speed is greatly accelerated, and its model volume is compressed, which solves a series of problems (slow recognition speed, large model volume, and difficulty of deployment within a traditional invoice recognition model). The experiments also show that the model has high recognition accuracy in its assessment of complex invoices, such as VAT invoices.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Precision** & **Recall** & **F1** & **Parameters** & **Time** \\ \hline LayoutLM\_BASE & 0.944 & 0.944 & 0.944 & 113 M & 4.5 s \\ LayoutLM\_LARGE & 0.952 & 0.952 & 0.952 & 343 M & 2.7 s \\ LayoutLMv2\_BASE & 0.963 & 0.963 & 0.963 & 200 M & 3.6 s \\ LayoutLMv2\_LARGE & 0.966 & 0.966 & 0.966 & 426 M & 2.1 s \\ StrucTexT & 0.967 & 0.968 & 0.969 & 107 M & 6.7 s \\ YOLOv4 [2] & 0.954 & 0.954 & 0.954 & 145 M & 4.6 s \\ LeNet-5 [3] & 0.941 & 0.942 & 0.941 & 92 M & 5.6 s \\ Tesseract-OCR [4] & 0.957 & 0.958 & 0.957 & 101 M & 7.3 s \\ StrucTexT\_slim & 0.947 & 0.948 & 0.947 & 80 M & 4.1 s \\ \hline \hline \end{tabular}
\end{table}
Table 1: Model performance comparison on SROIE.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Precision** & **Recall** & **F1** & **Parameters** & **Time** \\ \hline LayoutLM\_BASE & 0.760 & 0.816 & 0.787 & 113 M & 4.1 s \\ LayoutLM\_LARGE & 0.760 & 0.822 & 0.790 & 343 M & 2.9 s \\ LayoutLMv2\_BASE & 0.803 & 0.854 & 0.828 & 200 M & 3.7 s \\ LayoutLMv2\_LARGE & 0.832 & 0.852 & 0.842 & 426 M & 2.0 s \\ StrucTexT & 0.857 & 0.810 & 0.831 & 107 M & 6.5 s \\ YOLOv4 [2] & 0.837 & 0.852 & 0.841 & 145 M & 4.8 s \\ LeNet-5 [3] & 0.821 & 0.819 & 0.825 & 92 M & 6.1 s \\ Tesseract-OCR [4] & 0.836 & 0.841 & 0.837 & 101 M & 7.5 s \\ StrucTexT\_slim & 0.835 & 0.796 & 0.816 & 80 M & 3.9 s \\ \hline \hline \end{tabular}
\end{table}
Table 2: Model performance comparison on FUNSD.