# Intelligent Evaluation of Chinese Hard-Pen Calligraphy Using a Siamese Transformer Network
## 4 Results and Discussion
### Analysis of Calligraphy Evaluation Experiments  
The calligraphy evaluation aims to predict the similarity between two images using a Siamese network framework. VGG16 is employed as the backbone feature extraction network, while a Transformer serves as the enhanced feature extraction network to accomplish feature extraction. Then, the similarity between the written samples and standard \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **mPrecision (\%)** & **mRecall (\%)** & **mAP50 (\%)** \\ \hline \hline GoogLeNet & 99.84 & 99.90 & 99.70 \\ YOLOv5 & 99.99 & 100 & 99.95 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance comparison of different models. Figure 12: YOLOv5 training process. Figure 13: Comparison of test results. (**a**) GoogLeNet test results; (**b**) YOLOv5 test results. samples is calculated. The experimental environment is built on PyTorch and utilizes libraries such as NumPy, os, cv2, and PIL. The SGD optimizer is selected with a learning rate of 0.01, a momentum value of 0.9, and a weight decay of 0.0005. The batch size is set to two. The training process's loss variation is illustrated in Figure 14, and training concludes after 100 epochs with the minimum achieved loss of 0.178. The loss gradually decreases around the 80 th epoch, converging to a steady state, reaching an optimal value. To identify better-performing evaluation models, it is necessary to execute operations such as swapping backbone feature network models and removing network models to compare the resulting differences. The comparison involves ResNet50 and VGG16 as choices for the backbone feature extraction network. ResNet and VGG16 both belong to the category of deep convolutional neural networks (CNNs), yet they exhibit certain architectural distinctions. ResNet addresses the vanishing gradient problem through residual connections, enabling cross-layer information propagation that sustains performance in deep networks [39]. ResNet is notably deeper than VGG16, capable of surpassing 100 layers. Furthermore, to further assess the influence of the backbone feature extraction network on the calligraphy evaluation model, we exclude the VGG16 network. Instead, the transformer structure is directly employed for feature extraction, known as the ChangeFormer network -- a transformer-based Siamese network. The core concept of the ChangeFormer involves employing self-attention mechanisms to establish global dependencies within input sequences. It also concurrently employs variable-length sliding windows to capture local dependencies. Similarly, to verify the role of the enhanced feature extraction network in the calligraphy evaluation model, the transformer structure is removed. This involves utilizing solely the backbone feature network for feature extraction, omitting the enhanced feature extraction network and capturing more image-feature information. Consequently, the context and positional dependencies between input sequences are disregarded. This approach aims to observe the network training outcomes under such conditions. The variations in training loss for the calligraphy evaluation models are depicted in Figure 15. When comparing Figure 15a with Figure 14, it becomes apparent that the rate of loss decline is higher when VGG16 is utilized as the backbone feature extraction network. The minimum loss value achieved is 0.178 and is reached around the 70 th epoch, with relatively minor fluctuations thereafter, indicating near-convergence to an optimal state. Conversely, employing ResNet50 as the backbone feature extraction network results in a more gradual descent of loss, with a minimum loss value of 0.385. By the 70 th epoch, a similar steady state is achieved. From this, it can be deduced that VGG16 outperforms ResNet50 as the backbone network for calligraphy evaluation. Comparing Figure 15b with Figure 14, when the backbone feature extraction network is omitted and only the transformer structure is employed for feature extraction, the loss undergoes marginal fluctuations around 0.69.