# Intelligent Evaluation of Chinese Hard-Pen Calligraphy Using a Siamese Transformer Network
## 3 Proposed Methodology
### Encoder  
As shown in Figure 9, the encoding structure of the Transformer consists of multiple layers, each composed of multi-head self-attention mechanisms and feedforward neural networks. The input to this structure must be a sequence in time. Each time step of the sequence is processed by the CNN Encoder and then enriched with positional encoding. Every time step of the sequence is inputted into the multi-layer Transformer encoder. The self-attention mechanism captures the interdependencies between the components of the input sequence by computing self-attention for each time step. In the feedforward neural network, the model transforms each time step through two linear transformations and activation functions. After processing through the encoding structure, the sequence acquires  
Figure 8: Flowchart of Semantic Tokenizer Processing. its feature representation, which is then utilized by the decoder to generate the target image sequence. The processed tokens \(T^{i}\) (\(i\) = 1,2) from the Semantic Tokenizer module are concatenated, yielding the input tokens represented as \(X\) = {\(x\_1\), \(x\_2\),..., \(x\_2L\)}, where \(x\_t\) denotes the feature token at time step \(t\). Positional Encoding incorporates a d-dimensional vector into each time step \(t\) to enable the model to discern the distances between various time steps. This vector is defined by Equation (1):  
\[p_{i,j}=\left\{ \begin{aligned} sin\left(i/10,000^{2j/d}\right),& \text{ \emph{if} $j$ is even}\\ cos\left(i/10,000^{2(j-1)/d}\right),&\text{ \emph{if} $j$ is odd}\end{aligned} \right. \tag{1}\]  
where \(i\) represents the position in the sequence, \(j\) denotes the dimension in the vector, and \(d\) is the total dimension of the vector. Subsequently, \(x\_t\) is added to \(p\_t\), resulting in \(z\_t\). The matrix \(Z\)\(\in\)\(R^{2L\times C}\) is then subject to normalization. To enhance the processing of multi-modal data in this study, we employ a novel normalization method called \"Norm\". Norm consists of two main components: Instance Normalization (_IN_) and Layer Normalization (_LN_). _IN_ is designed for normalizing individual modalities and primarily operates on the channel dimension of a single modality's data. On the other hand, _LN_ is devised for normalizing multi-modal data, where it conducts normalization on the feature vectors at each position of each modality within every sample. The normalization formula for _IN_ is given in Equation (2) [37], wherein \(\mu\) signifies the mean located along the channel dimension, \(\sigma\) indicates the standard deviation along the  
Figure 9: Encoder structure diagram. same dimension, and \(\gamma\) and \(\beta\) are, in turn, recognized as the learnable scaling and shifting factors, respectively, within that dimension:  
\[IN(z)=\gamma\frac{z-\mu}{\sigma}+\beta \tag{2}\]  
The normalization formula of \(LN\) is shown in Equation (3) [38]:  
\[LN(z)=\gamma\frac{z-\mu}{\sqrt{\sigma^{2}+\varepsilon}}+\beta \tag{3}\]  
The input vectors utilizing this normalization technique comprise multiple batches, wherein \(\varepsilon\) denotes a minute constant (typically chosen as \(10^{-5}\)). Following the application of the normalization procedure, these vectors are projected into three distinct vector spaces: Query, Key, and Value. These projections are carried out through the utilization of three parameter matrices \(W_{Q}\), \(W_{K}\), and \(W_{V}\), each belonging to the domain \(R^{d\times L}\), as shown in Equation (4):  
\[Q=ZW_{Q}\ \ \ K=ZW_{K}\ \ \ V=ZW_{V} \tag{4}\]  
The corresponding attention scores are computed through the dot product operation between \(Q\) and \(K\) as follows:  
\[Attention(q_{t},K,V)=\sum_{i=1}^{T}\frac{exp\big{(}q_{t}k_{i}^{T}\big{)}}{\sum _{j=1}^{T}exp\big{(}q_{t}k_{j}^{T}\big{)}}^{v_{j}} \tag{5}\]  
Here, the \(exp\)(.) function computes the inner product between the query vector and the key vector. When calculating attention scores, the process first involves determining weights based on the similarity between the query and key vectors.