# Lightweight Transformer Network for Ship HRRP Target Recognition
## 2 Proposed Model
### Group Linear Transformations (GLTs)  
For a model, a larger number of parameters means higher equipment requirements and longer training time. A comparison of the Transformer encoder and the Transformer encoder combined with GLTs is shown in Figure 3. Compared with the standard Transformer encoder structure, the Transformer encoder combined GLTs uses GLTs at the input to process the input data. In two stages of expansion and reduction, the input data are first mapped to a high-dimensional spatial expansion to obtain more easily distinguishable feature, and then the dimension is reduced as the input to the Self-Attention layer. This transformation allows for more efficient parameter assignment in the expansion and reduction stages, yielding better results than simple linear layer coding. This is because GLTs are able to learn a wider range of features in different dimensions. Under the condition of ensuring good recognition accuracy, Multi-head Attention can be replaced by Self-Attention, while the input dimension of Self-Attention is half of that of Multi-head Attention in the standard Transformer. So, the number of multiplication and addition operations required for calculation is reduced by half, and the corresponding computational effort is greatly  
Figure 2: Local RNN module. reduced. The input dimension of the lightweight feedforward neural network in the Transformer encoder combined with GLTs is the same as that of the standard Transformer, but the dimension reduction process in the first layer reduces both the dimension and the size of the feedforward neural network to one-fourth of the original size, and the parameters become one-sixteenth of the original size. The hidden vector \(\mathbf{H}\), which is output by the local RNN module, is used as the input to the Transformer encoder with the GLTs. Suppose the number of layers of GLTs is \(N\). In the expansion stage, in \(\left\lceil N/2\right\rceil\) linear layers, the \(d_{m}\)-dimensional input is projected into the high-dimensional space with \(d_{\max}=w_{m}d_{m}\), where \(w_{m}\) is expanded dimensional coefficients; similarly, in the reduction stage, the \(d_{\max}\)-dimensional vector is projected into the \(d_{0}\)-dimensional space using the remaining \(N-\left\lceil N/2\right\rceil\) layers, projecting the \(d_{\max}\) dimensional vector into the \(d_{0}\) dimensional space. \[g^{l}=\left\{ \begin{matrix}\min\left(2^{l-1},g_{\max}\right),1\leq l\leq\left \lceil N/2\right\rceil\\ g^{N-1},Otherwise\end{matrix} \right. \tag{7}\]  
where \(g_{\max}\) is the set maximum number of linear transformation layers per layer. \(g^{l}\) denotes the number of linear transformation layers contained in the \(l\)th layer. Input the hidden vector \(\mathbf{H}\):  
\[\mathbf{Y}^{l}=\left\{ \begin{matrix}F\left(\mathbf{H},\mathbf{W}^{l},\mathbf{b}^{l},g^{l} \right),l=1\\ F\left(MIX\left(\mathbf{H},\mathbf{Y}^{l-1}\right),\mathbf{W}^{l},\mathbf{b}^{l},g^{l}\right),Otherwise\end{matrix} \right. \tag{8}\]  
where \(\mathbf{W}^{l}=\left[\mathbf{W}^{l}_{\mathbf{1}},\mathbf{W}^{l}_{\mathbf{2}},\ldots,\mathbf{W}^{l}_{g^{ l}}\right]\) and \(\mathbf{b}^{l}=\left[\mathbf{b}^{l}_{\mathbf{1}},\mathbf{b}^{l}_{\mathbf{2}},\ldots,\mathbf{b}^{l}_{g^{ l}}\right]\) are the weights and bias of the \(g^{l}\) group linear transformation function in the \(l\)th layer, respectively. \(F\) is a linear transformation function that divides the input \(H\) into \(g^{l}\) non-overlapping groups so that \(\mathbf{H}\) can be written as \(\mathbf{H}=\left[\mathbf{H}_{\mathbf{1}},\mathbf{H}_{\mathbf{2}},\ldots,\mathbf{H}_{g^{l}}\right]\), and the linear layer transforms each \(\mathbf{H}_{i}\) by the weights \(\mathbf{W}^{l}_{i}\) and \(\mathbf{b}^{l}_{i}\) to obtain an output \(\mathbf{Y}^{l}_{i}=\mathbf{H}_{i}\mathbf{W}^{l}_{i}+\mathbf{b}^{l}_{i}\). The output \(\mathbf{Y}^{l}_{i}\) of each group is then stitched to produce the final output \(\mathbf{Y}^{l}\). Figure 3: Comparison of Transformer encoder and Transformer encoder combined with GLTs. The left is the standard Transformer encoder.