## 2 PD Pattern Recognition Based on MCNN
### Lightweight MCNN Block  
The lightweight and efficient convolution module adopted in the MCNN is shown in Figure 5. Firstly, the number of input feature channels is extended to a large number of intermediate layers by 1 \(\times\) 1 point convolution, then features are extracted and optimized by 3 \(\times\) 3 depth convolution. Finally, 1 \(\times\) 1 point convolution is used to compress the features to the size given by the output channel. The specific implementation of each module of constructing this convolution network corresponding to Figure 6 is shown in Table 1. The number of input channels of the convolution module is N, and the number of output channels is M. For the input feature of a certain size W \(\times\) H, the expansion factor is \(t\), the convolution kernel size is 3, \(s\) denotes stride, and NL denotes the nonlinear function.  
Figure 4: Reverse residual unit.  
Figure 3: ResNet residual unit.  
Table 2 shows the overall structural parameters of the compact and computationally efficient lightweight MCNN. The rows in Table 2 describe the specific structural configuration of each convolution layer in the model [19]. The size of the input data of the first layer is 224 \(\times\) 224 \(\times\) 3, in which 3 represents the 3-channel image in RGB format. The standard convolution operation is carried out on this input data, in which the number of channels of the kernel is set to 16, which can minimize the operation time on the premise of ensuring accuracy. The 2 nd to 12 th convolution layers in Table 2 all adopt the reverse residual module with a linear bottleneck. The 13 th layer is point-by-point convolution. The 14 th layer is the pooling layer, which converts the output feature from the convolution layer pooling to \(1\times 1\times 576\). Then, the fully connected layer is used to reconstruct the final feature image into a lower-dimensional vector. Finally, \(k\) types of identification information are output in the 16 th layer.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Input** & **Operator** & **Output** \\ \hline W \(\times\) H \(\times\) N & \(1\times 1\) conv2d, NL & W \(\times\) H \(\times\) t N \\ W \(\times\) H \(\times\) t N & \(3\times 3\) dwise s = s, NL & W/s \(\times\) H/s \(\times\) t N \\ W/s \(\times\) H/s \(\times\) t N & Linear \(1\times 1\) conv2d & W/s \(\times\) H/s \(\times\) M \\ \hline \hline \end{tabular}
\end{table}
Table 1: Core module construction of MOBILENETS.  
Figure 5: Structural Diagram of MobileNets Block.  
Figure 6: The procedure of PD pattern recognition by MCNN.  
The \"conv2d\" in the Table 2 denotes the two-dimensional convolution operation; the \"Bneck\" is the bottleneck structure, the \"NBN\" indicates that the batch is not normalized, and the module with \"SE\" lightweight attention structure is marked in the \"SE\" column. The type of non-linear used in each convolution layer is introduced in \"NL\", in which \"HS\" means to use h-swish, \"RE\" means to use standard ReLU6.  
In some convolution layers of MCNN, h-swish is used to replace swish. This is because although swish can effectively improve the accuracy of the network, it will lead to a large amount of calculation and is takes too much time to contain the sigmoid function due to its characteristics of no upper bound and lower bound, and non-monotonicity. The h-swish can not only better approximate the original swish to reduce the amount of computation but can also improve the computational efficiency in the quantization mode, which makes the model more suitable for the embedded, low-power environment. The mathematical definitions of the swish and the h-swish are as follows. \(x\) is the input signal.  
\[\text{swish}\ x=X\frac{1}{1+e^{-x}} \tag{2}\]  
\[\text{h}-\text{swish}(x)=X\frac{\text{ReLU6}(x+3)}{6} \tag{3}\]  
\[\text{ReLU6}(x)=\left\{\begin{array}{ll}0&(x<0)\\ x&(0\leq x\leq 6)\\ 6&(x\geq 6)\end{array}\right. \tag{4}\]