## 3 Lip-Recognition Model Structure and Prune
#### Pruning Network Training and Testing  
The commonly used pruning algorithms generally follow the flow shown in Figure 5. Although the details of each algorithm may differ, the processes are similar [15]. Figure 4: Network structure diagram. The first step is the most important part of the pruning process. The importance of a pruned unit can be a weight parameter or an entire network layer, depending on the granularity of the pruning. The methods for measuring importance range from basic optimization-based algorithms to structure-based methods, with their own complexity and accuracy, and they can be chosen according to the needs. The second step is to cut out the unimportant connection parameters based on the results measured in the previous step. We can set the corresponding thresholds or rank them according to their importance to decide which parameters to cut. The third step is to fine-tune the network, i.e., to retrain the network. Retraining the pruned network can improve the accuracy and performance of the network, and the degree of fine-tuning can be adjusted according to your needs. The purpose of compression and acceleration is to transform the original network with high-density parameters into a sparse network, without sacrificing its recognition accuracy [16]. To maintain its accuracy, we need to prune the unimportant parameters in the model, so the core of the pruning problem is to set evaluation criteria for the importance of the parameters. There are two classical pruning methods, one is to set a threshold value to judge the weights according to the importance of the weights, the weights below the threshold value are unimportant, and the other is based on the Hessian matrix of the loss function on the weights, i.e., [17], the higher the value of the matrix, the more important the parameters are. Furthermore, the overall process of model channel pruning is shown in Figure 6, which is applied to the sparse channel. Channel-level based sparse pruning has good results in lip recognition. The application of sparsification can be implemented at different levels, including fine-strength-based sparsification, but fine-strength sparsification relies on special hardware and software equipment to perform sparsification, otherwise, the model runs slowly and the results are not satisfactory. Figure 5: Pruning flow chart. Figure 6: The pruning process. When applying channelization-based sparsification to a deep network (e.g., VGG16 in this paper), pruning the number of layers also achieves the desired effects, balancing speed and implementation. Furthermore, by adjusting the parameter size and channel weights in the model channels for joint training, channel-level sparsity can be achieved. Firstly, we trained the network sparsely to adjust the parameters. Secondly, we pruned the channels whose parameters were close to 0 after adjustment, and then we fine-tuned the network and retrained it. Through several iterations, we could obtain a network with low computational cost that requires little memory and retains the accuracy without cutting down on the parameters. The batch normalization layer enables the subsequent training of the input data to reach a steady state [18]. BN layers are widely used in convolutional neural networks for their ability to improve network convergence speed and learning performance. Thus, we proposed a channel-level network pruning method by borrowing the batch normalization layer (BN) concept to discriminate channels, the process of which is shown in Figure 7.  
During training, the update of parameters can constantly reset the proportion intensity of each layer. In other words, layers affect each other, while the pattern of input information to each layer is constantly changing. As a result, each layer needs to adapt continuously to the distribution of information during training, which may reduce training speed and lead to slower convergence. Moreover, this phenomenon occurs more frequently in deep neural networks, and it is called internal covariate shift [19].