# Hardware Acceleration of Satellite Remote Sensing Image Object Detection Based on Channel Pruning
## 3 Methodology
### YOLOv4-MobileNetv3 Model Channel Pruning  
The inference stage of the network model requires a large amount of memory bandwidth, and the YOLOv4-MobileNetv3 model still has redundant weights and activation values after lightweight processing, which leads to inefficient calculation in the model inference stage. In addition, due to the limitation of instruction sets and basic operations on hardware devices, many innovative modules or network layers will fail compilation, which greatly limits the deployment of lightweight networks on the embedded side and FPGA [49]. In view of the above situation, channel pruning is used to further compress the YOLOv4-MobileNetv3 network model. Compared to pruning methods that remove individual neurons, channel pruning does not introduce sparsity into the initial CNN model architecture, thus requiring no special hardware or software to implement [50]. This makes channel pruning extremely versatile in network model pruning, applicable to almost all model inference platforms, and suitable for hardware acceleration [51]. In channel pruning, the training acceleration of the YOLOv4-MobileNetv3 network model is achieved by using batch normalization (BN) between adjacent convolutional layers. The normalization operation is shown in Equation (8). \[y=\frac{\gamma(x-\bar{x})}{\sqrt{\theta^{2}+\epsilon}}+\beta \tag{8}\]  
where \(\bar{x}\) represents the mean of the input feature \(x,\theta^{2}\) represents the variance, \(\beta\) represents the bias, and \(\gamma\) represents the trainable scale factor. During training, \(\gamma\) is used to distinguish between important and non-important channels of the YOLOv4-MobileNetv3 network model. After selecting the pruning rate, non-critical channels are pruned, as shown in Figure 6. Channel sparsity training is performed by using L1 regularization on \(\gamma\); sparsity training loss \(f(\gamma)\) is shown in Equations (9) and (10). \[f(\gamma)=L(\gamma)+\mu||\gamma||_{1} \tag{9}\] \[||\gamma||_{1}=|\gamma_{1}|+|\gamma_{2}|+|\gamma_{3}|+|\gamma_{4 }|+\cdots+|\gamma_{n}| \tag{10}\]  
The \(L(\gamma)\) in the first part of the equation represents the loss of YOLOv4-MobileNetv3; \(\mu\) represents the penalty factor; \(||\gamma||_{1}\) represents the \(L1\) norm of \(\gamma\). Equation (11) is obtained by expanding Equation (9) at \(\gamma^{*}\), where \(H\) is represented as a Hessian matrix, and since the trainable scale factors \(\gamma\) in the parameter are independent of each other, \(H\) can become a diagonal matrix, as shown in Equation (12). \[f(\gamma)=L(\gamma^{*})+0.5(\gamma-\gamma^{*})\times(\gamma-\gamma^{*})^{T}\times H \tag{11}\]  
Figure 6: Channel pruning. \[H= \begin{bmatrix}H_{(1,1)}&&\\ &\ddots&\\ &&H_{(n,n)}\end{bmatrix}  \tag{12}\]  
Then, the original formula \(f(\gamma)\) can be expanded into Equation (13). \[f(\gamma)=L(\gamma^{*})+\sum_{i=0}[0.5H_{(i,i)}(\gamma_{i}-\gamma_{i}^{*})^{2}+ \mu|\gamma_{i}|] \tag{13}\]  
Combined with the \(\gamma\) assumption of mutual independence, we can obtain Equation (14). \[f(\gamma_{i})=L(\gamma_{i})+0.5H_{(i,i)}(\gamma_{i}-\gamma_{i}^{*})^{2}+\mu| \gamma_{i}| \tag{14}\]  
Equation (15) can be obtained by deriving the above Equation. For the Sgn function, if the input is \(\omega_{i}^{*}\) than 0, the Sgn function will return 1; if it is equal to 0, the Sgn function will return 0; if it is less than 0, the Sgn function will return \(-1\). \[H_{(i,i)}(\omega_{i}-\omega_{i}^{*})+\mu\times Sgn(\omega_{i}^{*})=0 \tag{15}\]  
In summary, it can be obtained \(\gamma_{i}\), as shown in Equation (16). \[\gamma_{i}=\left\{ \begin{array}{l}Sgn\big{(}\gamma_{i}^{*}\big{)}(|\gamma_{ i}^{*}-\frac{\mu}{H_{(i)}}),|\gamma_{i}^{*}|>\frac{\mu}{H_{(i,i)}}\\ 0,|\gamma_{i}^{*}|\leq\frac{\mu}{H_{(i)}}\end{array} \right. \tag{16}\]  
The scale factor of each BN layer of the YOLOv4-MobileNetv3 network before and after pruning is shown in Figures 7 and 8. It can be seen that after the sparse training is completed, the \(\gamma\) coefficient is concentrated around 0, which means that the sparse training is saturated.