# Time Delay Optimization of Compressing Shipborne Vision Sensor Video Based on Deep Learning
## 4 Time Delay Optimization of Compressing SVSV Based on Deep Learning
### Video Intra-Frame Coding Delay Optimization Algorithm Flow of Shipboard Vision Sensor  
The specific flow of the intra-frame coding delay optimization algorithm for SVSV is shown in Figure 12. It can be seen that the algorithm is mainly used to avoid the rate distortion cost calculation and comparison performed when deciding the CTU division structure in HEVC by directly predicting the CU division result of the shipboard vision sensor video image. Meanwhile, a threshold termination mechanism in the model is added, that is, when the predicted value is larger than the threshold, the model stops making CU division structure prediction and outputs the current CU hierarchical division structure prediction result to avoid the waste of computational resources to a certain extent and its specific workflow is shown as follows.  
1. Input the shipboard vision sensor video signal to be compressed into the encoder, and pre-process it to YUV format if the color space of the vision sensor video is RGB. Before formally starting encoding, the encoder will split each frame of the vision sensor video into N CTUs to be encoded.
2. The Y-component of CTU is fed into the trained prediction model with the network structure shown in Figure 11, and the pixel matrix is normalized to speed up the convergence. The model output is the probability of D(x,i,j), i.e., the corresponding dichotomous labels 0, 1.
3. The probability size of the model output D(0) is compared with the set threshold (this paper the threshold is set to 0.5) to determine whether the CU in the current CTU continues to be divided downward. If the probability of judging D(0) is less than the set threshold, it is directly determined that the current CTU is not divided, i.e., D(0) = 0 and output. Otherwise, D(0) = 1 continues to divide downward.  
4. Determine whether the probability size of D(1,i) is greater than the set threshold, if yes then D(1,i) = 1, the current sub-CU continues to divide downward. Otherwise, D(1,i) = 0, the current sub-CU is recorded, and the downward division is stopped. If the prediction results of all the current 4 sub-CUs are less than the threshold, i.e., \(\left\{\text{D}(x,i)\right\}_{i=1}^{4}=0\), the division structure of the current CTU is determined directly and output.
5. Similar to Step (4), determine whether the probability size of D(1,i,j) is greater than the set threshold, if yes, then D(1,i,j) = 1, the current sub-CU continues to divide downward. Otherwise, D(1,i,j) = 0, the current sub-CU is recorded, and the downward division is stopped. If the prediction results of all 16 current sub-CUs are less than the threshold value, i.e., \(\left\{\text{D}(x,i,j)\right\}_{i,j=1}^{4}=0\), the division structure of the current CTU is determined directly and output.
6. Integrate all recorded sub-CU division structures, corresponding to the CU hierarchical division structure model, the final division structure of the current CTU is determined and output.
7. In the process of compressing the SVSV, the process of calculating and comparing the cost of rate distortion is avoided by directly predicting the CU division structure of the shipboard vision sensor video image. For example, when the prediction result is \(\left\{\text{D}(x,i)\right\}_{i=1}^{4}=0\), the network model terminates the prediction of the division structure of 16 \(\times\) 16 size CUs and directly outputs the CTU division structure model containing four 32 \(\times\) 32 size CUs, which reduces the compression delay to a certain extent.  
Figure 12: Flow chart of intra coding time-delay optimization algorithm for shipborne vision sensor video.