# A Novel Video Transmission Latency Measurement Method for Intelligent Cloud Computing
## 2 Analysis of Video Transmission Latency
### Video Codec Latency  
Video coding standards have evolved over many generations. Codecs are also becoming more complex in order to deliver higher quality video with limited bandwidth. The codec latency is composed of encoding frame reordering latency, encoding processing latency, encoding buffering latency, decoding processing latency, and decoding frame reordering latency. The video to be encoded is fed into the encoder in chronological order. Video coding prediction includes intra-prediction, inter-prediction, and bidirectional prediction. Intra-prediction exploits the correlation between neighboring pixels by using the reconstructed pixels within the frame to derive predicted values through extrapolation from already coded pixels. Inter-prediction is the use of block-based motion compensation to generate a prediction model from one or more previously encoded image frames. Bidirectional prediction allows an encoder to obtain information from frames that are forward in the video stream, instead of just previously compressed frames. In typical applications, intra-prediction, inter-prediction, and bidirectional prediction are mixed to perform frame level prediction to achieve a balance between latency and quality. This will cause the order of video entering the encoder to be inconsistent with the actual encoding and decoding order, and some frames need to wait for encoding and decoding, resulting in frame reordering latency. Figure 2 describes the reference relationship of the RA (Radom Access) mode of HEVC. In this case, a GOP contains eight video frames and the picture number (Picture Order Count, POC) is 0-7. Due to the predictive reference relationship, the playback order of images is inconsistent with the encoding order, so the encoding order of POC 1 frame is behind POC 4 and 2 frame. Therefore, a frame reordering latency of three frames will be introduced in the encoding stage. In a scene of 50 frames/s, the latency of 3 frames is 60 ms. Likewise, the frame cannot be decoded until those two frames are decoded. The overall DOC (decoding order count) is shown in Figure 2. This part of the latency depends on the data arrival speed and decoding speed because the encoding of the current frame needs to refer to the data of the encoded frame. The GOP structure can be dynamic. The theoretical maximum latency caused by reordering can reach the size of the GOP. In order to meet low-latency applications, HEVC has introduced the LDP (low-latency P) mode, in which bidirectional prediction frames are canceled. Each incoming frame can be encoded immediately without waiting for other reference frames. Therefore, no reordering delay is introduced. Using this mode reduces the reordering latency, but the encoding performance decreases by 9-42% [31]. Figure 1: Video transmission delay distribution. The squares 1–3 marked in the figure are the video frames transmitted. In the encoding process, a large number of redundant searches need to be performed in the current frame and the reference frame to determine the most efficient encoding mode. At the same time, in order to adapt to the local characteristics of the video frame content, the to-be-encoded image should be divided into a series of encoding blocks. Fixed size macroblocks are used in AVC. HEVC uses a quad-tree division method. In addition to the quad-tree division method, VVC also introduces multi-type tree division. By increasing the division method, the coding efficiency of image details is enhanced, and the corresponding division and search computational complexity are also significantly increased. Compared with HEVC, VVC increases the coding complexity by a factor of about 7. The increase in computational complexity extends the encoding time. In actual implementation, parallel operations can be used to optimize the encoding structure to reduce encoding time, but parallelism is essentially the scheduling optimization of encoding and does not reduce the processing volume of encoding and decoding itself.