# Versatile Video Coding-Post Processing Feature Fusion:
## 3 Proposed Method
### Insight of Data Ingestion  
Figure 2 shows the step-by-step process for training and testing by transforming MP4 videos into YUV format and subsequently reconstructing the original video using the open source-based VVC encoder (VVenC) [38] and decoder (VVdeC) [39]. Our experiment is based on the BVI-DVC [40] dataset, which is stored in an MP4 format. While we recognize that using the original non-compressed YUV format is common practice in video coding experiments, our research focused on a scenario where MP4-encoded videos are more representative of the conditions in real-world applications. BVI-DVC [40] is currently utilized by the Joint Video Exploration Team (JVET) standardization organization for developing neural network-based video coding technology. The key benefit of YUV is that it decouples luminance (i.e., brightness) from chrominance (i.e., color), which is not possible with RGB. This decoupling enables more effective compression, as the human visual system is more sensitive to brightness variations than color changes. By reducing the resolution of YUV color components, the amount of data required to represent color information is lowered, resulting in a more efficient use of bandwidth for data transmission. The widespread use of YUV format in real-world video data transmission makes it an ideal choice for training datasets, ensuring better compatibility with practical applications. Moreover, the Joint Video Exploration Team (JVET) neural network-based video coding (NNVC) common test condition (CTC) [41] test sequences, a benchmark for video coding network development, are also in YUV format, making it a sensible decision to convert training data to YUV for more effective testing and analysis. The preprocessing stage, depicted in Figure 3, involves converting (a) original videos into their constituent frames, known as original images, using FFmpeg (version 6.1.1) [42], and (b) applying the same process to reconstructed videos to obtain reconstructed images, which are then input into the neural network for subsequent analysis. Deep learning models, typically designed for 2D image processing, can be adapted for video analysis by breaking down videos into individual frames. This approach unlocks several advantages, including the creation of a larger training dataset from the numerous frames, stream  
Figure 1: Enhancing video quality with CNN based post-processing in conventional VVC coding workflow. Figure 2: MP4 to YUV conversion and reconstruction using VVenC and VVdeC. lined preprocessing tasks like data augmentation and feature extraction, and improved computational efficiency, leading to accelerated training and inference speeds. Figure 4 describes the process of up-sampling the YUV 4:2:0 format data to convert it into the YUV 4:4:4 format, which involves interpolating and expanding the chroma components to match the resolution of the luma component. Most image processing neural networks are designed to process input data with identical resolutions for all color channels. However, YUV 4:2:0 data fall short of this requirement, which can lead to issues during training and inference. To avoid these problems, up-sampling YUV 4:2:0 to YUV 4:4:4 ensures that the input data meet the neural network expectations, resulting in smoother and more accurate processing. The chroma subsampling in YUV 4:2:0 data leads to a lower color resolution, as the UV channels are downscaled relative to the Y channel. This reduction in color fidelity can degrade the performance of neural networks. Conversely, YUV 4:4:4 data retain the full resolution for all components, thereby preserving color accuracy and facilitating optimal neural network outcomes. Figure 5 illustrates the process of reducing the resolution of the chroma components in the neural network output, transforming it from the YUV 4:4:4 format to the YUV  
Figure 4: Illustration of the conversion process from YUV 4:2:0 format to YUV 4:4:4 format before feeding data into the deep learning network.