# Object Detection-Based Video Compression
## 2 Related Works  
As a result of the evolution of IoT, object detection, and AI technologies, MPEG is attempting to create a standard through VCM by merging those technologies with video coding technology. Therefore, VCM architectures are being conceived through collaboration of these technologies. In addition, experiments are conducted through calls for evidence (CfEs) within VCM, and efforts are being made to verify the effectiveness of the technology. In VCM, a CfE was conducted in October 2020, and a draft call for proposal (Cfp) was published in January 2022. It is expected that VCM will proceed in the form of adding new encoding technologies based on the adopted proposal.  
VCM intends to provide services to technologies such as surveillance video, intelligent transportation, smart cities, intelligent industries, and intelligent content. Some common requirements for these services are as follows [19; 21]:  
* Efficient compression of bitstreams: It should have a higher compression rate than VVC-compressed bitstreams with similar performance.
* Varying degrees of performance should be supported: The goal is to support different optimizations for scenarios supporting single and multiple missions.
* Both machine-only and hybrid machine and human applications should be supported.  
Several VCM pipelines are shown as examples. In Figure 2, (a) shows the pipeline for video coding, (b) shows the pipeline for feature coding, and (c) shows the pipeline for hybrid coding. Figure 2a shows video encoding, video decoding, and then a machine analysis process performed after the video decoding. Thus, no additional procedure is required in the video encoder and decoder to support numerous machine-vision tasks because the reconstructed video after the video decoding is used as input for the machine analysis. Figure 2b shows a method to compress both the traditional video plus feature information after the feature conversion using the video encoder, where features are obtained from machine analysis. Since the video codec is a compression method optimized for the characteristics of image data, it is necessary to consider whether it is an efficient method. In Figure 2c, feature maps after machine analysis on the encoder side are compressed by the feature encoder, and the video encoder and decoder do not utilize the feature information in the encoding and decoding processes in order to use the existing video coding standard, in which the machine analysis in the decoder infers object detection, object segmentation, object tracking, event recognition, etc., from the decoded feature maps.  
Along with VCM, a large number of related studies are in progress. For example, [PERSON] et al. [23] mentioned the need for targeting collaborative optimization of compressing and transmitting multiple tasks in VCM. Currently, video compression is targeted only for human vision, rather than machine vision. Therefore, joint optimization of the VVC and feature streams is necessary. The relationship between human vision and machine vision also requires study. [PERSON] et al. [24] proposed a method combining human vision and machine vision. As the video coding achieves improved performance using the RDO process, [PERSON] et al. designed a feature-based RDO to achieve better compression performance from the machine point of view. [PERSON] et al. [25] proposed a key module that shows the combination of a pretrained model, learned feature extractor, neural network, and multiple tasks. In VCM, multiple tasks need to be applied in a variety of technologies, such as surveillance video, intelligent transportation, smart cities, intelligent industries, and intelligent content. In addition to collaboration between human vision and machine vision, the complexity of encoder and decoder is a significant issue that should not be overlooked, as compared with the VVC encoder and decoder.