## 4 Deep Learning-Based Color Barcode Extraction  
For the data decoding process in the D2C system, it is necessary to extract only the color barcode area within the captured image on the monitor. In a previous study [20], the color barcode area, extracted from a differential image, resulted in inaccurate results due to the effect of ambient light and the image capture jitter problem. This problem is exacerbated by poor alignment between the display and the camera. Therefore, in this study, we propose a method for color barcode area extraction using YOLO, a real-time object detection model. Among the various YOLO versions, YOLOv4 was applied in this study for color barcode area extraction. As shown in Figure 4, the YOLOv4 model uses CSPDarknet53 as the backbone structure, while using the YOLOv3 model almost as is, and uses spatial pyramid pooling (SPP) and a path aggregation network (PAN) as the neck structure. In addition, to improve the existing YOLOv3 model, YOLOv4 uses techniques called Bag of Freebies and Bag of Specials. Bag of Freebies is used to improve learning accuracy by increasing training costs. A typical example of this is data augmentation, which uses methods such as Random Erase to add noise to some areas of an image, CutOut to remove some areas, MixUp to mix two images with different labels, and CutMix to cut and paste two images with labels. Bag of Specials is a method to increase prediction accuracy, and various methods are used to enhance specific features, such as expanding the receptive field, introducing an attention mechanism, strengthening the feature integration capability, and selecting the bounding box for the prediction results. The input for YOLOv4-based color barcode area extraction is the image captured from the display device that transmits the color barcode. The input image is converted to 512 \(\times\) 512 pixels, which is the input size for CSPDarknet53. Here, the color barcode area to be extracted in the converted image is significantly reduced. However, CSPDarkent53 used  
Figure 4: Architecture of YOLO-based color barcode detection. Figure 3: Transmit packet structure of CCB-OCC. as a backbone of YOLOv4 has a large receptive field and a large number of parameters, so it shows excellent detection performance even for small objects. In addition, we were able to successfully extract color barcodes of various sizes through the SPP block, which increases the receptive field without affecting the operating speed of the model, and through the PAN, which receives information from different backbone levels. This helps to robustly extract color barcode areas at various distances between the display and the camera. The ideal form of the received color barcode would be one in which the data-encoded primary and complementary colors are smoothly connected within the color barcode area. However, it is common for actual extracted color barcodes to appear as trapezoids rather than rectangles. This could be due to the shape of and the alignment angle between the display and the camera, causing a large distortion in the captured image. This happened in most capture results, and the distance between the display and the camera makes the distortion even greater, which also affects the size of the color barcode. The area extracted using YOLO contains ambient noise, in addition to the color barcode, causing performance degradation during synchronization and decoding. Therefore, a process to remove noise is required, but simply reducing the area extracted by YOLO is not effective because the packet area extracted using YOLO is not constant in consecutive frames. For effective noise removal, methods such as differential images and morphological operation are performed in the overall process shown in Figure 5.  
As shown in Figure 5, the display where packet information is transmitted through the color barcode area is captured, and the YOLO model is trained using data containing annotations of the color barcode area to extract the approximate area of the color barcode from the received display image.