## 6 CNN-Based Synchronization and Color Pattern Classification  
For synchronization in the DeepCCB-OCC system, a pilot symbol classification model is used, and only pilot symbols must be accurately identified from the input of several color barcode patterns. This model should be able to distinguish a pilot symbol from a data symbol and should also be able to recognize what kind of pilot symbol it is. For this, the patterns of R, G, and B pilot symbols are designated as individual classes, and various patterns appearing in data symbols are grouped into a single class and are trained. Therefore, the pilot symbol classification model is trained to classify a total of four classes. In DeepCCB-OCC, color barcodes appear in various patterns in the image received from the same transmitted signal due to synchronization mismatch and alignment problems between the transmitting and receiving devices. In a real-world CCB-OCC system, the received color barcode appears in four patterns for a specific symbol, as shown in Figure 7. As explained in the previous section, 11 classes of data (R, G, B, and 0-7) are encoded in color barcodes, including pilot symbols and data symbols. In addition, since images taken at five different angles were used, the total of the types of color barcodes collected is the number of image capture angles (5) times the number of color barcode patterns (4) times the number of types of encoded data (11); thus, there are 220 types. To build a data set for CNN model training, we collected 20 images for each color barcode type--a total of 4400. When the DeepCCB-OCC technique is actually used, the size and shape of the color barcode may appear in various forms depending on the various distances and angles between the display and the camera. To improve model generalization, data augmentation, which has become a standard component of deep neural network training, is used. Figure 8 shows the synthetic data used to train the DeepCCB-OCC model. As can be seen, synthetic data were generated using data augmentation techniques such as enlargement/reduction, rotation, position transformation, and vertical inversion. To support the diversity of the synthetic data, 80 synthetic color barcodes for each color barcode type were created, resulting in a dataset of 22,000. Of those 22,000 color barcode images, 6000 with R, G, and B pilot signals encoded were used to train the CNN model for pilot symbol classification. The remaining 16,000 color barcodes with 0-7 data symbols encoded were used to train the CNN model for data symbol classification. The structure of the CNN model is shown in Figure 9. Since the size of the color barcode images used in the pilot symbol classification model and the data symbol classification model were the same, regardless of the encoded information, the two structures were configured identically. As input for the model, an RGB three-channel image at 100 \(\times\) 900 was used, composed of three 2D convolution layers, two maxpo  
Figure 7: Four different patterns for received color barcode symbol 001 (R, G, B = 255, 191.5, 0). nected layer, and a dropout layer. Since the training data for each classification model were configured differently, each model was trained independently. As shown in the figure, the pilot symbol classification model has four classes, and the data symbol classification model has eight classes. When consecutive color barcode patterns of RR'GG'BB' are obtained using the CNN model for pilot symbol detection, the CNN model for data detection is used as much as the data symbol length of the packet, and the transmitted signals are sequentially decoded. The entire process for decoding data using DeepCCB-OCC is described in Algorithm 1.