# Edge-AI-Based Real-Time Automated License Plate Recognition System
## 2 Materials and Methods
### License Plate Frame Detection and License Plate Character Recognition
#### 2.4.3 License Plate Character Recognition through Proposed M-YOLOv4  
In deep learning, small and large objects are defined based on the proportions of an object in an image rather than on the size of the object in an image. To avoid any misjudgment of license plate characters or the inclusion of erroneous judgments in the recognition results, two approaches are applied. The first approach retrieves the license plate frame as an image and subsequent application of character recognition for the license plate; the second approach involves the identification of a license plate frame and direct implementation of license plate recognition for a full image. In the first approach, the cut license plate frame turns the characters in the license plate into a large object, which cannot be converged during training through YOLOv4. Therefore, the present study applied the second approach to select only characters within the license plate frame in the full image for recognition. With this strategy, only the characters within a license plate frame are selected  
Figure 4: Distance (red line) between center point of license plate frame and virtual judgment line (blue line).  
Figure 3: YOLOv4-Tiny model. The red rectangular represents several modules commonly used in YOLOv4.  
for identification and the proportions of the license plate characters in the image are small. The original YOLOv4 allows for the recognition of license plate characters. However, the characteristics of small objects are lost after the multilayer convolution of the backbone layer is performed. Therefore, in the system of the present study, the original YOLOv4 is modified to improve the recognition of license plate characters. As illustrated in Figure 5, to avoid losing the characteristics of small objects through multiple cross-stage-partial (CSP) layers during training, all Conv + BN + Leaky_relu (CBL) blocks are moved up to form connections with the CSP layer and the final CSP convolutional layer of the original YOLOv4 backbone layer is removed (Figure 5 gray background area). CBL is the smallest component in the YOLOv4 network structure. This modified YOLOv4 model is called M-YOLOv4, which performs well in the recognition of license plate characters and is executed as fast as YOLOv4.  
The proposed M-YOLOv4 (Figure 5) comprises several blocks (Figure 6). Conv refers to a convolution layer that serves to slide the convolution kernel over the input image to obtain a feature map. Bn refers to batch normalization, and Mish denotes the activation function. Compared with the commonly used ReLU, Mish has a smoother function and superior generalization ability. Leaky ReLU is an improved activation function that allows ReLU to solve the problem of a gradient disappearing when the input is negative. ADD is the addition of a tensor, and a dimension is not expanded after an addition. Concat is the splicing of a tensor, and a dimension is expanded after splicing. CBM is the smallest component in a YOLOv4 structure, and it comprises Conv, Bn, and Mish. CBL consists of Conv, BN, and Leaky ReLU. Res unit imitates a ResNet model architecture, which comprises CBM, CBM, and ADD, to enable the deeper construction of the model. CSPX imitates the CSPNet model structure, which comprises CBM, Res unit, and Concat.  
In addition, for the proposed M-YOLOv4 training process, Distance-based IoU-Non-Maximum Suppression (DIoU-NMS) is adopted. DIoU-NMS not only considers the IoU between two object frames but also the distance between the center points of two object frames. The original YOLOv4 produces numerous overlapping recognition frames. Because license plate characters do not overlap, the DIoU-NMS used in the present study is reduced to suppress the overlapping of multiple recognition frames.  
Figure 5: Proposed M-YOLOv4 model. (a*b means that the number of a is b).