## 3 Proposed Methodology
### License Plate Character Recognition  
We propose a deep learning (DL) framework to accomplish the license plate character classification task as DL is most recently been introduced in approaches in the area of artificial intelligence. Deep learning is powered by neural networks. Convolutional neural networks (ConvNets or CNNs) are a category of neural networks that have proven very effective in areas such as image recognition and classification. They are made up of neurons that have learnable weights and biases. Each neuron receives some inputs, performs a dot product and optionally follows it with a non-linearity. The whole network still expresses a single differentiable score function: from the raw image pixels on one end to class scores at the other. And they still have a loss function on the last (fully-connected) layer. Figure 11 represents the proposed structure for character recognition of multinational vehicles LPs. First, the segmented image is decomposed into red, green and blue channels and then it is passed through polarity matching module to process data uniformly in order to eliminate the impact of foreground polarity that varies in multinational VLPs. To hierarchically learn features, these separated channels images are fed to CNNs and get output vectors, which are further concatenated to acquire enhanced image's feature information. Before sending this output feature vector to the classifier we introduce data normalization module which has a better impact on output scores. Finally, this normalized feature vector is fed to classifier in order to obtain target labels. Figure 8: Required objects detection part based on character height estimation filter: (**a**) Angle detection, (**b**) Angle correction, (**c**) Border touched character separation, (**d**) Bounding boxes on required objects after removing a small object. Figure 10: Failed segmentation samples of license plate images. Figure 9: LP images with noise, blur, shadow and various illumination conditions. The pre-trained network is used as a starting point to learn a new task. Fine-tuning a network with transfer learning is usually much faster than training a network with randomly initialized weights from scratch. Learned features can be transferred quickly to a new task using a smaller number of training images. Some of the well-known pre-trained networks (AlexNet [37], VGG-16 [38], GoogleNet [39], ResNet-18 [40], Inception v3 [41]) have been trained over a million images and can classify images into 1000 object categories and have learned rich feature representations for a wide range of images. The speed is one of the most important parameters as we are working for real-time applications. By keeping this constraint in mind, we choose AlexNet CNN as our base network for transfer learning as it is time efficient among the rest of the four well known CNNs, as shown in Figure 12. The test time is considered for whole test database to test the speed of networks for our particular application. The test database for recognition part contains 21717 license plate characters images that are extracted at segmentation stage from 3718 license plate images of eight different countries. In deep neural networks, higher-level convolutional layers have rich features representation while the spatial information is deficient. In contrast, in shallow layers, spatial information is reserved at the cost of less expressive features. Generally, features from the last convolutional layer are used by FC layers which are further used for classification. Max-pooling is a down sampling strategy in convolutional neural networks. Therefore, we may have a chance to get spatial information that might be lost during down sampling. Vanishing gradient is another problem that may occur in deep networks. For recognition and detection tasks, performance can be enhanced and information loss can also be reduced by using collective feature information of different convolutional layers, as the authors suggested in [42; 43; 44; 45].