3983

Sensors and Materials, Vol. 33, No. 11 (2021) 3983–3990
MYU Tokyo
S & M 2739

Recognition of Vehicle License Plate
Based on Hopfield Artificial Neural Network
Tian-Syung Lan,1,2* Jiawei Li,1 Xuan-Jun Dai,1** Ho-Sheng Chen,3 and Ruiming Liu4
1College of Mechanical and Control Engineering, Guilin University of Technology, Guangxi 541004, China
2Department of Information Management, Yu Da University of Science and Technology,

Miaoli County 36143, Taiwan

3College of Mechanical and Electrical Engineering, Guangdong University of Petrochemical Technology,Maoming Guangdong 525000, China

4Jiangsu Ocean University, Jiangsu Province 222005, China(Received July 1, 2021; accepted October 7, 2021)

Keywords: Hopfield, general license plate number recognition, image recognition

Vehicle license plate recognition has become widely used to control and efficiently managetransportation systems. The increasing number of vehicles and worsening congestion on the roadmake the accuracy of the traditional recognition system low. Thus, introducing a neural networkin license plate recognition has been paid considerable attention. In this paper, we aim toenhance recognition accuracy by using the Hopfield artificial neural network (ANN). Test
results showed that the recognition method based on the Hopfield ANN is found to be efficientand accurate.

1.

Introduction

Artificial intelligence (AI) neural network is an emerging technology as it has great potential
and adaptability in various fields of industry.(1) One of its applications is image processing that isused in daily lives. The license plate recognition of vehicles adopts image processing
technology.(2) For license plate recognition, several steps in image processing, such asgrayscaling, binarization, and edge detection, are required.
Grayscaling converts an original image into a gray-scaled image for a more accurate andfaster recognition of characters on license plates than the original method.(3) There are several
methods of grayscaling: maximum value method, average method, and weighted averagemethod.(4) Among these methods, the weighted average method is regarded as the mostappropriate. Image binarization is part of image segmentation.(5) As the segmentation of imagesby a threshold value has advantages over edge segmentation in terms of calculation andefficiency, threshold segmentation is used in general. There are several methods of thresholdsegmentation, such as the iterative method, maximum inter-class error method, and localsegmentation method.(6) Among them, the maximum inter-class error method is the mostpractical as it adaptively determines threshold values.
*Corresponding author: e-mail: tslan888@yahoo.com.tw**Corresponding author: e-mail: daixuanjun@163.com

https://doi.org/10.18494/SAM.2021.3518

ISSN 0914-4935 © MYU K.K.
https://myukk.org/

3984

Sensors and Materials, Vol. 33, No. 11 (2021)

Edge detection reflects the main feature and characteristics of an image.(7) Generally,significant changes in brightness are important in processing an image(8) and thus are used todetect the edge of the image to extract its subsequent features. Edge detection requires
appropriate algorithms and operators, such as Sobel, Robert, Prewitt, Log, Laplacian, and Cannyoperators.
For processing images effectively through the above-mentioned method, an artificial neuralnetwork (ANN) is widely used. Among recently developed ANNs, dynamic feedback neuralnetworks are mostly adopted, especially the Hopfield neural network as it is a nonlinear dynamic
feedback neural network and is known as the most effective so far in processing images. It has
stable performance and reaches a stable state regardless of the initial state by adjusting weights
to maintain its stability. The Hopfield neural network enhances the accuracy of recognizing
license plates regardless of road conditions and weather. It also processes images taken even bycomplementary metal-oxide-semiconductor (CMOS) camera modules. A CMOS sensor converts
photons to electrons for digital processing and is widely used in digital imaging devices. It ismainly used for low-price devices such as cameras for closed-circuit TVs and digital video
cameras. For higher quality images, charge-coupled devices (CCD) are used instead.
In this study, we aim to provide an effective method of recognizing license plates of vehicles
even for low-resolution images from CMOS cameras. By developing an appropriate algorithm,
we suggest a quick and easy method of recognizing license plates effectively.

2.

Methodology

The main idea of the proposed method is to distinguish target parts from their background in
an image according to a threshold value. For example, after a certain value for a gray pixel isassigned as a threshold, the pixel with the largest difference from the gray pixel is defined as thethreshold again. Then, in the cluster that contains the pixel, the inter-cluster difference between
the target part and its background is calculated. Repeating the process makes differences invalues large and errors small. By doing this, the inter-cluster difference in grayscale is calculated
to determine the threshold values with the largest difference.
2.1

Grayscaling images

We used the weighted average method in grayscaling. The method assigns the values of red(R), green (G), and black (B) colors in a gray image to the R, G, and B colors of an original
image before processing. This method is flexible as it adjusts the output of the gray image byusing the weights of colors as follows:R ===G B ( ϕ R R + ϕG G + ϕ B B ) / 3 ,(1)where φR, φG, and φB are the relative weights of R, G, and B, respectively. Adjusting the weights
outputs the transformed gray image. Since the eyes are most sensitive to the green color and leastsensitive to blue, the weight of G is the largest, whereas that of B is the smallest. An example ofan output image processed by the maximum inter-class error method is shown in Fig. 1.

Sensors and Materials, Vol. 33, No. 11 (2021)

Fig. 1.

2.2

3985

Binary image output by maximum inter-class error method.

Image binarization

Image binarization segments an image by a threshold for fast calculation and high efficiency.
Among several methods, we used the maximum inter-class error method for image binarization.
The maximum between-class error method is practical as it is adaptive to determine thethreshold value. The main idea of this method is to distinguish between what is necessary andnot based on the characteristics of the threshold value itself. Then, the difference is calculated
between the target and the background. A large difference means that the values of thebackground and target in the image are large with a small error. Thus, how to calculate thedifference efficiently is important in image binarization.
2.3

Image edge detectionA log operator is required to detect the edge of an image. The log operator is generated by
combining Laplacian and Gaussian to suppress noise, which sometimes disturbs several edges
instead.
2.4Hopfield neural network

The Hopfield ANN is trained to reduce the noise of information in images by creating aseries of digital dot matrices and creating a neural network in MATLAB. MATLAB provides
functions for the Hopfield ANN. newhop() function creates a discrete neural network with thecall format of net = newhop(T), where T is a matrix of R × Q (Q is a target vector with elementsof −1 or 1). As it is not easy to recognize characters on weathered license plates, it is necessary totrain the neural network with unchanged points in T. As the Hopfield network has one layer, its
activation function needs to be satlins() function that generates the digital dot matrix with noise.
Among several methods for simulating the generation of the digital matrix with noise, we use therandom noise generation method. When the digital dot matrix with noise is input into theHopfield neural network, the output of the network becomes the closest target vector to thematrix to realize the associative memory function. Then, the test results are analyzed, compared,
and used to verify the Hopfield network, as shown in Fig. 2.
The network state and operating mode of the discrete Hopfield network are defined asfollows.

3986

Sensors and Materials, Vol. 33, No. 11 (2021)(a)Fig. 2.

(b)(Color online) (a) Original image and (b) processed image with log operator.

Network statex j (t + 1) =f (net j ) ,(1)where f(∙) is the transition function and netj is the net input of the neuron j.
The symbolic linear function is defined as
1,x j=(t + 1) sgn(=net j ) −1,net j ≥ 0,
net j < 0.

(2)

The symmetric saturated linear function is defined as
1,
x=(net j ) net j ,=j (t + 1) satlins
−1,

net j < −1,− 1 ≤ net j ≤ 1,(3)net j > 1.

Operating mode
According to the network state, the first weight adjustment is determined as
 net j (t )  , j 1, 2,..., n ,=x j (t + 1) sgn =(4) net j (t )  , j 1, 2,..., n.
=
x j (t + 1) satlins=(5)

In the process, the last weight adjustment determines the weight adjustment that determines
the next weight adjustment. When the adjustment no longer changes the weight, the system
becomes stable. The next weight adjustment is carried out using the following equations:sgn  net j (t )  ,x j (t + 1) = x j (t ),j = i,j ≠ i,(6)Sensors and Materials, Vol. 33, No. 11 (2021)

 satlins  net j (t )  ,x j (t + 1) = x j (t ),2.53987

j = i,j ≠ i.

(7)

Vertical projection method

We use the vertical projection method because it is known to be appropriate for the character
segmentation of the license plate and for images with a left–right layout. The purpose of verticalprojection is to obtain the left and right edge positions of the license plate image. The verticalprojection counts the columns in the vertical direction of an image and the number of white
pixels in each column. By comparing with a predetermined threshold value, T, a row larger than
the threshold value is detected and defined as a valid row.

3.

Results

The first character of the license plate in China is a Chinese character for each province.
Thus, the database for network training is different from training alphabets and numbers. The
second character in the alphabet of the license plate represents a city in a province. Then,
numbers and letters are mixed into five characters.
3.1

License plate location

A license plate is located by the following steps: The first step is to grayscale an image,followed by edge detection. Then, the image is divided into several regions by image
segmentation. Finally, the license plate is located according to its characteristics. Next, image
processing is performed to obtain the edge image using the Hopfield neural network for showing
the parts for the license plate in the image. Figure 3 shows the image with the first edge
detection.
The figure still needs to fill small blanks and sharpen boundaries in the edges of the parts in
the image by open operation, which allows for the extraction of the image of the license plate.

(a)

(b)

Fig. 3. (Color online) Edge detection result of license plate. (a) Original image of vehicle. (b) Image with closedoperation.

3988

Sensors and Materials, Vol. 33, No. 11 (2021)

The operation for the corrosion and then expansion of an image is called opening operation. It
eliminates small objects, separates objects at delicate places, and smoothens the boundaries oflarger objects. Closed operation processes an image first with expansion and then with corrosion.
It fills small cavities in objects, connects adjacent objects, and smoothens their boundaries.
The result is shown in Fig. 3. The edges become clearer by removing the areas that areinconsistent with the license plate. After the opening operation, the image is denoised andexpanded. Finally, the license plate is located in the processed image (Fig. 4).
3.2

Character segmentation of license plate

For character segmentation, the original image is transformed into a binary image that isprojected vertically. After obtaining the histogram of the vertical projection of the license plate,the minimum values of the parts in the image are calculated. The minimum values are combinedwith the width and distance of the characters on the license plate to segment the characters of thelicense plate (Fig. 4). On the basis of the vertical projection, the spacing and characters on thelicense plate are segmented clearly for subsequent character recognition.
Character recognition is carried out by grayscaling an image first. Figure 5 shows the resultof grayscaling. By using the Rgb2gray code in MATLAB and the weighted average method, theweight of the red color is adjusted to be greater than those of the green and blue colors. The
processed gray image is small in size and easy to store for effective subsequent image
recognition.

(a)

(b)

Fig. 4. (Color online) Character segmentation of license plate. (a) Original image of license plate. (b) Segmentedcharacters.

(a)Fig. 5.

(b)(Color online) Results of grayscaling the image. (a) Original image. (b) Grayscaled image .

Sensors and Materials, Vol. 33, No. 11 (2021)

3989

Then, edge detection is applied to locate the license plate. The morphological processing
filters the candidate regions of the license plate on the basis of the features of the license plate.
The following results show that the positioning accuracy is still relatively high. First, edgedetection is carried out to locate the license plate clearly (Figs. 6 and 7).
After locating the license plate, it is necessary to segment the characters of the license plate.
The segmentation method is the vertical projection method mentioned above. This method
segments the characters of the license plate quickly and effectively. Figure 8 shows the result of
the character segmentation of the license plate.
The result shows that the accuracy of the recognition is acceptable but not complete. In thiscase, tilting needs to be added as an additional step for license plate recognition. Thus, we adopt
a tilt correction method. In recognizing the characters of license plates, tilted images may betaken because of camera shooting angles when vehicles turn and drive on slopes. Thesesituations lead to tilted images of the license plate, which hinders accurate image processing.
Therefore, it is necessary to correct the tilted images for license plate recognition. There areseveral tilt correction methods, such as Hough transform, Radon transform, least square method,and two-point method.
Radon transform is a projection method that projects the image of a tilted target within therange of different tilt angles of 0‒179°, and then finds the projection angle for the best result.
Then, the projection angle is recorded as the angle of inclination. The image corrected by themethod is shown in Fig. 9. Although the correction result does not transform the tilted image into
a perfect rectangle, the corrected image has sufficient quality for subsequent image recognition.

Fig. 6.

Edge detection of vehicle.

(a)Fig. 8.

Fig. 7.

(Color online) License plate location.

(b)(a) Character segmentation and (b) character identification by vertical projection method.

3990

Sensors and Materials, Vol. 33, No. 11 (2021)

(a)

(b)

Fig. 9. (Color online) Correction of tilted image for license plate recognition. (a) Original image. (b) Correctedpicture.

4.

Discussion and Conclusions

Low accuracy and efficiency have been problems in license plate recognition. To solve these
problems, we adopt the Hopfield neural network. By using the Hopfield neural network, Chinesecharacters and mixed letters of alphabets and numbers are recognized effectively. Chinesecharacters are more complicated than the English alphabet and difficult to recognize. The
background colors of license plates in China are white, black, green, and yellow, and two colors
are mixed in general. Grayscaling in this study is performed by using the weighted averagemethod for weight distribution. The proposed method also recognizes characters in tilted
images. The result shows that the proposed method allows accurate license plate recognition.
The license plate in China is difficult to recognize as it has Chinese characters, numbers, andalphabets that code for provinces, cities, and registration, respectively. Training an algorithm forrecognition requires several databases, including Chinese, English, and digital databases. By
using the Hopfield neural network, we tested the proposed method for license plates and obtained
a higher accuracy than previous methods. Despite that, the algorithm in this study needs
improvement for recognizing characters on old and worn license plates, characters masked byrain or snow, and characters in low-quality images due to the wrong location of cameras.
Enhancing the recognition accuracy for various types of license plates should also be studied.

References
1 H. Salehinejad, S. Sankar, J. Barfett, E. Colak, and S. Valaee: https://arxiv.org/abs/1801.0107 (accessedSeptember 2021).
2 R. K. Varma, P. S. Ganta, H. Krishna B, and P. Svsrk: Procedia Comput. Sci. 167 (2020) 2623. https://doi.
org/10.1016/j.procs.2020.03.324
3 Z. Liu, Y. Cai, L. Chen, H. Wang, and Y. He: Proc. Inst. Mech. Eng. D: J. Auto. Eng. 233 (2019) 2284. https://doi.org/10.1177/0954407019851339
4 J. Tan, Y. Tan, Y. Guo, and J. Feng: Physica A 545 (2020) 123782. https://doi.org/10.1016/j.physa.2019.123782
5 T. Björklund, A. Fiandrotti, M. Annarumma, G. Francini, and E. Magli: Pattern Recognit. 93 (2019) 134.
https://doi.org/10.1016/j.patcog.2019.04.007
6 R. Islam, M. R. Islam, and K. H. Talukder: Multimed. Tools Appl. 79 (2020) 20107. https://doi.org/10.1007/
s11042-020-08629-8
7 S. G. Patel: Int. J. Cybern. Inf. 2 (2013). 1. https://doi.org/10.5121/ijci.2013.2101
8 Y. Pan, L. D. Zhang, L. Duan, and F. Duan: Complex Syst. Complex. Sci. 15 (2018) 75. https://doi.org/10.13306/j.1672-3813.2018.01.011