Hybrid strategy for traffic light detection by combining classical and self‐learning detectors

IET Intelligent Transport SystemsVolume 14, Issue 7 p. 735-741 Research ArticleFree Access Hybrid strategy for traffic light detection by combining classical and self-learning detectors Feng Gao, Corresponding Author Feng Gao gaofeng1@cqu.edu.cn School of Automotive Engineering, Chongqing University, No. 174, Shazheng Street, Chongqing, People's Republic of ChinaSearch for more papers by this authorCaimei Wang, Caimei Wang School of Automotive Engineering, Chongqing University, No. 174, Shazheng Street, Chongqing, People's Republic of ChinaSearch for more papers by this author Feng Gao, Corresponding Author Feng Gao gaofeng1@cqu.edu.cn School of Automotive Engineering, Chongqing University, No. 174, Shazheng Street, Chongqing, People's Republic of ChinaSearch for more papers by this authorCaimei Wang, Caimei Wang School of Automotive Engineering, Chongqing University, No. 174, Shazheng Street, Chongqing, People's Republic of ChinaSearch for more papers by this author First published: 22 April 2020 https://doi.org/10.1049/iet-its.2019.0782Citations: 7AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Detection of the traffic light is a key function of the automatic driving system for urban traffic. Considering the characteristics of classical and self-learning algorithms, a fusion logic is proposed to make up the shortcoming of learning algorithms by combining the known knowledge with the learning features to detect the red and yellow–green traffic light without turn indicator. The relationship of detection performance among different detectors is established analytically. Then the improvement of detection performance by fusion is analysed theoretically and optimised numerically. According to the analysis results, the hybrid detector is designed by using the colour information in hue-saturation-intensity to extract the candidate region, the hog feature to identify the shape information of traffic light classified by a support vector machine, and a comparatively simple convolutional neural network (CNN) with the classical AlexNet structure to act as the self-learned detector. The effectiveness of the hybrid method is validated by several comparative tests with single CNN detectors and other fusion methods on the training dataset, and the extensibility to new application conditions is evaluated by vehicle tests. 1 Introduction Traffic light plays an important role in driving for both human and automatic driving [1]. The data from the Insurance Institute of Highway Safety shows that more than 890 people were killed by the crashes related to running red lights in 2017 [2]. Timely detection of traffic light state accurately is very important for automated vehicles [3]. Normally the detection process is firstly to find the candidate region, then in which the traffic light is detected using image processing algorithms [4]. The algorithm at the first step should be concise enough to focus on the interested area fast and accordingly reduces the difficulty of detection procedure effectively. Since the colour characteristic is very prominent, it is always used to extract the candidate region according to different colour spaces, such as red–green–blue (RGB) [5] and hue-saturation-value (HSV) [6]. Compared with RGB, HSV is more robust to environmental factors, such as illumination, shadow, and so on [7]. To reduce the influence of colour change, Ji et al. combined A-channel and S-channel according to the visual selective attention mechanism [8], but the frequent change of colour space increases its computational cost [9]. Within the candidate regions, the features known by humans were selected to detect the traffic light early including shape [5], colour [6], structure [10], and area and aspect ratio [11]. To overcome the limitations of human knowledge, some intelligent classifiers were proposed thereafter. Binangkit and Widyantoro trained a classifier using HSV colour [12]. Only using colour information makes it sensitive to other similar objects, e.g. braking light etc. Furthermore, some general features, such as Haar and histogram, were trained by support vector machine (SVM) in [13, 14], respectively. It is a trend to combine more information to train the classifier, such as light intensity, gradient, and so on, to increase the accuracy [15]. In [16], the detection precision and recall rate of the trained detector reaches 65.2 and 50%, respectively, using the public traffic lights dataset, LISA. Moreover, such artificial intelligent algorithms have also been applied to find the candidate region more accurately [17]. Though the detection performance of traffic light can be improved by combining multiple features, the comprehensiveness and correctness of the artificially selected features and their parameter values are greatly limited by the experience of the engineer. Moreover, compared with micro-features, the detection of such macro-features is easily influenced by a complex background. In recent years, with the rapid development of computing power and new algorithms, deep learning has attracted much attention and been applied to object detection widely because of its superiority of automatic micro-feature extraction [18-21]. Kim et al. used a deep network to found the candidate region [22] and Wang et al. designed a classifier using a convolutional neural network (CNN) to detect traffic light. To extend the applicability to deformed light, Weber et al. designed a deep CNN, which contains the correlation of pixels and realises the detection with only local information [23]. Considering the fact that it is very complex and time-consuming to build a completely new network, some popular models, such as faster-rcnn [20], Inception-v2 [24] etc., are also adopted to detect traffic light [25]. On a privacy dataset, the precision and recall rate of traffic lights detector based on CaffeNet reaches 96.8 and 96.9%, respectively [26]. In addition, they can detect other objects simultaneously, such as car, pedestrian etc. The aforementioned detection models show good performance when the object occupies enough area in the image. In practice, traffic light becomes very small when it is far away. To overcome this challenge, an improved single shot multi-box detector was proposed by replacing VGG-16 with Inception-v3 [27]. The minimum width of the light image required to accomplish detection is only 10 pixels, which was further reduced to 4 by combining the Yolo framework with a small CNN in [28]. To combine the advantages of different methods, fusion has become a trend in recent years. In [6], the classical method and CNN are fused to detect traffic lights. More detectors including a multi-size detector, a blub detector, and a classical detector are combined in [29]. However, the theoretical performance, the interrelations among detectors and technical guidance for the design of fusion logic are little investigated. Deep learning algorithms can automatically extract micro-features more comprehensively based on the dataset and the interference of other similar objects can be attenuated sufficiently. However, the unexplained attribute of the learned network causes it to be a black box to the user and leads to the following disadvantages: (i) The detection performance greatly depends on the dataset used to learn the network, while there is no effective way to evaluate its comprehensiveness especially for automatic driving. (ii) The learned features are unexplained, which may be the common attribute of all positive data but not the detected object. This causes the performance of deep learning to be poor when it is applied to the condition out of the dataset. (iii) The network is a black box to the user and it is still not known how to modify the network parameters directly to adapt to new application conditions. It is hard to meet the demand for the vehicle development cycle by relearning a new network. Compared with the disadvantages of deep learning mentioned above, the traditional method uses the known features to detect and have explicit physical meaning. This makes it become a beneficial supplement to deep learning in such aspects as interpretability, scalability, and expansibility. To make full use of the advantages of both classical and deep learning methods, a hybrid detection method of the traffic light is proposed by combining known knowledge with learning features in this study. The relationship of detection performance among different detectors and the improvement of detection accuracy is analysed theoretically. Under the guidance of theoretical results, the colour information in HSV space is used to extract the candidate region, the hog feature is adopted to identify the shape information which is further classified by a trained SVM, and a comparatively simple CNN with the classical AlexNet structure acts as the self-learned detector. The effectiveness of the hybrid method has been validated by comparative tests on the training dataset and vehicle tests to show the advantage of extensibility to new application conditions. The remainder of this paper is organised as follows: the proposed method is presented in Section 2; its performance is analysed theoretically in Section 3 and the design of the hybrid detector is introduced in Section 4; Sections 5 and 6 validate the performance and concludes the paper, respectively. 2 Hybrid detection method Considering the characteristics of different algorithms and the requirements of detection process of the traffic light, the framework of the proposed hybrid detector is shown in Fig. 1, which includes three parts: part 1) extraction of candidate regions; part 2) two detectors using self-extracted and known features, respectively; part 3) scheduling logic to combine the two detectors in part 2. Fig. 1Open in figure viewerPowerPoint Diagram of a hybrid detection approach Considering the concise requirement of extraction of candidate regions, the HSV colour is used to find the preliminary regions [7], which are further enlarged using the known knowledge that traffic light has a rectangular backplane as shown by Fig. 2 to retain more positive information to the next step. Meanwhile, this approach can avoid the amplification of the extraction error of the preliminary region, because it is independent of the state, installation mode and distance of traffic light. It should be noted that only the red and yellow–green traffic light without turn indicator is considered in this study. Fig. 2Open in figure viewerPowerPoint Expansion of the preliminary region In part 2, two detectors are designed to identify the light state to realise the advantage complement of the two methods based on known features (denoted by the classical detector) and self-extracted (denoted by learning detector), respectively, which are designed in Section 4 in detail. Considering the following reasons, a switching logic is proposed to schedule the two detectors in part 2: (i) algorithms such as CNN have an excellent performance on object detection, while this ability can hardly migrate to new application conditions, which are significantly different to its learning dataset; (ii) some known features of a traffic light are invariant with environments, such as shape etc. To simultaneously improve the detection accuracy and extension ability to new conditions, the switching logic is shown in Fig. 3. Fig. 3Open in figure viewerPowerPoint Switching logic 3 Performance analysis 3.1 Relationship among different methods The following statistical indices are used to evaluate the detection performance [3]: (1) where and are the accuracy and miss rate whose subscript '#' denotes the different methods by 'c' for classical detector, 'l' for learning detector and 'f' for the fused one. The symbol is the total number of candidate regions, where is the number of true classifications for positive samples, is the number of true classifications for negative samples, is the number of false classifications for positive samples and is the number of false classifications for negative samples. With definition (1), the performance relationship among different methods is summarised by the following theorem. Theorem 1.The performances of different detection methods have the following relationships: (2) where is the number of regions containing traffic light. Proof.According to (1) and the scheduling logic described by the pseudo-code in Section 2, the performances of different methods can be described by (3) (4) (5) According to the definition of , it satisfies (6) Combining the first equation in (6) with the second equation in (4) yields (7) Then can be represented by substituting (7) to the first equation in (4) as (8) Further substituting (8) to the second equation in (6) yields the following equation: (9) Then the number of different classifications for learning detector can be represented as following by substituting (7) and (9) into (5): (10) The first two equations in (2) can be obtained by substituting (10), (8) and to (3).From the second equation in (4) and the first equation in (6), we have and accordingly, the third inequality in (2) is established.The following equation can be obtained by combining the first equation in (6), the second equation in (2) and (4) (11) The fourth inequality in (2) is obtained from (11) because .Finally, from the second equation in both (3) and (4), we have and then the fifth inequality in (2) establishes. ▪ Theorem 2 establishes the performance relationship among different detection methods, which provides a theoretical basis to guide the design of the hybrid detection system. It can be concluded from Theorem 2 directly that (C1) Increasing the detection accuracy of the classical detector is beneficial to the hybrid detection including its accuracy and miss rate. (C2) Both detection accuracy and miss rate of the hybrid method are improved by a smaller . This implies that the candidate regions should contain all possible traffic lights as far as possible. (C3) The miss rate of the hybrid method will not be better than the classical detector. 3.2 Numerical analysis of detection performance This section numerically analyses the performance improvement of the hybrid detection method based on Theorem 1: the indices, and , both change with and monotonically because and are positive. The numerical analysis is conducted under the condition that and to show the trend of detection performances. The results are shown in Fig. 4. Fig. 4Open in figure viewerPowerPoint Relationship of detection performances (a) Detection accuracy , (b) Miss rate In the shadow area of Fig. 4a, the detection accuracy of the hybrid method is better than both classical and learning detectors. The miss rate of the hybrid one is only better than learning detector in the shadow area of Fig. 4b. Moreover, the improvement areas of both accuracy and miss are extended by using the classic detector with a lower miss rate. Combining this numerical analysis results and Theorem 2, the hybrid method can improve the detection accuracy and achieve a lower miss rate than the learning detector, but its miss rate cannot be better than the classical detector. On the other hand, only a simpler self-learning detector is required to achieve the same detection performance. 4 Design of hybrid detector 4.1 Decomposition of detection requirement To make a balance between performance and costs, the following index is defined to measure the improvement of accuracy: (12) The condition for the best is summarised by the following theorem. Theorem 3.The maximum improvement of detection accuracy is (13) and the required condition is (14) Proof.The improvement of detection accuracy is analysed under two conditions, i.e. and . Then is rewritten as the following by substituting the first equation in (2) to (12) (15) Since and , at both two conditions reaches the following maximum value when : (16) It is a quadratic polynomial with as its parameter and when (14) is established it takes the maximum value (13). ▪ Theorem 3 gives out the maximum improvement condition and provides a way to determine the requirement of detection accuracy for classical and learning detectors, whose relationship is shown in Fig. 5. Fig. 5Open in figure viewerPowerPoint Maximum improvement of accuracy From Fig. 5, it can be found that when , the fusion method has a fine improvement of accuracy and compare the performance requirements for others can be achieved in practice. From Theorem 1, it is known that and to get a comparative low , we take and to balance performances and technical difficulties. At this condition, the maximum improvement of accuracy is achieved when according to Theorem 3. Then the improvement of miss rate becomes (17) From (17), the miss rate of the hybrid method is smaller than the learning detector when . A lower requirement of miss rate for learning detector is beneficial to its costs of both computation and learning. Summarising the above theoretical and numerical analysis, the requirements for candidate regions extractor, classical and learning detectors are set to the following values to make a balance between performance and costs (18) 4.2 Extraction of the candidate region As proposed in Section 2, HSV colour space is used to extract the preliminary regions. From conclusion C2 from Theorem 1 and the performance requirement (18), to achieve a lower , the following two datasets are used to obtain the ranges of HSV for the extraction of the preliminary regions statistically: (i) LARA dataset collected on bright days [30]; and (ii) LISA dataset collected on dark cloudy days [31]. The statistical results are shown in Fig. 6, from which it can be found that there is no obvious boundary for different light states. It is impossible to distinguish light state directly using HSV, so the range is designed to extract all areas possibly containing traffic lights, not to identify light state. The final ranges for extraction are selected as Table 1 according to the statistical results in Fig. 6. Table 1. HSV range for extraction H S V 0–150 & 225–255 75–255 45–255 Fig. 6Open in figure viewerPowerPoint Statistical results of colour components 4.3 Design of classical detector From the analysis in Section 4.1, it is known that should be as small as possible and the final detection accuracy can be improved by learning detector. Classical detector is required to avoid missing the candidate region containing traffic light. Considering that shape is a notable feature of traffic light and image is easily deformed in practice, hog feature is selected to identify edge and shape because it can count occurrence of gradient orientation in local area. Then a SVM classifier is trained to identify traffic light from the candidate regions [32]. To get a lower , the SVM classifier is designed to be binary, which is only required to identify whether the candidate region contains a traffic light and the state of the traffic light is not detected. 4.4 Design of learning detector It is known from the theoretical and numerical analysis in Section 3 that both detection accuracy and miss rate of the hybrid one may be better than a single learning detector. On the other hand, learning detector can be designed to be simpler to reduce the costs of computation and training. The CNN with a classical AlexNet structure is chosen to act as learning detector because of its simple structure [33]. Since learning detector only needs to process the images classified to traffic light by classical detector, only the true classifications of classical detector are used to train the network for efficiency. The final structure of the network is shown in Table 2. Table 2. AlexNet architecture Conv1 Pooling Conv2 Pooling Conv3 Conv4 Conv5 FC1 FC2 FC3 input size 32 × 12 × 3 32 × 12 × 32 16 × 6 × 32 16 × 6 × 64 8 × 3 × 64 8 × 3 × 96 8 × 3 × 96 8 × 3 × 128 1024 × 1 1024 × 1 kernel size 3 × 3 × 32 2 × 2 3 × 3 × 64 2 × 2 3 × 3 × 96 3 × 3 × 96 3 × 3 × 128 — — — padding same valid same valid same SAME SAME — — — stride 1 × 1 2 × 2 1 × 1 2 × 2 1 × 1 1 × 1 1 × 1 — — — output size 32 × 12 × 32 16 × 6 × 32 16 × 6 × 64 8 × 3 × 64 8 × 3 × 96 8 × 3 × 96 8 × 3 × 128 1024 × 1 1024 × 1 4 × 1 5 Experimental validation and analysis To show the advantages of the proposed hybrid detector, two comparative tests have been carried out. One is to compare with different detectors (including single CNN detectors and other fusion detectors) using the training dataset in Section 4. The other is conducted by real vehicle tests to show the extensibility to new application conditions under the single CNN detector and the proposed fusion method has similar performance on the training dataset. 5.1 Comparative analysis of performance To evaluate the performance of the proposed fusion detector, it has been compared with the following four detectors: (i) A single CNN with 19 layers, which has a similar performance with the hybrid method. (ii) The single CNN with eight layers used in the hybrid one and designed in Section 4.4. (iii) The hybrid classical detector composed of a multi-size detector, a bulb detector, and a colour detector [29]. (iv) The hybrid CNN detector containing two CNN detectors for the classification of positive/negative samples and the types of positive samples, respectively [34]. The single CNN with 19 layers is used to verify that the proposed hybrid detector can simplify the CNN structure without sacrificing the performance of the CNN. The single CNN with eight layers is used to show the improvement of detection performances by fusion. The hybrid classical and CNN detectors are used to compare with the proposed fusion method. The comparative results are shown in Table 3, from which it can be seen that the comprehensive performance of the proposed hybrid detector is best. Table 3. Test results on the training dataset Detector Accuracy, % Miss, % hybrid 85.30 1.67 single CNN with 19 layers 84.60 3.67 single CNN with eight layers 82.50 7.14 hybrid classical 64.83 27.58 hybrid CNN 87.50 5.00 Though the hybrid detector has better comprehensive performance, there still exist some typical detection errors as shown in Fig. 7, which should be further investigated. In Fig. 7a, though the region containing the traffic light is successfully extracted by SVM, it is not correctly detected by CNN due to the serious deformation. In Fig. 7b, the traffic light is covered by about 2/3, which causes a wrong classification of SVM because it uses the shape information of traffic light. Fig. 7Open in figure viewerPowerPoint Typical detecting errors (a) Deformed traffic light, (b) Covered traffic light 5.2 Extensibility evaluation One motivation of designing the hybrid detector is to use the known knowledge to keep an acceptable detection performance when it is applied to new conditions. The two detectors: the proposed hybrid detector and the single CNN detector with 19 layers, which have same performance on training dataset in Table 3 have been directly tested with our development platform of automatic driving as shown in Fig. 8 [35]. The high definition camera installed on the vehicle roof outputs the image and the detection algorithm runs in an industrial process computer (CPU I7 6700TE, GPU NVIDIA GeForce GTX1050) fixed in the trunk. During the experiment, the detection rate is set to be 30 fps, and the average detection times of the hybrid detector and the single CNN detector are about 13 and 21 ms, respectively. The route is selected from the campus, which is about 3 km and contains eight traffic lights in one cycle. Fig. 8Open in figure viewerPowerPoint Experimental vehicle (a) Installation of camera, (b) Installation of IPC, (c) Data set image, (d) Acquired image Fig. 9 shows two pictures from the training dataset and acquired by the on-board system, which are differences in the scene complexity, hue, brightness of traffic light, and image resolution. Fig. 9Open in figure viewerPowerPoint Comparison of detection condition (a) Dataset image, (b) Acquired image The vehicle runs along the route ten times with the speed smaller than 25 km/h. The statistical results of two detectors are compared in Fig. 10. The detection accuracy of the hybrid one is reduced from 85.3 to 79.6%. The accuracy of the single CNN detector is only 45% under this new application condition because of the change of image characteristics. The hybrid method combines the shape information of traffic light, which is not affected by the environmental condition and image parameters. Fig. 10Open in figure viewerPowerPoint Detection performance of vehicle test To further analyse the performance degradation, some detailed detection results are shown in Fig. 11 comparatively. In this image, there exist several areas whose colour is red or green (denoted by A and B in Fig. 11a for example). These areas confuse the signal CNN detector successfully and they are classified as traffic light wrongly. The colour information of these areas is similar to traffic light, but they do not have the shape feature. The hybrid method can remove such disturbances finely by the known features of shape and detect the traffic light correctly as shown in Fig. 11b. Fig. 11Open in figure viewerPowerPoint Comparison of detection results (a) Single CNN detector, (b) Hybrid detector 6 Conclusion In this study, we propose a fusion algorithm for the detection of a traffic light to enhance the comprehensive performance by combining the known knowledge with the self-learned features. The theoretical, numerical and test results show that (i) The performance of the self-learned method greatly depends on the training dataset and degrades obviously when the application condition becomes totally different. (ii) Comparatively, the known features of the traffic light, e.g. shape, are not affected by environments easily. This is a beneficial supplement to the learning approach. (iii) The presented fusion logic can effectively combine the advantage of the classical method with the self-learned algorithms. The comprehensive detection performance is improved by properly designing the classical and learning detectors. Some open questions are worthy to be further investigated as follows: (i) In this study, the detection object is a simple type of traffic light. The application to the detection of other types of traffic lights, even traffic participants, is worth trying. (ii) The two detectors are combined by a switching logic, which is comparatively simple and might not be the best one. The structure for combination and its optimisation needs to be further studied. 7 Acknowledgments This work was supported by the National Key R&D Program of China under grant no. 2016YFB0100900 and the Natural Science Foundation of Chongqing under grant no. cstc2019jcyj-zdxmX0018. 8 References 1Gao F. Lin F. Liu B.: 'Distributed hinf control of platoon interacted by switching and undirected topology', Int. J. Automot. Technol., 2020, 21, (1), pp. 259– 268 2' Red light running'. Insurance Institute of Highway Safety. Available at http://www.iihs.org/iihs/topics/t/red-light-running/topicoverview 3Jensen M.B. Philipsen M.P. Møgelmose A. et al.: 'Vision for looking at traffic lights: issues, survey, and perspectives', IEEE Trans. Intell. Transp. Syst., 2016, 7, pp. 1800– 1815 4Omachi M. Omachi S.: ' Traffic light detection with color and edge information'. 2nd IEEE Int. Conf. on Computer Science and Information Technology, Beijing, People's Republic of China, 2009, pp. 284– 287 5Widyantoro D.H. Saputra K.I.: ' Traffic lights detection and recognition based on color segmentation and circle hough transform'. Int. Conf. on Data and Software Engineering, Yogyakarta, Indonesia, 2015, pp. 237– 240 6Ouyang Z. Niu J. Liu Y. et al.: 'Deep CNN-based real-time traffic light detector for self-driving vehicles', IEEE Trans. Mob. Comput., 2020, 19, (2), pp. 300– 313 7Cheng H. Jiang X. Sun Y. et al.: 'Color image segmentation: advances and prospects', Pattern Recogn., 2001, 34, (12), pp. 2259– 2281 8Bachmann T. Prinz W. Stoffer T.H.: ' Visual selective attention'. doi:10.1007/978-1-4419-1428-6_6153, 2012 9Ji Y. Yang M. Lu Z. et al.: ' Integrating visual selective attention model with HOG features for traffic light detection and recognition'. IEEE Intelligent Vehicles Symp., Seoul, Republic of Korea, 2015, pp. 280– 285 10Omachi M. Omachi S.: ' Detection of traffic light using structural information'. 10th IEEE Int. Conf. on Signal Processing, Beijing, People's Republic of China, 2010, pp. 809– 812 11Said A.F. Hazrati M.K. Akhbari F.: ' Real-time detection and classification of traffic light signals'. IEEE Applied Imagery Pattern Recognition Workshop, Washington, DC, USA, 2016, pp. 1– 5 12Binangkit J.L. Widyantoro D.H.: ' Increasing accuracy of traffic light color detection and recognition using machine learning'. 10th Int. Conf. on Telecommunication Systems Services and Applications, Denpasar, Indonesia, 2016, pp. 1– 5 13Lee S. Kim J. Lim Y. et al.: ' Traffic light detection and recognition based on Haar-like features'. Int. Conf. on Electronics, Information, and Communication, Honolulu, HI, USA, 2018, pp. 1– 4 14Du X.P. Li Y. Guo Y. et al.: 'Vision-based traffic light detection for intelligent vehicles'. Int. Conf. on Information Science & Control Engineering, doi:10.1109/ICISCE.2017.275, 2017 15Waisakurnia W. Widyantoro D.H.: ' Traffic light candidate elimination based on position'. 10th Int. Conf. on Telecommunication Systems Services and Applications, Denpasar, Indonesia, ISSN: 2372-7314, 2016 16Jensen M.B. Philipsen M.P. Moelmose A. et al.: ' Traffic light detection at night: comparison of a learning-based detector and three model-based detectors'. 11th Symp. on Visual Computing, Las Vegas, NV, USA., 2015, vol. 9474, pp. 774– 783 17Shen X. Andersen H. Ang M.H. et al.: ' A hybrid approach of candidate region extraction for robust traffic light recognition'. 20th IEEE Int. Conf. on Intelligent Transportation Systems, Yokohama, Japan, 2017, pp. 1– 8 18Zhao Z. Zheng P. Xu S. et al.: 'Object detection with deep learning: a review', IEEE Trans. Neural Netw. Learning Syst., 2019, 30, (11), pp. 3212– 3232 19Liu W. Anguelov D. Erhan D. et al.: ' SSD: single shot multibox detector'. 14th European Conf. on Computer Vision, Amsterdam, Netherlands, doi:10.1007/978-3-319-46448-0_2, 2016 20Ren S. He K. Girshick R. et al.: 'Faster R-CNN: towards real-time object detection with region proposal networks'. IEEE Trans. Pattern Anal. Mach. Intell., 2017, 39, (6), pp. 1137– 1149 21Redmon J. Farhadi A.: ' YOLOv3: an incremental improvement'. arXiv:1804.02767[cs.CV], 2018 22Kim J. Cho H. Hwangbo M. et al.: ' Deep traffic light detection for self-driving cars from a large-scale dataset'. 2018 21st Int. Conf. on Intelligent Transportation Systems, Maui, HI, USA, 2018, pp. 280– 285 23Weber M. Wolf P. Zöllner J.M.: ' DeepTLR: a single deep convolutional network for detection and classification of traffic lights'. 2016 IEEE Intelligent Vehicles Symp., Gothenburg, Sweden, 2016, pp. 342– 348 24Szegedy C. Vanhoucke V. Ioffe S. et al.: ' Rethinking the inception architecture for computer vision'. 2016 IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 2818– 2826 25Janahiraman T.V. Subuhan M.S.M.: ' Traffic light detection using tensorflow object detection framework'. 2019 IEEE 9th Int. Conf. on System Engineering and Technology (ICSET), Shah Alam, Malaysia, 2019, pp. 108– 113 26Wang J. Zhou L.: 'Traffic light recognition with high dynamic range imaging and deep learning'. IEEE Trans. Intell. Transp. Syst., 2019, 20, (4), pp. 1341– 1352 27Müller J. Dietmayer K.: ' Detecting traffic lights by single shot detection'. 2018 21st Int. Conf. on Intelligent Transportation Systems, Maui, HI, USA, 2018, pp. 266– 273 28Behrendt K. Novak L. Botros R.: ' A deep learning approach to traffic lights: detection, tracking, and classification'. 2017 IEEE Int. Conf. on Robotics and Automation, Singapore, 2017, pp. 1370– 1377 29Li X. Ma H. Wang X. et al.: 'Traffic light recognition for complex scene with fusion detections', IEEE Trans. Intell. Trans. Syst., 2018, 19, (1), pp. 199– 208 30' Traffic lights recognition public benchmarks'. Available at http://www.lara.prd.fr/benchmarks/trafficlightsrecognition 31Møgelmose A. Trivedi M.M. Moeslund T.B.: 'Vision based traffic sign detection and analysis for intelligent driver assistance systems: perspectives and survey', IEEE Trans. Intell. Transp. Syst., 2012, 13, (4), pp. 1484– 1497 32Dalal N. Triggs B.: ' Histograms of oriented gradients for human detection'. Computer Vision and Pattern Recognition, IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, San Diego, CA, USA., 2005, vol. 1, pp. 886– 893 33Krizhevsky A. Sutskever I. Hinton G.: ' ImageNet classification with deep convolutional neural networks'. NIPS'12 Proc. 25th Int. Conf. on Neural Information Processing Systems, Lake Tahoe, NV, USA., 2012, vol. 1, pp. 1097– 1105 34Luo H. Yang Y. Tong B. et al.: 'Traffic sign recognition using a multi-task convolutional neural network', IEEE Trans. Intell. Trans. Syst., 2018, 19, (4), pp. 1100– 1111 35Duan J. Gao F. He Y.: 'Test scenario generation and optimization technology for intelligent driving systems', IEEE Intell. Transp. Syst. Mag., 2020, doi:10.1109/MITS.2019.2926269 Citing Literature Volume14, Issue7July 2020Pages 735-741 FiguresReferencesRelatedInformation