Package: traffic-light-classifier
Traffic Light ClassificationA computer vision project to classify traffic light signals as either red, yellow or green.Table of ContentsProject OverviewData DescriptionMethodologyResultsPython packagetraffic_light_classifierPackage UsageProject OverviewTraffic light classification plays an important role in Advanced Driver Assist as well as self-driving vehicle systems which ensures timely and appropriate reaction to traffic lights in cross sections.In this project, a robust probabilistic approach based classifier has been implemented from scratch to classify traffic light signal's status using computer vision and machine learning techniques.Several data cleaning steps, features extraction and a probabilistic metric has been utilized.The classifier has been validated on a testing dataset with aaccuracy of 99.66 %.All training stages and prediction stages has been throughly visualized & analyzed and thus improvised.The methodology utilized in this project can be generalized and applied to many other computer vision applications.The project presentation notebook isNotebook Traffic_Light_Classifier.The implemented Python package code istraffic_light_classifier.Data DescriptionThis traffic light dataset consists of 1484 number of color images in 3 categories - red, yellow, and green. As with most human-sourced data, the data is not evenly distributed among the types. There are:904 red traffic light images536 green traffic light images44 yellow traffic light imagesNote: All images come from thisMIT self-driving car courseand are licensed under aCreative Commons Attribution-ShareAlike 4.0 International License.Each image is a numpy array of shape (n_row, n_col, 3) i.e each image has a 3 channels (RGB - red, green, blue color space) where n_col and n_row is the height and the width of the image.Each image gives information about its colors of the pixels and their location.The RGB color space of the images can be converted to HSV color space channels (hue, saturation, value) i.e. the information about the hue, saturation and brightness of the pixels of the images.MethodologyFeature extraction from the datasetThe features are in the form of either rgb values or hsv values of the pixels of the image and their locations.Some feature extracted:Average hsv image for red images, yellow images & green images.1D range of high saturation along width and along height for each image.1D range of high brightness along width and along height for each image..2D Region of high saturation for each image.2D Region of high brightness for each image.Location of light for each image.Extraction of average red light hues, average yellow light hues, average green light hues from each image.Average s (saturation) of light located.Average v (bright) of light located.etc.The classifier's training stages:Get average image for red, yellow and green training images.Get region of high saturation in average red, yellow & green images to narrow down the search window for individual images to look for the red, yellow, and green lights.Crop all training images at their respective color's average image's high saturation region.Locate lights in all images by using high saturation and high brightness regions.Crop images at their respective light's positionGet average image of red light, yellow light & average green light to see the distribution of hue, saturation and brightness in red lights, yeellow light, and green lights.Get distribtution of hues, saturations and brightnesses of red lights, average yellow lights & green lights.Optimize classifier's metric's parameters for red, yellow and green lights.Predict & get accuracy of training dataset for classifier's optimized parameters.Probabilities calculations and prediction.Hues extractionFor a single image, the classifier extracts the following hues from the 3 regions:Model's red    hues from the light located in model's red    light region.Model's yellow hues from the light located in model's yellow light region.Model's green  hues from the light located in model's green  light region.Probabilistic modelThis classifier classifies an input image either red, yellow or green based onprobabilities.For a single input image, 3 lights will be located in the 3 regions (i.e. red, yellow and green light regions). Then the classifier calculates3 probabilities:Probability of image being redProbability of image being yellowProbability of image being greenAnd propobilities are calculated by,$Probability\ of\ image\ being\ red    = \dfrac {strength_{red}}    {strength_{red} + strength_{yellow} + strength_{green}}$$Probability\ of\ image\ being\ yellow = \dfrac {strength_{yellow}} {strength_{red} + strength_{yellow} + strength_{green}}$$Probability\ of\ image\ being\ green  = \dfrac {strength_{green}}  {strength_{red} + strength_{yellow} + strength_{green}}$where,$strength_{red}    = \mu_{saturation_{red}}^a    \times \mu_{brightness_{red}}^b$$strength_{yellow} = \mu_{saturation_{yellow}}^a \times \mu_{brightness_{yellow}}^b$$strength_{green}  = \mu_{saturation_{green}}^a  \times \mu_{brightness_{green}}^b$and,$\mu_{saturation_{red}}$   : mean saturation of the light located in model's red    light region$\mu_{brightness_{red}}$   : mean brightness of the light located in model's red    light region$\mu_{saturation_{yellow}}$: mean saturation of the light located in model's yellow light region$\mu_{brightness_{yellow}}$: mean brightness of the light located in model's yellow light region$\mu_{saturation_{green}}$ : mean saturation of the light located in model's green  light region$\mu_{brightness_{green}}$ : mean brightness of the light located in model's green  light region$a$ & $b$     : model's parametersDetailed analysis and visualization of each stage has been given inNotebook Traffic_Light_Classifier.ResultsA custom made Python packagetraffic_light_classifierhas been implemented for this project.All training stages and prediction stages has been throughly visualized & analyzed and thus improvised.The classifier has been validated on a testing dataset with a accuracy of 99.66 %.The project results and package usage have been clearly demonstrated in theNotebook Traffic_Light_Classifier.Python packagetraffic_light_classifierThis project utilizes a custom-made packagetraffic_light_classifierwhich contains a classifier, plotting & feature extraction functionalities, and datasets for the project.Libraries used:OpenCV-Python,scipy,matplotlib,numpy.This library offers tools which enables to analyze and visualize the entire training and prediction process stages.Installation# Install package from PyPI >>!pipinstalltraffic_light_classifier# or# Install package from GitHub >>!pipinstallgit+https://github.com/ShashankKumbhare/traffic-light-classifier.git#egg=traffic-light-classifierPackage UsageImport packagetraffic_light_classifier# Import package `traffic_light_classifier` >>importtraffic_light_classifierastlcCreate an instance of class Model# Create an instance of class Model provided in the packagemodel=tlc.Model()Train model# Call `compile` method of model object to train the model# Note: Use parameter `show_analysis = True` to see the detailed process of the training/compiling stages.model.compile()model.compile(show_analysis=True)Predict image# Get a random red image from the test dataset provided in the packageimportrandomimage_red=random.choice(tlc.datasets.train.images_std.red)tlc.plots.plot_images(image_red)# Note: Use parameter `show_probabilities = True` to see the classification probabilities.#       Use parameter `show_analysis = True` to see the detailed process of the prediction stages.label_predicted=model.predict(image_red)label_predicted=model.predict(image_red,show_probabilities=True)label_predicted=model.predict(image_red,show_analysis=True)# For yellow and green test imagesimage_yellow=random.choice(tlc.datasets.train.images_std.yellow)image_green=random.choice(tlc.datasets.train.images_std.green)label_predicted=model.predict(image_yellow,show_analysis=True)label_predicted=model.predict(image_green,show_analysis=True)Predicting an entire dataset# Use `predict_dataset()` method to predict an entire dataset >>images_std=tlc.datasets.test.images_std.alllabels_std=tlc.datasets.test.labels_std# optionalaccuracy=model.predict_dataset(images_std,labels_std)print(accuracy)Analyzing & visualizing compilation/training processAn ardent user might want to see what is happening in the compiling/training process. He might also want to revisit or play with them.# After compilation/training, all the compilation stages are stored in `model.compilation` attribute >># To access them:# Compilation-Stage 1: Average image for red, yellow and green training imagesimage1=model.compilation.stg1_image_avg.redimage2=model.compilation.stg1_image_avg.yellowimage3=model.compilation.stg1_image_avg.greentlc.plots.plot_images([image1,image2,image3])tlc.plots.plot_channels(image1,"hsv")tlc.plots.plot_channels(image2,"hsv")tlc.plots.plot_channels(image3,"hsv")# Compilation-Stage 2a: Region of high saturation in average red, yellow & green imagesprint(model.compilation.stg2a_region_high_s.red)print(model.compilation.stg2a_region_high_s.yellow)print(model.compilation.stg2a_region_high_s.green)# Compilation-Stage 2b: Masked average images at their respective high saturation regionimage1=model.compilation.stg2b_image_avg_masked_region_high_s.redimage2=model.compilation.stg2b_image_avg_masked_region_high_s.yellowimage3=model.compilation.stg2b_image_avg_masked_region_high_s.greentlc.plots.plot_images([image1,image2,image3])tlc.plots.plot_channels(image1,"hsv")tlc.plots.plot_channels(image2,"hsv")tlc.plots.plot_channels(image3,"hsv")# Compilation-Stage 3: Cropped average images at high saturation regionimages1=model.compilation.stg3_dataset_images_cropped_high_s_region.red[0:10]images2=model.compilation.stg3_dataset_images_cropped_high_s_region.yellow[0:10]images3=model.compilation.stg3_dataset_images_cropped_high_s_region.green[0:10]tlc.plots.plot_images(images1)tlc.plots.plot_images(images2)tlc.plots.plot_images(images3)# Compilation-Stage 4: Locations of lights in all training imagesprint(model.compilation.stg4_locations_light.red[0:5])print(model.compilation.stg4_locations_light.yellow[0:5])print(model.compilation.stg4_locations_light.green[0:5])# Compilation-Stage 5: Cropped images at their respective light's positionimages1=model.compilation.stg5_dataset_images_light.red[0:10]images2=model.compilation.stg5_dataset_images_light.yellow[0:10]images3=model.compilation.stg5_dataset_images_light.green[0:10]tlc.plots.plot_images(images1)tlc.plots.plot_images(images2)tlc.plots.plot_images(images3)# Compilation-Stage 6: Average image of red lights, yellow lights & green lightsimage1=model.compilation.stg6_image_light_avg.redimage2=model.compilation.stg6_image_light_avg.yellowimage3=model.compilation.stg6_image_light_avg.greentlc.plots.plot_images([image1,image2,image3])tlc.plots.plot_channels(image1,"hsv")tlc.plots.plot_channels(image2,"hsv")tlc.plots.plot_channels(image3,"hsv")# Compilation-Stage 7: Hues, saturations and brightnesses of average red light, average yellow light & average green lightprint(model.compilation.stg7a_hue_avg_light.red.mu)print(model.compilation.stg7a_hue_avg_light.red.sigma)print(model.compilation.stg7a_hue_avg_light.red.dist)print(model.compilation.stg7b_sat_avg_light.red.mu)print(model.compilation.stg7b_sat_avg_light.red.sigma)print(model.compilation.stg7b_sat_avg_light.red.dist)print(model.compilation.stg7c_brt_avg_light.red.mu)print(model.compilation.stg7c_brt_avg_light.red.sigma)print(model.compilation.stg7c_brt_avg_light.red.dist)# Compilation-Stage 8: Optimized parameters  a & b for maximum accuracyprint(model.compilation.stg8_params_optimised)# Compilation-Stage 9: Prediction analysis & accuracy of training dataset for classifier's optimized parametersprint(dir(model.compilation.stg9a_dataset_train_analysis.green[0]))print(dir(model.compilation.stg9a_dataset_train_analysis.green[0]))print(dir(model.compilation.stg9b_misclassified.green[0]))print(f"Accuracy red     ={model.compilation.stg9c_accuracy_train.red}")print(f"Accuracy yellow  ={model.compilation.stg9c_accuracy_train.yellow}")print(f"Accuracy green   ={model.compilation.stg9c_accuracy_train.yellow}")print(f"Accuracy overall ={model.compilation.stg9c_accuracy_train.all}")Analyzing & visualizing prediction processAn ardent user might want to see what is happening behind the prediction process. Analyzing misclassified images might give user the understanding of the flaws of the classifier model and help improve the algorithm.importrandomimage_red=random.choice(tlc.datasets.train.images_std.red)tlc.plots.plot_images(image_red)label_predicted=model.predict(image_red)# After prediction, all the compilation stages are stored in `model.prediction` attribute >># To access them:# Compilation-Stage 1: Croped image at model's optimal high saturation region for red, yellow, green light's positionimage1=model.prediction.stg1_image_high_s_region.redimage2=model.prediction.stg1_image_high_s_region.yellowimage3=model.prediction.stg1_image_high_s_region.greentlc.plots.plot_images([image1,image2,image3])tlc.plots.plot_channels(image1,"hsv")tlc.plots.plot_channels(image2,"hsv")tlc.plots.plot_channels(image3,"hsv")# Compilation-Stage 2: Located lights in model's optimal regions of red, yellow, green lightsmodel.prediction.stg2_locations_light.redprint(model.prediction.stg2_locations_light.red)print(model.prediction.stg2_locations_light.yellow)print(model.prediction.stg2_locations_light.green)# Compilation-Stage 3: Cropped lightsimage1=model.prediction.stg3_image_light.redimage2=model.prediction.stg3_image_light.yellowimage3=model.prediction.stg3_image_light.greentlc.plots.plot_images([image1,image2,image3])tlc.plots.plot_channels(image1,"hsv")tlc.plots.plot_channels(image2,"hsv")tlc.plots.plot_channels(image3,"hsv")# Compilation-Stage 4: Extracted model's red, yellow, green light's colors from the respective cropped parts of the input imageimage1=model.prediction.stg4_image_colors_extracted.redimage2=model.prediction.stg4_image_colors_extracted.yellowimage3=model.prediction.stg4_image_colors_extracted.greentlc.plots.plot_channels(image1,"hsv")tlc.plots.plot_channels(image2,"hsv")tlc.plots.plot_channels(image3,"hsv")# Compilation-Stage 5: Distribution of hues extracted from the model's red, yellow & green light regionprint(model.prediction.stg5a_hue_input_light.red.mu)print(model.prediction.stg5a_hue_input_light.red.sigma)print(model.prediction.stg5a_hue_input_light.red.dist)print(model.prediction.stg5a_hue_input_light.yellow.mu)print(model.prediction.stg5a_hue_input_light.yellow.sigma)print(model.prediction.stg5a_hue_input_light.yellow.dist)print(model.prediction.stg5a_hue_input_light.green.mu)print(model.prediction.stg5a_hue_input_light.green.sigma)print(model.prediction.stg5a_hue_input_light.green.dist)# Similarly for saturation and brightness# Compilation-Stage 6: Probabilities of image being red, yellow and greenprint(model.prediction.stg6_probabilities.image_being_red)print(model.prediction.stg6_probabilities.image_being_yellow)print(model.prediction.stg6_probabilities.image_being_green)# Compilation-Stage 7: Predicted labelprint(model.prediction.stg7_label_predicted)print(model.prediction.stg7_label_predicted_str)The package usage have been clearly demonstrated in theNotebook Traffic_Light_Classifier.