[image] [image] Ressource protégée - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu

[image] [image]

* Documentation

* FR
  Français (FR)

* Anglais (EN)
  Se connecter

* Portail HAL Inria

* Recherche
  Loading...

## Recherche avancée

Information de documents

Titres

○ Titres

○ Sous-titre

○ Titre de l'ouvrage

○ Titre du volume (Série)

○ Champ de recherche par défaut (multicritères) + texte intégral des PDF

○ Résumé

○ Texte intégral indexé des documents PDF

○ Mots-clés

○ Type de document

○ Sous-type de document

○ Tous les identifiants du document

○ Identifiant HAL du dépôt

○ Langue du document (texte)

○ Pays (Texte)

○ Ville

○ À paraître (true ou false)

Ajouter
# Self Driving Car in a Constrained Environment

B. Harish, D. Thenmozhi

► To cite this version:

B. Harish, D. Thenmozhi. Self Driving Car in a Constrained Environment. 6th International Conference on Computer, Communication, and Signal Processing (ICCCSP), Feb 2022, Chennai, India. pp.325-339, 10.1007/978-3-031-11633-9_23. hal-04388146

HAL Id: hal-04388146

https://inria.hal.science/hal-04388146v1

Submitted on 11 Jan 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
## Auteur

### Auteur (multicritères)

○ Auteur (multicritères)

○ Auteur : Nom complet

○ Auteur : Nom de famille

○ Auteur : Prénom

○ Auteur : Complément de nom, deuxième prénom

○ Auteur : Organisme payeur

○ Auteur : IdHal (chaîne de caractères)

○ Auteur : Fonction

○ Auteur : personID (entier)

○ Auteur : Identifiant de l'organisme payeur

○ Auteur : Identifiant de la structure

○ Directeur de thèse

○ Éditeur

○ Éditeur scientifique

○ Éditeur de la série

## Ajouter

### Structure

### Structure (multicritères)

○ Structure (multicritères)

○ Structure : Acronyme

○ Structure : Nom

○ Structure : Code

○ Structure : Pays

○ Structure : Type

○ Structure : État dans le référentiel

○ Structure : Identifiant HAL de la structure

○ Structure/équipe de recherche : Pays

○ Structure/regroupement d'équipes : Pays

○ Structure/laboratoire : Pays

○ Structure/regroupement de laboratoires : Pays

○ Structure/institution : Pays

○ Structure/regroupement d'institutions : Pays

## Ajouter
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
Autres

Champ de recherche par défaut (multicritères)

○ <u>Champ de recherche par défaut (multicritères)</u>

○

○ <u>Revues (multicritères)</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Titre abrégé</u>

○ <u>Revue : Titre</u>

○ <u>Revue : Identifiant interne</u>

○ <u>Revue : Date de début de publication</u>

○ <u>Revue : ISSN électronique</u>

○ <u>Revue : ISSN</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Couleur dans SHERPA/RoMEO</u>

○ <u>Revue : État dans le référentiel</u>

○

○ <u>Colloque (multicritères)</u>

○ <u>Colloque : Titre</u>

○ <u>Colloque : Organisateur</u>

○ <u>Colloque : date de début (Année)</u>

○ <u>Colloque : date de fin (Année)</u>

○ <u>Conférence invitée (oui/non)</u>

○

○ <u>Projets ANR (multicritères)</u>

○ <u>Projet ANR : Acronyme</u>

○ <u>Projet ANR : Acronyme du programme</u>

○ <u>Projet ANR : Code décision (référence)</u>

○ <u>Projet ANR : Nom</u>

○ <u>Projet ANR : Identifiant interne</u>

○ <u>Projet ANR : État dans le référentiel</u>

○

○ <u>Projets européens (multicritères)</u>

○ <u>Projet européen : Acronyme</u>

○ <u>Projet européen : Identifiant de l'appel à projet</u>

○ <u>Projet européen : Référence</u>

○ <u>Projet européen : Nom</u>

○ <u>Projet européen : Date de fin</u>

○ <u>Projet européen : Financement</u>

○ <u>Projet européen : Date de début</u>
# SELF DRIVING CAR IN A CONSTRAINED ENVIRONMENT

B.Harish and Durairaj Thenmozhi*

SSN COLLEGE OF ENGINEERING

harish17049@cse.ssn.edu.in, theni_d@ssn.edu.in

**Abstract.** The purpose of the research is to build a machine learning model which can drive a car on the tracks of Udacity's Car simulator without any human intervention. This is achieved by mimicking the human driving behaviour in the training mode on a track. A dataset is generated by the simulator based on the human driving behaviour in the training mode and a deep learning model is built using this dataset which is then used to drive the car autonomously on any unseen track. Initially the model performed well only on the already seen track and failed to perform well on new unseen tracks. The simulator track in which the car was trained with didn't consist of any sharp turns or elevations or any other road barriers, but the real world tracks do contain them, so in order to overcome this problem image processing techniques like zooming,changing brightness,flipping images, panning were used and in order to avoid over-fitting problem more dataset was generated using data augmentation techniques. Finally a model was built which was able to generalise the tracks and drive the car autonomously on the unseen track of the simulator.

**Keywords:** Self-Driving Car, Deep Learning, Automation, Machine Learning, Udacity, Unity, Track, Roads, ELU Activation Function, Constrained Environment.

## 1 Introduction

This section describes the need of autonomous cars and the objectives of our research work.

### 1.1 Motivation

Today, driving has become a stress full job, people going on long trips get tired because of driving and lose their focus many times due to which accidents happen, this can be solved through model-based and learning-based [3] approaches in order to achieve full vehicle autonomy without or with minimum human intervention. Perceiving the scenes, controlling the vehicles and choosing a best and safe path to drive remains a challenge. A machine learning model is needed which can aid the car to drive autonomously in the real world tracks which has
○ Projet européen : État dans le référentiel

○ Projet européen : Identifiant interne

○

○ Date de production : année

○ Date de mise en ligne : année

○ Date de publication : année

○ Date d'écriture : année

○ Date de modification du dépôt : année

○ Date de dépôt : année

○ Date de publication électronique : année

○

○ Collection HAL (multicritères)

○ Collection HAL : catégorie

○ Collection HAL : Code

○ Collection HAL : Nom

○ Collection HAL : Identifiant interne

○

○ Identifiant interne du contributeur/déposant

○ Nom complet du contributeur/déposant

○

○ Domaines

○ Domaine primaire

○ Domaine racine

○ Sous-domaine niveau 1

○ Sous-domaine niveau 2

○ Sous-domaine niveau 3

○

○ Statut du document

○ Version du document

○ Type de dépôt

○ Type de document

○ ISBN

○ Numéro - référence

○ Identifiant DOI

○

○ Classification

○ Audience

○ Vulgarisation

○ Comité de lecture - texte (oui ou non)

○ Actes de colloque
lot of constraints like pot holes, elevations, speed breakers etc. With advancement in technologies and many things getting automated it's time to automate the driving task by building a model which can handle a lot of edge cases and the error produced by it should be extremely small.

## 1.2 Objectives

The governing objectives of the study are:

- To create a driving dataset using Udacity Simulator which can provide high definition pictures and video for the development of a deep learning model, similar to the famous NVIDIA model [1].

- To identify the right tools and techniques which can help in providing a safe vehicle autonomy.

- The car must be capable of driving within the lanes of the track.

# 2 Related Work

Muddassir Ahmed Khan [2] has undertaken this research work 4 years ago. For data generation, images were picked from the metadata provided driving_log.csv file and passed through the augmentor to get the training data. The top and bottom 25 pixels of the images are ignored, in order to get rid of the front of the car from the images, he has used various augmentation techniques like image flipping, brightness variation, adding shadows to the images. When he trained the network using such data alone the network went off the track and wasn't able to drive properly. He then included the left and right images with various offsets such as 0.3, 0.2, but these values led to large shifts in the steering and the car would wear off the track. 0.3 worked fine when training the network for few epochs but when the number of epochs was increased the vehicles started to move in a zig-zag fashion across the road with more and more hard turns. At the last moment he changed the offset to 0.09 and started getting better results.

In 2017, Albin Falk and David Granqvist from Chalmers University of Technology, University of Gothenburg [8], Sweden proposed that by integrating deep learning and conventional computer vision based techniques, redundancy can be introduced and it can minimize the unsafe behaviour in autonomous vehicles. A control algorithm was constructed to combine the advantages of a lane detection algorithm with a deep neural network which ensures the vehicle stays within the appropriate lane. The proposition makes it evident that the system shows better performance when a combination of the two technologies were run in a simulator than using them independently.

In 2018, Aditya Babhulkar from California State University, Sacramento has undertaken this research work [9]. The dataset was collected by driving the car on Track_1 of the simulator in the training mode, and the simulator provided
○ Référence interne

○ Financement

○ Collaborations

Ajouter

Lancer la recherche
Recherche experte (SolR)

Recherche experte (SolR)

Lancer la recherche
Vers la recherche avancée

• Déposer

Portail INRIA.HAL.SCIENCE
Les publications, logiciels... des scientifiques Inria

• Accueil

• Publications Inria

○ Consulter les derniers dépôts

○ Par équipe de recherche

○ Par auteur

○ Rapports de recherche et techniques

○ Livres blancs

○ Rapports d'activité des équipes

○ Statistiques

• Boite à Outils

○ X2Hal : importer par lot

○ Logiciels : créer le Codemeta.json

○ Créer mon IDHal

○ Créer mon CVHal

○ Haltools : créer ma page de publications

○ Haltools : extraire des données (réservé Inria)

○ Les bonnes pratiques

■ Comment bien s'affilier ?
the dataset in a csv file which consisted of steering angle and the path to then
directory in which the images of the track were stored. During the autonomous
mode the simulator acts as a server and the images captured from the single
center camera of the car are sent to the client python program which uses the
built neural network model and send back a steering angle to the car to drive
autonomously on the Track_2 of the simulator in autonomous mode. The model
was built using sequential models provided by Keras.

## 3 Proposed Methodology

<figure><img src="image_3.png" /><figcaption>Fig. 1. Bird's Eye view of the System's Working</figcaption></figure>

Figure 1 shows an overview of our system. The images from the three cameras attached to the car in the training mode of the simulator, are fed to the CNN model which predicts a steering angle for that part of the track, this predicted steering angle is compared to the desired steering angle proposed by human for that part of the track. And the weights of the CNN are adjusted to bring the CNN output closer to the desired output. Weight adjustment is accomplished using back propagation. Once trained, the network is able to generate steering commands for driving left, straight or right from the video images of a single center camera provided by the simulator in autonomous mode. Figure 2 shows this configuration.

## 4 Algorithm

- Open Unity Self Driving car simulator powered by Udacity.

- Select Track-1, the primitive track and choose training mode.

- Turn on recording mode and start collecting the data - track images and steering angle.

- Perform Data Manipulation i.e changing or altering the collected data.

- Perform Data Augmentation.
■ Quelle version puis-je déposer ?

■ Choisir le bon « type de document »

■ Mettre en valeur ses publications

○ Les tutos IES Inria

■ Lier les notices dans HAL

■ Umbrhal Inria pour l'accessibilité

■ Personnaliser son CVHAL

■ Créer et Personnaliser sa collection

• Documentation

○ Aide en ligne HAL

○ Aide en ligne Haltools

○ Aide en ligne X2hal

○ Tutoriels video HAL

○ Documentation API HAL

○ Ajouter des vignettes

○ Identifiant auteur IdHAL et CV

○ Compléter son dossier ORCID avec ses dépôts HAL

○ Inria et la Science Ouverte

X

X

X

Loading...

X

Le document que vous demandez est en access restreint
jusqu'au 2025-01-01 !

Revenir a la notice

Retourner à l'accueil

Contact

•

Ressources
<figure><img src="image_4.png" /><figcaption>**Fig. 2.** The CNN predicts a steering angle for the images from the single center camera of the car in the autonomous mode of the simulator.</figcaption></figure>

- Pre-process the data.

- Train the data using CNN and build the ML model.

- In the simulator, select Track-2 the unseen track and choose autonomous mode.

- In this mode simulator acts as a server and a python code built using Flask framework contains the built ML model acts as a client.

- The track images are sent to the python script containing the ML model.

- ML model sends the predicted steering angle for that point of the track back to the simulator.

- Test the performance of the model in the simulator.

## 4.1 Udacity Simulator

Unity simulator by Udacity [7] is used for training the self-driving car. The simulator provides two types of tracks, Track-1 is a circular track with a few sharp turns and no elevations, this is the track which was used for data collection purposes. Track-2 is a very complex track with a lot of sharp turns, ups and downs through out the end of the track, this is the track where we tested the car using the built machine learning model. The simulator has two modes - Training mode, where an user can move the car using his keyboard and Autonomous mode where the car is moved by instructions given by ML model. The car in the training mode has 3 cameras attached to it and in autonomous mode has a single camera attached to it.

## 4.2 Assumptions and Limitations

- No other cars are present along the track.

- No speed breakers are present.

- No human interventions along the track.

- Performance is affected by hardware specification, performance will be good only if run on machines with atleast 8GB of Ram and quad cores.
* Documentation

* FAQ

* API

* OAI-PMH

* AuréHAL

## Informations

* À propos

* Données personnelles

* Mentions légales

* Accessibilité

* Conformité RGAA

## Questions juridiques

* Je publie, quels sont mes droits ?

* Loi pour une République numérique

* Stratégie de non-cession des droits

* SHERPA/RoMEO

## Portails

* Portails institutionnels

* HAL

* HAL SHS

* HAL Thèses

* MédiHAL

## CCSD

* CCSD

* Episciences

* Sciencesconf
- No split lanes in the track. Assuming just a one way track.

- Udacity has released a newer version where we can train the car to handle the lane changes, it will be considered in the future works.

- Road traffic with few vehicles, hurdles and risks will be considered in the future works.

- A speed limit variable is set to control the speed of the car while driving autonomously. Its value can range from 5MPH to 30MPH(max speed of the car).

## 4.3 Data Collection:

Steps involved in the data collection are

- Training mode is turned on for Track-1 and data collection process is started.

- Various images of the track are captured.

- The simulator also records the Steering angle and speed of the car.

- The simulator provides the recorded data in CSV format.

- Size of the collected dataset = 25641 images of Track-1

- Amount of data used for Training = 15384

- Amount of data used for Testing on Track-2 = 10257

## 4.4 Data Manipulation:

The car provided by the simulator in the training mode contains 3 cameras attached in its front part. The cameras are located on the left, center and right side in the front part of the car. The idea is to make the images captured by the left and the right cameras as though they were captured by the center camera, this is done by adding a bias to their steering angles, this is necessary because in the autonomous mode the simulator provides only a single center camera to the car. Also by utilizing all the 3 cameras we get a bigger dataset of the track images to train our model, which helped in improving the accuracy of our machine learning model. The distance arc length between center and left camera(or right camera) is called as bias After trial and error we came up with 0.09 bias, which gave good predictions when trained with our machine learning model. For the images captured by the left camera, 0.09 bias was added to its steering angle. For the images captured by the right camera, -0.09 bias was added to its steering angle. These images from left and right cameras after adding the bias were appended to the images captured by the center camera.

In Figure 3, x-axis shows the range of steering angle and y-axis shows the count of images for that particular steering angle.

Since amount of images available for 0.0, 0.09 and -0.09 steering angle is more, our model will always tend to predict these steering angles making our model predict poorly, so, we deliberately delete some images containing these steering angles from various parts of the track, Figure 4 depicts the new distribution of steering angle after that deletion.
<figure><img src="image_5.png" /><figcaption>Fig. 3. Histogram of steering angle-1</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 4. Histogram of steering angle-2</figcaption></figure>

## 4.5 Data Augmentation

The biggest challenge was to generalize the car behaviour on Track_2 (unseen track) of the simulator. In real world, we can never train a machine learning model for every available track as it will require a humongous amount of data to train and process it. Real world contains a variety of weather conditions, and it is impossible to build a dataset for all such weather conditions and roads, however by using data augmentation techniques we can create a dataset closer to the real world.

**Zooming** Images were zoomed, to get a better view of the track. OpenCV techniques were used to achieve this. Figure 5 shows the original and zoomed track.

**Changing Brightness** By changing the brightness of the track images we can generalize the weather conditions. By increasing brightness we can get a track which looks as if it was captured during a sunny day and by decreasing the brightness we can make the track image look like as though it was captured during night time or a cloudy day or a shadow of a building/tree falls on the track. The change in brightness for various images of the track can be seen in the Figure 6.

**Panning/Translation** The image is moved along the x-axis or y-axis, which make the neural network to look everywhere in the image to capture the vital data. The original and translated images are shown in Figure 7.
<figure><img src="image_7.png" /><figcaption>Fig. 5. Zoomed Image</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 6. Brightness Altered Image</figcaption></figure>

**Horizontal Flipping** Sometimes when we collect data, the data may be skewed to left side(or right side), i.e if the track that we trained contained more no. of left turns compared to the no. of right turns, our model will always predict a left steering angle. Where as in real life tracks have an equal no. of left and right turns. To solve this problem, we randomly select some tracks and flip it horizontally and negating it's corresponding steering angle. This helped in making the dataset to contain nearly equal counts for left and right steering angles. Figure 8 shows the flipped image.

## 4.6 Pre-Processing

The height of the image is reduced to bring focus on the track thereby ignoring the landscapes surrounding the track. The color images(RGB) are converted to YUV(Y-Brightness and UV - color) encoding pattern. Gaussian Blur of size 3 by 3 was applied to the image. Images were resized to the shape 200 by 66. Normalization of the images are done.

## 4.7 Training

The recorded data is splitted into x_train and y_train. Figure 12 shows the split-
ted data, x-axis represents the steering angle, y-axis represents the count of
<figure><img src="image_9.png" /><figcaption>Fig. 9. Pre-processed image</figcaption></figure>

stride and a 5×5 kernel, and a non-strided convolution with a 3×3 kernel size in the last two convolution layers.

The five convolution layers are shown in Figure 13. The three fully connected layers which are designed to act as controller for steering, outputs the steering angle. To avoid over-fitting problem, we need a large amount of data to train on, so using data augmentation technique the batch generator creates large amount of data on the fly i.e only during training after this the training begins. The Rectified Linear Unit- ReLU activation function- $R(z)=max(0,z)$ was used during the initial stages of building and testing the model, however ReLU was not a good choice for this problem because ReLU function returns zero for values less than zero but our steering angles consist of both positive and negative values. So if a neuron gets an input of negative number it will return a value of zero, so the gradient at this point is zero and the weight of this neuron will never be changed because the back propagation uses the gradient value to change the weight values of the neurons. This phenomenon is called as dead ReLU. Due to this the neuron will always receive a value of zero and will always feed forward the value of zero and there is no learning. Sigmoid function can't be used as it would create vanishing gradient problem as our network is complex. So we chose Exponential Linear Unit -ELU activation function which is similar to ReLU in positive region but in negative region it returns the negative number. So unlike ReLU, ELU has a non-zero gradient value in the negative region, which means that it can always recover and fix its weight parameters to decrease its error i.e it is always capable of learning and contributing to the model, unlike ReLU which can essentially die. Figures 10 and 11 shows the ELU, ReLU and Sigmoid functions. Based on the accuracy, the CNN model is modified by changing the no. of neurons, by adding dropout layers, changing the learning rate, data augmentation etc to improve the accuracy.

Using PYTHON and Flask the created CNN model is given to the simulator and the car is driven in autonomous mode in the selected track. A speed limit is set to the car, the speed of the car increases till that limit and begins to gradually slow down after that.
<figure><img src="image_10.png" /><figcaption>Fig. 10. ELU Function</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Fig. 11. ReLU and Sigmoid Functions</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Fig. 12. Histogram Distribution of steering angles after train-test split</figcaption></figure>
<figure><img src="image_13.png" /><figcaption>Fig. 13. Architecture</figcaption></figure>
# 5 Testing

The built model is used in a python application which uses socketio to connect to the udacity simulator(server) by acting as a client. The Udacity simulator in autonomous mode acts as a server. In terminal the python driver file is run after loading the model. Inside the simulator, autonomous mode is selected and Track-2 - the unseen track is chosen. Unseen track is the one which the car has never seen before while training. The simulator sends the track images to the client (python application), which uses the built model to predict the steering angle for the car at that point of the track. The client code returns the steering angle to the simulator. To avoid the car going very fast, a speed_limit variable is used to control the throttle of the car. Response time is the sum of, time to send the track image to ML model, time for ML model to predict the steering angle for that point of track, time to send back the steering angle to the simulator. Here sending back steering angle takes very less time compared to other two factors. It takes a lot of time to send an image from simulator to model, but however since the model is available in local machine we can decrease this time by using it on a machine with decent hardware specs like atleast 8GB ram and quad cores. And the ML model predicting the steering angle also depends on the hardware specs, so by using the above hardware specs we can altogether reduce the latency i.e the response time. Figures 14,15 and 16 shows the car driving without human intervention in an unseen track i.e in Track-2 of the simulator.

<figure><img src="image_14.png" /><figcaption>Fig. 14.Car driving in autonomous mode Fig. 15.Car driving in autonomous mode</figcaption></figure>

# 6 Results

This section shows how our approach is different from the existing approaches and how well the built model has predicted in various parts of the unseen track (Track-2).

Our approach though it uses the same augmentation techniques as with the existing approaches mentioned in the literature survey, how we have used it matters. The existing approaches use all the augmentation techniques at once during
data augmentation, whereas in our approach we apply the 4 augmentation techniques (zooming, panning, flipping images and changing brightness) randomly, we select an image at random from the training dataset and then we generate a random number between 0 and 1, if the value is greater than 0.5 we apply zooming technique to it, or else we don't apply zooming to that image, and again we repeat the same for other augmentation techniques. So a particular image chosen for augmentation may be applied with all the 4 augmentation techniques or 3 or 2 or 1 or even none. This approach produces a dataset with increased randomness in the images compared to the existing approaches. Also, we have used ELU activation function which drastically improved the performance in the unseen track i.e Track-2.

Images below on the left column represent the true value of steering angle (TSA) for that point in the track.

Images on the right column represent the predicted value of steering angle (PSA) for that point in the track.

## 6.1 Left Curved Roads

<figure><img src="image_15.png" /><figcaption>Fig. 16. True SA = -0.148</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Fig. 17. Predicted SA = -0.106</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Fig. 18. True SA = -0.238</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>Fig. 19. Predicted SA = -0.283</figcaption></figure>
## 6.2 Right Curved Roads

<figure><img src="image_19.png" /><figcaption>Fig. 20. True SA = 0.257</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>Fig. 21. Predicted SA = 0.256</figcaption></figure>

<figure><img src="image_21.png" /><figcaption>Fig. 22. True SA = 0.381</figcaption></figure>

<figure><img src="image_22.png" /><figcaption>Fig. 23. Predicted SA = 0.430</figcaption></figure>
<table><thead><tr><th>S.No</th><th>Figure No</th><th>TSA</th><th>PSA</th><th>||PSA - TSA||</th><th>||PSA - TSA||<sup>2</sup></th></tr></thead><tbody><tr><td>1</td><td>17,18</td><td>0.148</td><td>-0.106</td><td>-0.254</td><td>0.064516</td></tr><tr><td>2</td><td>19,20</td><td>-0.238</td><td>-0.283</td><td>-0.045</td><td>0.00202</td></tr><tr><td>3</td><td>21,22</td><td>0.257</td><td>0.256</td><td>-0.001</td><td>1.0000e-06</td></tr><tr><td>4</td><td>23,24</td><td>0.381</td><td>0.430</td><td>0.049</td><td>0.002401</td></tr></tbody></table>

$$ \text{MSE(Mean Squared Error)} = \sum_{i=1}^{4} \frac{\|PSA - TSA\|^2}{4} = 0.017235 $$

$$ \text{RMSE(Root Mean Squared Error)} = \sqrt{\sum_{i=1}^{4} \frac{\|PSA - TSA\|^2}{4}} = 0.1312 $$

The table above shows the difference between True Steering Angle (TSA) and Predicted Steering Angle (PSA) i.e $\|PSA - TSA\|, \|PSA - TSA\|^2$, for the various road tracks.

# 7 Scope of Deployment in Real World

Since this research focused on working in a constrained environment, in the real world it could be used in places like industries where unmanned cars can be used for transporting goods. The minimum hardware requirement is a camera to capture the track images, servo/induction motors to drive the car based on predicted steering angle and a computer to run the model and give instructions to the car. We can use a Programmable Logic Controller (PLC) as a computer which would cost a minimum of Rs. 20000. The cost of a good quality single camera compatible with PLC would be around Rs.5000. The cost of servo motors for real cars would cost around Rs.20000. The cost of building the entire setup would be around Rs.20000. So the total cost to deploy this work with minimum hardware configurations in a constrained environment in the real world would be around Rs.65000.

# 8 Conclusions and Future Work

This research started with capturing images of the tracks,using computer vision [6] OpenCV techniques to add shadows to tracks,zooming of tracks,etc. Initially the model performed well only on the already seen track and failed to perform well on new unseen tracks. So, many models were built by changing the parameters like the number of neurons or the activation function so that the model was able to generalize road conditions and achieve the similar performance on different tracks. The spatial features were obtained by using CNN. In this research we used only the data obtained from the simulator, future works can be carried out by combining simulator data with the real world data and already many experimental configurations are being carried out in the field of autonomous cars. The models used in this research were built sequentially using Keras, future works can be done by trying parallel network layers for learning the specific track behaviour which can lead to an increase in performance. In this research
the simulated environment didn't consist of traffic environment, multiple lanes, obstacles or other cars along the track, when these are placed on the tracks they would make it more closer to a real world environment and a challenge for the self-driving cars.

## References

[1] Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316, 2016.

[2] Muddassir Ahmed Khan, Making A Virtual Self-Driving Car, Published on April 1 2017, https://muddassirahmed.medium.com/making-a-virtual-self-driving-car-2d81f3f539e8

[3] Lex Fridman, Daniel E Brown, Michael Glazer, William Angell, Spencer Dodd, Benedikt Jenik, Jack Terwilliger, Julia Kindelsberger, Li Ding, Sean Seaman, et al. Mit autonomous vehicle technology study: Large-scale deep learning based analysis of driver behavior and interaction with automation. arXiv preprint arXiv:1711.06976, 1, 2017.

[4] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pages 764–773, 2017.

[5] Nadia Jmour, Sehla Zayen, and Afef Abdelkrim. Convolutional neural networks for image classification. In: 2018 International Conference on Advanced Systems and Electric Technologies (ICASET), pages 397–402. IEEE, 2018.

[6] V. Badrinarayanan, F. Galasso, and R. Cipolla, Label propagation in video sequences, in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, 2010, pp. 3265–3272.

[7] Naoki, Introduction to Udacity Self-Driving Car Simulator, Published on Feb 25 2017, https://naokishibuya.medium.com/introduction-to-udacity-self-driving-car-simulator-4d78198d301d

[8] Falk, A. and Granqvist, D., 2017. Combining Deep Learning with traditional algorithms in autonomous cars.

[9] Babhulkar, A., 2019. Self-driving car using udacity's car simulator environment and trained by deep neural networks.