Simulating self-driving cars using Unity-ML
By: Surajit Saikia
Tags: Self Driving Cars, Deep Learning, AI, Autonomous Cars, Machine Learning

Most importantly, we need to create an environment that is similar to what the real car would see on the road. The key parts of a simulation environment are:

Real Physics: In order to make the self-driving more realistic, the simulated environment should have the same attributes as in the real world. For instance, the gravity of 9.81 meters per second square applied to the environment, assigning real weights to cars and so on Observation: This part comprises the elements that are present in an environment in which the car needs to consider in order to drive and take appropriate actions. Sensor feed: The car needs to perceive the environment or observation in order to drive. The sensor feed can be obtained from various cameras and LIDAR devices installed in the car.

We will next see three such basic simulation environment that where I am training the cars to be self-driving vehicles.

Self-driving AI-race

The 3D cars learn to drive and race on their own using deep reinforcement learning. After several rounds of training using proximal policy optimization update, the cars can drive themselves to complete the circuit. Each of the cars in this environment weighs 1500 kg and are assigned with wheel colliders. Moreover, it has two reward function. First, if the car manages to drive without colliding it receives a positive reward. Secondly, I have assigned 18 checkpoints in the race track, and if the cars cross them then again a positive reward is given. If you want to create your own Unity-Environment with colliders and checkpoints, then you can have a look in this link.

Self-driving Reacher (Zombie Hunter)

In this video, a car learns to drive using deep reinforcement learning and Unity ML. The environment was created using UNITY 3D, and C# code was used to assign colliders and reward functions. The car receives a positive reward if it gets nearer to the zombies, and the environment is reset if the car goes out of bound. After training for several rounds using proximal policy optimization update, the car learns to reach the target (or to smash zombies). To create a basic car and to make it driveable you can follow this link. In this tutorial, you can learn how to make a very basic car and to control it using your keyboard. Bellow is the controller script that I created to control the car as shown in the video above.

Autonomous navigation using Deepdrive

Deepdrive is a realistic simulator for self-driving cars create by [1]. The observation space of the AI-agent or the car is multidimensional, to name a few, the states consist of image data, depth data, angular velocity and so on. The goal of the car is to complete its lap in minimum time by avoiding accidents. In this project, to train the self-driving car, I am experimenting with several observation states to come up with an efficient model. Moreover, I am using Deep reinforcement learning with Actor-critic approach and Proximal Policy Optimization. You can find more details about the deep-drive environment here.

Next, I will explain how to train a self-driving car in a simulation environment. So, stay tunned.

References: