The Workings of Self-Driving Cars
By: Zoe Petroianu
Tags: Self Driving Cars, AI, Technology, Artificial Intelligence, Computer Science

What does it need?

A self-driving car needs to be able to replicate a human’s brain; this includes reactions to hazards and decision making. You would want to avoid other cars, but you wouldn’t want to change course for a flying plastic bag. All of the decisions that a human would make need to be replicated by computers, excluding human error, in order for self-driving cars to be safe.

Every self-driving car is built differently depending on the company behind it, but it can be concluded that generally all of them have the same basic properties…

Limarc Ambalina / Source

Vision: What does the world look like?

In order for a car to process what to do in a certain situation, it first needs to be able to see for itself. This part of a self-driving car is implemented to function as human eyes. There are two main components involved: cameras and radars.

One of the most challenging and complex parts of self-driving cars is computer vision. Since computers only see numbers and process pixels of an image at a time, programmers use numerous neural networks in order for the computer to figure out what it is seeing. This is all a part of deep learning, a subset of machine learning. In order for a self-driving car to recognize other cars, it needs to analyze many images of cars. Through machine learning, we are able to train our model to detect what an image is showing.

Here is a fun online resource which we used during the InspiritAI camp which demonstrates basic machine learning.

Cameras are able to collect that data for machine learning and are placed all over the car. Tesla’s Autopilot features a total of eight cameras to provide 360 degree vision. However, cameras can only go so far, which is why self-driving cars also rely on radars and sensors to determine what the environment looks like. These tools are used to augment camera data. By using radars and sensors, self-driving cars able to determine the speed and distance of various objects.

Enrique Dans / Source

One of the widely used systems in self-driving cars is LIDAR (Light Detection and Ranging). First used in the 1960s, it now has a lot of attention for it’s incredibly fast and accurate data collection. By releasing short beams of infrared light at an object, it determines the time it takes for the beam to return to calculate the depth resolution of that object. LIDAR systems are generally complex and big spinning sensors on top of the car, but with integrated photonics, they have shrunk in size. One day, they are aimed to fit in the headlight of your car.

LIDAR Advantages:

High speed processing

Not affected by light intensity of the environment

High resolution imaging

Works smoothly with other data methods

No distortions

LIDAR Disadvantages:

Relatively expensive

Performs poorly in certain weather conditions

Requires high computational resources

Jeremy Cohen / Source

Where are we?

By utilizing localization, self-driving cars are able to determine where they are in regards to other objects. Now that we know what the world looks like, we need to know where we are. If you are driving yourself, you probably takes paths you already know. You already know the speed limits, stop signs, and speed bumps. Similarly, a map tells a self-driving car when and where it should expect them. GPSs aren’t always completely accurate, so companies use high definition maps and sophisticated mathematical algorithms to pinpoint the car’s location more precisely.

Internal mapping is able to track the current and predicted location of all surrounding objects, both static and in motion. By using pre-determined shape and motion descriptors, the self-driving car can classify what the object is and make smarter decisions on what actions to take when approaching it. By predicting the future locations of obstacles, the car can incorporate this information into its path planning.

Government Technology / Source

Put it into action!

Once the self-driving car has a clear vision of the world and its positioning relative to its surrounding, the car needs to determine its course of action. Path planning informs the car where we want to go, what to do in response too predictions of other cars’ movements, and whether it should slow down, speed up, or shift right or left. All of this is done while keeping track of the speed limit, other limitations, and obstacles in the road. One method of path planning is to create waypoints for the car to follow.

After the decisions have been made, the car needs to follow the directions by hitting the break and turning the wheel. The goal of the car is to stay in the middle of the lane, much like a human would do. This is done by trying to match up a line of the car’s predicted trajectory with a line of the car’s ideal trajectory. The closer the better!