Real-Time Lane Detection and alerts for Autonomous Driving
By: Priya Dwivedi
Tags: Self Driving Cars, Data Science, Deep Learning, Artificial Intelligence, Computer Vision

On this dataset, we can train a semantic segmentation model to segment out the pixels that belong to the lane class. U-Net model would be an ideal fit here since it is a lightweight model with real-time inference speed. U-Net is an encoder-decoder model with skip connections b/w encoder and decoder blocks. The model architecture is shown below.

U-Net model architecture from the U-Net Paper

However, the loss function should be modified to a Dice Loss coefficient. The lane line segmentation problem is an extremely imbalanced data problem. The majority of the pixels in the image belong to the background class. Dice Loss is based on Sorenson-Dice coefficient and it attaches similar importance to false positives and false negatives which allows it to perform better for imbalanced data problems. You can read more about the Dice Loss in their paper. The Dice loss tries to match the lane pixels in the ground truth and predicted models and is able to get a crisp boundary prediction. I really enjoyed this blog on using Dice Loss for boundary prediction.

LaneNet Model

For this blog, I used the LaneNet model to generate lane lines. The LaneNet model is a two-stage lane line predictor. The first stage is an encoder-decoder model to create a segmentation mask for the lane lines. The second stage is a lane localization network that takes as input the lane points extracted from the mask and uses LSTM to learn a quadratic function that predicts the points for the lane.

The image below shows the two stages in action. The left panel is the original image, the center panel is the lane mask output from stage 1 and the right panel is the final output from stage 2.

LaneNet Model explained (Picture from the paper — https://arxiv.org/pdf/1807.01726.pdf)

I used the LaneNet model implementation from this repo. The code is well maintained and runs without errors.

Generating Smart Alerts

I have combined the predictions of lane lines with object detection to generate smart alerts. These smart alerts can be related to:

Detecting the presence of other vehicles in Vehicle’s Ego Lane and measuring the distance to them

Detecting the presence of a vehicle in the adjacent lanes

Understanding the radius of a turn for curved roads

Here, I have used YOLO-v5 to detect cars and humans on the road. If you are interested in testing YOLO-v5, please check out my blog here.

YOLO-v5 does quite well in detecting other cars on the road. The inference time is also very fast.

Below we are using YOLO v5 to measure the distance b/w ego vehicle and the closest vehicle in front. The models return the distance in pixels which can be converted to meters based on the camera parameters. Since camera parameters are not known for the TUSimple data set, I have estimated pixels to meter conversion based on the standard width of lane lines.

Alerts for distance measurement

We can similarly calculate the radius of curvature of the lane and use that for the steering module of the car.

The radius of curvature measurement

Conclusion

In this blog, we explore the problem of detecting lane lines accurately and quickly for applications in Autonomous Driving. We then use YOLOv5 to build an understanding of other objects on the road. This can be used to generate smart alerts.

At Deep Learning Analytics, we are extremely passionate about using Machine Learning to solve real-world problems. We have helped many businesses deploy innovative AI-based solutions. Contact us through our website here if you see an opportunity to collaborate.

References