Inspiring Use of Artificial Intelligence (AI) in Highway Transportation

David Kuehn, Federal Highway Administration

October 6, 2022

© USchools / iStock.

Key Point: I am pleased to be able to speak about how the Federal Highway Administration Exploratory Advanced Research program is exploring the use of artificial intelligence (AI) to improve the safety and reliability of highway transportation.

Notes: 

1

AI Use in Transportation

Past: Expert systems.

Present: Machine learning:

Supervised learning.

Energy-intensive computing.

Large, annotated datasets.

Future:

Semisupervised, unsupervised, and one-shot learning.

Low-energy edge computing.

2

Key Point: Artificial intelligence in transportation is not new.  The highway transportation industry has been using expert systems for decades.  What is new is the collection of vast amounts of data that can support supervised learning.

Notes:

There are large amounts of data coming from roadside sensors, vehicle-based sensors, and personal mobile devices I

Roadside sensors include cameras, magnetometers (loop sensors), radar, LIDAR

Vehicle sensors include GPS, inertial, pavement traction, rain (for automated windshield wipers), 

Personal devices include GPS, inertial, acoustical, etc.  

There are many limits to applying current machine learning algorithms

They require large amounts of annotated data

Data may contain personally identifiable information or business proprietary information

They often require energy intensive, large scale computing facilities with the transfer and storage of large amounts of data.

The Federal Highway Administration is seeking the next generation of AI, which includes

Algorithms that can work with little or no annotated data

Running on low-energy, edge devices

2

Traffic Safety

Cooperative perception and control for freeway traffic system operations.

© University of California, Berkeley.

Three Class 8 trucks during a field test. 

Federal Highway Administration.  2021. EAR Program Research

Results: Updated through 2021. Report No. FHWA-HRT-22-034. Washington, DC: Federal Highway Administration. https://www.fhwa.dot.gov/publications/research/ear/22034/22034.pdf.

3

Key Point: A key focus of the Federal Highway Administration is supporting a safe systems approach that can eliminate traffic deaths and serious injuries.  Artificial intelligence is one factor in 

Notes: 

The research project aims to develop next-generation Traffic System Management and Operations (TSMO) solutions for freeway systems based on cooperative driving automation (CDA). The proposed solution concerns both cooperative perception (i.e., estimation and prediction using heterogeneous sources of data based on machine learning and filtering methods) and cooperative control (i.e., advanced artificial intelligence algorithms customized for vehicle and infrastructure level control, i.e., cooperative merge, platooning, and speed harmonization). The proposed work focuses on computational applications that could substantially increase freeway system safety and mobility, meeting the following two objectives: 1.) Integrate traditional and non-traditional highway data to better explain and predict system performance; 2.) Provide decision support to assist experts in the design, operations, or management of highway systems.  

More information is located at https://highways.dot.gov/research/projects/cooperative-perception-and-control-freeway-traffic-system-operations.  

Reaching zero deaths requires the implementation of a Safe System approach, which was founded on the principles that humans make mistakes and that human bodies have limited ability to tolerate crash impacts. In a Safe System, those mistakes should never lead to death. Applying the Safe System approach involves anticipating human mistakes by designing and managing road infrastructure to keep the risk of a mistake low; and when a mistake leads to a crash, the impact on the human body doesn’t result in a fatality or serious injury. Road design and management should encourage safe speeds and manipulate appropriate crash angles to reduce injury severity.

There are six principles that form the basis of the Safe System approach: deaths and serious injuries are unacceptable, humans make mistakes, humans are vulnerable, responsibility is shared, safety is proactive, and redundancy is crucial.

More information on the safe systems approach is located at Zero Deaths - Safety | Federal Highway Administration (dot.gov)

Image from EAR Program Research Results updated through 2021 located at FHWA-HRT-22-034: EAR Program Research Results (dot.gov)

3

Traffic Reliability 

Predictive real-time traffic management in large-scale networks: Uses model-based artificial intelligence.

Traffic incident detection and analysis system. 

Autonomous winter road maintenance decisionmaking: Enabled by boosting existing transportation data infrastructure with deep and reinforced learning.

4

Key Point:  There are many opportunities to improve system reliability with artificial intelligence.  

Notes: 

This research project proposes to develop theories, models and algorithms of AI guided by transportation network flow models, to achieve two main goals: to predict non-recurrent traffic conditions in large-scale networks at least 30 minutes ahead, and to proactively recommend operational management strategies in real-time. Prediction and operational strategies are intimately coupled. The prediction will be made by a machine that learns not only historical multi-source traffic data but also considers operational strategies that are currently being recommended or will be recommended/engaged. Operational strategies are made and updated in real time using model-based AI with ahead-of-curve prediction. Case studies will be conducted in one small municipality network and two large-scale regional networks. More information is located at https://highways.dot.gov/research/projects/predictive-real-time-traffic-management-large-scale-networks-using-model-based.  

Existing highway incident detection technologies fail to use available footage from traffic cameras installed on the highways. On the other hand, monitoring and analyzing the overwhelming quantity of camera data without assistive automated methods is intractable. Using Artificial Intelligence (AI), models can be trained to enhance images and provide robust detection and classification of traffic incidents, resulting in more cost-effective deployment of incident response resources. This research focuses on solving challenges including: 1) the lack of a robust automatic incident detection system capable of emphasizing key events with minimal false alarms, 2) overcoming the problems inherent in current learning algorithms, which significantly degrade in performance under adverse weather conditions, 3) the lack of availability of a dataset with diverse footage of highway incidents to foster the development and validation of AI algorithms. Here, a novel framework using AI and image processing algorithms for exploiting the potential of currently installed highway camera infrastructures for highway incident detection is presented, including detecting wrong-way driving, traffic congestion, crashes, and bicyclists or pedestrians in tunnels. More information is located at https://highways.dot.gov/research/projects/traffic-incident-detection-and-analysis-system-tidas.  

This research aims to boost the current model-driven winter maintenance with an artificial-intelligence (AI)-enhanced framework in which data can be analyzed in real-time for autonomous decision-making, and such decision-making ability can be improved continuously as more data is obtained. For this purpose, recurrent neural networks will be used to obtain a data-driven environment (i.e., road conditions) prediction ability; deep reinforced learning will be employed to reach an autonomous decision-making ability; convolutional neural networks (CNN) will be investigated for accurate and real-time road condition sensing. The proposed AI work will form a closed-loop consisting of data acquisition, condition predictions, decision-making, validation, and human intervention. If successful, the project deliverables will significantly improve winter road maintenance, directly benefit safety, operations, mobility and workforce training, and indirectly contribute to pavement design and management. More information is located at https://highways.dot.gov/research/projects/autonomous-winter-road-maintenance-decision-making-enabled-boosting-existing and at https://www.fhwa.dot.gov/publications/research/ear/21090/index.cfm.  

4

Computer Vision

Video analytics for automatic annotation of driver behavior and driving situations in naturalistic driving data. 

Deep insight: Deep extraction of driver state from naturalistic driving dataset.

© Volpe National Transportation Systems Center.

Researchers training neural networks to detect traffic signals and signal states. 

Federal Highway Administration. 2022. The Role of Artificial Intelligence and Machine Learning in Federally Supported Surface Transportation: 2022 Updates. Report No. FHWA-HRT-22-026. Washington, DC: Federal Highway Administration. https://www.fhwa.dot.gov/publications/research/ear/22026/22026.pdf.

5

Key Point: Computer vision has proved useful as a research tool in studying the behavior of drivers and pedestrians.  With new faster, edge algorithms it has the potential for supporting traffic control systems at intersection and other locations where there are potential conflicts.

Notes

The goal of the project is to develop a system that can analyze naturalistic driving videos and automatically produce annotations and descriptors for events, behavior, and driving scenarios that relate to transportation safety. Using large datasets obtained in naturalistic driving situations, the system will use innovative machine-learning techniques to recognize safety-related aspects of the driver, passengers, and outside environment. The project has four primary objectives that address 1) characterization of high-level driver behavior, such as eating or attending to a phone; 2) classification of extra-vehicular context, such as construction zones; 3) interactions and dependencies between drivers and the surrounding environment, such as looking at a passing vehicle or looking at a billboard; and 4) conclusions as to how the video analytics techniques developed under the other objectives enable human factors researchers to address key research questions in novel ways.

Computer vision algorithms have been improved significantly to provide accurate detection, tracking, and recognition of objects. Available public datasets are crucial for extracting key features thanks to ever-developing machine learning and deep learning algorithms in general. Since cameras are becoming vastly available to be employed on vehicles to extract useful information for both autonomous driving and intelligent driver assistance, the researchers are developing intelligent driver state estimation algorithms based on state-of-the-art detection and recognition using computer vision. One of the main drawbacks for naturalistic driving data is having low-resolution and noisy video data that limits the overall accuracy when tested with the models trained on clear images. The researchers will use a comprehensive AI platform for data management, modeling, and enhanced annotations; video quality enhancement using deep models; face detection at acute angles; and recurring network-based driver state estimation. 

Image from FHWA-HRT-22-026: The Role of Artificial Intelligence and Machine Learning in Federally Supported Surface Transportation 2022 Updates (dot.gov). The researchers are training neural networks to detect traffic signals and signal states. © Volpe National Transportation Systems Center.

5

Getting Involved

Exploratory Advanced Research Overview | FHWA (dot.gov)

Source: FHWA.

Federal Highway Administration. 2022. The Role of Artificial Intelligence and Machine Learning in Federally Supported Surface Transportation: 2022 Updates. Report No. FHWA-HRT-22-026. Washington, DC: Federal Highway Administration. https://www.fhwa.dot.gov/publications/research/ear/22026/index.cfm.

6

Key Point: The Exploratory Advanced Research Program seeks experts from government, academia, and industry to participate in the program from scouting of new topics through transition of research results.  

Notes

6

Questions?

7

Key Point:

Questions:

7

Disclaimer

The U.S. Government does not endorse products or manufacturers. Trademarks or manufacturers’ names appear in this presentation only because they are considered essential to the objective of the presentation. They are included for informational purposes only and are not intended to reflect a preference, approval, or endorsement of any one product or entity.

8

# image4.jpeg

# image1.png

# image2.svg

# image3.png

# image10.png

# image11.jpeg

# image6.png

# image7.png

# image12.png

# image13.png

# image14.png

# image8.jpeg