NIST Workshop

September 8, 2023

Digital Infrastructure Research in AZ

Jeffrey Wishart, Ph.D.
2

Overview

• The IAM has been researching Digital 

Infrastructure and how it can be used to 

improve intersection safety. 

• SMARTDRIVE Testbed in Anthem, AZ. Data 

collected from:

• Infrastructure-based cameras

• Drone-based camera

• Ground-based cameras

• Differential GPS

• Infrastructure-based LIDAR

• SPaT

• Next Data Collection Day: October 31, 2023

• Infrastructure-based RADAR

• ADS-equipped vehicle perception system
3

Camera-Based Detection and Tracking Algorithm

https://youtube.com/clip/UgkxBlIia4F2ZbxM7NbNLLrlWliTRNB1jBIc
4

SAE J3237 Driving Assessment Metrics

• Defined set is a mix of existing, 

adapted, and novel metrics

• Each metric has:

1. Taxonomy (Black/Grey/White)

2. Leading/Lagging

3. Origin

4. Definition

5. Observable Variables

6. Formulation

7. Subjective Assumptions/Thresholds 

8. Applicability

9. Example Usage

• SAE J3237 Recommended Practice 

(V&V TF under ORAD Committee)

• Black Box Metrics can be used for 

human- and ADS-driven vehicles.

* The Safety Envelope Metrics are intended to be neutral to the Safety 

Envelope Formulation selected to calculate the spatio-temporal boundaries.
5

Location Selection

• The IAM is currently identifying additional, prioritized 

locations for future digital infrastructure. 

Criteria of interest include:

• Equipped with infrastructure-based cameras

• VRU prevalence

• AV testing presence

• Collision incidents

• Traffic volume

• Weather diversity

• Extreme glare locations

• Some 3,262 signalized locations have been identified with existing cameras.

• Prioritized list being created, and locations currently being selected.
6

Intersection Data Collection Issues

There are several issues that must be considered when taking data at an intersection:

1. Measurement Objectives

2. Sensor Modalities

i. Measurement Uncertainty

ii. Lighting Conditions

iii. Environmental Conditions

iv. Sensor Fusion

3. Detection and Tracking Algorithms

4. Occlusions and Odd Shapes

5. Sensor Movement

6. Compute:

i. Post hoc 

ii. Real time: Edge, Cloud, or NOC
7

Next Steps

• Trial additional sensor modalities and suites

• Refine sensor fusion and detection and tracking algorithms

• Identify additional AZ-specific use cases

• Install digital infrastructure at selected locations

• Identify and trial warning techniques

• Submit entry/application to ARPA-I Intersection Safety Challenge and USDOT 

SMART grant

• Support the development of a national database of use cases and validated 

digital infrastructure solutions
8

IAM Publications

1. Wishart, J., Como, S., Elli, M., Russo, B. Weast, J., et al. (2020). “Driving Safety Performance Assessment Metrics for ADS-Equipped Vehicles,” SAE Technical 

Paper 2020-01-1206, 2020, doi:10.4271/2020-01-1206.

2. Elli, M., Wishart, J., Como, S., Dhakshinamoorthy, S., and Weast, J. (2021). “Evaluation of Operational Safety Assessment(OSA) Metrics for Automated Vehicles 

in Simulation,” SAE Technical Paper 2021-01-0868, doi:10.4271/2021-01-0868.

3. Altekar, N., Wishart, J., Como, S., Bruyere, D., Head, L., and Saleem, F. (2021)., “Infrastructure-based Sensor Data Capture Systems for Measurement of 

Operational Safety Assessment (OSA) Metrics,” SAE Technical Paper 2021-01-0175, doi:10.4271/2021-01-0175.

4. Lu, D., Jammula, V., Como, S., Wishart, J., Elli, M, Chen, Y., and Yang, Y., “CAROM – Vehicle Localization and Traffic Scene Reconstruction from Monocular 

Cameras on Road Infrastructure,” 2021 ICRA conference.

5. Weast, J., Elli, M., Alvarez, I., and Kovesdy, S., “To Err is Human: The Role of Human Safety Metrics in an Age of Automated Vehicles”, Accepted for 2021 SAE 

WCX.

6. Kidambi, N., Wishart, J., Elli, M., and Como, S. (2022). “Sensitivity of Automated Vehicle Operational Safety Assessment (OSA) Metrics to Measurement and 

Parameter Uncertainty,” SAE Technical Paper 2022-01-0815, doi:10.4271/2022-01-0815.

7. Como, S., Wishart, J., Elli, M., Kidambi, N. (2022). “Evaluating the Severity of Safety Envelope Violations in the Proposed Operational Safety Assessment (OSA) 

Methodology for Automated Vehicles,” SAE Technical Paper 2022-01-0819, doi:10.4271/2022-01-0819.

8. Jammula, V., Wishart, J., Yang, Y. (2022). “Evaluation of Operational Safety Assessment (OSA) Metrics for Automated Vehicles Using Real-World Data,” SAE 

Technical Paper 2022-01-0062, doi:10.4271/2022-01-0062.

9. Srinivasan, A., Mahartayasa, Y., Jammula, V., Lu, D., Como, S., Wishart, J., Yang, Y., and Yu, H. (2022). “Infrastructure-Based LiDAR Monitoring for Assessing 

Automated Driving Safety,” SAE Technical Paper 2022-01-0081, doi:10.4271/2022-01-0081.

10. Das, S., Yu, H., Rath, P., Wishart, J., Smith, T. (2023). “Comparison of Infrastructure- and Onboard Vehicle-Based Sensor Systems in Measuring Safety Metrics,” 

SAE Technical Paper 2023-01-0858, doi:10.4271/2023-01-0858.

11. Como, S., Wishart, J. (2023). “Evaluating Automated Vehicle Scenario Navigation using the Operational Safety Assessment (OSA) Methodology,” SAE Technical 

Paper 2023-01-0797, doi:10.4271/2023-01-0797.

12. Lu, D., Eaton, E., Weg, M., Wang, W., Como, S., Wishart, J., Yu, H., Yang, Y. (2023). “CAROM Air - Vehicle Localization and Traffic Scene Reconstruction from 

Aerial Video,” 2023 ICRA conference.

13. Chen, X., Wang, H., Razi, A., Russo, B., Pacheco, J., Roberts, J., Wishart,, J., Head, L., Baca, A. (2022). “Network-level Safety Metrics for Overall Traffic Safety 

Assessment: A Case Study,” IEEE Access, Vol. 11, pp. 17756-17778.
9

Thank You

• Jeffrey Wishart, PhD

jeffw@azcommerce.com