# Surrogate Safety Analysis of Pedestrian-Vehicle Conflict at Intersections Using Unmanned Aerial Vehicle Videos

Authors: Peng Chen; Weiliang Zeng; Guizhen Yu; Yunpeng Wang
Journal: Journal of Advanced Transportation
 (2017)

---

## Abstract

Conflict analysis using surrogate safety measures (SSMs) has become an efficient approach to investigate safety issues. The state-of-the-art studies largely resort to video images taken from high buildings. However, it suffers from heavy labor work, high cost of maintenance, and even security restrictions. Data collection and processing remains a common challenge to traffic conflict analysis. Unmanned Aerial Systems (UASs) or Unmanned Aerial Vehicles (UAVs), known for easy maneuvering, outstanding flexibility, and low costs, are considered to be a novel aerial sensor. By taking full advantage of the bird’s eye view offered by UAV, this study, as a pioneer work, applied UAV videos for surrogate safety analysis of pedestrian-vehicle conflicts at one urban intersection in Beijing, China. Aerial video sequences for a period of one hour were analyzed. The detection and tracking systems for vehicle and pedestrian trajectory data extraction were developed, respectively. Two SSMs, that is, Postencroachment Time (PET) and Relative Time to Collision (RTTC), were employed to represent how spatially and temporally close the pedestrian-vehicle conflict is to a collision. The results of analysis showed a high exposure of pedestrians to traffic conflict both inside and outside the crosswalk and relatively risking behavior of right-turn vehicles around the corner. The findings demonstrate that UAV can support intersection safety analysis in an accurate and cost-effective way.


---

## Body

## 1. Introduction

Pedestrian safety at intersections remains a critical issue. With the dramatic increasing of urban traffic flow, the major threat to pedestrians comes from frequent interactions with turning vehicles at crosswalk. Though crosswalks are operated to give pedestrians prioritized right of way over vehicles, still around 30% of the total traffic accident fatalities in China are pedestrians according to the accident statistics from the Ministry of Public Security of China [1]. The National Highway Traffic Safety Administration Report [2] indicated that pedestrians account for 15% of the fatalities in traffic accidents in the US. Japan National Policy Agency [3] stated that more than one-third of the total traffic accident fatalities in Japan are pedestrians at signalized and unsignalized crosswalks. Pedestrian safety has become a major concern worldwide.So far the reactive strategies for the purpose of improving pedestrian safety have been primarily based on identifying sites with high crash rates. It is subject to less crash records or validity losing due to changes of road system and operation. On the other hand, traffic conflict technique (TCT) represents an efficient approach to enable a preventive strategy development. Surrogate safety measures (SSMs) serve as near-crash indicators to measure spatial and temporal proximity of road users. In the context of safety assessment and improvement of urban intersections, the conflict between pedestrians and turning vehicles needs special attention. However, there are still limited applications of SSM on pedestrian-vehicle conflict assessment [4]. One possible reason is that pedestrian exposure to the risk of collision is difficult to measure directly, since this would involve tracking the movements of all people at all time [5]. Data collection and processing remains a common challenge to pedestrian studies.Field surveys of pedestrian-vehicle conflict are costly to conduct and suffer from inter- and intraobserver variability for the repeatability and consistency of results [6]. Video detection, as alternative data collection procedure to relieve the issues and limitations of manual data collection, has attracted considerable interest. It provides a reliable way to collect road users’ positions in time and space, that is, trajectories, that benefit the detailed analysis of pedestrian-vehicle conflict [7]. However, this method is relatively expensive and in practice it is difficult to collect and process video data at a large scale over a long period of time. In order to enable a view of both pedestrians and conflicting vehicles at the monitored intersection, video cameras should be installed high enough, for example, mounted on existing poles located near the intersection. This usually brings heavy labor work and high cost of maintenance and even is not allowed due to security restrictions. Furthermore, the synchronization among multiple cameras for one intersection is complicated and requires much extra effort. Last, to extract pedestrian and vehicle trajectory data from videos with a desirable accuracy and efficiency remains a difficult problem.Unmanned Aerial Systems (UASs) or Unmanned Aerial Vehicles (UAVs), known for easy maneuvering, outstanding flexibility, and low costs, are considered to be a novel aerial sensor. UAVs can be launched and deployed within minutes and exchange with the control center in real time. While in the last decade UAVs have been frequently employed in the military, civilian applications of UAVs still face several technical and institutional barriers, for example, strict airspace and route restrictions. In recent years, an increasing number of countries such as China and US have begun to consider and evaluate flexible air traffic control rules. For instance, the China Air Traffic Control Center promised to open up the low attitude space (lower than 1000 m) management in the following years. Such emerging trend presents a great opportunity for the transportation departments to fully explore the potential of UAVs in road traffic network surveillance. The equipped sensors on the UAVs such as high-resolution camera, radar, and infrared camera can provide bird’s eye view over an intersection or a large area. The entire images and video can be further processed to monitor traffic flow interaction and evaluate traffic state evolution. Thus, UAVs can be an effective aerial traffic information gathering platform. This study will investigate the potential of applying UAV videos for surrogate safety analysis of pedestrian-vehicle conflicts in an accurate and cost-effective way. To the best of the authors’ knowledge, it will be the first attempt to employ UAVs for detailed safety assessment at intersections.The remainder of the paper is organized as follows. A thorough literature review on UAV applications in transportation engineering and operation as well as SSMs for pedestrian-vehicle conflict assessment is presented first. Then the process of data acquisition using UAVs is introduced and the procedures of trajectory extraction are elaborated. Next, postextracted SSMs at one urban intersection in Beijing, China, are investigated in detail by referring to intersection geometry, traffic volume, and signal control strategy. Last, conclusions are drawn and recommendations are provided for future consideration.


## 2. Literature Review

### 2.1. UAV Applications in Transportation Engineering and Operation

UAV, as an aerial traffic information gathering platform, has been becoming more prominent in transportation engineering and operation. For instance, by utilizing aerial images captured from UAVs, the Washington State Department of Transportation evaluated the use of a UAV as an avalanche control tool on mountain slopes above state highways [8]. The Michigan Department of Transportation tested five main UAV platforms with a combination of optical, thermal, and LiDAR sensors to assess critical transportation infrastructure and issues such as bridges, confined spaces, traffic flow, and roadway assets [9]. The Utah Department of Transportation examined the use of high-resolution aerial photography obtained from UAVs to monitor and document State Roadway structures and associated issues [10]. The Florida Department of Transportation investigated the feasibility of using surveillance video from UAVs for traffic control and incident management [11].Perhaps the most important role that UAVs could fill is providing a rapid response to incidents [12]. Since time of response is vital to victim survivability and eventual health state, a UAV could fly directly to an incident ahead of emergency responders. The timely aerial video images transmitted back to the operators will allow rapid assessment of the situation and proper allocation of emergency response resources. Besides emergency-based applications, UAVs are also valuable for traffic management and monitoring applications. Coifman et al. (2006) demonstrated several applications by using data from a UAV flying in an urban environment, for example, determining level of service, estimating average annual daily travel, documenting intersection operations, and measuring Origin-Destination flows. Cheng et al. [13] presented a method for detecting and counting vehicles from UAV video flow. Hart and Gharaibeh [14] used micro-UAVs as a tool for collecting condition and inventory data for roadside infrastructure assets. Yu and David [15] investigated the feasibility of using high-resolution images acquired by the small UAV in work zone management, traffic congestion, safety, and environmental impact studies.The spatial perspectives offered by UAVs from the air demonstrate to be more promising than presently available ground-based views for traffic management and monitoring. Useful information can be derived from UAV video for both offline planning and real-time management. To this end, vision-based detection and frame-to-frame matching to track road users are important. However, in practice accurate detection and tracking from the UAV platform is a challenging task due to platform motion, image instability, the relatively small size of the objects, varied appearance, and so forth. Such technical issues may impose limitations to transportation professionals in a variety of intensive research and applies uses. Recently, by using UAV images, Xu et al. [16] developed a new hybrid vehicle detection scheme which integrated the Viola-Jones and linear Support Vector Machine (SVM) classifier with Histogram of Oriented Gradient (HOG) feature methods; Ma et al. [17] developed a pedestrian detection and tracking system using thermal infrared images recorded from UAVs. The proposed detection and tracking approaches would facilitate a more detailed analysis of road users’ behavior and interaction based on accurate trajectory data extracted from UAV video. As an extension of the above work, the aim of this paper is to apply UAV video for surrogate safety analysis of pedestrian-vehicle conflict at intersections.


### 2.2. SSMs for Pedestrian-Vehicle Conflict Assessment

As an alternative to crash risk estimation based on limited crash data, SSMs serve as near-crash indicators to measure the severity and frequency of traffic conflict events. Numerous SSMs have been suggested for safety evaluation of traffic facilities as shown in Allen at el. [18], Gettman and Head [19], and HSM [20]. In general, a SSM is supposed to satisfy two conditions in order to be useful for safety applications [4]: (1) a measurable or observable noncrash event that is physically related in a predictable and reliable way to crashes and (2) a practical method for converting or calibrating the noncrash event into a corresponding crash frequency and/or severity.In the case of pedestrian-vehicle conflict at intersections, turning vehicles typically have to filter through conflicting pedestrian flow at crosswalk during permitted signal phase as implemented in China and US. Under the mixed impact of surrounding environment, crosswalk geometry, signal operation, and pedestrians moving in different directions, turning vehicles might take risky behavior by not yielding to pedestrians or passing through small gaps in pedestrian flow, which poses a threat to pedestrian safety. The most commonly used SSMs for pedestrian conflict assessment include but not limited to the following measures:(i)
Time to Collision (TTC), which is defined as the time that remains until a collision between two road users would have occurred if the collision course and speed difference are maintained [21].(ii)
Postencroachment Time (PET), which is defined as the time difference between the moment when an offending road user leaves an area of potential collision and the moment of arrival of a conflicted road user possessing the right of way [22].(iii)
Time to Zebra (TTZ), which is a variation of TTC in order to estimate frequency and severity of critical encounters between crossing pedestrians and vehicles that are approaching the crosswalk [23].(iv)
Deceleration-to-Safety Time (DST), which is the necessary deceleration to reach a nonnegative PET value if the movement of the conflicting road users remains unchanged [24].(v)
Gap Time (GT), which is defined as the time lapse between the completion time of encroachment by one road user and the arrival time of the interacting road user if they continue with the same speed and path [25].In general, Allen et al. [18], Gettman and Head [19], and Gettman et al. [26] found that TTC and PET are ranked as the most accurate measures for the analysis of safety at intersections in light of ease of measurement, consistency over time, and relation to other measures. TTC requires estimating the time remaining to the conflict point at each time instant in the case of pedestrian-vehicle conflict, while to measure PET, only their passing times at conflict point are necessary. Due to its simplicity, PET is also amenable to automated measurement methods using techniques such as video image processing. Another important property of PET is that it is continuous from crash-free operations to crash occurrences with a distinct boundary at zero. The smaller value of PET implies a greater risk of vehicle-pedestrian collisions. In practice, Songchitruksa and Tarko [27] demonstrated the usefulness of the number of short PETs in explaining the variability of crash counts and concluded that the frequency of short PETs is a potential indicator in discriminating varying safety levels across survey sites.However, PET has inherent drawbacks in its ability to accurately capture conflict severity [6]. For example, the events in which the approaching vehicle decelerated to near-stop to avoid collision with the conflicting pedestrian may have PET values that do not reflect that true severity of the interaction. On the other hand, the main advantage of TTC is its ability to capture the severity of an interaction in an objective and quantitative way. Thus, a combination of these SSMs would be necessary to help identify all the dangerous interactions between vehicles and pedestrians.


## 2.1. UAV Applications in Transportation Engineering and Operation

UAV, as an aerial traffic information gathering platform, has been becoming more prominent in transportation engineering and operation. For instance, by utilizing aerial images captured from UAVs, the Washington State Department of Transportation evaluated the use of a UAV as an avalanche control tool on mountain slopes above state highways [8]. The Michigan Department of Transportation tested five main UAV platforms with a combination of optical, thermal, and LiDAR sensors to assess critical transportation infrastructure and issues such as bridges, confined spaces, traffic flow, and roadway assets [9]. The Utah Department of Transportation examined the use of high-resolution aerial photography obtained from UAVs to monitor and document State Roadway structures and associated issues [10]. The Florida Department of Transportation investigated the feasibility of using surveillance video from UAVs for traffic control and incident management [11].Perhaps the most important role that UAVs could fill is providing a rapid response to incidents [12]. Since time of response is vital to victim survivability and eventual health state, a UAV could fly directly to an incident ahead of emergency responders. The timely aerial video images transmitted back to the operators will allow rapid assessment of the situation and proper allocation of emergency response resources. Besides emergency-based applications, UAVs are also valuable for traffic management and monitoring applications. Coifman et al. (2006) demonstrated several applications by using data from a UAV flying in an urban environment, for example, determining level of service, estimating average annual daily travel, documenting intersection operations, and measuring Origin-Destination flows. Cheng et al. [13] presented a method for detecting and counting vehicles from UAV video flow. Hart and Gharaibeh [14] used micro-UAVs as a tool for collecting condition and inventory data for roadside infrastructure assets. Yu and David [15] investigated the feasibility of using high-resolution images acquired by the small UAV in work zone management, traffic congestion, safety, and environmental impact studies.The spatial perspectives offered by UAVs from the air demonstrate to be more promising than presently available ground-based views for traffic management and monitoring. Useful information can be derived from UAV video for both offline planning and real-time management. To this end, vision-based detection and frame-to-frame matching to track road users are important. However, in practice accurate detection and tracking from the UAV platform is a challenging task due to platform motion, image instability, the relatively small size of the objects, varied appearance, and so forth. Such technical issues may impose limitations to transportation professionals in a variety of intensive research and applies uses. Recently, by using UAV images, Xu et al. [16] developed a new hybrid vehicle detection scheme which integrated the Viola-Jones and linear Support Vector Machine (SVM) classifier with Histogram of Oriented Gradient (HOG) feature methods; Ma et al. [17] developed a pedestrian detection and tracking system using thermal infrared images recorded from UAVs. The proposed detection and tracking approaches would facilitate a more detailed analysis of road users’ behavior and interaction based on accurate trajectory data extracted from UAV video. As an extension of the above work, the aim of this paper is to apply UAV video for surrogate safety analysis of pedestrian-vehicle conflict at intersections.


## 2.2. SSMs for Pedestrian-Vehicle Conflict Assessment

As an alternative to crash risk estimation based on limited crash data, SSMs serve as near-crash indicators to measure the severity and frequency of traffic conflict events. Numerous SSMs have been suggested for safety evaluation of traffic facilities as shown in Allen at el. [18], Gettman and Head [19], and HSM [20]. In general, a SSM is supposed to satisfy two conditions in order to be useful for safety applications [4]: (1) a measurable or observable noncrash event that is physically related in a predictable and reliable way to crashes and (2) a practical method for converting or calibrating the noncrash event into a corresponding crash frequency and/or severity.In the case of pedestrian-vehicle conflict at intersections, turning vehicles typically have to filter through conflicting pedestrian flow at crosswalk during permitted signal phase as implemented in China and US. Under the mixed impact of surrounding environment, crosswalk geometry, signal operation, and pedestrians moving in different directions, turning vehicles might take risky behavior by not yielding to pedestrians or passing through small gaps in pedestrian flow, which poses a threat to pedestrian safety. The most commonly used SSMs for pedestrian conflict assessment include but not limited to the following measures:(i)
Time to Collision (TTC), which is defined as the time that remains until a collision between two road users would have occurred if the collision course and speed difference are maintained [21].(ii)
Postencroachment Time (PET), which is defined as the time difference between the moment when an offending road user leaves an area of potential collision and the moment of arrival of a conflicted road user possessing the right of way [22].(iii)
Time to Zebra (TTZ), which is a variation of TTC in order to estimate frequency and severity of critical encounters between crossing pedestrians and vehicles that are approaching the crosswalk [23].(iv)
Deceleration-to-Safety Time (DST), which is the necessary deceleration to reach a nonnegative PET value if the movement of the conflicting road users remains unchanged [24].(v)
Gap Time (GT), which is defined as the time lapse between the completion time of encroachment by one road user and the arrival time of the interacting road user if they continue with the same speed and path [25].In general, Allen et al. [18], Gettman and Head [19], and Gettman et al. [26] found that TTC and PET are ranked as the most accurate measures for the analysis of safety at intersections in light of ease of measurement, consistency over time, and relation to other measures. TTC requires estimating the time remaining to the conflict point at each time instant in the case of pedestrian-vehicle conflict, while to measure PET, only their passing times at conflict point are necessary. Due to its simplicity, PET is also amenable to automated measurement methods using techniques such as video image processing. Another important property of PET is that it is continuous from crash-free operations to crash occurrences with a distinct boundary at zero. The smaller value of PET implies a greater risk of vehicle-pedestrian collisions. In practice, Songchitruksa and Tarko [27] demonstrated the usefulness of the number of short PETs in explaining the variability of crash counts and concluded that the frequency of short PETs is a potential indicator in discriminating varying safety levels across survey sites.However, PET has inherent drawbacks in its ability to accurately capture conflict severity [6]. For example, the events in which the approaching vehicle decelerated to near-stop to avoid collision with the conflicting pedestrian may have PET values that do not reflect that true severity of the interaction. On the other hand, the main advantage of TTC is its ability to capture the severity of an interaction in an objective and quantitative way. Thus, a combination of these SSMs would be necessary to help identify all the dangerous interactions between vehicles and pedestrians.


## 3. Methodology

### 3.1. Detection and Tracking

In order to investigate pedestrian-vehicle conflict, road users should be detected and then tracked frame-to-frame in UAV video. In this study, we extract the trajectories for vehicles and pedestrians, respectively, at intervals of every 0.04 s by using the detection and tracking system developed in our previous studies [16, 17]. A brief introduction is provided below.An image processing system for automated vehicle trajectory extraction was developed based on UAV videos [16]. As shown in Figure 1, the system mainly includes three modules: (1) video stabilization; (2) vehicle detection; and (3) vehicle tracking. Note that, due to UAV motions, image registration algorithm [29] was first applied to stabilize UAV videos. The object detection framework of faster R-CNN [30] was applied for vehicle detection. Then the algorithm of kernelized correlation filters (KCF) [31] was applied for vehicle tracking. The trajectory of one vehicle will be derived after the tracking was finished.Figure 1
The workflow of vehicle detection and tracking.Similar to the work of Beymer et al. [32], several entry and exit regions were set as shown in Figure 2. Commonly, the entry and exit regions are set at the upstream and downstream of a signalized intersection. Once a vehicle entered the entry region it will be detected and tracked; the tracking will be finished until the vehicle entered the exit region. Left-turn and right-turn vehicles can be distinguished by the information of entry and exit included in the trajectories. For example, trajectories of left-turn vehicles from the southbound approach can be obtained between Entry 1 and Exit 2 by distinguishing from through and U-turn vehicles as shown in Figure 2(a); similarly, trajectories of right-turn vehicles from the northbound approach can be obtained between Entry 2 and Exit 2 as in Figure 2(b).Figure 2
The configuration of entry and exit regions for turning vehicles.
(a)
Left-turn
(b)
Right-turnPedestrian detection and tracking from the UAV platform is a challenging task due to the small size of the objects and the high-density crowd. A semiautomatic pedestrian detection and tracking system was developed for pedestrian trajectory data extraction from UAV aerial images [17]. The developed system consists of two components, that is, detector and tracker. The detector is responsible for automatic aerial pedestrian detection. The tracker is used for pedestrian tracking and extracting trajectory coordinate data. Figure 3 illustrates the overall workflow of the system. First, pedestrians are detected by detector in detection area (green region as shown in Figure 3), which is manually set up according to the crosswalk location. A general and machine learning-based method is employed for constructing pedestrian detector. It includes two stages, that is, pedestrian feature descriptor extraction and pedestrian classification. HOG feature, as a kind of local gradient feature, has been proved to perform well in pedestrian detection problems [17, 33]. Hence, our pedestrian detector employs HOG feature as pedestrian descriptor. By utilizing multisize sliding window method, detector scans the detection area. For each window, a linear SVM classifier [34] is used to classify window as pedestrian or nonpedestrian. When pedestrians are detected, their coordinates will be fed as inputs for initialization of trackers. Then the trackers apply the pyramidal Lucas–Kanade method [35] to compute the local sparse optical flow, together with a secondary detection in the search region for correcting the drift when tracking pedestrians. A tracker is responsible for only one pedestrian object. After tracking, the pedestrian trajectory data can be saved for further analysis. Figure 4 shows the pedestrian tracking results and trajectory visualization.Figure 3
The workflow of pedestrian detection and tracking.Figure 4
(a) Tracking trajectory visualization and (b) tracking point visualization.
(a)
(b)Note that the tracked positions or trajectories might contain measurement errors. Kalman filtering (KF) was used to correct the errors and smooth the raw trajectory data. The KF computes the best estimate of the state vector (i.e., position coordinates) by minimizing the squared error according to the estimation of the past state and the present state. The image coordinates were converted to geographic coordinates by projective transformation.The available observations are trajectory profiles based on time series. From these data, all relevant quantities of vehicles and pedestrians, such as positions, velocities, and acceleration, can be derived either directly or by applying finite differences. The ordinary differential equations for speed and acceleration can be solved as follows:(1)v→αt=v→αxt,v→αyt=xαt+1-xαt-12Δt,yαt+1-yαt-12Δt,f→αt=f→αxt,f→αyt=xαt+1-xαt-xαt-xαt-1Δt2,yαt+1-yαt-yαt-yαt-1Δt2,where v→αt is the speed vector for vehicle/pedestrian α at time t, f→αt is the acceleration vector for α at time t, xα and yα are the positions in the x and y directions, respectively, and Δt is the time interval for trajectory extraction, that is, 0.04 s.


### 3.2. SSM Measurement

Traditional traffic conflict technique usually use PET and TTC to represent the probability of collision or how close the conflict is to a collision [7, 36].In the context of vehicle conflict assessment, PET is defined as the time difference between the moment when the first vehicle passed the conflict area and the moment of arrival of the second vehicle subsequently at the same area. In the context of vehicle-pedestrian conflict assessment, PET can be similarly defined as the time difference between the departure of the encroaching pedestrian from the potential collision point and the arrival of the conflicting vehicle at the point, or vice versa. However, as PET only considers the last moment of the interaction, it has limitations in indicating pedestrian safety during the course of vehicle-pedestrian interaction.Alternatively, TTC has been commonly implemented as a measure of conflict severity for the whole interaction process. It was originally defined as the time that remains for the paired vehicles before they collide, if both continue at their present speeds along their respective trajectories. TTC can be easily detected in the rear-end conflict situation because the trajectories of the paired vehicles are assumed to be overlapped. However, it cannot be detected (or does not exist) in most of the interactions if the trajectories of the paired users intersect, for example, the pedestrian-vehicle conflict and the conflict between left-turn and opposing through vehicles. In the rear-end conflict, the following vehicle will definitely collide with the leader vehicle if the speed of the follower is higher. However, for the pedestrian-vehicle conflict, the cases that the pedestrian and the vehicle occupy the trajectory intersection point at the same moment are rare. To overcome this problem, we use the Relative Time to Collision (RTTC) as the indicator to measure the conflict severity. As shown in Figure5, RTTC is defined as the time difference between the first road user arriving at the potential conflicting location and the second road user arriving at this location if they keep their current speeds. It should be noted that the TTC can be detected only when the RTTC equals zero. The RTTC can be formulated as follows: (2)RTTCt=P→ct-P→vtv→vt-P→ct-P→ptv→pt,where P→c(t) is the potential conflicting point at the moment t, which is the intersecting point determined by the moving directions of two road users [37], as shown in Figure 5; P→vt is the location of the vehicle at the moment t; P→pt is the location of the pedestrian at the moment t; v→vt is the speed of the vehicle at the moment t; v→pt is the speed of the pedestrian at the moment t.Figure 5
RTTC and PET of pedestrian-vehicle conflict identified on the time-space diagram (modified from [28]).In data processing, both RTTC and PET are obtained as a function of paired vehicle-pedestrian speeds and spacing. A time-space diagram identifying RTTC and PET for a pedestrian-vehicle conflict event is illustrated in Figure5. The trajectories of the crossing pedestrian and the turning vehicle are represented by curve A and curve B, respectively. In such cases, RTTC at instant t1 can be obtained as(3)RTTCt1=t5-t3.The PET for such a conflict event can be obtained as(4)PET=t4-t2Note that RTTC is instant varying and continually calculated between conflicting vehicles and pedestrians. Thus, a set of RTTC values will be obtained for each conflict. The minimum RTTC (RTTCmin) can be extracted from this set to indicate the maximum severity of this interaction. In this study, only traffic events with associated minimum RTTC of less than 3 s are considered for safety assessment. This value was selected by considering the close proximity of road users in space and time based on the work of Sayed and Zein [38]. On the other hand, by referring to the empirical work by Ni et al. [36], only the conflicting events with PET values of less than 3 s are included for safety assessment.


## 3.1. Detection and Tracking

In order to investigate pedestrian-vehicle conflict, road users should be detected and then tracked frame-to-frame in UAV video. In this study, we extract the trajectories for vehicles and pedestrians, respectively, at intervals of every 0.04 s by using the detection and tracking system developed in our previous studies [16, 17]. A brief introduction is provided below.An image processing system for automated vehicle trajectory extraction was developed based on UAV videos [16]. As shown in Figure 1, the system mainly includes three modules: (1) video stabilization; (2) vehicle detection; and (3) vehicle tracking. Note that, due to UAV motions, image registration algorithm [29] was first applied to stabilize UAV videos. The object detection framework of faster R-CNN [30] was applied for vehicle detection. Then the algorithm of kernelized correlation filters (KCF) [31] was applied for vehicle tracking. The trajectory of one vehicle will be derived after the tracking was finished.Figure 1
The workflow of vehicle detection and tracking.Similar to the work of Beymer et al. [32], several entry and exit regions were set as shown in Figure 2. Commonly, the entry and exit regions are set at the upstream and downstream of a signalized intersection. Once a vehicle entered the entry region it will be detected and tracked; the tracking will be finished until the vehicle entered the exit region. Left-turn and right-turn vehicles can be distinguished by the information of entry and exit included in the trajectories. For example, trajectories of left-turn vehicles from the southbound approach can be obtained between Entry 1 and Exit 2 by distinguishing from through and U-turn vehicles as shown in Figure 2(a); similarly, trajectories of right-turn vehicles from the northbound approach can be obtained between Entry 2 and Exit 2 as in Figure 2(b).Figure 2
The configuration of entry and exit regions for turning vehicles.
(a)
Left-turn
(b)
Right-turnPedestrian detection and tracking from the UAV platform is a challenging task due to the small size of the objects and the high-density crowd. A semiautomatic pedestrian detection and tracking system was developed for pedestrian trajectory data extraction from UAV aerial images [17]. The developed system consists of two components, that is, detector and tracker. The detector is responsible for automatic aerial pedestrian detection. The tracker is used for pedestrian tracking and extracting trajectory coordinate data. Figure 3 illustrates the overall workflow of the system. First, pedestrians are detected by detector in detection area (green region as shown in Figure 3), which is manually set up according to the crosswalk location. A general and machine learning-based method is employed for constructing pedestrian detector. It includes two stages, that is, pedestrian feature descriptor extraction and pedestrian classification. HOG feature, as a kind of local gradient feature, has been proved to perform well in pedestrian detection problems [17, 33]. Hence, our pedestrian detector employs HOG feature as pedestrian descriptor. By utilizing multisize sliding window method, detector scans the detection area. For each window, a linear SVM classifier [34] is used to classify window as pedestrian or nonpedestrian. When pedestrians are detected, their coordinates will be fed as inputs for initialization of trackers. Then the trackers apply the pyramidal Lucas–Kanade method [35] to compute the local sparse optical flow, together with a secondary detection in the search region for correcting the drift when tracking pedestrians. A tracker is responsible for only one pedestrian object. After tracking, the pedestrian trajectory data can be saved for further analysis. Figure 4 shows the pedestrian tracking results and trajectory visualization.Figure 3
The workflow of pedestrian detection and tracking.Figure 4
(a) Tracking trajectory visualization and (b) tracking point visualization.
(a)
(b)Note that the tracked positions or trajectories might contain measurement errors. Kalman filtering (KF) was used to correct the errors and smooth the raw trajectory data. The KF computes the best estimate of the state vector (i.e., position coordinates) by minimizing the squared error according to the estimation of the past state and the present state. The image coordinates were converted to geographic coordinates by projective transformation.The available observations are trajectory profiles based on time series. From these data, all relevant quantities of vehicles and pedestrians, such as positions, velocities, and acceleration, can be derived either directly or by applying finite differences. The ordinary differential equations for speed and acceleration can be solved as follows:(1)v→αt=v→αxt,v→αyt=xαt+1-xαt-12Δt,yαt+1-yαt-12Δt,f→αt=f→αxt,f→αyt=xαt+1-xαt-xαt-xαt-1Δt2,yαt+1-yαt-yαt-yαt-1Δt2,where v→αt is the speed vector for vehicle/pedestrian α at time t, f→αt is the acceleration vector for α at time t, xα and yα are the positions in the x and y directions, respectively, and Δt is the time interval for trajectory extraction, that is, 0.04 s.


## 3.2. SSM Measurement

Traditional traffic conflict technique usually use PET and TTC to represent the probability of collision or how close the conflict is to a collision [7, 36].In the context of vehicle conflict assessment, PET is defined as the time difference between the moment when the first vehicle passed the conflict area and the moment of arrival of the second vehicle subsequently at the same area. In the context of vehicle-pedestrian conflict assessment, PET can be similarly defined as the time difference between the departure of the encroaching pedestrian from the potential collision point and the arrival of the conflicting vehicle at the point, or vice versa. However, as PET only considers the last moment of the interaction, it has limitations in indicating pedestrian safety during the course of vehicle-pedestrian interaction.Alternatively, TTC has been commonly implemented as a measure of conflict severity for the whole interaction process. It was originally defined as the time that remains for the paired vehicles before they collide, if both continue at their present speeds along their respective trajectories. TTC can be easily detected in the rear-end conflict situation because the trajectories of the paired vehicles are assumed to be overlapped. However, it cannot be detected (or does not exist) in most of the interactions if the trajectories of the paired users intersect, for example, the pedestrian-vehicle conflict and the conflict between left-turn and opposing through vehicles. In the rear-end conflict, the following vehicle will definitely collide with the leader vehicle if the speed of the follower is higher. However, for the pedestrian-vehicle conflict, the cases that the pedestrian and the vehicle occupy the trajectory intersection point at the same moment are rare. To overcome this problem, we use the Relative Time to Collision (RTTC) as the indicator to measure the conflict severity. As shown in Figure5, RTTC is defined as the time difference between the first road user arriving at the potential conflicting location and the second road user arriving at this location if they keep their current speeds. It should be noted that the TTC can be detected only when the RTTC equals zero. The RTTC can be formulated as follows: (2)RTTCt=P→ct-P→vtv→vt-P→ct-P→ptv→pt,where P→c(t) is the potential conflicting point at the moment t, which is the intersecting point determined by the moving directions of two road users [37], as shown in Figure 5; P→vt is the location of the vehicle at the moment t; P→pt is the location of the pedestrian at the moment t; v→vt is the speed of the vehicle at the moment t; v→pt is the speed of the pedestrian at the moment t.Figure 5
RTTC and PET of pedestrian-vehicle conflict identified on the time-space diagram (modified from [28]).In data processing, both RTTC and PET are obtained as a function of paired vehicle-pedestrian speeds and spacing. A time-space diagram identifying RTTC and PET for a pedestrian-vehicle conflict event is illustrated in Figure5. The trajectories of the crossing pedestrian and the turning vehicle are represented by curve A and curve B, respectively. In such cases, RTTC at instant t1 can be obtained as(3)RTTCt1=t5-t3.The PET for such a conflict event can be obtained as(4)PET=t4-t2Note that RTTC is instant varying and continually calculated between conflicting vehicles and pedestrians. Thus, a set of RTTC values will be obtained for each conflict. The minimum RTTC (RTTCmin) can be extracted from this set to indicate the maximum severity of this interaction. In this study, only traffic events with associated minimum RTTC of less than 3 s are considered for safety assessment. This value was selected by considering the close proximity of road users in space and time based on the work of Sayed and Zein [38]. On the other hand, by referring to the empirical work by Ni et al. [36], only the conflicting events with PET values of less than 3 s are included for safety assessment.


## 4. Experiment Results and Analysis

### 4.1. Study Site

The selected study site is the intersection of Huayuan Road and Beitucheng Road in the Haidian District of Beijing, China. The intersection is located on a key route to the downtown area, characterized by higher vehicle volume and medium-to-high pedestrian demand during peak hours. For signal control, this intersection is fixed-time controlled with a cycle length of approximately 120 s. The yellow time durations are 3 s and the all-red durations are 1 s at all the approaches. The three-phase control plan is presented in Figure6. Note that, for the northbound and southbound approaches, both left-turn and right-turn phases are permitted, indicating that potential conflicts exist when turning vehicles filter through conflicting pedestrian streams at crosswalk. The length and width of the crosswalks in the north-south directions are 35 m and 5.5 m, respectively. The green phase for pedestrians at the two crosswalks is 45 s. The light volume of bicycles at the site ensures that bicycle interference with pedestrian flow can be roughly neglected.Figure 6
Signal phasing at study site.Experiments were conducted using aerial videos captured by an optical camera (Gopro Hero Black Edition 3) with a 1920 × 1080 resolution mounted on a quadrotor UAV (model: Phantom 2). Figure7 shows the basic components of the UAV platform. A 3-axis gimbal is mounted on the UAV to stabilize the videos and eliminate video jitters caused by UAV, thus greatly reducing the impact from external factors, for example, wind. Besides, an on-screen display, an image transmission module and a video monitor are installed in the system for data transmission and airborne flying status monitoring and control. The flexibility of this device enables wide coverage of the scene from a top-down view. The flight altitude was set approximately 100 m above the ground in this experiment. Aerial video was recorded by the UAV from 5 PM to 6 PM on 17 of April 2015.Figure 7
Quadrotor UAV for video collection.


### 4.2. Data Extraction

In total, the dataset consists of the trajectories of 1494 pedestrians and 282 right-turn vehicles. The visualization of the extracted trajectories is presented in Figure8. The significant variation of pedestrians and turning vehicles’ trajectories can be identified. It is worth mentioning that not a small number of pedestrians walked outside the crosswalk during pedestrian green phase and some rushed into crosswalks without necessarily heeding approaching turning vehicles during pedestrian flashing green phase, which may increase the occurrence probability of severe conflicts. Besides, by taking the westbound approach as an example, Figure 9 compares the distributions of the extracted right-turn vehicle trajectories at three cross-sections. Though right-turning vehicles entered the intersection centered around the middle point of the through-right lane, as shown in Figure 9(a), the exiting positions of right-turning vehicles are widely distributed through the cross-sections 2 and 3, as in Figures 9(b) and 9(c). It may lead to widely distributed conflict points on crosswalk. In reality, pedestrians and vehicles behave by anticipating the behavior of each other to avoid collision. On signalized crosswalk, pedestrian trajectories are supposed to be under the interaction between pedestrian flow, conflicting vehicles, pedestrian signal control, intersection geometry, and so forth [37, 39]. Turning vehicle trajectories are also sensitive to intersection corner radius, turning angle (i.e., the angle between entering and exit approaches), and vehicle speed. The accurately extracted trajectories from UAV video offer a good basis for intersection safety assessment.Figure 8
Visualization of extracted trajectories.
(a)
Pedestrians
(b)
Right-turn vehiclesFigure 9
Comparison of extracted right-turn vehicle trajectories at cross-sections.
(a)
(b)
(c)


### 4.3. SSM Analysis

Based on extracted trajectory data, traffic conflicts between pedestrians and right-turn vehicles were identified and SSMs, that is, PET and RTTC, were calculated accordingly. In terms of SSMs, the conflict analysis aims to identify conflict frequency, severity and location (conflict points).The spatial distribution of small PETs (which are less than 3 s in this study) is shown in Figure10. A smaller PET value indicates a higher probability of collision occurring at the end of the vehicle-pedestrian interaction. It shows that most of the small PETs as well as the related conflict points are widely distributed on the area in front of downstream exiting approaches. The details of the statistics are shown in Figure 11. It is found that 57% of the small PETs occur inside the crosswalk, while 43% outside the crosswalk. Even though the pedestrians are supposed to walk inside the crosswalk, not a small number of them walk outside the crosswalk under complex interactions with opposing pedestrians and turning vehicles. If we look into the percentage of severe conflicts (PET < 1 s), it is found that 20% of the PETs are less than 1 s for the pedestrians walking inside the crosswalk, while there are 25% of the PETs that are less than 1 s for the pedestrians walking outside the crosswalk. It indicates that walking outside the crosswalk is more dangerous. A possible reason is that drivers might not recognize and yield to the pedestrians if the pedestrians do not walk inside the crosswalk. Figure 11 shows that almost 64% of the small PETs and 70% of the sever conflicts (PET < 1 s) occur at the northern and eastern crosswalks due to the large pedestrian flow.Figure 10
Spatial distribution of the number of small PETs.Figure 11
The number of small PETs inside and outside of the crosswalk.
(a)
North
(b)
South
(c)
East
(d)
WestDifferent from PET, RTTC reflects the potential conflict severity during the course of vehicle-pedestrian interaction. Figure12 presents the spatial distribution of pedestrian-vehicle conflicts with the minimum RTTC of less than 3 s by heat mapping. It clearly shows that most of the severe conflicts occur outside the crosswalk. Two reasons may lead to this phenomenon. First, not a small number of pedestrians walk outside the crosswalk, which may cause the severe conflict with right-turn vehicles. Second, the potential conflict point for calculating RTTC is determined by the intersection of the current speed vectors of the paired users, which may distribute sparsely due to the flexible change of pedestrian movement.Figure 12
Spatial distribution ofRTTCmin.In general, there are two types of vehicle-pedestrian conflict, that is, vehicle yielding to pedestrian, also known as pedestrian passing first (PPF), and pedestrian yielding to vehicle, also known as vehicle passing first (VPF). The PPF and VPF cases are compared because these two types of conflicts can result in different safety performance. Figure13 provides the details of statistics of the critical PPF and VPF (RTTC < 1 s) inside and outside the crosswalk. It is found that VPF occupies 55%, while PPF occupies 45% for the total critical RTTCs. It indicates that VPF is more dangerous than PPF. This is because both the vehicle and the pedestrian do not yield to each other and the vehicle tries to accelerate to pass the conflict area first. For the PPF cases, most of the vehicles decelerate to yield to the pedestrians, even though the vehicle may not totally stop and reaccelerate quickly once the pedestrians pass the conflict area. It was also found that a large percent of VPF occur outside the crosswalk at the northbound and eastbound due to the high traffic flow of right-turn vehicles and pedestrians at these two crosswalks. The statistics of PET in Figure 11 also show the potential risk at these crosswalks. Because drivers may not notice the pedestrian outside the crosswalk, the frequent critical confliction may lead to accidents. Thus, we propose the recommendation that a special pedestrian phase should be set to separate the right-turn vehicles from the southbound and eastbound. In practice, varying pedestrian and vehicular behavior may lead to misunderstanding of others’ decisions, which may result in safety problems. Considering widely distributed conflict points, both pedestrians and turning vehicles must pay attention to a broader area where conflicts may occur.Figure 13
The number of critical RTTC inside and outside of the crosswalk.
(a)
North
(b)
South
(c)
East
(d)
West


## 4.1. Study Site

The selected study site is the intersection of Huayuan Road and Beitucheng Road in the Haidian District of Beijing, China. The intersection is located on a key route to the downtown area, characterized by higher vehicle volume and medium-to-high pedestrian demand during peak hours. For signal control, this intersection is fixed-time controlled with a cycle length of approximately 120 s. The yellow time durations are 3 s and the all-red durations are 1 s at all the approaches. The three-phase control plan is presented in Figure6. Note that, for the northbound and southbound approaches, both left-turn and right-turn phases are permitted, indicating that potential conflicts exist when turning vehicles filter through conflicting pedestrian streams at crosswalk. The length and width of the crosswalks in the north-south directions are 35 m and 5.5 m, respectively. The green phase for pedestrians at the two crosswalks is 45 s. The light volume of bicycles at the site ensures that bicycle interference with pedestrian flow can be roughly neglected.Figure 6
Signal phasing at study site.Experiments were conducted using aerial videos captured by an optical camera (Gopro Hero Black Edition 3) with a 1920 × 1080 resolution mounted on a quadrotor UAV (model: Phantom 2). Figure7 shows the basic components of the UAV platform. A 3-axis gimbal is mounted on the UAV to stabilize the videos and eliminate video jitters caused by UAV, thus greatly reducing the impact from external factors, for example, wind. Besides, an on-screen display, an image transmission module and a video monitor are installed in the system for data transmission and airborne flying status monitoring and control. The flexibility of this device enables wide coverage of the scene from a top-down view. The flight altitude was set approximately 100 m above the ground in this experiment. Aerial video was recorded by the UAV from 5 PM to 6 PM on 17 of April 2015.Figure 7
Quadrotor UAV for video collection.


## 4.2. Data Extraction

In total, the dataset consists of the trajectories of 1494 pedestrians and 282 right-turn vehicles. The visualization of the extracted trajectories is presented in Figure8. The significant variation of pedestrians and turning vehicles’ trajectories can be identified. It is worth mentioning that not a small number of pedestrians walked outside the crosswalk during pedestrian green phase and some rushed into crosswalks without necessarily heeding approaching turning vehicles during pedestrian flashing green phase, which may increase the occurrence probability of severe conflicts. Besides, by taking the westbound approach as an example, Figure 9 compares the distributions of the extracted right-turn vehicle trajectories at three cross-sections. Though right-turning vehicles entered the intersection centered around the middle point of the through-right lane, as shown in Figure 9(a), the exiting positions of right-turning vehicles are widely distributed through the cross-sections 2 and 3, as in Figures 9(b) and 9(c). It may lead to widely distributed conflict points on crosswalk. In reality, pedestrians and vehicles behave by anticipating the behavior of each other to avoid collision. On signalized crosswalk, pedestrian trajectories are supposed to be under the interaction between pedestrian flow, conflicting vehicles, pedestrian signal control, intersection geometry, and so forth [37, 39]. Turning vehicle trajectories are also sensitive to intersection corner radius, turning angle (i.e., the angle between entering and exit approaches), and vehicle speed. The accurately extracted trajectories from UAV video offer a good basis for intersection safety assessment.Figure 8
Visualization of extracted trajectories.
(a)
Pedestrians
(b)
Right-turn vehiclesFigure 9
Comparison of extracted right-turn vehicle trajectories at cross-sections.
(a)
(b)
(c)


## 4.3. SSM Analysis

Based on extracted trajectory data, traffic conflicts between pedestrians and right-turn vehicles were identified and SSMs, that is, PET and RTTC, were calculated accordingly. In terms of SSMs, the conflict analysis aims to identify conflict frequency, severity and location (conflict points).The spatial distribution of small PETs (which are less than 3 s in this study) is shown in Figure10. A smaller PET value indicates a higher probability of collision occurring at the end of the vehicle-pedestrian interaction. It shows that most of the small PETs as well as the related conflict points are widely distributed on the area in front of downstream exiting approaches. The details of the statistics are shown in Figure 11. It is found that 57% of the small PETs occur inside the crosswalk, while 43% outside the crosswalk. Even though the pedestrians are supposed to walk inside the crosswalk, not a small number of them walk outside the crosswalk under complex interactions with opposing pedestrians and turning vehicles. If we look into the percentage of severe conflicts (PET < 1 s), it is found that 20% of the PETs are less than 1 s for the pedestrians walking inside the crosswalk, while there are 25% of the PETs that are less than 1 s for the pedestrians walking outside the crosswalk. It indicates that walking outside the crosswalk is more dangerous. A possible reason is that drivers might not recognize and yield to the pedestrians if the pedestrians do not walk inside the crosswalk. Figure 11 shows that almost 64% of the small PETs and 70% of the sever conflicts (PET < 1 s) occur at the northern and eastern crosswalks due to the large pedestrian flow.Figure 10
Spatial distribution of the number of small PETs.Figure 11
The number of small PETs inside and outside of the crosswalk.
(a)
North
(b)
South
(c)
East
(d)
WestDifferent from PET, RTTC reflects the potential conflict severity during the course of vehicle-pedestrian interaction. Figure12 presents the spatial distribution of pedestrian-vehicle conflicts with the minimum RTTC of less than 3 s by heat mapping. It clearly shows that most of the severe conflicts occur outside the crosswalk. Two reasons may lead to this phenomenon. First, not a small number of pedestrians walk outside the crosswalk, which may cause the severe conflict with right-turn vehicles. Second, the potential conflict point for calculating RTTC is determined by the intersection of the current speed vectors of the paired users, which may distribute sparsely due to the flexible change of pedestrian movement.Figure 12
Spatial distribution ofRTTCmin.In general, there are two types of vehicle-pedestrian conflict, that is, vehicle yielding to pedestrian, also known as pedestrian passing first (PPF), and pedestrian yielding to vehicle, also known as vehicle passing first (VPF). The PPF and VPF cases are compared because these two types of conflicts can result in different safety performance. Figure13 provides the details of statistics of the critical PPF and VPF (RTTC < 1 s) inside and outside the crosswalk. It is found that VPF occupies 55%, while PPF occupies 45% for the total critical RTTCs. It indicates that VPF is more dangerous than PPF. This is because both the vehicle and the pedestrian do not yield to each other and the vehicle tries to accelerate to pass the conflict area first. For the PPF cases, most of the vehicles decelerate to yield to the pedestrians, even though the vehicle may not totally stop and reaccelerate quickly once the pedestrians pass the conflict area. It was also found that a large percent of VPF occur outside the crosswalk at the northbound and eastbound due to the high traffic flow of right-turn vehicles and pedestrians at these two crosswalks. The statistics of PET in Figure 11 also show the potential risk at these crosswalks. Because drivers may not notice the pedestrian outside the crosswalk, the frequent critical confliction may lead to accidents. Thus, we propose the recommendation that a special pedestrian phase should be set to separate the right-turn vehicles from the southbound and eastbound. In practice, varying pedestrian and vehicular behavior may lead to misunderstanding of others’ decisions, which may result in safety problems. Considering widely distributed conflict points, both pedestrians and turning vehicles must pay attention to a broader area where conflicts may occur.Figure 13
The number of critical RTTC inside and outside of the crosswalk.
(a)
North
(b)
South
(c)
East
(d)
West


## 5. Conclusions and Future Work

Despite the prominent advantage of UAVs for emergency and traffic monitoring, there has been no research yet to employ UAVs for detailed safety assessment at intersections. In practice, accurate detection and tracking from UAVs is a challenging task due to platform motion, image instability, the relatively small size of the objects and varied appearance, and so forth. This study, as a pioneer work, investigated the feasibility of applying UAV video for surrogate safety analysis of pedestrian-vehicle conflict at intersections. By taking full advantage of the bird’s eye view offered by UAV, the image processing systems for automated vehicle trajectory extraction and semiautomatic pedestrian trajectory extraction were developed, respectively. Based on the trajectory data collected from one urban intersection in Beijing, China, two SSMs, that is, PET and RTTC, were employed to represent the frequency, severity, and location of pedestrian-vehicle conflict. The results of analysis showed a high exposure of pedestrians to traffic conflict both inside and outside the crosswalk and relatively risking behavior of right-turn vehicles around the corner. The findings demonstrate that UAV can support intersection safety analysis in an accurate and cost-effective way.Still, there are some limitations of this study. Firstly, due to the limitations of top-down views, the characteristics of pedestrian heterogeneity, for example, gender and age, cannot be identified in the video and thus are not discussed in this study. In pedestrian safety analysis, a recognized key issue [40] is that pedestrians of different gender and age behave differently on crosswalk. This aspect will be investigated in future studies with aerial videos captured at a lower flight altitude and with better visibility. Secondly, it is necessary to further study the interactions between pedestrian groups and vehicle (platoons), since the safety assessment for group interactions and pairwise intersections can be different [36]. Thirdly, the threshold values of PET and RTTC in this study were derived from previous studies, which may be influenced by the configuration of intersections and crosswalks and need further investigation. The pedestrian-vehicle conflict at other intersections with different intersection geometries, traffic volumes, and signal control strategies is supposed to be analyzed and compared in terms of SSMs with more UAV videos. Last but not least, it is essential in practice for transportation authorities to equip and manage UAVs in order to quickly detect and evaluate intersection safety, especially for the sites with frequent and severe traffic conflicts. Thus, it will be interesting to analyze the cost-benefit of establishing an integrated UAV network with fixed detectors by accounting for various UAV speed, admissible airspace, and operational budget constraints.



---

*Source: 5202150-2017-05-18.xml*