# Division of Area of Fixation Interest for Real Vehicle Driving Tests

Authors: Qian Xu; Tang-yi Guo; Fei Shao; Xue-jiao Jiang
Journal: Mathematical Problems in Engineering
 (2017)

---

## Abstract

The area of interest (AOI) reflects the degree of attention of a driver while driving. The division of AOI is visual characteristic analysis required in both real vehicle tests and simulated driving scenarios. Some key eye tracking parameters and their transformations can only be obtained after the division of AOI. In this study, 9 experienced and 7 novice drivers participated in real vehicle driving tests. They were asked to drive along a freeway section and a highway section, wearing the Dikablis eye tracking device. On average, 8132 fixation points for each driver were extracted. After coordinate conversion, the MSAP (Mean Shift Affinity Propagation) method is proposed to classify the distribution of fixation points into a circle type and a rectangle type. Experienced drivers’ fixation behavior falls into the circle type, in which fixation points are concentrated. Novice drivers’ fixation points, which are decentralized, are illustrated in the rectangle type. In the clustering algorithm, the damping coefficientλ determines the algorithm convergence, and the deviation parameter p mainly affects the number of clusters, where larger p values generate more clusters. This study not only provides the cluster type and cluster counts, but also presents the borderlines for each cluster. The findings provide significant contribution to eye tracking research.


---

## Body

## 1. Introduction

Drivers’ visual attention characteristics are some of the most important issues in conducting research on human factors affecting driving, and they play an important role in surrogate safety analysis, behavioral intention recognition, and early risk warnings. The development of eye tracking devices, such as Eyelink, Tobii, and Dikablis, made capturing eye movement features possible and easier than ever before. The most commonly used parameters include, but are not limited to, the saccade features, fixation features, pupil size, and perclos. Such data can be collected by means of real vehicle tests or simulators, where participants are required to wear a pair of eye tracking glasses. Driving simulations are easier to organize, and large sample data can be obtained. Data collected in vehicle tests, though, is real and exact, which is important and necessary for the validation of driving simulations.In real vehicle tests, Henning et al. [1] used turn signals and rearview observations as the indication of lane change intentions. Bhise et al. [2] found that drivers rotate their heads before searching visually in real vehicle driving scenarios. Lethaus and Rataj [3] conducted both real and simulated driving tests. After collecting data on drivers’ visual characteristics in real vehicle tests, they proposed driving behavioral predictors using vision related parameters, drawing the conclusion that eye movements precede other behavioral features. In their simulation study, they compared the accuracies of different driving behavior prediction algorithms [4]. In another simulation, Salvucci and Liu [5] found that drivers have different degrees of attention to rearview mirrors during lane keeping and lane changing which is consistent with Henning’s results [1].Psychophysiological research using eye gaze data has been a popular method for measuring drivers’ attention allocation [6], situation awareness [7, 8], and hazard perception [9, 10]. The gaze dispersion patterns provide some understanding of what information drivers use to keep themselves engaged in the driving task and what information they use to keep driving. In Tyron Louw’s study, drivers’ horizontal and vertical gaze dispersion during both conventional and automated driving were compared, concluding that, during automation, drivers’ horizontal gaze was generally more dispersed than that observed during manual driving, and it was more concentrated when drivers performed a visual secondary task. Drivers’ vertical gaze was most dispersed when the road scene and dashboard were completely occluded during automation [11]. The eye movement feature has been an important measurement for driving behavior research. For example, Navarro et al.’s research analyzed the variance of time spent looking in the tangent point area with the assistance factor of automated steering [12]. Tivesten and Dozza investigated the effect of both driving context and visual-manual phone tasks on drivers’ glance behavior in naturalistic driving and concluded that drivers indeed spend more time looking at the road and have a lower proportion of long off-road glances [13]. However, while gaze concentration has been used successfully in manual driving to distinguish between the effects of visual and cognitive load and it is commonly accepted that drivers have different degrees of interest in different targets or areas while driving, there is scarce criteria for the area of interest (AOI) division established in the literature. Dividing the visual area into several subareas according to drivers’ interests helps researchers to have better understanding of drivers’ eye behavior. The division of AOI is visual characteristic analysis required in real vehicle tests and simulated driving scenarios. Based on the AOI division we can obtain statistical eye movement parameters, including the fixation duration, fixation counts, saccade counts, time to first glance, and first glance duration. These basic parameters and their transformations, such as the total glance time, AOI attention ratio, glance rate, glance location probability, percentage of eyelid closure, and saccade trajectory, are key explanatory variables used to interpret drivers’ behavior. If we treat the visual area as a whole, without dividing it into subareas according to the study purposes, most of basic visual parameters, especially their transformations, are difficult to obtain. Some researchers just provide AOI division results, including AOI counts and AOI borders, arbitrarily. The NTHSA [14] divided drivers’ visual area into eight subareas, namely, the front left, front, front right, left window, right window, rearview mirror, left mirror, and right mirror, to analyze the time window of lane changes. To compare the differences in visual characteristics between left lane change and going straight, Olsen et al. [15] divided the visual area into nine regions, namely, the front, rearview mirror, left window, right window, left mirror, right mirror, left blind spot, right blind spot, control panel, other interior, and indeterminate, and the results showed that before a left lane change drivers pay attention to left AOIs twice more often than to front AOIs. Although these studies showed intuitively sound conclusions, it is still not clear how AOIs were divided.Using the front left, left window, and left mirror as the indicators of left lane change intention is widely accepted. The problem is how to determine the borderline for each AOI. For example, if the area of the rearview mirror is too large, fixation points that in fact belong to the windshield view may be mistakenly assumed as rearview fixation points. If the area is smaller than it should be, some rearview fixation points may be processed incorrectly. A vague borderline between different AOIs may lead to wrong conclusions. Therefore, determining the number of AOIs and their borderlines is of great significance.Based on fixation data from real vehicle driving tests, this paper proposes a clustering method to divide the visual area into a certain number of subareas and determine the borderline of each AOI.


## 2. Methodology

Cluster analysis (clustering) is the process of grouping data into classes or clusters based on similarity. Objects similar to one another are grouped within the same cluster, while different objects are placed in different clusters. That is to say, clusters formed this way should be highly homogeneous internally and highly heterogeneous externally. Cluster analysis methods are mostly used when there are no any prior hypotheses in the exploratory phase of the study. In a sense, the cluster analysis finds most significant solutions. As an important tool of data mining, cluster analysis is widely applied in statistics, image processing, information retrieval, and machine learning.Commonly used algorithms of cluster analysis include theK-means clustering [16], hierarchical clustering [17], fuzzy clustering [18], spectral clustering [19], and density-based clustering [20]. Due to the complexity of gaze distribution and the larger amount of the data, one clustering method can only solve a particular type of problems. The improved Affinity Propagation (AP) Algorithm, Mean Shift Affinity Propagation (MSAP) Algorithm, is used in this paper, and the result is compared to the K-means clustering. Then the paper gives the suggestion of critical AOI division.


### 2.1. Division Algorithm for Area of Visual Interest

#### 2.1.1. Affinity Propagation (AP) Algorithm

For the fixation area clustering problem, the output of the division algorithm is the area where fixation points concentrate. This concept is consistent with the AP (Affinity Propagation) cluster algorithm. The AP cluster method, proposed by Frey and Dueck [21], obtains high quality clusters by transmitting information among nearby data points. It classifies data points on the basis of similarity among data points. The AP method assumes all fixation points as potential cluster centers. In other words, each point can be treated as an initial representative point. This approach eliminates the need for selection of initial points and also leads to more stable and higher quality clustering results. The AP method is described below.k
i
,
j denotes the similarity between fixation points i and j, which is determined by the negative Euclidean distance between i and j. The closer the distance between the two points, the greater the similarity between them.(1)ki,j=-xi-xj2.By traversing through alln fixation points the similarity matrix K is obtained:(2)K=ki,jn×n,where the diagonal element ks,s in K is the deviation parameter p. The larger the value of p is, the more likely s is a cluster center.Two information parameter matricesR and A are based on K, where R=[ri,s]n×n is the attractiveness matrix and A=[ai,s]n×n is the matrix of membership degrees. Attractiveness is defined as the likelihood that s is the cluster center of fixation point i. The element ri,s is the point-to-information from fixation point i to potential cluster center s, representing the attractiveness of s. The membership degree is defined as the likelihood that i chooses s as its cluster center. The element ai,s is the information sent from s to i, representing the grade of membership for fixation point i. The attractiveness and membership degrees determine the iterations of the clustering algorithm and thus become the core idea of the algorithm.(3)ri,s⟵ki,s-maxsi⁡ai,si+ki,siai,s⟵min⁡0,rs,s+∑i′max⁡0,ri′,sifi≠s,as,s⟵maxi′⁡0,ri′,k.Given thats is the cluster center of fixation point i, the assignment of each non-center point to its cluster center represents the clustering results.In the information iteration process, the damping coefficientλ plays an important role in the algorithm convergence. The larger the value of λ, the better the convergence. In each iteration step, the updated results of ri,s and ai,s are the weighted sums of the last and current iteration outcomes, where the corresponding weights are λ and 1-λ, respectively. Given the number t of the current iteration, the weighted iteration equations are as follows:(4)rti,s⟵1-λki,s-maxs′≤t⁡at-1i,s′+ki,s′+λrt-1i,sati,s⟵1-λmin⁡0,rt-1s,s+∑i′max⁡0,rti′,s+λat-1i,sats,s=1-λ∑i′max⁡0,rti′,s+λat-1i,s,where λ∈(0,1) can suppress possible artifacts of the algorithm.


#### 2.1.2. Mean Shift Affinity Propagation (MSAP) Algorithm

Through iterative deployment, the AP algorithm uses the information transfer of each point-to-point pair to reach stable cluster classification. The computational complexity isOn2log⁡n, which is large and high-dimensional. Regarding drivers’ fixation points, the large data size makes the calculation of the similarity matrix more complicated and thus increases the time and space complexity of the clustering process. To avoid the influence of complexity on the clustering of the fixation interest area and to reduce the scale of the similarity matrix, the time and space complexity needs to be decreased in order to accelerate the clustering process.Accordingly, we propose an improved AP method, the Mean Shift Affinity Propagation (MSAP). First, we use the mean shift method to preprocess input fixation points. The number of data sets is replaced by the number of areas, where each area can be treated as a whole. Then, instead of fixation point coordinates, mean coordinate values of fixation points in one area are considered as data points. Using the number of areas rather than the number of data sets, the AP algorithm conducts a redivision process to improve clustering performance.The AP algorithm clusters fixation points based on similarity.n fixation points constitute an n×n similarity matrix K. Before calculating K, we need to define the feature space. Considering the correlation between eye movements and sight lines, the scan angle of the sight line is introduced as the main feature of the feature space. After preprocessing data using the mean shift method, any fixation point entered is expressed as a three-dimensional information vector x=(xs,xr), where xs represents the two-dimensional coordinates and xr is the scan angle of the sight line of the entered fixation point.The core function can be written as follows:(5)Khs,hr=Chsdhrpkxshs2kxrhr2,where C is a constant, p and d denote the spatial dimension (in this research p=1, d=2), hs is the radius of the core function, representing the core size of the spatial domain, and hr is the radius of the feature space, representing the core size of the value domain.


## 2.1. Division Algorithm for Area of Visual Interest

### 2.1.1. Affinity Propagation (AP) Algorithm

For the fixation area clustering problem, the output of the division algorithm is the area where fixation points concentrate. This concept is consistent with the AP (Affinity Propagation) cluster algorithm. The AP cluster method, proposed by Frey and Dueck [21], obtains high quality clusters by transmitting information among nearby data points. It classifies data points on the basis of similarity among data points. The AP method assumes all fixation points as potential cluster centers. In other words, each point can be treated as an initial representative point. This approach eliminates the need for selection of initial points and also leads to more stable and higher quality clustering results. The AP method is described below.k
i
,
j denotes the similarity between fixation points i and j, which is determined by the negative Euclidean distance between i and j. The closer the distance between the two points, the greater the similarity between them.(1)ki,j=-xi-xj2.By traversing through alln fixation points the similarity matrix K is obtained:(2)K=ki,jn×n,where the diagonal element ks,s in K is the deviation parameter p. The larger the value of p is, the more likely s is a cluster center.Two information parameter matricesR and A are based on K, where R=[ri,s]n×n is the attractiveness matrix and A=[ai,s]n×n is the matrix of membership degrees. Attractiveness is defined as the likelihood that s is the cluster center of fixation point i. The element ri,s is the point-to-information from fixation point i to potential cluster center s, representing the attractiveness of s. The membership degree is defined as the likelihood that i chooses s as its cluster center. The element ai,s is the information sent from s to i, representing the grade of membership for fixation point i. The attractiveness and membership degrees determine the iterations of the clustering algorithm and thus become the core idea of the algorithm.(3)ri,s⟵ki,s-maxsi⁡ai,si+ki,siai,s⟵min⁡0,rs,s+∑i′max⁡0,ri′,sifi≠s,as,s⟵maxi′⁡0,ri′,k.Given thats is the cluster center of fixation point i, the assignment of each non-center point to its cluster center represents the clustering results.In the information iteration process, the damping coefficientλ plays an important role in the algorithm convergence. The larger the value of λ, the better the convergence. In each iteration step, the updated results of ri,s and ai,s are the weighted sums of the last and current iteration outcomes, where the corresponding weights are λ and 1-λ, respectively. Given the number t of the current iteration, the weighted iteration equations are as follows:(4)rti,s⟵1-λki,s-maxs′≤t⁡at-1i,s′+ki,s′+λrt-1i,sati,s⟵1-λmin⁡0,rt-1s,s+∑i′max⁡0,rti′,s+λat-1i,sats,s=1-λ∑i′max⁡0,rti′,s+λat-1i,s,where λ∈(0,1) can suppress possible artifacts of the algorithm.


### 2.1.2. Mean Shift Affinity Propagation (MSAP) Algorithm

Through iterative deployment, the AP algorithm uses the information transfer of each point-to-point pair to reach stable cluster classification. The computational complexity isOn2log⁡n, which is large and high-dimensional. Regarding drivers’ fixation points, the large data size makes the calculation of the similarity matrix more complicated and thus increases the time and space complexity of the clustering process. To avoid the influence of complexity on the clustering of the fixation interest area and to reduce the scale of the similarity matrix, the time and space complexity needs to be decreased in order to accelerate the clustering process.Accordingly, we propose an improved AP method, the Mean Shift Affinity Propagation (MSAP). First, we use the mean shift method to preprocess input fixation points. The number of data sets is replaced by the number of areas, where each area can be treated as a whole. Then, instead of fixation point coordinates, mean coordinate values of fixation points in one area are considered as data points. Using the number of areas rather than the number of data sets, the AP algorithm conducts a redivision process to improve clustering performance.The AP algorithm clusters fixation points based on similarity.n fixation points constitute an n×n similarity matrix K. Before calculating K, we need to define the feature space. Considering the correlation between eye movements and sight lines, the scan angle of the sight line is introduced as the main feature of the feature space. After preprocessing data using the mean shift method, any fixation point entered is expressed as a three-dimensional information vector x=(xs,xr), where xs represents the two-dimensional coordinates and xr is the scan angle of the sight line of the entered fixation point.The core function can be written as follows:(5)Khs,hr=Chsdhrpkxshs2kxrhr2,where C is a constant, p and d denote the spatial dimension (in this research p=1, d=2), hs is the radius of the core function, representing the core size of the spatial domain, and hr is the radius of the feature space, representing the core size of the value domain.


## 2.1.1. Affinity Propagation (AP) Algorithm

For the fixation area clustering problem, the output of the division algorithm is the area where fixation points concentrate. This concept is consistent with the AP (Affinity Propagation) cluster algorithm. The AP cluster method, proposed by Frey and Dueck [21], obtains high quality clusters by transmitting information among nearby data points. It classifies data points on the basis of similarity among data points. The AP method assumes all fixation points as potential cluster centers. In other words, each point can be treated as an initial representative point. This approach eliminates the need for selection of initial points and also leads to more stable and higher quality clustering results. The AP method is described below.k
i
,
j denotes the similarity between fixation points i and j, which is determined by the negative Euclidean distance between i and j. The closer the distance between the two points, the greater the similarity between them.(1)ki,j=-xi-xj2.By traversing through alln fixation points the similarity matrix K is obtained:(2)K=ki,jn×n,where the diagonal element ks,s in K is the deviation parameter p. The larger the value of p is, the more likely s is a cluster center.Two information parameter matricesR and A are based on K, where R=[ri,s]n×n is the attractiveness matrix and A=[ai,s]n×n is the matrix of membership degrees. Attractiveness is defined as the likelihood that s is the cluster center of fixation point i. The element ri,s is the point-to-information from fixation point i to potential cluster center s, representing the attractiveness of s. The membership degree is defined as the likelihood that i chooses s as its cluster center. The element ai,s is the information sent from s to i, representing the grade of membership for fixation point i. The attractiveness and membership degrees determine the iterations of the clustering algorithm and thus become the core idea of the algorithm.(3)ri,s⟵ki,s-maxsi⁡ai,si+ki,siai,s⟵min⁡0,rs,s+∑i′max⁡0,ri′,sifi≠s,as,s⟵maxi′⁡0,ri′,k.Given thats is the cluster center of fixation point i, the assignment of each non-center point to its cluster center represents the clustering results.In the information iteration process, the damping coefficientλ plays an important role in the algorithm convergence. The larger the value of λ, the better the convergence. In each iteration step, the updated results of ri,s and ai,s are the weighted sums of the last and current iteration outcomes, where the corresponding weights are λ and 1-λ, respectively. Given the number t of the current iteration, the weighted iteration equations are as follows:(4)rti,s⟵1-λki,s-maxs′≤t⁡at-1i,s′+ki,s′+λrt-1i,sati,s⟵1-λmin⁡0,rt-1s,s+∑i′max⁡0,rti′,s+λat-1i,sats,s=1-λ∑i′max⁡0,rti′,s+λat-1i,s,where λ∈(0,1) can suppress possible artifacts of the algorithm.


## 2.1.2. Mean Shift Affinity Propagation (MSAP) Algorithm

Through iterative deployment, the AP algorithm uses the information transfer of each point-to-point pair to reach stable cluster classification. The computational complexity isOn2log⁡n, which is large and high-dimensional. Regarding drivers’ fixation points, the large data size makes the calculation of the similarity matrix more complicated and thus increases the time and space complexity of the clustering process. To avoid the influence of complexity on the clustering of the fixation interest area and to reduce the scale of the similarity matrix, the time and space complexity needs to be decreased in order to accelerate the clustering process.Accordingly, we propose an improved AP method, the Mean Shift Affinity Propagation (MSAP). First, we use the mean shift method to preprocess input fixation points. The number of data sets is replaced by the number of areas, where each area can be treated as a whole. Then, instead of fixation point coordinates, mean coordinate values of fixation points in one area are considered as data points. Using the number of areas rather than the number of data sets, the AP algorithm conducts a redivision process to improve clustering performance.The AP algorithm clusters fixation points based on similarity.n fixation points constitute an n×n similarity matrix K. Before calculating K, we need to define the feature space. Considering the correlation between eye movements and sight lines, the scan angle of the sight line is introduced as the main feature of the feature space. After preprocessing data using the mean shift method, any fixation point entered is expressed as a three-dimensional information vector x=(xs,xr), where xs represents the two-dimensional coordinates and xr is the scan angle of the sight line of the entered fixation point.The core function can be written as follows:(5)Khs,hr=Chsdhrpkxshs2kxrhr2,where C is a constant, p and d denote the spatial dimension (in this research p=1, d=2), hs is the radius of the core function, representing the core size of the spatial domain, and hr is the radius of the feature space, representing the core size of the value domain.


## 3. Fixation Data Acquisition and Characteristics Analysis

### 3.1. Pixel Coordinates and Reference Markers

In a real driving test, scenes captured by the cameras of the Dikablis eye tracker (in Figure1) always change with head turning during lane changing, turning, accelerating, and decelerating. To obtain the relative position of a fixation point, the pixel coordinates and reference marker (pasted on the control panel or windshield) are introduced. The camera coverage is divided into 1920∗1080 small squares. Regardless of how scenes change in the camera, the relative coordinates of the extracted fixation point (the reference coordinate origin is fixed at the upper-left corner O) never change. The coordinates of the fixation point are labeled as (xpixel,ypixel), where 0≤xpixel≤1920 and 0≤ypixel≤1080.Figure 1
The coordinate system based on the camera mounted on the Dikablis.Assuming that the camera projection area isϑcam, which is constant with length Lcam and width Wcam, define the driver’s vision coverage as ϑsce with length Lsce and width Wsce. The necessary and sufficient condition for the transformation from pixel coordinates to two-dimensional coordinates is(6)nmar·ncam≥0∑m=1nmarSi,m·ϑcam≥ϑcami=1,2,…,K,…,ncam∑i=1ncam∑m=1nmarSi,m·ϑcam≥ϑseci≥0,m≥0,Si,m∈0,1,where ncam and nmar represent the number of scenes and markers, respectively, and S(i,m) denotes the presence of the mth marker in the ith scene. If marker m exists in scene i, S(i,m)=1; otherwise, S(i,m)=0.


### 3.2. Conversion of Pixel Coordinates to Two-Dimensional Coordinates

If the setting of reference markers satisfies the first condition in (6), then there is at least one reference marker in the scene. As the marker’s pixel coordinates change with different scenes, a single reference marker may have more than one pixel coordinate. However, given a pasted marker on the windshield, like in Figure 2, there should be only one fixed two-dimensional coordinate. Although the pixel coordinate of the fixation point shifts, its direction relative to the reference marker does not change. We define the direction of fixation point A relative to reference marker B as vector AB⃑. Given the two-dimensional coordinate of marker B, the only relative two-dimensional coordinate of fixation point A can be computed using the following equation:(7)mfix,nfix=mmar,nmar+xfix,yfix-xmar,ymar,where mfix,nfix and mmar,nmar are the two-dimensional coordinates of fixation points and reference markers and (xfix,yfix) and (xmar,ymar) are the pixel coordinates of fixation points and reference markers in the same scene.Figure 2
The two-dimensional coordinate system relative to the reference marker.


### 3.3. Types of Fixation Points

The real vehicle driving tests involved 16 drivers, whose basic driving attributes are recorded in Table1. A SUV is employed in the test to maintain a good visual condition for drivers. In order to ensure driving environment is consistent for all the participants, each experiment lasted for at least 30 minutes and conducted on workdays in good weather during free hours (10:00 am–11:00 am; 15:00 pm–16:30 pm) included the same freeway and highway scenarios using D-Lab driving analysis system (made in Ergoneers company, Germany). During the test, all participants were almost exposed to the same natural lighting condition which is about 30000 Lux. Drivers wear the Dikablis eye tracking glasses which can track drivers’ eyes movement and collect data related to eyes, such as fixation point, and the data can be output and basically processed by D-Lab system. Before data recording, a five-minute warm-up driving was conducted, and during the tests, the participants were required to behave normally to provide a driving behavior as realistic as possible. In total, more than 8000 fixation points (average 8132, sample rate 81.36%) were extracted for each driver using the D-LAB Studio v3.0. Then, the correlation analysis of the fixation points was conducted to figure out the visual characteristics of all participants.Table 1
Summary of drivers’ attributes.
Attribute
Novice driver (7)
Experienced driver (9)
p value
Age
21.2
49.25
<0.01
Driving years
2.1
15.1
<0.01
Mileage/year
3288
4146
>0.05
I
n
t
e
l
l
i
g
e
n
c
e
a
28.50
27.50
<0.01
Eyesight
1.24
1.48
>0.05
aIntelligence is obtained using the method of simple mental state checklist (MMSE).
Novice drivers and experienced drivers are determined using the driving years.In the bivariate correlation analysis, the Pearson correlation, confidence intervals, and significance tests were used. The Pearson correlation coefficientr is given as follows:(8)r=covx,yσxσy=Ex-μxy-μyσxσy,where μx and μy are the means of x and y, respectively, and σx and σy represent the standard deviation of x and y, respectively.Given the null hypothesisH0:r=0, the statistic zr is computed after Fisher’s z-transformation.(9)zr=tan⁡h-1r=12log⁡1+r1-r,where zr follows the normal distribution P(x) with mean 0 and standard deviation 1/(n-3).The significance testp value is given by(10)p=2min⁡pX<zr,pX>zr.The(1-α) confidence interval is given as follows:(11)rl=tan⁡hξl=exp⁡2ξl-1exp⁡2ξl+1ru=tan⁡hξu=exp⁡2ξu-1exp⁡2ξu+1ξl=zr-r2n-1+z1-α21n-3ξu=zr-r2n-1-z1-α21n-3.Givenα=0.1 (95% confidence interval), the computed values of r and p are presented in Table 2. The D-Lab analysis system automatically overlays drivers each fixation spot and draws the hot spot figure as shown in Table 3. Further analysis shows two types of the fixation point distribution. One type corresponds to the case where x and y are uncorrelated, and the other type corresponds to the case of weak negative correlation. According to the distribution of fixation points in Table 3, the negative correlation case is defined as the circle type and the uncorrelated case as the rectangle type.Table 2
The correlations among fixation points.
Correlation
⁢Negatively correlated
⁢Uncorrelated
x
y
x
y
x
Pearson (r)
1
−.020
1
−.011
Sig. (2-tailed,p)
0.074
0.346
y
Pearson (r)
−.020
1
−.011
1
Sig. (2-tailed,p)
0.074
0.346Table 3
The two types of fixation points distribution.
Type
Sketch
Coordinates of fixation points
Hot spot
Circle
Rectangle


#### 3.3.1. Circle Type

In this case, the coordinates of fixation points present weak negative correlation. The high frequency gazed area (also called the hot spot area) is concentrated in a certain region where drivers look straight forward. The most focused point (actually a small area) is the center of the circle. The closer the fixation point to the center, the more the driver gazes at that point. Experienced drivers’ fixation points are classified into this type. These drivers focus on the lanes ahead, having less head turns and quicker scanning speeds. For prejudging and lane changes, compared to novice drivers, experienced drivers spend less time watching the left/right rearview mirrors, and their sightline returns to the front windshield quicker. Some experienced drivers even watch the rearview mirror before changing a lane. Their eye movement behavior explains why experienced drivers’ fixation points are more concentrated in the hot spot area and distributed in a circle shape.


#### 3.3.2. Rectangle Type

In this case, the coordinates have no significant correlation. The hot spot is not concentrated and shows a rectangular shape. Novice drivers’ fixation points fall into this category. With less experience, these drivers rarely watch control panels or rearview mirrors. During a lane change, they spend more time (longer duration or repeated views) watching the left/right rearview mirrors. Their eye behavior forms a long strip distribution.


## 3.1. Pixel Coordinates and Reference Markers

In a real driving test, scenes captured by the cameras of the Dikablis eye tracker (in Figure1) always change with head turning during lane changing, turning, accelerating, and decelerating. To obtain the relative position of a fixation point, the pixel coordinates and reference marker (pasted on the control panel or windshield) are introduced. The camera coverage is divided into 1920∗1080 small squares. Regardless of how scenes change in the camera, the relative coordinates of the extracted fixation point (the reference coordinate origin is fixed at the upper-left corner O) never change. The coordinates of the fixation point are labeled as (xpixel,ypixel), where 0≤xpixel≤1920 and 0≤ypixel≤1080.Figure 1
The coordinate system based on the camera mounted on the Dikablis.Assuming that the camera projection area isϑcam, which is constant with length Lcam and width Wcam, define the driver’s vision coverage as ϑsce with length Lsce and width Wsce. The necessary and sufficient condition for the transformation from pixel coordinates to two-dimensional coordinates is(6)nmar·ncam≥0∑m=1nmarSi,m·ϑcam≥ϑcami=1,2,…,K,…,ncam∑i=1ncam∑m=1nmarSi,m·ϑcam≥ϑseci≥0,m≥0,Si,m∈0,1,where ncam and nmar represent the number of scenes and markers, respectively, and S(i,m) denotes the presence of the mth marker in the ith scene. If marker m exists in scene i, S(i,m)=1; otherwise, S(i,m)=0.


## 3.2. Conversion of Pixel Coordinates to Two-Dimensional Coordinates

If the setting of reference markers satisfies the first condition in (6), then there is at least one reference marker in the scene. As the marker’s pixel coordinates change with different scenes, a single reference marker may have more than one pixel coordinate. However, given a pasted marker on the windshield, like in Figure 2, there should be only one fixed two-dimensional coordinate. Although the pixel coordinate of the fixation point shifts, its direction relative to the reference marker does not change. We define the direction of fixation point A relative to reference marker B as vector AB⃑. Given the two-dimensional coordinate of marker B, the only relative two-dimensional coordinate of fixation point A can be computed using the following equation:(7)mfix,nfix=mmar,nmar+xfix,yfix-xmar,ymar,where mfix,nfix and mmar,nmar are the two-dimensional coordinates of fixation points and reference markers and (xfix,yfix) and (xmar,ymar) are the pixel coordinates of fixation points and reference markers in the same scene.Figure 2
The two-dimensional coordinate system relative to the reference marker.


## 3.3. Types of Fixation Points

The real vehicle driving tests involved 16 drivers, whose basic driving attributes are recorded in Table1. A SUV is employed in the test to maintain a good visual condition for drivers. In order to ensure driving environment is consistent for all the participants, each experiment lasted for at least 30 minutes and conducted on workdays in good weather during free hours (10:00 am–11:00 am; 15:00 pm–16:30 pm) included the same freeway and highway scenarios using D-Lab driving analysis system (made in Ergoneers company, Germany). During the test, all participants were almost exposed to the same natural lighting condition which is about 30000 Lux. Drivers wear the Dikablis eye tracking glasses which can track drivers’ eyes movement and collect data related to eyes, such as fixation point, and the data can be output and basically processed by D-Lab system. Before data recording, a five-minute warm-up driving was conducted, and during the tests, the participants were required to behave normally to provide a driving behavior as realistic as possible. In total, more than 8000 fixation points (average 8132, sample rate 81.36%) were extracted for each driver using the D-LAB Studio v3.0. Then, the correlation analysis of the fixation points was conducted to figure out the visual characteristics of all participants.Table 1
Summary of drivers’ attributes.
Attribute
Novice driver (7)
Experienced driver (9)
p value
Age
21.2
49.25
<0.01
Driving years
2.1
15.1
<0.01
Mileage/year
3288
4146
>0.05
I
n
t
e
l
l
i
g
e
n
c
e
a
28.50
27.50
<0.01
Eyesight
1.24
1.48
>0.05
aIntelligence is obtained using the method of simple mental state checklist (MMSE).
Novice drivers and experienced drivers are determined using the driving years.In the bivariate correlation analysis, the Pearson correlation, confidence intervals, and significance tests were used. The Pearson correlation coefficientr is given as follows:(8)r=covx,yσxσy=Ex-μxy-μyσxσy,where μx and μy are the means of x and y, respectively, and σx and σy represent the standard deviation of x and y, respectively.Given the null hypothesisH0:r=0, the statistic zr is computed after Fisher’s z-transformation.(9)zr=tan⁡h-1r=12log⁡1+r1-r,where zr follows the normal distribution P(x) with mean 0 and standard deviation 1/(n-3).The significance testp value is given by(10)p=2min⁡pX<zr,pX>zr.The(1-α) confidence interval is given as follows:(11)rl=tan⁡hξl=exp⁡2ξl-1exp⁡2ξl+1ru=tan⁡hξu=exp⁡2ξu-1exp⁡2ξu+1ξl=zr-r2n-1+z1-α21n-3ξu=zr-r2n-1-z1-α21n-3.Givenα=0.1 (95% confidence interval), the computed values of r and p are presented in Table 2. The D-Lab analysis system automatically overlays drivers each fixation spot and draws the hot spot figure as shown in Table 3. Further analysis shows two types of the fixation point distribution. One type corresponds to the case where x and y are uncorrelated, and the other type corresponds to the case of weak negative correlation. According to the distribution of fixation points in Table 3, the negative correlation case is defined as the circle type and the uncorrelated case as the rectangle type.Table 2
The correlations among fixation points.
Correlation
⁢Negatively correlated
⁢Uncorrelated
x
y
x
y
x
Pearson (r)
1
−.020
1
−.011
Sig. (2-tailed,p)
0.074
0.346
y
Pearson (r)
−.020
1
−.011
1
Sig. (2-tailed,p)
0.074
0.346Table 3
The two types of fixation points distribution.
Type
Sketch
Coordinates of fixation points
Hot spot
Circle
Rectangle


### 3.3.1. Circle Type

In this case, the coordinates of fixation points present weak negative correlation. The high frequency gazed area (also called the hot spot area) is concentrated in a certain region where drivers look straight forward. The most focused point (actually a small area) is the center of the circle. The closer the fixation point to the center, the more the driver gazes at that point. Experienced drivers’ fixation points are classified into this type. These drivers focus on the lanes ahead, having less head turns and quicker scanning speeds. For prejudging and lane changes, compared to novice drivers, experienced drivers spend less time watching the left/right rearview mirrors, and their sightline returns to the front windshield quicker. Some experienced drivers even watch the rearview mirror before changing a lane. Their eye movement behavior explains why experienced drivers’ fixation points are more concentrated in the hot spot area and distributed in a circle shape.


### 3.3.2. Rectangle Type

In this case, the coordinates have no significant correlation. The hot spot is not concentrated and shows a rectangular shape. Novice drivers’ fixation points fall into this category. With less experience, these drivers rarely watch control panels or rearview mirrors. During a lane change, they spend more time (longer duration or repeated views) watching the left/right rearview mirrors. Their eye behavior forms a long strip distribution.


## 3.3.1. Circle Type

In this case, the coordinates of fixation points present weak negative correlation. The high frequency gazed area (also called the hot spot area) is concentrated in a certain region where drivers look straight forward. The most focused point (actually a small area) is the center of the circle. The closer the fixation point to the center, the more the driver gazes at that point. Experienced drivers’ fixation points are classified into this type. These drivers focus on the lanes ahead, having less head turns and quicker scanning speeds. For prejudging and lane changes, compared to novice drivers, experienced drivers spend less time watching the left/right rearview mirrors, and their sightline returns to the front windshield quicker. Some experienced drivers even watch the rearview mirror before changing a lane. Their eye movement behavior explains why experienced drivers’ fixation points are more concentrated in the hot spot area and distributed in a circle shape.


## 3.3.2. Rectangle Type

In this case, the coordinates have no significant correlation. The hot spot is not concentrated and shows a rectangular shape. Novice drivers’ fixation points fall into this category. With less experience, these drivers rarely watch control panels or rearview mirrors. During a lane change, they spend more time (longer duration or repeated views) watching the left/right rearview mirrors. Their eye behavior forms a long strip distribution.


## 4. Division of Area of Fixation Interest

According to the clustering algorithm proposed above, the procedure to classify the fixation points (more than 8000 for each participant) is described as follows.


### 4.1. Data Preprocessing

Extract the feature space based on the scan angle of the sight line using the MS (mean shift) method. The initial data is divided into multiple data sets. Then, the central coordinates of data sets (areas) can be calculated and input into the AP algorithm as improved initial fixation points.


### 4.2. Initialization

Calculate the negative Euclidean distances between input fixation points to obtain the similarity matrixK. Initialize variables by assigning reasonable values. We hereby set ri,s=0, ai,s=0, the maximum iterations = 1000, the successive iterations for a given cluster center = 100, and p = a certain value.


### 4.3. Iterations

In each iteration, compute the values ofri,s and ai,s according to (3) and (4) to find potential clusters. There are two criteria to end iterations, namely, the maximum number of iterations (1000) and the number of successive iterations for a given cluster center (100).


### 4.4. Result Judgment

If the clustering results, including the distribution and clustering number, do not meet requirements, change thep value and repeat the clustering procedure until satisfactory results are achieved.In the clustering algorithm, two parameters affect the number of iterations and the number of clusters. One is the damping coefficientλ, which determines the algorithm convergence. The other one is the deviation parameter p, which mainly affects the number of clusters. Different p values lead to different cluster numbers. Larger p values generate more clusters.Different values of the damping coefficientsλ result in different numbers of iterations and different turbulence curves. Taking one driver’s fixation points as an example (Figure 3), a smaller λ (0.50) leads to dramatic fluctuations and a lower number of iterations. A larger λ (0.90) leads to the opposite results. To explain this phenomenon, we need to consider the relationship among ri,s, ai,s, and λ. In each iteration, ri,s and ai,s are both affected by λ. For smaller values of λ, the net similarity between two successive iterations fluctuates sharply, possibly leading to a local optimum or instability. Larger values of λ lead to slower convergence speeds but make iterations stable, avoiding the shortcomings of smaller λ.Figure 3
The iteration process with different damping coefficients.
(a)
λ
=
0.5
(b)
λ
=
0.9In this study, there are more than 8000 fixation points for each participant. To make the iteration process as stable and convergent as possible, a largerλ (=0.90) is advisable. Although the number of iterations increases, the changing trend of net clustering similarity levels off.In Figures4(a), 4(b), and 4(c), the median (S) is the diagonal element in the similarity matrix. We can see that the larger the p value (absolute value), the larger the number of clusters. We do not consider less than two or more than ten clusters, since a too small or too large number of clusters lose internal homogeneity and external heterogeneity. By assigning the p value of 2 × medians (S), median (S), and median (S)/2, we can obtain two, four, and six clusters, respectively. The center of each cluster is labeled by the black star.Figure 4
The clustering results for different fixation types with different damping coefficients.
(a)
p = 2 × median (S); number of clusters = 2
(b)
p = median (S); number of clusters = 4
(c)
p = median (S)/2; number of clusters = 6Different numbers of clusters have different explanations. Givenp = 2 × median (S), the fixation area on the windshield for both circle and rectangle types is divided into two parts. The dividing line for the circle type is slightly to the left of the center and for the rectangle type just the opposite. In Figure 5, a driver looks straight at the target (a car in front or the lane) while driving normally. As the driver is placed on the left side of the car, the driver’s sight line intersects the windshield to the left of the center line. However, usually, a novice driver, who drives slower than average, pays more attention to passing-by vehicles or other information on the left and right sides, making his fixation points diluted.Figure 5
Driver’s sight line under the two divider lines’ situation.Whenp = median (S), clear differences appear. For the circle type (experienced drivers), the area of fixation interest is divided into four parts, which have one-to-one correspondence to the left rearview, right rearview, rearview, and control panel views. Novice drivers’ eye movement characteristics show little attention to the rearview mirror and control panel; therefore, the area of fixation interest is split into four parts horizontally and almost evenly. Figure 5 shows the division lines’ relationship between the novice and experienced drivers on two AOIs situation. It is clear that two types of driver divide attention to left and right differently.Givenp = median (S)/2, the number of clusters increases to six. Figure 4(c) shows trends similar to Figure 4(b). For the circle type, the only difference is that the areas of rearview panel are both divided into two subareas, which is hard to explain reasonably.


## 4.1. Data Preprocessing

Extract the feature space based on the scan angle of the sight line using the MS (mean shift) method. The initial data is divided into multiple data sets. Then, the central coordinates of data sets (areas) can be calculated and input into the AP algorithm as improved initial fixation points.


## 4.2. Initialization

Calculate the negative Euclidean distances between input fixation points to obtain the similarity matrixK. Initialize variables by assigning reasonable values. We hereby set ri,s=0, ai,s=0, the maximum iterations = 1000, the successive iterations for a given cluster center = 100, and p = a certain value.


## 4.3. Iterations

In each iteration, compute the values ofri,s and ai,s according to (3) and (4) to find potential clusters. There are two criteria to end iterations, namely, the maximum number of iterations (1000) and the number of successive iterations for a given cluster center (100).


## 4.4. Result Judgment

If the clustering results, including the distribution and clustering number, do not meet requirements, change thep value and repeat the clustering procedure until satisfactory results are achieved.In the clustering algorithm, two parameters affect the number of iterations and the number of clusters. One is the damping coefficientλ, which determines the algorithm convergence. The other one is the deviation parameter p, which mainly affects the number of clusters. Different p values lead to different cluster numbers. Larger p values generate more clusters.Different values of the damping coefficientsλ result in different numbers of iterations and different turbulence curves. Taking one driver’s fixation points as an example (Figure 3), a smaller λ (0.50) leads to dramatic fluctuations and a lower number of iterations. A larger λ (0.90) leads to the opposite results. To explain this phenomenon, we need to consider the relationship among ri,s, ai,s, and λ. In each iteration, ri,s and ai,s are both affected by λ. For smaller values of λ, the net similarity between two successive iterations fluctuates sharply, possibly leading to a local optimum or instability. Larger values of λ lead to slower convergence speeds but make iterations stable, avoiding the shortcomings of smaller λ.Figure 3
The iteration process with different damping coefficients.
(a)
λ
=
0.5
(b)
λ
=
0.9In this study, there are more than 8000 fixation points for each participant. To make the iteration process as stable and convergent as possible, a largerλ (=0.90) is advisable. Although the number of iterations increases, the changing trend of net clustering similarity levels off.In Figures4(a), 4(b), and 4(c), the median (S) is the diagonal element in the similarity matrix. We can see that the larger the p value (absolute value), the larger the number of clusters. We do not consider less than two or more than ten clusters, since a too small or too large number of clusters lose internal homogeneity and external heterogeneity. By assigning the p value of 2 × medians (S), median (S), and median (S)/2, we can obtain two, four, and six clusters, respectively. The center of each cluster is labeled by the black star.Figure 4
The clustering results for different fixation types with different damping coefficients.
(a)
p = 2 × median (S); number of clusters = 2
(b)
p = median (S); number of clusters = 4
(c)
p = median (S)/2; number of clusters = 6Different numbers of clusters have different explanations. Givenp = 2 × median (S), the fixation area on the windshield for both circle and rectangle types is divided into two parts. The dividing line for the circle type is slightly to the left of the center and for the rectangle type just the opposite. In Figure 5, a driver looks straight at the target (a car in front or the lane) while driving normally. As the driver is placed on the left side of the car, the driver’s sight line intersects the windshield to the left of the center line. However, usually, a novice driver, who drives slower than average, pays more attention to passing-by vehicles or other information on the left and right sides, making his fixation points diluted.Figure 5
Driver’s sight line under the two divider lines’ situation.Whenp = median (S), clear differences appear. For the circle type (experienced drivers), the area of fixation interest is divided into four parts, which have one-to-one correspondence to the left rearview, right rearview, rearview, and control panel views. Novice drivers’ eye movement characteristics show little attention to the rearview mirror and control panel; therefore, the area of fixation interest is split into four parts horizontally and almost evenly. Figure 5 shows the division lines’ relationship between the novice and experienced drivers on two AOIs situation. It is clear that two types of driver divide attention to left and right differently.Givenp = median (S)/2, the number of clusters increases to six. Figure 4(c) shows trends similar to Figure 4(b). For the circle type, the only difference is that the areas of rearview panel are both divided into two subareas, which is hard to explain reasonably.


## 5. Conclusions and Discussion

Using real vehicle driving tests, eye movement data for 16 drivers was obtained. In this research, fixation points were extracted in the D-LAB Studio to divide the area of visual interest into subareas with the purpose to find out how drivers watch objects in sight and determine the distribution of AOIs and the difference in fixation outcomes between different types of drivers (experienced and novice). According to the clustering results using the MSAP method, the following conclusions can be drawn.The clustering method called the MSAP is suitable for the division of AOIs. Generally, both cluster number and AOI distribution show consistency with our driving common sense. The most important finding is that experienced drivers’ fixation points are distributed in a circle, in which the center is the most-watched region. At the same time, novice drivers’ fixation points are relatively decentralized, showing a rectangular shape. In this rectangle, there is no clear concentration region in both horizontal and vertical directions. Therefore, in eye movement or eye tracking research, it is imperative to distinguish experienced and novice drivers; otherwise, a bias may arise. To obtain a reasonable division of AOIs, using data on experienced participants is preferable.According to Figure4, dividing the front vision area into four subareas (AOIs), in addition to the right mirror area and left mirror area, makes most sense. As shown in Figure 6, the front four AOIs are the left vision region (left windshield and left rearview mirror), right vision region (right windshield and right rearview mirror), rear vision region, and control panel vision region. Since Lethaus and Rataj [3] concluded that eye movements precede drivers’ behavioral features, the four AOIs are consistent with driving habit. Drivers would be fixed at the left windshield area on usual driving, and pay more attention to the rear mirror area when they try to understand the traffic information behind themselves, or they would transform their fixation to the control panel when they like to know whether they were speeding. The differences between experienced and novice drivers are reflected in the rearview mirror and control panel regions. Watching the rearview mirror makes lane change safer, and in-vehicle entertainment devices (located in the control panel area) make driving more relaxed. However, due to relatively limited driving experience, novice drivers rarely look at rearview mirrors and control panels. This phenomenon requires a reasonable design of control panels intended for human-machine interactions requiring as little visual attention as possible.Figure 6
Division of AOIs (number of clusters = 4, experienced drivers).Compared with the AOI division methods mentioning in Introduction, the division of AOIs in Figure6 is more closely related to driver glance behavior. Since Olsen et al. [15] made no difference between experienced and novice drivers and they just divided the AOIs mechanically as shown in Figure 7, actually, drivers sit at the left side in the car and usually pay more attention to the left windshield, so the AOIs division cannot reflect driver concerns when it just use the “front” as an AOI of front windshield in Figure 7.Figure 7
Erik divisions of AOIs.In addition, the AOI division result is not consistent with any previous study. This could be explained by the fact that the gaze dates are collected as usual driving, which include going straight and lane changing, not only one special driving condition. Since Fitch was aimed at analyzing the difference between lane change crashes and near-crashes, the font area was qualitatively divided into the front left, front, and front right [14], but the borderlines between different AOIs are not clear. So it is worth noting that the number of clusters depends on the purpose of study. If we predict lane change intentions, it is suggested to divide the left/right view regions into two (or more) subareas, namely, the windshield and left/right rearview mirror subareas. And the MSAP clustering method would be an effective way to find the AOI borderlines.Together with further investigation on drivers’ eye movement characteristics, the AOI division can provide useful information for control panel design, objects placement in vehicles, and so forth. More importantly, the concept of the AOI can provide a connection between drivers’ behavioral intentions and eye tracking data. For example, a fixation point projected on the left rearview mirror may be an implication of a left lane change. Based on the real-time distribution of fixation points, we can predict the lane change probability during driving. However, this task is for future research.



---

*Source: 3674374-2017-05-21.xml*