# An Image Processing-Based Method to Analyze Driver Visual Behavior Using Eye-Tracker Data
## 3 Methodology
### Data Collection and Pre-Processing  
The Pupil Labs eye tracker device's handbook describes the World Camera as the physical scene camera that records the subject's field of vision (called world video), referring to the driver's perspective. The world video was acquired from the Pupil Labs 3.0 software, together with the timestamps of each frame. Because the gaze locations are given in 2D normalized space, a program was built to convert them to 2D image space. To determine the  
Figure 3: The road circuit’s segmentation. The rectilinear segment 5, and the curvilinear segment 2 were selected for the analysis.  
Figure 2: A SketchUp 3D design of the La Masa area, Bovisa Campus.  
exact position of the gaze on the image of the frames, the gaze coordinates were projected onto the image frame of 1280 \(\times\) 720 pixels. Consequently, precise gaze positions on the frame photos were obtained. Constructing a reference array addressed the difference in timestamps between gaze locations and video frames. This reference array facilitated the selection of the nearest frames and gaze positions to a new reference array that still had a 100-millisecond time step. The choice of this time step for the reference array was aligned with the CarMaker vehicular data sampling time step. The matching procedure involved filtering the nearest time steps of the frames and gaze positions based on the reference array. The results of the matching procedure indicated a satisfactory degree of precision, with the most significant error being on the order of 10 milliseconds. An example of the matching procedure is provided in Table 1, where the accuracy (\(\mid\) gaze_time-frame_time\(\mid\)) column displays the errors, and the frame_time and gaze_time columns display the time steps of the nearest frames and gaze positions chosen relative to the reference array.  
This sequence of operations is synthesized in the flow chart in Figure 4. Table 2 reports the total number of available frames and those used for the analysis. It is easy to see that the percentage of frames analyzed in the curvilinear segment is generally lower than in the rectilinear one due to the substantial change in the driver's perspective and the shorter length of the curvilinear segment. Obstructions to the driver's view, such as the front and side windows or side-view mirrors, may prevent the capture of road boundary coordinates, resulting in frames where one or both road borderlines cannot be discerned and are excluded from the analysis. In addition, the analysis utilized only a tiny fraction of the total frames in the segments, ranging from 9% to 22%. This limited proportion was mainly caused by the need to match eye-tracker video camera frames with vehicular data sampled at a 100-millisecond timestep.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Segment** & \begin{tabular}{c} **Gaze Location** \\ **in the X-Axis** \\ \end{tabular} & \begin{tabular}{c} **Gaze Location** \\ **in the Y-Axis** \\ \end{tabular} & \begin{tabular}{c} **Time** \\ **(Reference)** \\ \end{tabular} & \begin{tabular}{c} **Gaze\_Time** \\ **(Reference)** \\ \end{tabular} & \begin{tabular}{c} **Frame\_Time** \\ **(Reference)** \\ \end{tabular} & \
\begin{tabular}{c} **Accuracy** \\ \end{tabular} &
\begin{tabular}{c} **Frame\_Number** \\ \end{tabular} \\ \hline
2 & 677.391 & 415.385 & 103.2 & 103.204 & 103.195 & 0.0087 & 4861 \\
2 & 677.391 & 415.385 & 103.3 & 103.300 & 103.296 & 0.0041 & 4866 \\
2 & 677.391 & 415.385 & 103.4 & 103.396 & 103.396 & 0.0004 & 4872 \\
2 & 677.391 & 415.385 & 103.5 & 103.501 & 103.514 & 0.0130 & 4878 \\ \hline \hline \end{tabular}
\end{table}
Table 1: A partial listing of nearest frame timestamps to gaze position timestamps for segment 2.  
Figure 4: Flowchart summarizing the matching of eye-tracker video camera frames with vehicular data.