## Materials and Methods
### Obtaining the Gaze Direction \(\varphi_{g}\)
#### 2.2.2 Obtaining the Gaze Point in the RGB Camera Coordinate System \((x_{g}^{c},y_{g}^{e})\)  
World camera coordinates of the gaze point were converted into the RGB camera coordinate system to facilitate acquisition of the gaze point in the RGB camera image. To this end, an image-matching exercise, called template matching, was performed, and coordinates of the gaze point were translated and converted using the results of template matching. This image matching is used because the processing speed is fast.  
Template matching is a method of checking whether patterns similar to those existing in a certain region of a template image are present in a whole image. Examples of a template matching are shown in Figure 4. The similarity \(S_{NCC}(x^{e},y^{e})\) is calculated using the luminance value of the image while sequentially moving the template image over the whole image to identify the region demonstrating highest similarity. Here, determination of the similarity utilizes the Normalized Cross-Correlation (NCC). NCC is calculated using Equation (1). \(T(i,j)\) represents the luminance value of the template image while \(I(i,j)\) represents that of the whole image. NCC indicates that the closer its value is to 1, the higher the similarity is.  
\[S_{NCC}(x^{c},y^{c})=\frac{\sum\limits_{j=0}^{N-1}\sum\limits_{i=0}^{M-1}I(i,j)T(i, j)}{\sqrt{\sum\limits_{j=0}^{N-1}\sum\limits_{i=0}^{M-1}I(i,j)^{2}\cdot\sum \limits_{j=0}^{N-1}\sum\limits_{i=0}^{M-1}T(i,j)^{2}}} \tag{1}\]  
In the proposed study, a part of the world camera image was cut out as the template image, and the RGB camera image was considered the whole image. In order to speed up the processing, the size of the RGB camera image was compressed from 1920 \(\times\) 1080 pixels to 384 \(\times\) 216 pixels while that of the world camera image was reduced from 1280 \(\times\) 720 pixels to 344 \(\times\) 194 pixels. From the size of the world camera image, cut out 172 \(\times\) 96 pixels which is about half size in both length and width. The part to be cut off forms the part centered at the point of regard. Figure 4a-c depicts examples of the RGB camera image, gaze point captured in the world camera image, and the template image in the crossroad.  
Template matching is performed using the above images. The pixel coordinate when the similarity \(S_{NCC}(x^{c},y^{c})\) is highest is defined as \(\left(x_{NCC}^{c},y_{NCC}^{c}\right)\). At this time, the gaze point in the world camera image is transformed into the RGB camera image using Equation (2).  
\[\left(x_{g}^{c},y_{g}^{c}\right)=\left(x_{g}^{c},y_{g}^{c}\right)+\left(x_{NCC }^{c},y_{NCC}^{c}\right) \tag{2}\]  
In this study, we use of only the matching result when \(\left(x_{NCC}^{c},y_{NCC}^{c}\right)\) is less than 50 pixels compared with the value before a time as shown in the following equation.  
\[\left\{\begin{array}{l}\left|x_{NCC}^{c}(t)-x_{NCC}^{c}(t-1)\right|<50\\ \left|y_{NCC}^{c}(t)-y_{NCC}^{c}(t-1)\right|<50\end{array}\right. \tag{3}\]  
Figure 4: Example of determination of gaze-point coordinates (**a**) RGB camera image (Whole image); (**b**) Gaze point in the world camera coordinate system; (**c**) Template image; (**d**) Checking similar patterns; (**e**)Result obtained from template-matching exercise; (**f**) Gaze point in the RGB camera coordinate system.  
Figure 4d-f show examples of the result obtained via template matching and gaze point in the RGB camera image in the crossroad.  
Through use of only the matching result \(S_{NCC}\left(x_{NCC}^{c},y_{NCC}^{c}\right)>0.9\), we obtained more reliable gaze information. This value \(S_{NCC}\left(x_{NCC}^{c},y_{NCC}^{c}\right)\) is the other of the two values affecting the gaze detection accuracy.