# Contactless Real-Time Eye Gaze-Mapping System Based on Simple Siamese Networks
## 3 Eye Gaze-Mapping System
### Facial Detection, Alignment, and Recognition  
We deployed the entire gaze-mapping system to edge devices. Therefore, facial detection and alignment suitable for low-power computing was adopted, as shown in Figure 3. A facial detection module suitable for the purpose of real-time gaze-mapping was modified and applied to MobileNet [38], which is typically used for edge-device computing.  
In the following equation, \(D_{K}\) is the kernel size, \(D_{F}\) is the input channel size, \(M\) is the input channel, and \(N\) is the output channel; the general convolutional computation is shown in Equation (1):  
\[FLOPs(Conv2D)=D_{K}\cdot D_{K}\cdot M\cdot N\cdot D_{F}\cdot D_{F} \tag{1}\]  
However, when the factorized depth-wise separable convolution of MobileNet was applied, the calculation was expressed, as follows:  
\[FLOPs(DepthwiseSeparableConv2D)=D_{K}\cdot D_{K}\cdot D_{F}\cdot D_{F}\cdot M+D_{F }\cdot D_{F}\cdot M\cdot N \tag{2}\]  
According to Equations (1) and (2), the ratios in the calculation were reduced, as compared to the general convolution.  
\[FLOPs.DepthwiseSeparableConv2D=\frac{D_{K}^{2}+N}{D_{K}^{2}N}=\frac{1}{N}+\frac{1}{D _{K}^{2}} \tag{3}\]  
By assuming a kernel size of 3, the MobileNet-based facial detector could achieve approximately 8 times the computational efficiency in FLOPS, and it effectively reduced the memory weight of the facial detection module in the proposed gaze-mapping system.  
\[FLOPs(+ResolutionMultiplier)=D_{K}\cdot D_{K}\cdot\beta D_{F}\cdot\beta D_{F} \cdot\alpha M+\beta D_{F}\cdot\beta D_{F}\cdot\alpha M\cdot\alpha N \tag{4}\]  
Figure 3: Core module of MobileNet was a Depth-wise Conv, followed by BN ReLU. Depth-wise Conv reduced the number of parameters and computational costs more than the normal convolutional 2D process. (**Left**): Depth-wise convolutional structure. (**Right**): Basic convolutional layer structure. Red area: Indicating the bottleneck layer.  
In addition, as shown in Equation (4), when the final calculation was applied, the width and resolution multipliers \(\alpha\) and \(\beta\), respectively, were used to reduce the facial recognition module, so it would be suitable and deployable when using the limited NVIDIA Jetson TX2 hardware. A thin network could be created based on the value of \(\alpha\). By adjusting \(\beta\), the size of the input image and all the internal layers could be reduced by the same ratio. The facial alignment task was closely related to the detection task. Therefore, it was possible to maximize the effectiveness of the sorting when learning by multi-tasking.  
The MTCNN proposed by [PERSON] et al. [34] consisted of three models, P-Net, R-Net, and O-Net, and used a cascaded inference structure. The network structure was designed to learn classification, landmark localization, and the multi-task loss of box regression in a joint-learning manner.  
The main characteristic of MTCNN was implemented in the form of an image pyramid, which was expected to improve the detection and recognition performance by aligning faces of various sizes. In particular, it could maximize the performance of the gaze-recognition module, which was dependent on the facial detection and alignment performance. The facial recognition module applied Inception-ResNet [39]. In this study, the effect of the residual module [40] reduced the convergence speed once a large dataset had been learned. When the facial recognition module was executed, diverse features could be recognized. The triplet loss [41] for learning the discriminative features within the similar texture information was defined according to Equation (5):  
\[Loss=max(0,\alpha+d(f(X_{i})),f(X_{pos}))-d(f(X_{i}),f(X_{neg}))) \tag{5}\]  
where \(f\) is an embedding function and \(d\) is a distance function that measures the distance between two inputs. The embedding distance of the data \(X_{pos}\) were similar to the anchor vector, as the reference point was expected to be greater than the distance from \(X_{neg}\), and the distance function was trained using the \(L2\) distance for the task.