# A Study of Eye-Tracking Gaze Point Classification and Application Based on Conditional Random Field
## 2 Proposed Method
### CRF Model Implementation Details  
In the gaze image \(I\), the target area is first manually marked:  
\[R_{T}=\big{\{}r\big{(}r_{x},r_{y}\big{)}\big{|}r_{x}\in\big{[}x,x^{\prime} \big{]},r_{y}\in[y,y^{\prime}]\big{\}}, \tag{6}\]  
where \(r\big{(}r_{x},r_{y}\big{)}\) denotes the area of the target in image \(I\), \(r_{x}\in[x,x^{\prime}],r_{y}\in[y,y^{\prime}]\) represents the range of values of the area, the location of the gaze point within this range. We classify and label the visual characteristics of the labeled gaze point as \(Y\).  
The training dataset contains gaze points \(D\), gaze image \(I\) and gaze points of classification markers \(Y\). Using the previously designed model, the corresponding relationship between these elements in the training samples is established, the classification of test gaze points is estimated, and the useful meaning gaze points are obtained. The CRF model is  
Figure 5: The input feature fusion module.  
developed for input feature decoding, according to Equation (1) of Section 2.2, and the probability function is given by:  
\[P(Y|G)=\frac{1}{Z}\exp\{-E(Y|G(D,I)\}, \tag{7}\]  
where \(G\) is the feature of the gaze point which is a fused visual feature to the gaze point representation. \(Y\) is the feature of the gaze point class marker, and \(E\) is expressed as:  
\[E(Y|G(D,I))=\sum\sum P_{U}(Y_{i},G(D,I))+\sum P_{V}(Y_{i-1},Y_{i},G(D,I)), \tag{8}\]  
where \(P_{U}\) and \(P_{V}\) are, respectively, unary feature function and binary neighborhood feature function of the gaze point, \(P_{U}\) denotes the function between feature \(G\) and class label \(Y\) at a moment of gaze, which is used to describe the influence of the feature on the class label.  
Using the previous analysis, \(P_{U}\) is expressed as:  
\[P_{U}(Y_{i},G(D,I))=P_{U}(Y_{i},G_{t}), \tag{9}\]  
where \(Y_{i}\) is the \(i\)-th feature of marked gaze points.  
The neighborhood feature function between each gaze point \(Y_{i-1}\) and \(Y_{i}\) is used to describe the relationship between neighboring gaze points:  
\[P_{V}(Y_{i-1},Y_{i},G(D,I))=P_{V}(Y_{i-1},Y_{i},G_{t}), \tag{10}\]  
The specific \(G_{t}\) describes the encoding and relationship building for various inputs in the previous section, using CRF as a decoder for the interpretation of the above relationship:  
\[y^{*}=\arg\max score_{Y\in Y_{x}}(Y,G(D,I)), \tag{11}\]  
The maximum fractional output is calculated using the Viterbi algorithm [24], where \(y^{*}\) denotes the predicted classification gaze points, \(Y\) presents the marked gaze points and \(Y_{x}\) denotes the possible gaze points.  
In summary, the proposed gaze point classification method of conditional random field based on visual characteristics is summarized as follows (Algorithm 1):  
```
Input: Training data and Testing data, containing gaze points D, gaze images I and marker gaze points Y. Output: The optimal classification gaze points \(y^{*}\) of Testing gaze points 1. Extracting the features of gaze images \(G^{s}_{t}\) and the features of gaze points \(G^{a}_{t}\) 2. The weight predictor based on LSTM to obtain the weight vector \(\omega_{t}\) of each feature channel of the gaze point (Equation (4)), fusing the features of gaze point and gaze image 3. Inputting all feature information into CRF to decode and acquire the corresponding training parameters 4. Extracting feature information for the gaze point and gaze image in the test data (Equations (9) and (10)) 5. Calculating the function \(E\) (Equation (8)) using the step 3, to find the optimal classification gaze points \(y^{*}\) of the test gaze points (Equation (11)) 6. return Classification result of gaze points
```  
**Algorithm 1**A method for gaze point classification based on conditional random field and visual characteristics  
The parameters of the proposed method are shown in Table 2: