# Analysis of Facial Cues for Cognitive Decline Detection Using In-the-Wild Data
## 3 Methodology
### Feature Extraction
#### 3.3.1 Eye Blink Rate  
The calculation of the EBR is based on the eye aspect ratio (EAR) measure. Six eye landmarks (\(x\),\(y\)), where \(p_{1}\) and \(p_{4}\) represent the horizontal eye corners, and \(p_{2}\), \(p_{3}\), \(p_{5}\) and \(p_{6}\) represent the vertical positions of the eyelids, (as shown in Figure 3), were used to compute the EAR according to Equation (1) [30]. Figure 2: Pipeline of visual feature extraction approach. The top part of the figure shows the calculation of the EAR and then the calculation of the EBRs (\(EBR_{n}\)) based on the different thresholds (\(T_{n}\)) of the whole video. The lower part of the figure illustrates how the three head orientations are extracted and used for two different visual features. \[EAR=\frac{\|p_{2}-p_{6}\|+\|p_{3}-p_{5}\|}{2\|p_{1}-p_{4}\|} \tag{1}\]  
Both eyes' EARs are calculated, and then the average is taken. This average EAR is compared with a particular threshold to decide whether there is an eye closure (i.e., the EAR value is lower than the given threshold) or not in each frame. Following [12], a state machine (SM) is used to determine whether an observed eye closure is a genuine blink. The length of the closure is measured in consecutive frames, ranging from 2 to 30 frames. This range allows for variability in the blink duration, as participant tiredness increases during the recording session, leading to longer or more varied eye closure. For example, shorter eye closures may occur early in the session, while longer closures might be observed as the participant becomes fatigued. The EBR is calculated automatically using an approach based on multiple thresholds (MTs) [12]. The MTs approach calculates the minimum and maximum EAR value over all the participants (OAPs) to generate a range of thresholds for each video. However, the threshold's upper limit might be dominated by very high EAR values caused by large HMs or turns and occluded faces (i.e., outliers). To address this, any value above the third standard deviation (\(\mu+(3\times\sigma)\)) is considered an outlier and is removed. After the outliers have been removed, different step sizes (0.1, 0.01, and 0.001) are then used to produce the resulting thresholds for the MTs approach. For example, with a minimum of 1 and a maximum of 3, using a step size of 0.1, the thresholds would range from 1.0 to 3.0, resulting in 21 thresholds (1.0, 1.1, 2., 2.9, 3.0). To account for variation in video length across participants, we normalized all extracted features, including the EBR, based on the number of frames in each video. This ensures comparability and mitigates potential bias due to differing video durations. In contrast to the previous work that calculated over all participants (MTs) [12], in this work, the MTs approach was developed to become participant-dependent (PD). Thus, MTs-PD calculates the thresholds based on the minimum and maximum for each participant. This approach was chosen to address the issues related to the calculation of the EAR when using the IVA\({}_{34}\) dataset. Figure 4 shows the mean (blue) and the third SD (orange) of each participant. Notably, the mean and the SD of participants in IVA\({}_{18}\) are higher than in IVA\({}_{34}\), which shows that the IVA\({}_{34}\) participants have a lower range for the EAR than the participants in IVA\({}_{18}\). The approach that worked for IVA\({}_{18}\)[25] cannot achieve high performance on the combined IVA\({}_{52}\) data due to the variations in the recording devices and environments and the other challenges in this dataset (see Section 3.1). Thus, the MTs-PD approach was developed. As stated previously, the IVA\({}_{52}\) data include many challenges that resulted in many extremely high values in the EAR. In addition, calculating the mean and the 3 rd SD of each participant using the EAR showed a significant difference between the IVA\({}_{18}\) and the IVA\({}_{34}\) datasets.