# Machine Learning for Cognitive Load Classification – A Case Study on Contact-Free Approach

Mobyen Uddin Ahmed, Shahina Begum, Rikard Gestlöf, Hamidur Rahman, Johannes Sörman

► To cite this version:

Mobyen Uddin Ahmed, Shahina Begum, Rikard Gestlöf, Hamidur Rahman, Johannes Sörman. Machine Learning for Cognitive Load Classification – A Case Study on Contact-Free Approach. 16th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2020, Neos Marmaras, Greece. pp.31-42, 10.1007/978-3-030-49161-1_3. hal-04050591

HAL Id: hal-04050591

https://inria.hal.science/hal-04050591

Submitted on 29 Mar 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Machine Learning for Cognitive Load Classification – a
Case Study on Contact-free Approach

Mobyen Uddin Ahmed, Shahina Begum, Rikard Gestlöf, Hamidur Rahman, Johannes
Sörman

School of Innovation Design and Engineering (IDT), Mälardalen University,
Västerås, Sweden Tel.: + 12345678, fax: + 87654321

E-mail: {mobyen.ahmed, shahina.begum, hamidur.rahman}@mdh.se

**Abstract.** The most common ways of measuring Cognitive Load (CL) is using physiological sensor signals e.g., Electroencephalography (EEG), or Electrocardiogram (ECG). However, these signals are problematic in situations e.g., in dynamic moving environments where the user cannot relax with all the sensors attached to the body and it provides significant noises in the signals. This paper presents a case study using a contact-free approach for CL classification based on Heart Rate Variability (HRV) collected from ECG signal. Here, a contact-free approach i.e., a camera-based system is compared with a contact-based approach i.e., Shimmer GSR+ system in detecting CL. To classify CL, two different Machine Learning (ML) algorithms, mainly, Support Vector Machine (SVM) and k-Nearest-Neighbor (k-NN) have been applied. Based on the gathered Inter-Beat-Interval (IBI) values from both the systems, 13 different HRV features were extracted in a controlled study to determine three levels of CL i.e., S0: low CL, S1: normal CL and S2: high CL. To get the best classification accuracy with the ML algorithms, different optimizations such as kernel functions were chosen with different feature matrices both for binary and combined class classifications. According to the results, the highest average classification accuracy was achieved as 84% on the binary classification i.e. S0 vs S2 using k-NN. The highest $F_1$ score was achieved 88% using SVM for the combined class considering S0 vs (S1 and S2) for contact-free approach i.e. the camera system. Thus, all the ML algorithms achieved a higher classification accuracy while considering the contact-free approach than contact-based approach.

**Keywords:** Cognitive Load (CL), contact-free approach, k-nearest-neighbor (k-NN), Support Vector Machines (SVM), Machine Learning (ML)

## 1 Introduction

Driving is a complex task that requires high concentration with simultaneous skills and abilities [1]. Research shows that, most of the traffic accidents are caused by the human error. [1]. Most of the cases a road accident occurs when a driver loose concentration on driving due to look outside or thinking anything rather than driving task. Therefore, there is a connection between low concentration level and traffic accidents has been
identified in [2]. Research shows that the number of road deaths increased by 28% in Sweden, by 14% in the Czech Republic and by 11% in the Netherlands in 2018 compared to 2017 [3]. Over 20% of those 90% are due to fatigue or CL (e.g. use of in-vehicle devices which can lead to cognitive overload for the driver) [4]. CL is also known as cognitive workload, which is a concept that describes the relationship between the cognitive demands from a task and the environment that influence the user's cognitive resources [5, 6].

To identify CL, psycho-physiological measures are mainly used, as they assess interactions between physiological and psychological states of a human. The most common psychophysiological measurement to detect CL is EEG, because it objectively identifies the cognitive cost of performing tasks [6]. Other parameters are also used to detect CL, such as: pupil dilation, HRV, galvanic skin response (GSR), and EEG etc. [7]. There are disadvantages with these contact-based approaches as most of them used electrodes and they should be connected to the human body, which can be cumbersome to use in situations such as driving. In many occasions the measurement signals are affected due to movements and artifacts [8]. That is why a contact-free way of measuring CL could be ideal for these situations. On the contrary, the contact-free approach e.g. camera systems have been incessantly growing in the research community [9]. Here, the camera system have developed based on color schemas such as RGB and the Lab color space to extract heart rate (HR) and IBI by detecting variation in facial skin-color caused by cardiac pulse [9]. McDuff et. al [10] has built a remote person independent classifier and predict CL with 85% classification accuracy, here, a contact-free camera-based system was used in order to measure HR, HRV and breathing rate (BR).

In order to detect or classify CL, HRV features are used, since the HR fluctuates with varying levels of CLs which can be detected with HRV metrics [11]. In medical diagnoses and stress detection HRV is a commonly used parameter, there are many different features in HRV that can be calculated and can be used as a parameter to estimate CL. There are three established ways to calculate the HRV, either by time-domain, frequency-domain or non-linear measurements [12]. CL detection is a very inefficient for humans to manually review large data sets because of the factors that occur naturally in everyday life situations [13]. This is why the ML algorithms are an essential factor in classifying CL.

The data collection for cognitive load research are generally conducted either in in-door environment using simulator driving or in outdoor environment using real road driving scenario [14-16]. The authors in [17] have focused on the feasibility of using visual attention features to classify CL in three different states i.e., low, medium and high cognitive load. Four different classification models such as k-NN, logistic regression, SVM, adaptive boosting and random forest have used and the k-NN received the highest accuracy of 81 %. A study performed by Wang et. al. [18] has received a 97.2 % of classification accuracy in detecting high CL.

This paper presents a case study on contact-free approach for CL classification based on the features extracted through HRV analysis and using on ML. Here, the goal of the study is to compare the contact-free approach i.e., camera-based system with the contact-based approach i.e., a Shimmer GSR+ system to determine how well the contact-
free approach performs in detecting CL. To classify the CL, two different ML algo-
rithms, mainly, SVM and k-NN are used. This study includes a collection of data for
both the systems in a controlled environment. Thirteen HRV features were extracted to
determine three states of CL based on the gathered IBI values from both the systems.
The three levels are, S0: reference point for low CL, S1: normal CL where the test
subjects performed easy puzzles and drove normally in a video game and S2: high CL
where the test subjects completed hard puzzles and drove on the hardest course of a
video game while answering math questions [19]. To get the best classification accu-
racy with the ML algorithms, different optimizations such as kernel functions i.e., radial
basis function (RBF), and sigmoid function were chosen with different feature matrixes
both for binary and combined class classifications.

## 2 Materials and Methods

In the contact-free approach, a Logitech HD Webcam C615 camera with the resolution of 1920x1080, and a frame rate of 30fps equipped with auto focus was used to record faces of the test subjects. The recorded videos were saved in MP4 format. The camera was adjusted according to the height of the head of the test subject so that it can capture the facial image properly. The test subjects were sat roughly 70cm in front of the camera during the different phases of the controlled experiment, thus the desired ROI of the test subjects and facial features were extracted [9]. During the data collection, the test subjects were asked to move their heads as normal but not too much.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Overview of the workflow for measuring CL</figcaption></figure>

In the contact-based approach, a Shimmer GSR+ system was wired up to the sub-
jects' body through a clamp to the earlobe, which provided the corresponding IBI val-
ues. The data recorded by the shimmer sensor was connected to a laptop via Bluetooth.
The standard settings of Consensus were recorded as Unix_Timestamps, PPG_A13 as
IBI and PPGtoHR. The IBI value was recorded by the shimmer system and saved in
Excel sheets through a software called Consensus to monitor and manage the data gath-
ered by the shimmersensing¹. The shimmer had a sampling rate of roughly 40000 after

¹ http://www.shimmersensing.com/products/consensys
a 5-minute test run. Here the frequency of the sensor reading was 133 Hz. An overview of the workflow used in this study is presented in Figure, that describes the different parts i.e., Data collection, Data Processing, Data Set Preparation, Classification Learner, ML Algorithms, Model Classifier, Classification Results.

## 2.1 Data collection

This study includes 11 test subjects with age between 21 and 33 years. Among them 9 were men with average age of 26 and two were women with average age of 27. Again, four of them had lowest education in math, three of them had highest math education and the rest had average math courses. The mean value of how good they see themselves in mental arithmetic were 5,9 in a scale of 10. The mean value of time spent playing digital games per week was 13 hours, but the time variation is high between the test subjects. The one with the highest hours per week had forty hours, and the lowest had one hour. Two of the eleven test subjects only played on mobile, and the rest played on console/computer.

The data was collected through a controlled study consists of seven phases [19], between phases there were a minor pause to set the next phase up and in the fifth phase there were a 15-minute break. The 1st phase of this experiment was a five-minute rest while sitting normal and watching TV silently. The 2nd Phase was to complete the two different easy metal puzzles while sitting normal. Here, the test subject played an easy puzzle first, and when they have disassembled the first one, they started over the next one. When both the puzzles were disassembled the test subject would start to reassemble both the puzzles in the same order. This process was repeated for five minutes. The 3rd Phase was about to play both the metal puzzles with a higher difficulty. The 4th phase was about the test subject rested for 15 minutes and were offered coffee and snacks. The 5th phase was about to sit a normal watching TV silently same as in phase one, for five minutes. In the 6th phase the test subjects play a game 'Mario Kart Double Dash' on the 150cc course BOWER'S CASTLE competing against seven bots. The time of this phase was five minutes; however, the time was varied depending on how well the test subjects performed. Finally, the 7th phase is about to play the 'Mario Kart Double Dash' 150cc once again, but on RAINBOW ROAD, which is the final and the hardest course of the game. Here, instead of competing against seven bots the test subjects had to answer math questions while playing. The math questions had to be answered before the next question was asked, the questions were asked in every 30 seconds and five questions in total for each time. Table 1 illustrates the summary of the phases.

<table><caption>**Table 1.** Summary of each phase during the data collection</caption><thead><tr><th>Phases</th><th>1<sup>st</sup></th><th>2<sup>nd</sup></th><th>3<sup>rd</sup></th><th>4<sup>th</sup></th><th>5<sup>th</sup></th><th>6<sup>th</sup></th><th>7<sup>th</sup></th></tr></thead><tbody><tr><th>Duration (min)</th><td>5</td><td>5</td><td>5</td><td>15</td><td>5</td><td>5</td><td>5</td></tr></tbody></table>
<table><tr><td>Description</td><td>Normal sitting and watching TV</td><td>Normal sitting and solving easy puzzle</td><td>Normal sitting and solving hard puzzle</td><td>Rest and drinking coffee/tea</td><td>Normal sitting and watching TV</td><td>Playing driving game</td><td>Playing driving game and solve math</td></tr><tr><td>Goal</td><td>Low cognitive level</td><td>Medium High cognitive level</td><td>High cognitive level</td><td>Relax and recover state</td><td>Low cognitive level</td><td>Medium High cognitive level</td><td>High cognitive level</td></tr><tr><td>Datasets</td><td>S0</td><td>S1</td><td>S2</td><td></td><td>S0</td><td>S1</td><td>S2</td></tr></table>

## 2.2 Data Pre-processing

For each phase of every test subject, the sample size was 40 000 on an average. A python program with the library OpenPyxl was created in order to handle samples, the program both transformed the UNIX time to 'date time' and removed the samples where the IBI was -1.0. Thus, a smaller sample size is reduced to 250-500 depending on the recorded time as well as the precision of the device. The reason of this huge decrease was that the IBI was calculated approximately once per second, since the HR was around 60-80 BPM for all the test subjects. This resulted in roughly 300 recorded IBI's for each phase of every test subject, since the length of each experiment phase was five minutes long.

For the camera data, IBI was extracted from the recorded video and saved in a separate file. From each of the recorded video, first, the face of the test subject was detected and then a region of interest (ROI) was selected from the detected face [20]. Each ROI was converted into three color frames which are red, green and blue which are transferred into Lab color space or Lab signal. Three signal processing algorithm First Fourier Transform (FFT) [21], Independent Component Analysis (ICA) [22] and Principle Component Analysis (PCA) [23] were used to extract IBI from the Lab signal by applying a band pass filter of 40-120 Hz [24]. An overview of the camera system that extracts IBI from a video recording is presented in Figure 2.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Different steps of a camera system to extract IBI from camera data</figcaption></figure>

## 2.3 Feature extraction

Once the collected data were structured in a manner that would make it easier for a script to extract features from the recorded data the implementation of the feature extraction session started with the so-called trial and error phase in order to find a way of extracting all the HRV features properly. The purpose of the script was to reduce the total duration of the recorded time into three minutes and only consider the middle of the recording and remove the data from the beginning and end of the recording to avoid faulty recording. These three minutes were then be further divided into three parts of one minute each. Then for each minute of these three minutes data, the feature function in MATLAB was used to calculate the 13 features where six time domain features and seven frequency domain features as shown in Table 2 and Table 3 respectively. Then the features for each phase was calculated for each test subject. This was resulted in a 3*14 matrix linked to one class (in the controlled experiment phase), where SittingNormal = S0, Driving 1 or Puzzle 1 = S1, Driving 2 or Puzzle 2= S2.

<table><caption>Table 2. List of different time-domain HRV Features</caption><thead><tr><td>Feature</td><td>Description</td><td>Unit</td></tr></thead><tbody><tr><td>MeanNN</td><td>Is the average of all NN-intervals</td><td>ms</td></tr><tr><td>SDNN</td><td>Standard deviation of NN-intervals</td><td>ms</td></tr><tr><td>RMSSD</td><td>Root mean square successive RR interval differences</td><td>ms</td></tr><tr><td>SDSD</td><td>Standard deviation of successive differing between neighbor NN-intervals</td><td>ms</td></tr><tr><td>NN50</td><td>Number of pairs of successive NN-intervals differing by more than 50ms</td><td>int</td></tr><tr><td>pNN50</td><td>Percentage of successive RR-intervals differing by more than 50ms</td><td>%</td></tr></tbody></table>

<table><caption>Table 3. List of different frequency-domain HRV Features</caption><thead><tr><td>Feature</td><td>Description</td><td>Unit</td></tr></thead><tbody><tr><td>VLF</td><td>Absolute power of very-low-frequency (0.003- 0.04 Hz)</td><td>ms<sup>2</sup></td></tr></tbody></table>
<table><tr><td>LF</td><td>Absolute power of low-frequency (0.04- 0.15 Hz)</td><td>ms<sup>2</sup></td></tr><tr><td>HF</td><td>Absolute power of high-frequency (0.15- 0.40 Hz)</td><td>ms<sup>2</sup></td></tr><tr><td>TotalPower</td><td>Variance of all NN-intervals (approximately ≤ 0,4 Hz)</td><td>ms<sup>2</sup></td></tr><tr><td>LF/HF ratio</td><td>Ratio of LF-HF power</td><td>%</td></tr><tr><td>LF (nu)</td><td>Relative power of low-frequency (0.04- 0.15 Hz) in normal units</td><td>nu</td></tr><tr><td>HF (nu)</td><td>Relative power of high-frequency (0.15- 0.40 Hz) in normal units</td><td>nu</td></tr></table>

## 2.4 Classification

For the classification using k-NN and SVM, the whole data sets were divided into four groups both for shimmer and camera data separately as follows:

* S0 vs S1, Normal sitting vs Driving normal

* S0 vs S2, Normal sitting vs Driving with distractions

* S1 vs S2, Driving normal vs Driving with distractions

* S0-S1-S2, Combined classification

First, 80% of the data for each group of data set was randomly selected for training and 20% for data was selected for testing. Binary classification was conducted for this paper works and two ML algorithms e.g. k-NN and SVM were considered an instance of supervised learning in which an algorithm learns to classify new observations from examples of labeled data [25]. SVM constructs a hyperplane or a set of hyperplane to analyses data to identify pattern and commonly used for classification and regression analysis [26]. K-NN is a nearest-neighbor classification model in which it can be altered both the distance metric and the number of nearest neighbors. Because a k-NN classifier stores training data, the model is used to compute re-substitution predictions. Alternatively, this model is used to classify new observations using the predict method [27]. Different kernel functions were investigated both for k-NN and SVM which were weighted, fine and medium kernel for k-NN and cubic, fine and medium kernel for SVM. The best cross-validation k-fold value was 10 for both k-NN and SVM. The binary SVM classifier was implemented according to [eq. 1], the fitcsvm2 is a built-in function in MATLAB that returns a trained SVM model, based on the features of the training set x and the list with the corresponding class y. The parameters 'KernelFunction' indicates that the kernel used in this model should be the variable k. The parameter 'KernelScale' indicates that the kernel scale used in this model should be the variable s. 'BoxConstraint' is the parameter indicating that the boxconstraint of the model should be the variable b.

$$trainedSVM=fitcs(x,y,'KernelFunction',k,'KernelScale',s,'BoxConstraint',b) \quad (1)$$

For k-NN model, [eq. 2] was implemented which is also a MATLAB built-in function. The function fitck-NN returns a trained K-NN model and based on the features of the training set x and the list with the corresponding class y. The parameter NumNeighbors' flags that the variable k should be used as the number of neighbors

<footnote n="2">² https://se.mathworks.com/help/stats/fitcsvm.html</footnote>
for this K-NN model. 'DistanceWeight' is a parameter in the fitck-NN that indicates
that the variable *w* should be used as the distance weight.

$$
\text{trainedKNN} = \text{fitck}(x, y, \text{'NumNeighbors'}, k, \text{'DistanceWeight'}, w) \quad (2)
$$

The optimized classifiers were implemented for each training set of the class combina-
tions based on the feature matrixes. The trained models were then given the features of
the test sets and based on how they predicted the class of the features, the accuracy of
each model could determine by how many correct class predictions it achieved. Each
model executed 20 times for each data set and mean value of the result was considered.

3 Experimental Results

The classification learner revealed that the two best classification algorithms for the
collected data were k-NN and SVM. Classification accuracy for the binary data sets (S0
vs S1), (S0 vs S2), (S1 vs S2) and (S0 vs S1 and S2) and class classification accuracy
for the combined data sets (S0, S1, S2) are calculated using ML algorithms i.e., k-NN
and SVM. Also, F₁-score is calculated for each of the data set using the ML algorithms.
A summary of the classification accuracy and F₁-score for the shimmer data is pre-
sented in Table 3 and Table 4 presents the classification accuracy for the camera data.
The highest value of F₁-score both for the shimmer and camera are highlighted in gray
color in the table.

<table><caption>**Table 4.** Classification results using Shimmer system</caption>
  <thead>
    <tr>
      <th rowspan="3">Classification</th>
      <th rowspan="3">Data Set</th>
      <th colspan="4">Shimmer</th>
    </tr>
    <tr>
      <th colspan="2">k-NN</th>
      <th colspan="2">SVM</th>
    </tr>
    <tr>
      <th>Mean accuracy with 20 runs</th>
      <th>F<sub>1</sub></th>
      <th>Mean accuracy with 20 runs</th>
      <th>F<sub>1</sub></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td rowspan="3">Binary</td>
      <td>S0 vs S1</td>
      <td>0.74</td>
      <td>0.7</td>
      <td>0.74</td>
      <td>0.78</td>
    </tr>
    <tr>
      <td>S0 vs S2</td>
      <td>0.78</td>
      <td>0.74</td>
      <td>0.66</td>
      <td>0.79</td>
    </tr>
    <tr>
      <td>S1 vs S2</td>
      <td>0.58</td>
      <td>0.41</td>
      <td>0.58</td>
      <td>0.65</td>
    </tr>
    <tr>
      <td>Combination</td>
      <td>S0 vs S1 and S2</td>
      <td>0.72</td>
      <td><strong>0.86</strong></td>
      <td>0.68</td>
      <td>0.85</td>
    </tr>
  </tbody>
</table>

<table><caption>**Table 5.** Classification results using Camera system</caption>
  <thead>
    <tr>
      <th rowspan="3">Classification</th>
      <th rowspan="3">Data Set</th>
      <th colspan="4">Camera</th>
    </tr>
    <tr>
      <th colspan="2">k-NN</th>
      <th colspan="2">SVM</th>
    </tr>
    <tr>
      <th>Mean Accuracy with 20 runs</th>
      <th>F<sub>1</sub></th>
      <th>Mean Accuracy with 20 runs</th>
      <th>F<sub>1</sub></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td rowspan="3">Binary</td>
      <td>S0 vs S1</td>
      <td>0.71</td>
      <td>0.71</td>
      <td>0.70</td>
      <td>0.68</td>
    </tr>
    <tr>
      <td>S0 vs S2</td>
      <td>0.81</td>
      <td>0.84</td>
      <td>0.74</td>
      <td>0.74</td>
    </tr>
    <tr>
      <td>S1 vs S2</td>
      <td>0.67</td>
      <td>0.57</td>
      <td>0.65</td>
      <td>0.66</td>
    </tr>
  </tbody>
</table>
<table><tr><td>Combination</td><td>S0 vs S1 and S2</td><td>0.79</td><td>0.87</td><td>0.72</td><td><b>0.88</b></td></tr></table>

For k-NN and SVM, a graphical representation of average *F*<sub>1</sub>-score between shimmer and camera is presented in Figure 3 and Figure 4. From the figures it can be seen that *F*<sub>1</sub>-score for camera is larger than Shimmer both for k-NN and SVM considering binary and combined class classification.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Graphical representation of *F*<sub>1</sub>-score between Shimmer and Camera data considering binary classification.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Graphical representation of *F*<sub>1</sub>-score between shimmer and camera data considering classification of combined class</figcaption></figure>

# 4 Discussion and Conclusion

The experimental result shows that highest accuracy has been achieved in a controlled environment using the camera recording during the driving phase. Also, this shows that binary classification using the k-NN algorithm with the class combination of S0 vs S2
achieves an accuracy of 81% which was higher than the expected accuracy from the classification learner by one percent. With the same class combination and setup recorded with the Shimmer sensor system only got 78% accuracy, which is higher than the expected accuracy 70%. For the class combination S0 vs (S1 and S2), the highest $F_1$ score 84% was achieved with medium gaussian SVM using the feature matrix recorded from the Camera system.

The results also demonstrate that the ML algorithms could classify CL in Driving using video games with equal or even higher accuracy than a traditional way (i.e., using metal puzzles). This is significant because controlled experiments that involve actual video games have not been extensively used in the state-of-the-art research. This research also reveals that the accuracy in classifying CL between driving normal and driving with distractions was greater than the accuracy between the Easy puzzle and Hard puzzle. The reason for this might have been that the difficulty between the easy puzzle and hard puzzle was too small. If math questions had been added to the hard puzzle phase of the controlled experiment the results could have been different. The best improvement was achieved from the testing with an increase of the accuracy with 3% based on the focus to feature selection. The MATLAB function sequentialfs (Sequential Feature selection) was used to determine the best features among the extracted features. This shows that the NFFT feature, was not necessary and therefore NFFT was removed from the feature sets. After analyzing more into the features, the MeanNN, RMSSD, VLF and TotalPower were found the most robust features while considering stable accuracies. The data set S2 is provided more accuracy compare to S0 using contact-free approach since it was the most difficult and the hardest difficulty, the driver usually ended up so far behind the bots on the course that they were not disturbed by the bots in the game. The math questions were used to make distraction of the driver which means there should have difference level of cognitive load; however, practically it was not found the difference between cognitive load levels due to less distraction of the drivers by math questions. The choice of an easier track could have made it possible to have the data differentiate even more than it did. Also, the time determined when a math question would be asked, every 30 seconds. This made most of the questions to be asked when the driver had a straight line without any worries and could therefore answer the question more easily. Also, more questions could have been held at hand in order for the questions to stretch across the whole course.

When conducting the controlled experiment, the two recording systems (i.e., Shimmer and Camera) had to be started at the same time. The challenge with this was that a button had to be pressed on each system in order to start the recordings. This made that the two system recordings had a minor time difference of a few ms for each recording for each of the test subjects. Also, when cutting the recordings from five minutes into three minutes the implementation cut at the one-minute mark. The same thing occurred when the implementation cut the last minute from the recording at the three-minute mark. Because of this cutting the time gap was small enough that it should not affect the result.

With these findings it seems plausible that the contact-free system can be used reliably in real driving scenario. However, it would also be interesting to investigate if there is something that increase the CL more than other things and try to categorize how much it increased the CL. Also, do more experiments with the game over a longer
period of time, to see how different test subjects will adapt to the Mario Kart game. Here, only the frequency and the time-domain features were considered. So, it would also be interesting to involve the non-linear HRV features in the future.

**Acknowledgment.** The authors would like to acknowledge Embedded Sensor Systems for Health Plus (ESS-H+) for their support of the research projects. The authors would also like to acknowledge all the participants for supporting in this study.

References

[1] J.-P. Kim and S.-W. Lee, *Time domain EEG analysis for evaluating the effects of driver's mental work load during simulated driving*. 2017, pp. 79-80.

[2] M. Miyaji, H. Kawanaka, and K. Oguri, "Study on effect of adding pupil diameter as recognition features for driver's cognitive distraction detection," in *2010 7th International Symposium on Communication Systems, Networks & Digital Signal Processing (CSNDSP 2010)*, 2010, pp. 406-411.

[3] "Road Safety Annual Report 2019, OECD Publishing, Paris," OECD/ITF (2019).

[4] P. Sena, M. d'Amore, M. Pappalardo, A. Pellegrino, A. Fiorentino, and F. Villecco, "Studying the influence of cognitive load on driver's performances by a Fuzzy analysis of Lane Keeping in a drive simulation," *IFAC Proceedings Volumes*, vol. 46, no. 21, pp. 151-156, 2013/01/01/ 2013.

[5] O. Palinko, A. L. Kun, A. Shyrokov, and P. Heeman, "Estimating cognitive load using remote eye tracking in a driving simulator," presented at the *Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications*, Austin, Texas, 2010. Available: https://doi.org/10.1145/1743666.1743701

[6] N. Kumar and J. Kumar, "Measurement of Cognitive Load in HCI Systems Using EEG Power Spectrum: An Experimental Study," *Procedia Computer Science*, vol. 84, pp. 70-78, 2016/01/01/ 2016.

[7] B. Reimer, B. Mehler, J. Coughlin, K. Godfrey, and C. Tan, *An on-road assessment of the impact of cognitive workload on physiological arousal in young adult drivers*. 2009, pp. 115-118.

[8] T. O. Zander et al., "Evaluation of a Dry EEG System for Application of Passive Brain-Computer Interfaces in Autonomous Driving," (in eng), *Frontiers in human neuroscience*, vol. 11, pp. 78-78, 2017.

[9] H. Rahman, M. Ahmed, and S. Begum, *Vision-Based Remote Heart Rate Variability Monitoring using Camera*. 2017.

[10] D. McDuff, S. Gontarek, and R. Picard, "Remote measurement of cognitive stress via heart rate variability," *2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014*, vol. 2014, pp. 2957-60, 08/01 2014.

[11] M. N. Sahadat, S. Consul-Pacareu, and B. I. Morshed, "Wireless ambulatory ECG signal capture for HRV and cognitive load study using the NeuroMonitor platform," in *2013 6th International IEEE/EMBS Conference on Neural Engineering (NER)*, 2013, pp. 497-500.

[12] F. Shaffer and J. P. Ginsberg, "An Overview of Heart Rate Variability Metrics and Norms," (in eng), *Frontiers in public health*, vol. 5, pp. 258-258, 2017.

[13] S. Hussain, S. Chen, R. Calvo, and F. Chen, "Classification of Cognitive Load from Task Performance & Multichannel Physiology during Affective Changes," 01/01 2007.
[14] E. Solovey, M. Zec, E. Garcia Perez, B. Reimer, and B. Mehler, "Classifying driver workload using physiological and driving performance data: Two field studies," *Conference on Human Factors in Computing Systems - Proceedings*, 04/26 2014.

[15] J. Engström, E. Johansson, and J. Östlund, "Effects of visual and cognitive load in real and simulated motorway driving," *Transportation Research Part F: Traffic Psychology and Behaviour*, vol. 8, no. 2, pp. 97-120, 2005/03/01/ 2005.

[16] B. Reimer and B. Mehler, "The Impact of Cognitive Workload on Physiological Arousal in Young Adult Drivers: A Field Study and Simulation Validation," *Ergonomics*, vol. 54, pp. 932-42, 10/01 2011.

[17] A. Liu and C. Chen, "Towards Practical Driver Cognitive Load Detection Based on Visual Attention Information," *Masters of Science, Univ. of Toronto, Canada*, 2017.

[18] C. Wang and J. Guo, "A data-driven framework for learners' cognitive load detection using ECG-PPG physiological feature fusion and XGBoost classification," *Procedia Computer Science*, vol. 147, pp. 338-348, 2019/01/01/ 2019.

[19] J. Sörman and R. Gestlöf, "CONTACT-FREE COGNITIVE LOAD CLASSIFICATION BASED ON PSYCHO-PHYSIOLOGICAL PARAMETERS," *Bachelor of Science*, Mälardalen University, Sweden, 2019.

[20] H. Rahman, S. Begum, M. U. Ahmed, and P. Funk, "Real Time Heart Rate Monitoring From Facial RGB Color Video Using Webcam," in *29th Annual Workshop of the Swedish Artificial Intelligence Society (SAIS) 2016*, Malmö, Sweden, 2016.

[21] H. Buijs, A. Pomerleau, M. Fournier, and W. Tam, "Implementation of a fast Fourier transform (FFT) for image processing applications," *Acoustics, Speech and Signal Processing, IEEE Transactions on*, vol. 22, no. 6, pp. 420-424, 1974.

[22] P. Comon, "Independent Component Analysis, a new Concept," *Signal Process.*, vol. 36, pp. 287-314, 1994.

[23] M. A. Kramer, "Nonlinear principal component analysis using autoassociative neural networks," *AIChE Journal*, vol. 37, no. 2, pp. 233-243, 1991.

[24] H. Rahman, M. U. Ahmed, and S. Begum, "Non-contact Heart Rate Monitoring Using Lab Color Space," in *13th International Conference on Wearable, Micro & Nano Technologies for Personalized Health (pHealth2016)*, Crete, Greece, 29-31 May. 2016.

[25] E. Alpaydin, *Introduction to Machine Learning*. MIT Press, 2010.

[26] C. Cortes and V. N. Vapnik, "Support-vector networks," *Machine Learning*, vol. 20, no. 3, pp. 273-297, 1995.

[27] P. Tamrakar, S. S. Roy, B. Satapathy, and S. P. S. Ibrahim, "Integration of lazy learning associative classification with k-NN algorithm," in *2019 International Conference on Vision Towards Emerging Trends in Communication and Networking (ViTECoN)*, 2019, pp. 1-4.