# Recognition of Blinks Activity Patterns during Stress Conditions Using CNN and Markovian Analysis
## 3 Materials and Methods
### IBI Sequence Classification  
The IBI behaviour prediction across stress phases was performed using a one- dimensional (1D) CNN under a 10 \(\times\) 10 nested cross-validation schemes with 10 repetitions. For generalization, task-independent prediction was performed. The 1D CNN was selected due to its effectiveness on the classification of timeseries with variable size. Different network architectures were applied and the one that gives the highest accuracy but keeps the computational complexity low is presented. The network architecture consists of two convolutional layers followed by non-linear activation function rectified linear units (ReLu), one max-pooling layer applied after the first convolutional layer and the global max-pooling layer after the second convolutional layer. This configuration enables the CNN to accept inputs of variable size. For the first convolutional layer, 64 filters with a kernel size of 1 were trained, whereas 128 filters with a kernel size of 1 were trained for the second convolutional layer. Finally, a linear layer with an output shape of 2 and a softmax activation returns the classification score. The convolutional and pooling layers are followed by a dense fully connected layer that interprets the features extracted by the convolutional part of the model. A flatten layer is used between the convolutional layers and the dense layer to reduce the feature maps to a single 1D vector. To improve generalization, the model has been regularized using a dropout on the outputs of the max-pooling and global max-pooling layers (values between \(p\) = [0.1-0.9] were checked and the value \(p\) = 0.1 was finally selected), L2-regularization (values of \(\lambda>10^{-4}\) were checked and \(\lambda\) = 0.01 was finally selected) using the weights of the second convolutional layer, and finally early-stopping the training after the validation loss has not improved for 5 out of 10 epochs during pre-training /fine-tuning. The learning rate range was [0.01-0.00001], values between 0.0001 and 0.001 were finally selected for each experimental phase. All models were trained using the Adam optimizer [20]. Hyperparameters (including learning rate, L2 regularization and dropout probability) were optimized on 90% of the training data, leaving 10% for validation. After determining the appropriate hyperparameters, the model performance was tested out-of-sample on the testing set prepared under the 10 fold nested cross-validation scheme. To increase robustness, all CNN experiments were repeated 10 times on the same data split, and thus training and testing accuracy were the average over all 10 trials. The code was implemented using Keras [21] with the TensorFlow [22] backend. The best method to select the number of convolutional layers is to increment one at a time until best accuracy is achieved. Two key hyperparameters that define the convolution operation are size and number of kernels. The former is typically 1 for 1D signals. The latter is arbitrary, and determines the depth of output feature maps. A convolutional layer can be seen as applying and sliding a filter over the time series. Unlike images, the filters exhibit only one dimension (time) instead of two dimensions (width and height). The filtercan also be seen as a generic non-linear transformation of a time series. Here, two feature maps were extracted from 1D signals with approximately 1200 time points, by upsampling the in-plane dimension by a factor of 64 in the first convolutional layer and 128 in the second. Adding more layers or increment the size of filters will produce more complex features. Different combination of filter sizes were tested and the accuracies were much lower. A motivation for applying several filters on an input time series would be to learn multiple discriminative features useful for the classification task. In addition, we developed a Long Short-Term Memory network model (LSTM) for the IBI sequence and the blink sequence classification [23].