Title: MENTAL FATIGUE MONITORING USING BRAIN DYNAMICS PREFERENCES

Abstract: Driver's cognitive state of mental fatigue significantly affects driving performance and more importantly public safety. Previous studies leverage the response time (RT) as the metric for mental fatigue and aim at estimating the exact value of RT using electroencephalogram (EEG) signals within a regression model. However, due to the easily corrupted EEG signals and also non-smooth RTs during data collection, regular regression methods generally suffer from poor generalization performance. Considering that human response time is the reflection of brain dynamics preference rather than a single value, a novel model called Brain Dynamics ranking (BDrank) has been proposed. BDrank could learn from brain dynamics preferences using EEG data robustly and preserve the ordering corresponding to RTs. BDrank model is based on the regularized alternative ordinal classification comparing to regular regression based practices. Furthermore, a transition matrix is introduced to characterize the reliability of each channel used in EEG data, which helps in learning brain dynamics preferences only from informative EEG channels. In order to handle large-scale EEG signals and obtain higher generalization, an online-generalized Expectation Maximum (OnlineGEM) algorithm also has been proposed to update BDrank in an online fashion. Comprehensive empirical analysis on EEG signals from 44 participants shows that BDrank together with OnlineGEM achieves substantial improvements in reliability while simultaneously detecting possible less informative and noisy EEG channels.

1. INTRODUCTION:

As reported by sleep health report (Adams et al., 2017) , mental fatigue is a major cause in 33% -45% of all road accidents. In general, mental fatigue (Boksem & Tops, 2008) refers to the inability to maintain optimal cognitive performance in continuous task of the high demand of cognitive activity. Such inability in the context of driver could lead to accidents with severe consequences. Individuals may find themselves in a mental fatigue state because of lack of sleep, continuous driving for longtime, midnight driving, monotonous driving, and driving during the influence of sleeping drugs or sleep disorders (Ji et al., 2004; Ting et al., 2008) .

In response to these critical issues, several methods (Jap et al., 2009; Wascher et al., 2014; Cook et al., 2007; Lal et al., 2003; de Naurois et al., 2017) have been proposed to estimate and predict the mental fatigue based on EEG and RT (Fig. 1a ). Some of these methods, however, performed considerably well for some participants but failed for others due to lack of robustness. There are several challenges behind such instability and one of such problem is how to use RT effectively. RT is the most resourceful piece of information to predict mental fatigue. However, it is easily affected by the instrumental error, mind wandering or any other task non-related factor. A previous study (Wei et al., 2015) tried to overcome this problem by providing different techniques to adjust RT, they also tried removing outliers nevertheless, failed to make it work for all participants. The regression assumption of this method between EEG signals and RT is not correct. Human's RT is usually the result of preference (Izuma & Adolphs, 2013) in brain dynamics during the task, than just a single value. These preferences of human can be affected by different cognition (Möckel et al., 2015) like mind wandering (Lin et al., 2016) , and/or lower level of attention (Chuang et al., 2018) . Therefore, the relationship between EEG and RT including the extreme/abnormal RT should be taken care in the way that reflect human brain dynamics preferences by the developed technique itself. Another important problem is related to different brain regions, which are normally responsible for different functionalities. There was an attempt to choose different brain regions (Wascher et al., 2014) for a method during mental fatigue prediction but these regions of the brain are not necessary same for all participants (Gramann et al., 2006) . For example, existing work (Wascher et al., 2014) used frontal theta to represent a different level of mental fatigue for all participants. In such case learning model's reliability would inevitably degrade because of possibly noisy channels chosen, on different brain regions, by the method. Some previous work (de Naurois et al., 2017) , attempted to solve this issue using artificial neural network models but still failed to provide convincing results. Again these mentioned work reflect developed methods should be based on brain dynamics preferences rather than fix model or regions of brain.

To overcome the above-mentioned problems, a new approach has been proposed. We called it BDrank such that brain dynamics ranking. This approach not only learns from brain dynamics preferences for mental fatigue but also other cognitive states (Lal et al., 2003) , while effectively preserving the exact ordering of RT (Fig. 1b ). This approach surprisingly improved over defects of previous models and their performance due to noisy and extreme RT. Furthermore, BDrank model also proposes to use transition matrix to evaluate the high confidence BDrank (HC-BDrank) sources among different EEG channels, which contributes highly toward task performance. In order to handle large-scale EEG signals and obtain higher generalization, an online-generalized Expectation Maximum (OnlineGEM) (Roche, 2011) algorithm also has been proposed. Comprehensive empirical experiments on EEG signals from 44 participants show that BDrank model together with Onli-neGEM algorithm delivers substantial improvements and robust performance while simultaneously providing the information about noisy or less informative EEG channels.

2. MATERIALS AND METHODS::

Huang et al.

Information transfer during drowsy driving processes X and Y. We assume that each of these processes can be approximated by a stationary Markov Process of finite order d. Thus, we can reconstruct the state space of the process X by a delay embedded vector of dimension d with past values. Representing the two time series as X = xt and Y = yt, the delay embedded vector is defined as

x d t = xt, xt-τ , xt-2τ , • • • , x t-(d-1

)τ ; similar representation can also be made for y d t . The dimension of the embedding space is d, and the delay is τ . Under the assumption that the system X can be approximated by a stationary Markov process of order d, the transition probabilities that describe the system are given by:

EQUATION

)

as TE(Y → X), is computed as (Schreiber, 2000; Hlaváèková-Schindler et al., 2007; Vicente et al., 2011 )

EQUATION

where

y m t = yt, yt-τ, • • • , y t-(m-1

)τ indicating that the process X depends on m states of Y. The Equation (3) can be rewritten in terms of differential entropy as the following

TE(Y → X) = H(x d t , y m t ) -H(xt+u, x d t , y m t )

Figure 2 : Sustained-attention driving task a) Experiment Paradigm: This paper utilized the 33-channels EEG data recorded in the previous study (Huang et al., 2015) from 44 adult participants while performing long sustained attention task. The experiment has been conducted using a virtual-reality (VR) dynamic driving simulator (Fig. 2D-E ). The task involves driving on the fourlane highway while lane-departure events were randomly induced deviation toward the side of the road from the original position. Each participant was instructed to quickly respond to steer back to original position. A complete trial in this study (Fig. 2A ), includes 10s baseline, deviation onset, response onset, and response offset (Fig. 2B-C ). The next trial occurs within an interval of 5-10s after finishing the current trial. Each participant completed T trials within 1.5h. For each trial i, the EEG signals {x n,i } N n=1 from N different channels were recorded simultaneously and the corresponding response time RT i was also collected afterwards. If a participant fell asleep during the experiment, there was no feedback to wake him up.

The response time is an intuitive indicator used to assess human mental fatigue. Therefore, the common practice for mental fatigue monitoring is to find a robust mapping for humans' response time (RT) to an emergent situation using the EEG signals recorded beforehand. The natural way to forecast the response time with the EEG signals is to formulate it as a regression task (Fig. 1a ), namely finding a nonlinear mapping (e.g. neural network, SVR) from the EEG signals x to the corresponding RT. However, due to the easily corrupted properties of the EEG signals and the existence of extreme values in response times during data collection, focusing on predicting the exact value of a noisy and non-smooth measurement (the response time) is easier to create a near-perfectly fitted model with poor generalization performance (See Table 1 and Fig. 3 for more details). This creates a dilemma: it requires a powerful learning model to predict response time with the complex EEG signals (indeed, it is exactly our target) but it is not so significant to excessively approximate the exact value of response time, especially the extreme values.

Here comes the problem, how to find an efficient way to learn from the noisy response time while the exact value is not so significant? Actually, the RTs are defined in the totally ordered space R. The totally ordered space owns its structure meanings, which are preserved by the pairwise comparisons between the RTs. The pairwise comparisons indeed preserve the whole relative structure information between the response times while ignoring their absolute numerical information. Therefore, predicting the orderings of the pairwise comparisons can be accepted as a regularized alternative of previous regression model. Further, it makes it more flexible to consider more powerful learning unit in the proposed model. Therefore, there is no longer requirement to estimate the RT with a regression model. Instead, the proposed approach will transform it into an ordinal classification problem, and focus on correctly preserving the whole orderings between the pairwise RT comparisons (Fig. 1b ). First, the preference propositions1 could be constructed as follows,

EQUATION

where M 1 and M 2 denote the number of type-1 and type-2 preference propositions. D 1 denotes the type-1 preference propositions that the orderings between the RTs are significant; D 2 denotes the type-2 preference propositions that the RTs in each comparison are comparable. Then, the brain dynamics preference was constructed for each proposition with the corresponding pairwise EEG signals recorded from each channel. For brevity of notations, new index (the notation in Eq.1) is adopted in the following, instead of the original index used in sustained-attention driving task (Fig. 1 ). Namely, the m-th proposition ρ m ∈ D 1 D 2 denotes the pairwise comparison RT m,1 > RT m,2 or RT m,1 ≈ RT m,2 . And the pairwise brain dynamics preference (x 1 n,m , x 2 n,m ) denotes the features recorded within the n-th channel for each preference proposition

ρ m ∀m = 1, 2, • • • , M 1 + M 2 .

In this paper, the 10s baseline (Fig. 2B ) as the feature vector has been adopted, which is assumed to be long enough to detect any significant changes in brain activity (Zhang, 2000) . This followed by exploring the relation between the 10s baseline x (∈ R k ) and the preference proposition ρ m under the following four assumptions: (1) different participants are independent during the data collection process; (2) Different EEG sensors used for recording are recorded independently from scalp without influencing other sensors (Homan et al., 1987; Teplan et al., 2002) ; (3) Different trials are conducted independently during the data collection process; (4) The collected response time are slightly corrupted by inherent (basically irremovable) sources of noise, but the ranking relationships are preserved to some extents. b) Brain Dynamics Preferences: Again, instead of modelling the dependence between the RT and the 10s baseline as a regression problem, we aim to predict the orderings of the pairwise comparisons using the brain dynamics preferences, namely f (x 1 , x 2 ) → ρ. Further, a preference proposition ρ has three states: 1, 0, -1, denoting win (RT 1 > RT 2 ), tie (RT 1 ≈ RT 2 ) and loss (RT 1 < RT 2 ), respectively. However, classical classification models, e.g. logistic ordinal regression (Harrell, 2001) , are infeasible for our problem, due to the lack of a normalized probability definition for three states. Since two types of preference propositions D 1 and D 2 are considered in our problem, we tailor-define a normalized probability definition, namely first normalizing the probabilities of the states (1, -1) (exclusively to D 1 ) to 1, then generalize the probability definition to state 0. To be specific, it can be mathematically formulated as follows,

P(ρ|w, x 1 , x 2 ) =      σ (w T ∆x)[1 -κ(w T ∆x)] ρ = 1 κ(w T ∆x) ρ = 0 σ (-w T ∆x)[1 -κ(w T ∆x)] ρ = -1 (2)

where σ (z) = 1/(1 + e -z ) is the sigmoid function and σ (-z) = 1σ (z). The symbol ∆x denotes the subtraction (x 1x 2 ) between the brain dynamics preferences (x 1 , x 2 ). Following Weng & Lin (2011) , the probability of a tie is modelled as the geometric mean between a win and a loss, namely

κ(w T ∆x) = σ (w T ∆x)[1 -σ (w T ∆x)].

c) Preference State Transition: Furthermore, considering the different functions of different regions in the human brain, the relative contributions of different channels to human response time may vary a lot. For example, the information conveyed by positive channels is positive related to the RT, while negative channels may convey the information which is negative related to the RT. There are also some noisy (non-relevant) channels which are independent to the learning task. Therefore, if we directly model the EEG preferences recorded in each channel without making any distinctions about the channel state (i.e. positive, noisy and negative), the model's reliability would inevitably degrade. Note that a channel is called as "noise" if the current algorithms could not extract useful brain information with EEG signals from this channel (Alharbi, 2018; Lin et al., 2018) .

In the following, a transition matrix Π n is introduced to characterize the reliability of each channel n w.r.t the corresponding task. Let ρ denote the preference proposition and ρ (n) denote the prediction from the n-th channel. They are defined on the finite state space S = {1, 0, -1}. Then we have

EQUATION

where P i, j (ρ|ρ (n) ) = P(ρ = S j |ρ (n) = S i ). Note that we do not consider the transition between the type-1 and type-2 preference propositions (i.e. P(ρ = {1, -1}|ρ (n) = 0) = 0 and P(ρ = 0|ρ (n) = {1, -1}) = 0), since the equal cases are hard to measure when do prediction. A promising approach to generalize the transition matrix Π n (Eq.3) is to introduce the concept of the confidence region to measure the equal cases (Pregibon et al., 1981) .

Remark: The parameter π n in the transition matrix Π n actually indicates the reliability of the n-th channel ∀n = 1, 2, • • • , N. It additionally helps to divide the channels into three states: (1) positive channels with π n close to 1, the ranking model (Eq.2) can extract enough information from the n-th channel, and exactly predict the state of the preference proposition.

(2) Noisy channels with π n approximating to 0.5, the ranking model can not extract any useful information from the n-th channel.

(3) Negative channels with π n close to 0, the ranking model can extract enough information from the n-th channel, but the prediction states are exactly opposite to the proposition states. The identified positive and negative channels are all considered as informative EEG channels, which helps in learning reliable models for the corresponding task.

With the introduced transition matrix Π n , the marginal likelihood function for each proposition ρ can be further represented as P(ρ) = E ρ (n) P(ρ|ρ (n) )P(ρ (n) ) . Specifically,

P(ρ|w, Π n , x 1 , x 2 ) =      [π n σ (w T ∆x) + (1 -π n )σ (-w T ∆x)][1 -κ(w T ∆x n,m )] ρ = 1 κ(w T ∆x n,m ) ρ = 0 [(1 -π n )σ (w T ∆x) + π n σ (-w T ∆x)][1 -κ(w T ∆x n,m )] ρ = -1 (4)

For sake of simplicity, the subscripts (n, m) are omitted. Let D = D 1 D 2 denotes all the preference propositions and X represents the recorded EEG signals from N different channels. We further extend our robust ordinal classification (Eq.4) for BDrank into the Bayesian version. A Gaussian prior is introduced for w (i.e., w ∼ N(µ, Σ)). Since the transition matrix Π n only depends on the parameter π n , we only focus on estimate the parameter π n ∀n = 1, 2, • • • , N in the following. Let π denote {π n } N n=1 , and we introduce a Beta prior for each π n (i.e., π ∼ B(α,

β ) = ∏ N n=1 B(α n , β n )).

Then, our BDrank can be formulated into a maximum a posteriori (MAP) estimate as follows, arg max π,w P 0 (π)P 0 (w)P(D|π, w, X) = P 0 (π)P 0 (w)P(D 1 |π, w, X)P(D 2 |π, w, X)

= P 0 (π)P 0 (w) N ∏ n=1 M 1 ∏ m=1 P(ρ m = 1|π n , w, ∆x n,m ) M 2 ∏ m =1 P(ρ m = 0|π n , w, ∆x n,m ) (5) = B(π|α, β )N(w|µ, Σ) N ∏ n=1 M 1 ∏ m=1 [π n σ (w T ∆x) + (1 -π n )σ (-w T ∆x)][1 -κ(w T ∆x n,m )] M 2 ∏ m =1 κ(w T ∆x n,m ) .

Note that due to the symmetry of the state probability (Eq.2) and transition matrix (Eq.3) w.r.t. state 1 and -1, the resultant marginal likelihood (Eq.4) remains symmetry w.r.t. state 1 and -1. For simplicity, D 1 is constructed using the preference propositions with state ρ = 1 only. M 1 and M 2 denote the number of type-1 and type-2 preference propositions, namely

|D 1 | = M 1 and |D 2 | = M 2 .

The variable n iterates over the channels. m and m iterate over two types of preference propositions, respectively. Now our aim is to estimate the model parameters (w and π) by maximizing Eq.5. In principle, any solution strategies for MAP can be considered to solve this problem. See Section 3 for optimization details.

d) Reliability analysis and channel state estimation Note that our BDrank (Eq.5) indeed trains a mixture of two complementary classifiers, which share the same parameter w. It is different from classical mixture models, since it clusters at the channel level instead of the sample level. In particular, in terms of the positive channels with π n close to 1, BDrank relies on the first classifier to update the shared parameter w. In terms of the negative channels with π n close to 0, Eq.5 automatically switches to the opposite classifier which can extract correct information from the negative channels and update the shared parameter w accordingly. Further, BDrank is robust to the noisy channels with π n approximating to 0.5, because Eq.5 gives up extracting information from the noisy channels by assigning a constant likelihood (i.e. 0.5) to each brain dynamics preference. Note that the estimated π n can be leveraged as the indicator to detect noisy channels with π n ≈ 0.5, ∀n = 1, 2, • • • , N. See Fig. 4 in the experimental section for more details.

3. OPTIMIZATION METHODS:

In this section, we describe a generalized Expectation-Maximization (EM) algorithm (Dempster et al., 1977) to solve the proposed BDrank (Eq.5). Since the feasible region of π n is restricted to [0, 1], the gradient-based optimization methods would make our solution inaccurate and inefficient.

The EM algorithm is an efficient iterative procedure to compute the MAP problem in presence of latent variable (ρ

(n)

m in Eq.5). EM avoids calculating the derivative to the expectation of the latent variable directly, and resorts to a surrogate lower bound to optimize. Therefore, EM, a silver bullet for MAP with latent variable, can significantly simplify the optimization over parameter π n for Eq.5.

3.1. GENERALIZED EM FOR BDRANK:

For each type-1 proposition ρ m , we introduce an auxiliary variable δ P(D, Ξ, π, w|X) = P 0 (π)P 0 (w)P(D 1 , Ξ|π, w, X)P(D 2 |w, π, X)

EQUATION

= P 0 (π)P 0 (w)

N ∏ n=1 M 1 ∏ m=1 π n σ (w T ∆x n,m ) δ (n) m (1 -π n )σ (-w T ∆x n,m ) 1-δ (n) m [1 -κ(w T ∆x n,m )] M 2 ∏ m =1 κ(w T ∆x n,m ) .

Now, we can deal with the joint distribution directly, which leads to significant simplifications for optimization. The complete log likelihood can be written as log P(D, Ξ, π, w|X) = log P 0 (π) + log P 0 (w)

+ N ∑ n=1 M 1 ∑ m=1 log[1 -κ(w T ∆x n,m )] + N ∑ n=1 M 2 ∑ m =1

log κ(w T ∆x n,m ) (7)

+ N ∑ n=1 M 1 ∑ m=1 δ (n) m log π n σ (w T ∆x n,m ) + (1 -δ (n) m ) log(1 -π n )σ (-w T ∆x n,m ) .

Expectation Step In the expectation step, we first calculate the expected value of the auxiliary variable δ

EQUATION

where γ(δ

(n)

m ) denotes the degree of the consistency between the prediction ρ

(n)

m and the preference proposition ρ m . Then, the expected value of the complete-data log likelihood function w.r.t the posterior expectation of the auxiliary variable Ξ = {δ

(n) m } M 1

m=1 can be represented as follows:

EQUATION

+ N ∑ n=1 M 1 ∑ m=1 γ(δ (n) m ) log σ (w T ∆x n,m ) + (1 -γ(δ (n) m )) log σ (-w T ∆x n,m ) + constant.

Maximization Step In the maximization step, we maximize the objective function Eq.9 w.r.t the model parameters π and w, respectively. In terms of π, we set the gradient of Eq.9 w.r.t π n to zero and obtain the following estimate for π n :

EQUATION

In terms of w, due to the complexity of the sigmoid function, we do not have a closed form solution for w and we need to use gradient-based optimization methods. In the following, we adopt the L-BFGS to optimize w, the objective function L (w) consists of the second, the third, the fourth and the sixth parts of Eq.9 and the gradient function g(w) can be represented as follows,

EQUATION

w new can be solved with L-BFGS using L (w) and g(w). The EM algorithm then iterates the E-step and M-step until convergence is achieved.

3.2. ONLINE GEM FOR BDRANK:

The generalized EM approach introduced in Section 3.1 is inefficient for large-scale datasets, because we need to iteratively calculate the gradient with respect to parameters π and w over all samples during each maximization step. Motivated from the stochastic approximation literature, we introduce an online-generalized Expectation Maximization (OnlineGEM) approach, which resorts to stochastic mini-batch optimisation to learn the parameters. To be specific, OnlineGEM approximates the updated π and w in batch EM with a single sample or mini-batch samples. Since a mini-batch samples cannot be a perfect approximation for the whole dataset, we interpolate between the new and former estimations with a decreasing step-size η k2 , as in Liang & Klein (2009) .

Before the k-th iteration, we randomly downsample a mini-batch D k from the preference propositions D. The number of two types preference propositions in D k are M * 1 and M * 2 , which are much smaller than the corresponding total size M 1 and M 2 , respectively. The expectation step remains similar. The difference is that we only need to calculate the posterior expectation of the auxiliary variable δ (n) m over the mini-batch D k . In the maximization step, we maximize the objective function, calculated on the mini-batch D k , with regard to model parameters π and w. In terms of parameter π n , since its marginal distribution belongs to exponential family, we could perform the stochastic update in the space of sufficient statistics (Cappé & Moulines, 2009) . φ * n denotes the noisy estimate of the sufficient statistic for π n .

EQUATION

EQUATION

EQUATION

In terms of parameter w, the above practice is infeasible due to its non-exponential marginal distribution. Inspired by the stochastic gradient EM algorithms in Cappé & Moulines (2009) , we perform the stochastic update in the original space. First, a local optima regression weight w (k) can be estimated via iterative optimization over the mini-batch D k , using L-BFGS algorithm. Then we interpolate between the local optima and former estimations to form a global approximation.

EQUATION

The convergence issues of the proposed online generalized EM algorithm are the analogues of the discussion given by Cappé & Moulines (2009) for their stochastic gradient EM Algorithms. The existence of such links is hardly surprising. In view of the discussions in Section 3 of Cappé & Moulines (2009) , the online update rule (Eq.13b) could also be seen as a stochastic gradient descent formula, namely w new = w old + η k (w (k)w old ) .

4. EMPIRICAL ANALYSIS:

In this section, we demonstrate the reliability of the proposed BDrank (Eq.5) with EEG signals from forty four participants.

Data Preprocessing: EEG preferences for each participant has been generated as follows: (1) the trials of each participant were randomly divides into two parts: 50% for training and 50% for test;

(2) EEG preferences were constructed according to the pairwise comparisons between the RTs. To be specific, the type-1 preference propositions D 1 were constructed with RT comparisons

(RT m,1 , RT m,2 ), which satisfies RT m,2 < min(RT m,2 + τ 1 , τ 2 × RT m,2 ) < RT m,1 ; the type-2 preference propositions D 2 were constructed with RT comparisons (RT m ,1 , RT m ,2 ), which satisfies RT m ,2 < RT m ,1 < min(RT m ,2 + τ 3 , τ 4 × RT m ,2 ).

It is notable that τ 1 > τ 3 > 0 and τ 2 > τ 4 > 1 control the difference in the RT comparisons simultaneously, we empirically set τ 1 = 1; τ 2 = 1.5; τ 3 = 0.8; τ 4 = 1.3 for all participants in our experiment setting. Considering the time delay among the channels in the time domain, Fourier transform (Welch, 1967) has been applied to EEG signals to transform timeseries into frequency domain. Further, to avoid overhead computation, EEG power within 0-30Hz has been selected, which is considered to be the most relevant to the RTs (Huang et al., 2015) .

Baselines and Metrics::

We compared BDrank with one regression model Support Vector Regression (SVR) (Smola & Schölkopf, 2004) and one ordinal classification model Logistics Ordinal Regression (LOR) (Harrell, 2001) . First, we aggregate the predictions from different channels using a simple voting scheme, namely ρm

= sign ∑ N n=1 ρ (n) m 1(π n > κ) -1(π n < 1 -κ) . ρ (n)

m denotes the predicted state (1 means win and -1 means loss) for (RT 1 m , RT 2 m ) by the n-th channel, using the brain dynamics preference (x 1 n,m , x 2 n,m ). ρm is the final estimated order for (RT 1 m , RT 2 m ) by aggregating the predictions ρ (n) m over all channels. 1() is an indicator that returns one if the argument is valid and returns zero otherwise.

Then, we introduce two metrics to measure the performance of BDrank model from different perspectives. First, we adapted the Wilcoxon-Mann-Whitney statistics (Yan et al., 2003) to evaluate the accuracy (in %, higher is better) over all pairs, namely 1

M 1 ∑ M 1 m=1 1(ρ m = ρm ).

Further, we investigate the reliability of BDrank in terms of preserving the global ordering w.r.t RTs. Note that a totally ordered set could be equally represented by a fully directed graph, where the fully directed graph can be further encoded by its degree sequence. Therefore, we first collected the indegree sequences3 (Becirovic, 2017) of the constructed directed graph using the predicted RTs and then measured the discrepancy between the predicted indegree sequences and ground truth using the root-mean-squared errors (smaller is better). See Supplementary for the detailed description.

Note that we only trust the predictions from informative channels with reliability π n > κ or π n < 1κ. κ is set to 0.85 for all participants in our experiment. In terms of SVR and LOR, consider the scale difference between the EEG signals between different channels, we train a SVR/LOR model for each channel and aggregate the results from different channels to calculate the final predictions using the majority voting scheme. Since there is no mechanism for SVR and LOR to evaluate the channel state, we trust all the channels by default. Further, we only calculate the two metrics on the type-1 preference propositions D 1 , since the state of the type-2 preference propositions D 2 is hard to evaluate when do prediction.

Parameter Initialization: In terms of the weight w, we randomly initialized w in [-1, 1]. In terms of the channel reliability π n , we assumed all channels were relevant to the task beforehand, and randomly initialized the channel reliability π n , ∀n = 1, 2, • • • , N in [0.5, 1]. The L-BFGS implementation was downloaded from Granzow, where we used the default parameters. In terms of the hyperparameters (µ, Σ) for w, we adopted the standard Gaussian distribution, namely w ∼ N(0, 1). In terms of the hyperparameters (α n , β n ), as we intended to eliminate the effects of noisy channels, we adopted a strong non-informative prior for π n , namely (2006) . In terms of the maximum iteration number, we set MaxIter = 30 in our experiment to ensure the algorithm converged for each participant. In terms of the minibatch size R, we set the sample ratio to 10% and 5% for type-1 and type-2 preference propositions, respectively. The parameters for LOR were optimized with L-BFGS with the default setting for all participants. In terms of SVR, we resorted to 5-fold cross validation to find the best parameters for each participant.

α n = β n = 20, ∀n = 1, 2, • • • , N, according to Bishop

4.1. EMPIRICAL RESULTS OF BDRANK ON BRAIN DYNAMICS PREFERENCES:

The accuracies of LOR and BDrank for training and test EEG preferences are presented in Table 1 . For SVR, we followed standard procedure and optimization for each participant to predict RTs with training and test EEG signals. We adopted the Wilcoxon-Mann-Whitney statistics to measure the accuracy of the predicted RTs w.r.t. ground truth (shown in Table 1 ). In particular, the number of participants with test accuracy above 75% are 17 and 25 for SVR and BDrank, respectively. This observation is consistent with our statement that classification-based models, served as a regularized alternative for the regression-based models, can effectively circumvent the overfitting caused by non-smooth RTs and preserve the ordering corresponding to RTs.

(3) However, simple classification-based extensions (e.g. LOR) do not enjoy the benefits, which only achieves comparable test accuracies with SVR (outperforming SVR on only 21/44 participants). It is because they are not robust to the noisy data, while it is a common phenomenon when dealing with EEG signals. (4) Note that, compared to the fine-tuned SVR for each participant, fixed parameters were adopted for all participants in the experiment of LOR and BDrank. It further verifies the superior robustness of ordinal classification models compared to regression-based models.

To further investigate the reliability of BDrank in terms of preserving the global ordering corresponding to RTs, we first collected the indegree sequences of the constructed directed graph using the predicted RTs and then measured the indegree discrepancy between the calculated indegree sequences and the ground truth using the root-mean-squared error (RMSE) (shown in Table . 2). We also investigated the reliability of our BDrank from the perspective of noisy channel detection.

According to our analysis, the parameter π n in the transition matrix Π n actually indicates the channel reliability. Hereafter, we leverage π n as the channel reliability indicator to detect noisy channels. Fig. 4 shows that: (1) the noisy channels is universally exist among the EEG signals. There are total 42 among 44 participants with at least one noisy channels detected. For example, the 33-th channel is recognized as the noisy channel by BDrank for almost all participants. It is reasonable since the 33-th channel is the non-EEG channel, which is generally acknowledged as the non-relevant channel to any tasks; (2) For each participant, most channels are reliable, which ensures we can always find enough support to training our BDrank; and (3) The detected noisy channels varies from participant to participant, and do not possess the transitivity property between participants. Because the noise can arise due to (i) intrinsic non-EEG channel, e.g. the 33-th channel; (ii) channels for lateral mastoid references, e.g. the 23-th and 29-th channel (Chatrian et al., 1985) ; and (iii) improper experimentation or artifacts (Lin et al., 2018) .

5. CONCLUSION:

This work proposes a BDrank model to assess the state of mental fatigue. The efficacy of BDrank model was demonstrated using EEG data collected in sustained driving task from 44 participants. This model has been further combined with an online-generalized Expectation Maximization (On-lineGEM) algorithm to provide a continuous update in the model. BDrank model utilized a unique methodology with a regularized alternative, i.e. ordinal classification, to circumvent overfitting to the extreme values of RTs. It has been demonstrated that the overall performance of BDrank can be significantly improved with the introduction of the transition matrix, which enables the technique to evaluate the reliability of informative EEG channels while detecting noisy EEG channels. Empirical results show that BDrank combined with the OnlineGEM algorithm delivers significant improvement over SVR and LOR in terms of global ranking preservation.

In this work, the cooperation mechanism among channels is simplified as a weighted voting system, while different trials are viewed independently. We intend to further formulate it with more complex mechanisms, such as Markov decision process (MDP), to conduct learning and decision making simultaneously. Some previous (Chen et al., 2016; 2015) studied the decision making process among crowd (noisy) workers, which is promising to our setting to investigate the cooperation mechanism among noisy channels. Efforts are underway to apply this approaches in future work.

:

We used the term "preference" intentionally to show that brain dynamics keep changing w.r.t. human behaviours and it happens because the human brain prefers one decision over others. Therefore, we prefer to call it"preference" than "classification".

η k = (k + 2) -τ 0 , where k is the number of iterations and 0.5 < τ 0 < 1. The smaller the τ 0 , the larger the update η k , and the more quickly we forget (decay) our old parameters. This can lead to swift progress but also generates instability.

We only consider the indegree sequence because the indegree and outdegree of a vertex can be uniquely determined when the overall degree of each vertex is fixed. The indegree of vertex v t can be calculated as deg-(v t ) = ∑ m∈N 1 (v t ) [1( ρm = 1) + 0.5 × 1( ρm = 0)] + ∑ m∈N 2 (v t ) [1( ρm = -1) + 0.5 × 1( ρm = 0)],where N 1 (v t ), N 2 (v t ) denote the index set of the pairwise comparisons with the RT of trial t (vertex v t ) appearing in the first and second position, respectively.

Figures and Tables:

Figure 1: (a) Regression model with EEG signals. (b) BDrank model with brain dynamics preferences.

FIGURE 1 | Sustained-attention driving task. (A) Event-related lane-deviation paradigm. (B-C) EEG and behavior were recorded simultaneously. (D) The experimental paradigm was implemented in a VR-based driving simulator, where a 3-D surrounding view simulated a monotonous highway scene and a vehicle was mounted on a six degree-of-freedom dynamic platform. (E) The car speed was fixed at 100 km/h and the subject was asked to keep the car cruising on the central of the lane.

{1, 0}) for each channel, representing the consistency between the preference proposition ρ m and the prediction ρ (n) m given by the n-th channel. Specifically, δ (n) m = 1 denotes the prediction ρ (n) m given by the first classifier is consistent with the preference proposition ρ m , and δ (n) m = 0 denotes the prediction ρ (n) m estimated by the second classifier is consistent with the preference proposition ρ m . We can therefore find an equivalent formulation of our BDrank model involving the auxiliary variable Ξ = {δ (n) m } M 1 m=1 .

Figure 4: Reliability of different channels for forty four participants estimated by BDrank. Each column denotes the states of 33 channels for each participant. The channels with estimated reliability 0.15 ≤ π n ≤ 0.85 are marked in red.

Training and test accuracy (in %). Higher is better, the best is marked in gray BDrank shows significant improvements over SVR and LOR in terms of test accuracies.

Table content: <table><tr><td colspan="2">Participant</td><td>P1</td><td>P2</td><td>P3</td><td>P4</td><td>P5</td><td>P6</td><td>P7</td><td>P8</td><td>P9</td><td>P10</td><td>P11</td><td>P12</td><td>P13</td><td>P14</td><td>P15</td><td>P16</td><td>P17</td><td>P18</td><td>P19</td><td>P20</td><td>P21</td><td>P22</td></tr><tr><td/><td>SVR</td><td colspan="2">98.16 98.45</td><td>100</td><td>99.40</td><td>100</td><td>100</td><td>100</td><td>100</td><td>100</td><td>100</td><td>100</td><td colspan="4">98.31 98.73 99.82 99.69</td><td>100</td><td>100</td><td>100</td><td>100</td><td colspan="2">98.78 99.64</td><td>100</td></tr><tr><td>Train</td><td>LOR</td><td colspan="18">91.66 94.73 77.24 88.65 81.31 97.10 96.37 85.42 84.35 73.67 82.42 89.97 89.79 85.19 91.83 99.01 88.38 86.06</td><td colspan="4">91.90 91.65 88.66 84.83</td></tr><tr><td/><td colspan="23">BDrank 98.96 97.13 95.26 99.08 94.92 99.85 97.45 97.40 97.33 93.60 99.43 97.99 98.48 96.47 98.60 99.88 98.95 100.00 99.68 99.03 96.23 85.95</td></tr><tr><td/><td>SVR</td><td colspan="18">68.56 76.81 68.49 61.20 76.04 78.04 69.59 66.90 67.01 79.23 73.34 67.58 69.45 69.47 85.38 73.73 75.38 55.60</td><td colspan="4">66.55 71.87 75.03 68.22</td></tr><tr><td>Test</td><td>LOR</td><td colspan="18">71.22 69.12 53.12 61.63 64.63 68.46 70.99 54.97 67.18 59.18 52.96 69.87 70.50 61.18 76.27 69.52 50.35 60.87</td><td colspan="4">82.73 60.67 65.10 69.98</td></tr><tr><td/><td colspan="19">BDrank 77.04 77.93 84.05 71.93 81.21 68.06 73.45 79.64 72.97 82.09 68.92 76.24 72.95 67.87 87.64 73.74 63.08 65.53</td><td colspan="4">83.85 74.62 74.38 77.28</td></tr><tr><td colspan="2">Participant</td><td>P23</td><td>P24</td><td>P25</td><td>P26</td><td>P27</td><td>P28</td><td>P29</td><td>P30</td><td>P31</td><td>P32</td><td>P33</td><td>P34</td><td>P35</td><td>P36</td><td>P37</td><td>P38</td><td>P39</td><td>P40</td><td>P41</td><td>P42</td><td>P43</td><td>P44</td></tr><tr><td/><td>SVR</td><td>100</td><td>100</td><td>97.13</td><td>100</td><td>99.47</td><td>100</td><td colspan="2">99.88 99.98</td><td>100</td><td>100</td><td>100</td><td>100</td><td colspan="3">99.93 99.97 99.95</td><td>100</td><td colspan="2">98.21 99.09</td><td>100</td><td>100</td><td>100</td><td>99.59</td></tr><tr><td>Train</td><td>LOR</td><td colspan="18">91.63 98.08 88.86 92.10 90.97 84.00 86.91 92.00 87.36 90.84 80.20 86.10 91.40 89.03 93.47 93.52 90.86 92.19</td><td colspan="4">93.42 82.37 81.87 81.80</td></tr><tr><td/><td colspan="19">BDrank 97.13 99.79 94.00 99.19 95.90 92.29 98.59 96.60 94.64 92.61 85.70 92.32 99.09 96.37 98.64 99.21 97.89 95.94</td><td colspan="4">96.74 93.41 96.20 76.04</td></tr><tr><td/><td>SVR</td><td colspan="18">69.70 72.84 67.25 80.89 72.71 66.17 77.24 76.59 81.11 79.54 80.64 73.83 78.39 70.99 75.34 61.9 86.92 76.51</td><td colspan="4">69.89 68.21 65.09 63.06</td></tr><tr><td>Test</td><td>LOR</td><td colspan="18">69.80 74.24 72.02 63.80 73.62 66.46 63.05 77.12 73.09 72.15 73.20 69.21 69.53 71.31 70.01 84.42 69.85 65.87</td><td colspan="4">70.02 77.20 84.89 70.41</td></tr><tr><td/><td colspan="19">BDrank 71.40 82.06 74.06 72.61 79.12 68.56 81.69 79.99 75.57 77.67 79.34 79.30 80.11 73.97 76.14 86.93 87.23 77.47</td><td colspan="4">70.40 77.38 88.47 74.16</td></tr><tr><td colspan="24">From Table 1, we observe that: (1) SVR achieves the highest accuracies (100% accuracy) on the</td></tr><tr><td colspan="24">training EEG data for almost all participants. However, test accuracies for most participants are</td></tr><tr><td colspan="24">inferior to other models, 21 participants for LOR and 35 participants for BDrank out of 44 partic-</td></tr><tr><td colspan="24">ipants. This observation is consistent with our previous discussion that regression-based models</td></tr><tr><td colspan="24">are easily overfitting to the training datasets, especially when extreme values (RTs in our problem)</td></tr><tr><td colspan="2">exist. (2)</td><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr></table>

Training and test RMSE (in #). Smaller is better, the best is marked in gray It means that SVR over-estimates the RTs with small values and under-estimates the RTs with large values. It is consistent with our claim that regression-based model is not suitable for the tasks with non-smooth response variable.4.2 NOISY CHANNEL DETECTION

Table content: <table><tr><td colspan="2">Participant</td><td>P1</td><td>P2</td><td>P3</td><td>P4</td><td>P5</td><td>P6</td><td>P7</td><td>P8</td><td>P9</td><td>P10</td><td>P11</td><td>P12</td><td>P13</td><td>P14</td><td>P15</td><td>P16</td><td>P17</td><td>P18</td><td>P19</td><td>P20</td><td>P21</td><td>P22</td></tr><tr><td/><td>SVR</td><td>1.52</td><td>2.51</td><td>0.00</td><td>1.23</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>2.01</td><td>1.26</td><td>0.39</td><td>0.64</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>3.84</td><td>1.56</td><td>0.00</td></tr><tr><td>Train</td><td>LOR</td><td>4.06</td><td>4.69</td><td>16.60</td><td>9.54</td><td>29.77</td><td>3.31</td><td>5.30</td><td colspan="4">13.79 28.86 42.19 16.04</td><td>6.38</td><td>5.53</td><td>11.66</td><td>8.42</td><td>1.25</td><td>8.12</td><td>4.90</td><td>8.63</td><td colspan="3">15.02 20.88 31.94</td></tr><tr><td/><td colspan="2">BDrank 0.92</td><td>3.78</td><td>3.16</td><td>0.74</td><td>7.52</td><td>0.11</td><td>4.67</td><td>1.82</td><td>3.85</td><td>11.38</td><td>0.65</td><td>1.73</td><td>0.91</td><td>3.18</td><td>1.71</td><td>0.39</td><td>0.52</td><td>0.00</td><td>2.91</td><td>2.15</td><td>7.51</td><td>34.12</td></tr><tr><td/><td>SVR</td><td colspan="13">14.30 19.52 22.95 30.11 39.45 16.80 29.69 27.01 53.23 38.90 23.56 20.32 13.71</td><td colspan="9">21.55 14.42 26.66 16.40 16.24 28.40 47.88 39.85 60.94</td></tr><tr><td>Test</td><td>LOR</td><td colspan="22">13.74 23.08 31.36 27.96 53.02 22.61 33.03 34.91 46.77 62.14 35.40 20.02 13.71 26.61 23.03 30.98 28.80 15.43 24.52 62.99 56.10 59.79</td></tr><tr><td/><td colspan="15">BDrank 10.94 19.45 10.68 19.37 25.12 10.60 28.59 12.31 32.92 27.63 16.13 14.82 12.67 21.75</td><td colspan="3">11.54 26.32 12.95</td><td>3.35</td><td colspan="4">19.38 32.09 40.92 52.11</td></tr><tr><td colspan="2">Participant</td><td>P23</td><td>P24</td><td>P25</td><td>P26</td><td>P27</td><td>P28</td><td>P29</td><td>P30</td><td>P31</td><td>P32</td><td>P33</td><td>P34</td><td>P35</td><td>P36</td><td>P37</td><td>P38</td><td>P39</td><td>P40</td><td>P41</td><td>P42</td><td>P43</td><td>P44</td></tr><tr><td/><td>SVR</td><td>0.00</td><td>0.00</td><td>6.08</td><td>0.00</td><td>2.22</td><td>0.00</td><td>0.98</td><td>0.22</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.26</td><td>0.19</td><td>0.26</td><td>0.00</td><td>2.10</td><td>1.38</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.67</td></tr><tr><td>Train</td><td>LOR</td><td>16.60</td><td>3.31</td><td colspan="10">15.98 19.20 17.59 27.85 22.97 14.68 28.61 17.74 36.52 23.72</td><td>7.71</td><td>11.56</td><td>7.46</td><td>6.22</td><td>6.05</td><td>5.88</td><td colspan="4">11.70 22.95 15.26 11.39</td></tr><tr><td/><td colspan="2">BDrank 5.05</td><td>0.45</td><td>7.70</td><td>1.94</td><td>9.65</td><td>13.36</td><td>2.79</td><td>6.21</td><td colspan="4">10.07 16.75 28.45 13.62</td><td>1.02</td><td>4.07</td><td>1.20</td><td>1.18</td><td>1.81</td><td>3.39</td><td>6.58</td><td colspan="3">22.79 12.28 15.43</td></tr><tr><td/><td>SVR</td><td colspan="16">42.89 25.60 40.55 41.12 44.93 49.09 39.85 36.77 39.03 37.63 35.72 39.49 16.90 24.19 21.14 15.44</td><td>9.15</td><td colspan="5">21.64 46.41 35.09 26.92 20.37</td></tr><tr><td>Test</td><td>LOR</td><td colspan="22">46.80 22.47 36.34 64.61 41.59 48.48 31.45 40.34 53.32 48.30 47.65 35.17 22.46 26.83 25.55 13.59 18.60 22.05 47.83 26.00 20.95 18.10</td></tr><tr><td/><td colspan="12">BDrank 44.58 14.89 32.64 32.46 37.91 39.16 24.13 29.46 38.94 40.12 39.23</td><td>34.7</td><td colspan="3">15.26 24.18 18.03</td><td>8.8</td><td>9.13</td><td colspan="4">14.26 42.25 23.86 9.09</td><td>16.40</td></tr></table>

Peer review (iclr2019):
 The paper proposes an algorithm for mental fatigue monitoring, relating a subjects* EEG signals to their reaction time (RT) during a simulated driving task, as an ordinal regression problem. The authors argue that RTs could be heavily skewed and/or non-smooth, making traditional regression approaches unstable due to outlier values. They propose a brain dynamic ranking algorithm, BDrank, using a generalized EM algorithm to estimate its parameters, and compare it to support vector regression and Logistic Ordinal Regression, where they show improved performance by accuracy and root mean squared error (RMSE) over a database of 44 subjects. General comments, in no particular order: 1. There are some minor grammatical errors throughout. The paper could benefit from another read-through to correct these errors. 2. It is unclear to me how the model works at test time; as the model is essentially building a relational structure in the data, does the user have to provide multiple EEG trials at time of prediction? 3. There is notation early on in the paper that doesn*t appear to be appropriately defined. For example, Equation (1) describes two sets of propositions, with M1 and M2 elements, respectively. How is M1 and M2, the total set of propositions, calculated? It appears to be all pair-wise comparisons of RTs but then it*s unclear why there are two indexes associated with them. Also, what does it mean for the orderings to be significant? (i.e.: that the "type-1 preference propositions that the orderings between the RTs are significant"). The authors then switch to a new notation x^1 and x^2 without defining them. Notational problems also persist throughout the paper, making it hard to gauge what is being done at each step. 4. The authors describe using an FFT to transform the EEG data into the frequency domain. I*m assuming they are doing the FFT on the entire 10-s interval but the paper does not make this clear. Also, the authors state using EEG power between 0-30Hz for their analysis; do they further sub-divide this range (for example, to the standard theta/alpha/beta power ranges) or just use the power across the entire 0-30Hz band? 5. I am concerned with the relatively sparse set of comparison algorithms the authors use. The authors only compare to relatively simple approaches (support vector regression and logistic ordinal regression), yet they cite many previous works in this area but do not compare against them, instead just leaving a pretty generic statement of "The regression assumption of this method between EEG signals and RT is not correct"; they do not elaborate on this aspect. Overall I think there is limited novelty in the approach; the idea to learn the structure of the data relationally instead of absolutely is pretty straight-forward, and is a standard practice for example in non-parametric statistical modeling. I am also not positive that ICLR is the best venue for this work; perhaps a better avenue for this would be in a more BCI/neural engineering-focused venue.