## 4 Results
### Forecasting Mental Effort Using the Deep-Learning Models  
The LSTM algorithm, as shown in Table 8, depicted that the F1 and R-squared values were about 0.99 for all test and train split ratios. This model was able to predict the number of transitions in the test data set accurately for the train ratio of 30% and above. However, after examining the graph of absolute error versus observations in seconds (Figure 3), it was found that the transition was not predicted correctly at the exact moment when it occurred, but rather predicted it a step later. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Train \%** & **Test \%** & \multicolumn{4}{c}{**Test Data**} \\ \hline
**First** & **Last** & **F1** & **RMSE** & \
\begin{tabular}{c} **No. of Transitions** \\ **Present in the Test Set** \\ \end{tabular} &
\begin{tabular}{c} **No. of** \\ **Transitions Detected** \\ \end{tabular} & **R-Squared \({}^{*}\)** \\ \hline
10 & 90 & 0.24 & 1.46 & 73 & 37.639 & 0.62 \\
20 & 80 & 0.06 & 1.78 & 65 & 6624 & 0.44 \\
30 & 70 & 0.30 & 1.18 & 58 & 12.688 & 0.73 \\
40 & 60 & 0.31 & 1.11 & 51 & 9968 & 0.74 \\
50 & 50 & 0.33 & 1.06 & 42 & 6615 & 0.75 \\
60 & 40 & 0.30 & 1.04 & 32 & 3967 & 0.76 \\
70 & 30 & 0.26 & 1.00 & 24 & 2567 & 0.72 \\
80 & 20 & 0.31 & 0.88 & 16 & 2205 & 0.75 \\
90 & 10 & 0.22 & 0.95 & 8 & 830 & 0.61 \\ \hline \hline \end{tabular}  
* R-squared for test data.  
\end{table} 
Table 7: Forecasting efficacy of the two-state Markov-switching effects dynamic regression model for different training and testing partitions. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Train \%** & **Test \%** & \multicolumn{4}{c}{**Test Data**} \\ \hline
**First** & **Last** & **F1** & **RMSE** & \
\begin{tabular}{c} **No. of Transitions** \\ **Present in the Test Set** \\ \end{tabular} &
\begin{tabular}{c} **No. of** \\ **Transitions Detected** \\ \end{tabular} & **R-Squared \({}^{*}\)** \\ \hline
10 & 90 & 0.9998 & 0.22 & 73 & 413 & 0.99 \\
20 & 80 & 0.9998 & 0.18 & 65 & 71 & 0.99 \\
30 & 70 & 0.9998 & 0.13 & 58 & 58 & 0.9967 \\
40 & 60 & 0.9997 & 0.13 & 51 & 51 & 0.9967 \\
50 & 50 & 0.9997 & 0.09 & 42 & 42 & 0.9983 \\
60 & 40 & 0.9998 & 0.08 & 32 & 32 & 0.9985 \\
70 & 30 & 0.9998 & 0.07 & 24 & 24 & 0.9987 \\
80 & 20 & 0.9998 & 0.06 & 16 & 16 & 0.999 \\
90 & 10 & 0.9998 & 0.05 & 8 & 8 & 0.9989 \\ \hline \hline \end{tabular}  
* R-squared for test data.  
\end{table} 
Table 8: Forecasting efficacy of the LSTM model for different training and testing partitions. Figure 2: Absolute error versus observations in seconds for the two-state Markov-switching dynamic regression model with a 30%/70% training/testing split. The RNN results (Table 9) showed that when the training ratio was 40% and above, the number of transitions predicted for the test data and the actual number of transitions were equal. Similarly to LSTM, the transition was not predicted at the exact moment when it occurred in the test data set. The F1 score and R-squared for the data test set was about 0.99 at a test ratio of 40% and above. CNNs were not originally developed for time-series modeling, but the layers can be defined in a way to support time-series-based predictions. When we implemented the CNN, the results as shown in Table 10 indicated that there was a significant difference in the number of transitions predicted on the test data and the actual number of transitions; however, the high R-squared values (>0.9 for when 20% or more of the initial data were used for training) and RMSE below 1 indicated that missed predictions tended to be quite close in proximity to the actual reported values. The prediction efficacy improved steadily as the proportion of training data increased. When more than 50% of the initial values were used for training, F1 scores above 0.9 were achieved, indicating that CNNs could be potentially useful for forecasting. Figure 3: Absolute error versus observations in seconds for the LSTM with a 30%/70% training/testing split.