## 5 Evaluation Protocol and Results
### Experiment Results  
From the various experiments we performed to select the best model, it is notable that the architecture of Model-1 (Table 1) with the Adam optimizer and ReLU activation function is the best choice. It is also notable from the brain regions experiments that all channels work better to distinguish between the subjects of each class. To address the concerns regarding overfitting, we implemented a rigorous evaluation protocol. We employed 10-fold cross-validation, the common evaluation approach in machine learning, to ensure the robustness of the model against overfitting. Using this approach, the dataset was split into ten stratified sets (folds). Ten experiments were performed in such a way that in every experiment, one-fold was held out, in turn, as an independent test set. The model was trained using the remaining eight folds as the training set and the ninth fold as a validation set. After training the model, it was tested on the holdout test. In this way, the model was trained on different sets and was tested on different independent test sets. The results presented in Tables 7 and 8 show the robustness of the model; in every experiment (i.e., for each fold), the results are almost similar, and there is no significant difference. \begin{table}
\begin{tabular}{c c c c c} \hline
**Pattern’s Length** & **Stride** & **Avg-Accuracy** & **Avg-Sensitivity** & **Avg-Specificity** \\ \hline \multirow{2}{*}{1 s} & 0.125 s & 91.17\% & 92.21\% & 89.98\% \\ \cline{2-5}  & 0.25 s & 65.64\% & 72.18\% & 58.09\% \\ \hline \multirow{2}{*}{2 s} & 0.125 s & 96.20\% & 97.18\% & 95.07\% \\ \cline{2-5}  & 0.25 s & 74.56\% & 79.36\% & 69.01\% \\ \hline \multirow{2}{*}{3 s} & 0.125 s & 99.88\% & 100\% & 99.74 \\ \cline{2-5}  & 0.25 s & 83.31\% & 86.92\% & 79.13\% \\ \hline \end{tabular}
\end{table} 
Table 6: Results of testing different pattern lengths and stride. \begin{table}
\begin{tabular}{c c c c} \hline
**Fold Number** & **Accuracy** & **Sensitivity** & **Specificity** \\ \hline Fold1 & 98.87\% & 98.13\% & 98.48\% \\ \hline Fold2 & 97.03\% & 96.98\% & 97.00\% \\ \hline Fold3 & 98.26\% & 98.04\% & 98.14\% \\ \hline Fold4 & 96.21\% & 98.58\% & 97.48\% \\ \hline Fold5 & 96.21\% & 98.84\% & 97.62\% \\ \hline Fold6 & 98.26\% & 97.87\% & 98.05\% \\ \hline Fold7 & 96.43\% & 98.58\% & 93.95\% \\ \hline Fold8 & 99.08\% & 98.22\% & 98.62\% \\ \hline Fold9 & 95.08\% & 99.47\% & 97.43\% \\ \hline Fold10 & 96.33\% & 97.51\% & 94.97\% \\ \hline
**Avg** & 97.18\% & 98.22\% & 97.17\% \\ \hline \end{tabular}
\end{table} 
Table 7: Results of testing the best model without majority voting. The optimizers used for the training of the CNN models are iterative algorithms that start with an initial guess of learnable parameters. However, parameter initialization is a problem in all iterative algorithms, and the model's learning depends on this initialization. To overcome this problem, we ran the model five times and selected the one with the best results. In the following, we present the results of testing the method with the best model. First, we tested the model on the testing set without using majority voting; we applied the data augmentation scheme on the training set. Second, we tested the model on the testing set using majority voting. Tables 7 and 8 show the test accuracy, sensitivity, and specificity scores of each fold of the two experiments, respectively. From Table 8, we can notice that the model achieved 100% accuracy and specificity scores for all folds except fold 7 and 100% sensitivity scores for all ten folds. Thus, the average scores were 99.88%, 100%, and 99.74% for accuracy, sensitivity, and specificity, respectively. These results indicate that the model is robust and has no overfitting, so it generalizes well to classify the unseen data. Table 9 shows the confusion matrices on the testing data for fold 7, which has one mistake by classifying one normal subject as a schizophrenic, and fold 10, which was perfectly classified.