## 4 Computational Results
### Results  
All of the experiments were run in an Intel (Intel Corporation, Santa Clara, CA, USA) i7 personal computer with 8 GB of RAM, and the problem was coded using MATLAB R2012a. Table 5 presents summary statistics from evaluating all 270 instances of every benchmark set. Here, the Average Deviation (Avg. Dev.) is computed as the average of all of the deviations when evaluating (\(M_{\mathrm{s}}-M^{\star}\)) \(M^{\star}\), where \(M^{\star}\) represents the reference makespan when comparing the results of Stage 1 and Stage 3 (\(M^{\star}\) can be either a confirmed optimal or the Best-Known Makespan), and \(M_{\mathrm{s}}\) denotes the makespan of the current stage. Furthermore, S1, S2 and BKO represent the makespan of Stage 1, Stage 2 and the Best-Known Optima (BKO), respectively. From Table 5, it is noticed that the number of optima found decreases as the number of activities increases. This is because of the increasing complexity of the problem; however, it is worth noting that these optima can be found in two different stages (Stages 1 and 3), which explains why there can be a negative average deviation when Stage 3 is compared against Stage 1 (e.g., Column MM100) or Stage 2 (MM30 through MM100). Looking at the last column, the overall average deviation of the schedules resulting from Stage 1 is 3.06%, which confirms the power of the discrete version of the ABC algorithm implemented. Furthermore, the average deviation of the schedules resulting from Stage 2 is 9.44% when compared to the BKO, but only 5.38% when compared against the input received from Stage 1. Finally, the average deviation of the schedules resulting from Stage 3 is 4.93% when compared to the BKO, only 0.59% when compared against Stage 1 and \(-\)5.16% when compared against Stage 2. In Table 5, the column \"overall average\" indicates that there is a difference between the instances' makespan when compare against the BKO. Therefore, to test if this difference is statistically significant, Table 6 presents the hypotheses to be tested for each benchmark set, and Table 7 summaries the results. \begin{table}
\begin{tabular}{c c c} \hline \hline \multicolumn{1}{c}{\(\mathbf{H_{0}}\): \(\mathsf{\mu_{BKO}}=\mathsf{\mu_{S1}}\)} \\ \(\mathbf{H_{1}}\): \(\mathsf{\mu_{BKO}}<\mathsf{\mu_{S1}}\) \\ \hline \(\mathbf{H_{0}}\): \(\mathsf{\mu_{BKO}}=\mathsf{\mu_{S2}}\) \\ \(\mathbf{H_{1}}\): \(\mathsf{\mu_{BKO}}<\mathsf{\mu_{S2}}\) \\ \hline \(\mathbf{H_{0}}\): \(\mathsf{\mu_{BKO}}=\mathsf{\mu_{S3}}\) \\ \(\mathbf{H_{1}}\): \(\mathsf{\mu_{BKO}}<\mathsf{\mu_{S3}}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Hypotheses tests for each stage against Best-Known Optima (BKO). \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Benchmark Set**} & \multicolumn{2}{c}{**\(\mathbf{\mu_{p}}\)-Value**} \\ \cline{2-5}  & **Stage 1** & **Stage 2** & **Stage 3** \\ \hline
**MM30** & 0.067 & 0.000 & 0.024 \\
**MM50** & 0.057 & 0.000 & 0.029 \\
**MM100** & 0.049 & 0.000 & 0.047 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: \(p\)-Values for the hypotheses’ tests. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Benchmark Set** & **MM30** & **MM50** & **MM100** & **Overall Average** \\ \cline{2-5}  & **Optima Found** & **260** & **123** & **84** & **Overall Average** \\ \hline
**Stage 1** & _Avg. Dev. vs. BKO_ & 0.18\% & 4.57\% & 4.42\% & 3.06\% \\ \hline
**Stage 2** & _Avg. Dev. vs. BKO_ & 9.69\% & 10.13\% & 8.50\% & 9.44\% \\  & _Avg. Dev. vs. S1_ & 6.02\% & 5.84\% & 4.27\% & 5.38\% \\ \hline
**Stage 3** & _Avg. Dev. vs. BKO_ & 5.04\% & 5.39\% & 4.37\% & 4.93\% \\  & _Avg. Dev. vs. S1_ & 1.10\% & 0.79\% & \(-\)0.12\% & 0.59\% \\  & _Avg. Dev. vs. S2_ & \(-\)5.36\% & \(-\)5.48\% & \(-\)4.62\% & \(-\)5.16\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Results summary for benchmark instances. Avg. Dev., Average Deviation. Based on the \(p\)-values obtained for the hypotheses tested, it is concluded that there is not enough statistical evidence to reject the null hypotheses for Stagel vs. BKO.