## 5 Computational Experiments

Table 2: Scaling factor computed for each CPU. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & & & \multicolumn{3}{c}{HGA\_RMA} & \multicolumn{3}{c}{PSOVNS} & \multicolumn{3}{c}{NEGAVNS} & \multicolumn{3}{c}{RDIS / RAMP} \\ \cline{3-10}
**Group** & **Instances** & **Size** & **Real** & **Scaled** & **Real** & **Scaled** & **Real** & **Scaled** \\ \hline
1 & ta001-ta010 & \(20\times 5\) & 4.5 & 5.31 & 300 & 381 & 10 & 10.9 & 10 & 10 \\
2 & ta011-ta020 & \(20\times 10\) & 9 & 10.62 & 300 & 381 & 20 & 21.8 & 20 & 20 \\
3 & ta021-ta030 & \(20\times 20\) & 18 & 21.24 & 300 & 381 & 40 & 43.6 & 40 & 40 \\
4 & ta031-ta040 & \(50\times 5\) & 11.3 & 13.334 & 300 & 381 & 25 & 27.25 & 25 & 25 \\
5 & ta041-ta050 & \(50\times 10\) & 25.5 & 26.55 & 300 & 381 & 50 & 54.5 & 50 & 50 \\
6 & ta051-ta060 & \(50\times 20\) & 45 & 53.1 & 300 & 381 & 100 & 109 & 100 & 100 \\
7 & ta061-ta070 & \(100\times 5\) & 22.5 & 26.55 & 600 & 762 & 50 & 54.5 & 50 & 50 \\
8 & ta071-ta080 & \(100\times 10\) & 45 & 53.1 & 600 & 762 & 100 & 109 & 100 & 100 \\
9 & ta081-ta090 & \(100\times 20\) & 90 & 106.2 & 600 & 762 & 200 & 218 & 200 & 200 \\
10 & ta091-ta100 & \(200\times 10\) & 90 & 106.2 & 600 & 762 & 200 & 218 & 200 & 200 \\
11 & ta101-ta110 & \(200\times 20\) & 180 & 212.4 & 600 & 762 & 400 & 436 & 400 & 400 \\
12 & ta111-ta120 & \(500\times 20\) & 450 & 531 & – & – & 1000 & 1090 & 1000 & 1000 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of maximum allowed running times for different metaheuristics. analyze their difference. For this purpose, these methods are initially ranked based on their average deviation from the best-known solution %DEVavg, for each instance group, as shown in Table 5. In the case of tied %DEVavg values, the average of the ranks with the assumption of no tied values, are calculated. The mean rank for NEGAVNS, RAMP, and RDIS are 1.5, 1.667 and 2.833 respectively. Friedman test has been performed on the ranked data set, showing a statistically significant difference in the ranks, with \(\chi^{2}(2)=15.2\), \(p=0.0005\). Further, for each pair of algorithms, post-hoc analysis using Wilcoxon signed-rank tests has been conducted with a significance level set at \(p<(0.05/3=0.017)\) (Bonferroni correction). It has been found that there were no significant differences between the ranks of NEGAVNS and RAMP (\(p\) = 0.482) but a significant difference between the ranks of NEGAVNS, and RDIS (\(p\) < 0.001 ) as well as between RAMP and RDIS (\(p\) < 0.001). This could indicate that out of the three compared methods, NEGAVNS and RAMP are the best performing ones and RDIS falls in second place. It is worth noting, these results should be interpreted with caution, as they are solely based on average deviation, and other factors such as best deviation, average and best running times, and implementation details have not been considered. An interested reader can see [49] for a tutorial on nonparametric statistical tests, used for comparing metaheuristics. For a more fine-grained analysis, an instance-by-instance performance comparison is presented in Tables 6 and 7. In these tables, the performance of the RDIS on these instances is shown and compared with that of the NEGAVNS presented in [43]. The NEGAVNS, which is a genetic algorithm (GA) and uses the variable neighborhood search (VNS) employs the NEH heuristic presented in [26] for constructing its initial solutions, and that is why it has been named as NEGAVNS. The NEH has also been named as the initial of its authors and is a famous algorithm. As further information, the performance of the NEH has also been provided in the table. In line with [43] ([PERSON], [PERSON] et al. 2009), we have set the running time of the RDIS to \((n*m)/10\) seconds for each run. Moreover, for removing the effect of the random seed and in line with other algorithms, the RDIS has been run 10 times for each instance, with different random seeds.