## 5 Computational Results
### Computational Results of Metaheuristics
These statistics indicate that VBIH generated much better results than both the \(\text{IG}_{\text{RS}}\) and \(\text{IG}_{\text{ALL}}\) algorithms. In order to see if these results are statistically significant, we provide the \(95\%\) confidence interval plot of algorithms in Figure 9, where we can observe that differences in ARPD values are statistically significant on the behalf of VBIH algorithm against both \(\text{IG}_{\text{RS}}\) and \(\text{IG}_{\text{ALL}}\) algorithms because their confidence intervals do not coincide. In other words, VBIH algorithm was statistically superior to both \(\text{IG}_{\text{RS}}\) and \(\text{IG}_{\text{ALL}}\) algorithm. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{\(n\times m\)} & \multicolumn{3}{c}{\(\text{IG}_{\text{RS}}\)} & \multicolumn{3}{c}{\(\text{IG}_{\text{ALL}}\)} & \multicolumn{3}{c}{\(\text{VBIH}\)} \\ \cline{2-10}  & **Avg. ** & **Min** & **Max** & **Avg. ** & **Min** & **Max** & **Avg. ** & **Min** & **Max** \\ \hline \(100\times 20\) & \(0.13\) & \(-0.14\) & \(0.39\) & \(-0.04\) & \(-0.21\) & \(0.1\) & \(-0.25\) & \(-0.44\) & \(-0.03\) \\  
In the Supplementary Materials, we summarize all the best-known solutions found for the first time by IGALL and VBIH algorithms. The VBIH algorithm further improves 230 out of 240 instances. In addition, 173 out of 240 instances are improved by the IGRS algorithm, while the IGALL algorithm further improves 222 out of 240 instances. The IGALL algorithm improves six instances that are not improved by VBIH algorithm. Ultimately, 236 out of 240 instances are further improved by all algorithms within \(45\times n\times m\) time limits with the remaining four solutions being equal. As mentioned before, IGALL algorithm is presented in [5], where they analyzed the performances of IGRS and IGALL on both [PERSON]'s [42] and large VRF instances. They observed that the results obtained by using [PERSON]'s benchmark set, both algorithms do not present very significant differences with respect to the RPDs obtained. In fact, they have shown that both algorithms did not show any statistically significant differences. However, statistically significant differences between IGRS and IGALL have been shown when large VRF instances are employed. In order to validate this observation, we have run three algorithms on [PERSON]'s benchmark set with a stopping criterion \(T_{max}=45\times n\times m\) milliseconds. Furthermore, we run three algorithms without the [PERSON]'s speed up method and they are denoted as IGRS*, IGALL* and VBIH*. The computational results are given in Table 9. As seen in Table 9, VBIH produced much better RPDs than IGRS and IGALL algorithms when the [PERSON]'s speed up method is employed since its overall RPD was 0.17 from the best-known solutions. However, IGRS and IGALL algorithms do not show so many differences in terms of RPDs. Interval plots of the algorithms in Figure 10 show that differences in RFDs are not statistically significant because their confidence intervals do coincide. This suggests a fact that researches on PFSP and its variants should employ VRF benchmark suite to see differences in algorithms newly presented. Figure 10 also shows that the [PERSON]'s speed up method is significantly effective for all three algorithms. During these runs, we were also able to find 3 new best-known solutions for the [PERSON]'s benchmark suite (ta054 = 3719, ta55 = 3610, ta56 = 3680) and their permutations are also provided in the Supplementary Materials. Figure 9: Interval plot at the 95% confidence level for large VRF instances.