## 6 Results
### Summary of Results  
Table 1 summarises the feasibility as success rate and solution quality as the mean objective value over the 30 runs and 18 problem instances.  
The first row (Success Rate) shows the average success rate, or feasibility, across all instances for all algorithms. This is calculated as the number of times a solution was found by the algorithm relative to the total number of runs. Similarly, the second and third rows (Mean Reliability, Mean Com. Overhead) show the average reliability and communication overhead, respectively, across all instances (where feasibility was found) for each algorithm. We see here that BACS (Col) is the best performing algorithm considering all measures, hence we conclude that using co-location bounds to guide the optimisation method towards feasible regions is the most effective method in finding not only feasible but also high-quality solutions.  
Due to the stochastic behavior of the algorithms, we conducted 30 runs per algorithm, which showed variability between runs in the results. Hence, the discrepancies are analysed for statistical significance considering the reliability and communication overhead criteria and the levels of violations seen. We performed an unpaired t-test on the means for all pairs of algorithms regarding reliability, communication overhead and the total number of violations.  
Let \(\mu_{diff}=\mu_{\text{alg1}}-\mu_{\text{alg2}}\) denote the difference between average reliability obtained from alg1 and alg2. The following competing hypotheses are tested against each other.  
\[\begin{cases}\mathcal{H}_{0}:\mu_{diff}=0\\ \mathcal{H}_{A}:\mu_{diff}\
eq 0\end{cases}\]  
The p-value of each test is obtained from the mean, the standard deviation, and the number of successful cases seen in each algorithm. The following matrix represents the p-values for each test given the 0.05 significance level.  
\[\begin{pmatrix}\text{BACS(Col)}&\text{BACS(Com)}&\text{BACS(Mem)}&\text{BACS(SS)} \\ \text{0.0003}&\text{0.0026}&\text{5.18}\times 10^{-6}&\text{3.61}\times 10^{-5}\\ &\text{0.8157}&\text{1.38}\times 10^{-1}&\text{8.39}\times 10^{-1}\\ &&\text{1.15}\times 10^{-1}&\text{6.5}\times 10^{-1}\\ &&&\text{1.22}\times 10^{-1}\\ &&&\text{BACS(Mem)}\\ &&&\text{BACS(SS)}\end{pmatrix}\]  
The first row of the matrix, clearly, shows that all versions of BACS are statistically different from ACS.  
The same argument is valid in comparing the performance of different algorithms considering the number of violations. In particular, the following matrix is the result of a pair-wise test of different algorithms regarding violations.  
\begin{table}
\begin{tabular}{l c c c c c} \hline  & **ACS** & **BACS (SS)** & **BACS (Mem)** & **BACS (Col)** & **BACS (Com)** \\ \hline Success Rate & 0.5 & 0.72 & 0.89 & **0.94** & 0.61 \\ Mean Reliability & 0.44 & 0.44 & 0.44 & **0.56** & 0.5 \\ Mean Com. Overhead & 0.62 & 0.62 & 0.71 & **0.92** & 0.86 \\ \hline \end{tabular}
\end{table}
Table 1: A summary of the performance of the algorithms across all instances. The results highlighted in boldface are statistically significant.  
\[\begin{array}{c c c c}\text{BACS(Col)}&\text{BACS(Com)}&\text{BACS(Mem)}& \text{BACS(SS)}\\ 0.004&7.1\times 10^{-5}&0.005&0.006\\ &7.73\times 10^{-3}&0.602&0.451\\ &&0.002&0.001\\ &&0.812\\ &&\text{BACS(Mem)}\\ &&\text{BACS(SS)}\end{array}\]  
The matrices of \(p\)-values, in particular the first rows, provide strong evidence of significant differences between the algorithms, hence we conclude that the superior performance of BACS is statistically significant.