## 7 Methodology and Experimentation
### Experimentation
#### 7.4.2 Performance  
As in Section 7.4.1, the current section is also divided into two subsections, one regarding human-designed heuristics results and the other for the RL agent. In Figures 8 and 9, we present the performance measured in GIPS for each human-designed heuristic using random walk as the model to generate connection events for 20 simulations. As seen in Figure 8, the values obtained for each simulation across all heuristics are not spread out but are instead close, which translates to small differences between the proposed heuristics with the reliability score and those that do not have the score. On the other hand, Figure 9 shows a different scenario where ReleSEAS and RelBPA can separate from the competitors  
\begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Job Set** & **Random** & **BPA** & **eSEAS** & **RelBPA** & **ReleSEAS** \\ \hline
1500 & 38.69\% & 40.38\% & 40.40\% & 48.59\% & 50.98\% \\
36,000 _(full set)_ & 5.75\% & 7.56\% & 7.63\% & 24.05\% & 26.39\% \\ \hline \hline \end{tabular}
\end{table}
Table 1: Average job completion throughout all simulations using random walk.  
Figure 6: Average job completion for each simulation using random walk to generate connection events and the full set of jobs.  
Figure 7: Average job completion for each simulation using random waypoint to generate connection events and the full set of jobs.  
in a significant way in most cases. In Figures 10 and 11, which present the performance when using random waypoint, we can see the same behavior but with less strength.  
In this metric, we use the complete and small datasets to explore whether the same relation is in job completion. Even though both metrics are highly related, having more completed jobs does not necessarily mean that the performance in GIPS will be equally high. This is because modeled jobs can--as in real life--be simple or complex; we might want to process a simple math calculation or run an image through an object detection model. Nevertheless, the exact relationship between the original human-designed heuristics and the proposed versions remains for both metrics.  
In Figure 8, the values obtained for each simulation across all heuristics are not scattered but are instead close, which translates to slight differences between ReleSEAS and RelBPA with regard to the original versions. On the other hand, Figure 9 shows a different scenario where ReleSEAS and RelBPA can significantly separate from the rest of the policies when using the complete set of jobs. In regard to the performance metric, we can see in Table 3 a difference of 14.98 GIPS for BPA and RelBPA, whereas for eSEAS and ReleSEAS, the difference is 19.07 GIPS. On the other hand, when using the complete set of jobs, the difference between BPA and RelBPA rises up to 631.9 GIPS, and then again, for eSEAS and ReleSEAS, the difference is 724.46. This also holds when using random waypoint, as can be seen in Table 4 and in Figures 10 and 11. Still, on a smaller scale, since this algorithm is more stable than the random walk, the new versions of human-designed heuristics presented in this paper thus do not outperform the classic versions in the same ways as in the random walk scenarios, which are more dynamic or unstable.  
Figure 8: Average performance in GIPS for each simulation using random walk to generate connection events and the 1500 jobs set.  
Figure 9: Average performance in GIPS for each simulation using random walk to generate connection events and the full set of jobs.