# A Hybrid Metaheuristic for Multi-Objective Scientific Workflow Scheduling in a Cloud Environment
## 5 Performance Evaluation
### Simaulation Results  
The proposed HBMMO algorithm was evaluated against a set of well-known techniques to solve multi-objective optimization problem, including NSGA-II [40], MOPSO [46], and MOHEFT [3]. For the purpose of comparison, all algorithms employed the same number of evaluation functions. The parametric values for NSGA-II are set as: population size \(n=100\), maximum iterations \(i=500\), crossover rate \(cr=0.8\), and mutation rate \(mr=0.01\); for MOPSO: population size \(n=100\), learning factors \(c_{1}=1.5\) and \(c_{2}=1.5\), and inertia weight \(w=[0.9-0.7]\); and for MOHEFT: number of trade-off solutions \(n=100\). To achieve the Pareto optimal solutions with the algorithms, the scheduling is repeated 50 times for each algorithm. The results are obtained by taking the average. The VMs are selected randomly such that the fastest VM is three times faster than the slowest one as well as three times more expensive.  
Figure 4 shows the multi-objective non-dominated solutions obtained for Montage, CyberShake, Epigenomics, and LIGO workflows, respectively. It shows that a lower makespan is correlated with a higher cost and vice versa. We can see that the solutions obtained using HBMMO have a better search ability due to the uniform distribution of solutions than those of MOHEFT, MOPSO, and NSGA-II. It can be seen that the Pareto fronts obtained using HBMMO are superior for all of the workflow instances under consideration. Even in the case of CyberShake, where MOHEFT performs significantly better, HBMMO is still able to maintain better convergence and a uniform distribution of solutions.  
Figure 5 shows the results obtained by computing the mean GD for Montage with 25 tasks, CyberShake with 30 tasks, Epigenomics with 24 tasks, LIGO Inspiral Analysis with 30 tasks, and SIPHT with 30 tasks. It is observed that the GD value for the proposed HBMMO algorithm is lower as compared to other algorithms. It implies that the solution set generated by the proposed method has a better ability to converge towards the true Pareto front.  
Figure 4: _Cont._  
As can be seen from Figure 6, the mean \(HV\) of the HBMMO against the comparative algorithms is statistically better in most of the scenarios. Compared with NSGA-II, the performance gain is over 50% in most of the cases whereas the improvement rate of HBMMO over MOHEFT is slightly better for small- and medium-size workflows. It can be concluded that HBMMO has better search efficiency and can achieve better non-dominated fronts.  
Figure 4: Bi-objective Pareto fronts for benchmark workflows: (a) Montage_25; (b) CyberShake_30; (c) Epigenomics_24; (d) LIGO_30.  
Figure 5: Generational distance obtained after 15 runs of the benchmark workflows.  
Figure 6: _Cont._  
Figure 6: Comparison of \(HV\)-metric for different algorithms on benchmark workflows: (**a**) Montage; (**b**) CyberShake; (**c**) Epigenomics; (**d**) LIGO; (**e**) SIPHT.  
Figure 7 shows a graph of the average Degree of Imbalance (\(DI\)) of tasks among VMs. It describes the fairness of the tasks' distribution across the VMs. HBMMO has the best load distribution of tasks between the VMs. Therefore, the smaller value of DI reveals the better performance of the achieved solution set.  
Figure 8 shows that HBMMO achieves a better average makespan than the comparative algorithms for all of the benchmark workflows. Figure 9 shows the improvement rate of HBMMO over the MOHEFT algorithm in terms of makespan and cost of the workflow applications. In the case of makespan, the HBMMO gains an improvement by 7.26%, 14.04%, 4.54%, and 16.6% compared with the MOHEFT, using the Montage, CyberShake, Epigenomics, and LIGO workflows, respectively. The cost of HBMMO was better by 14%, 18%, 8%, and 4% over the MOHEFT algorithm in the case of the Montage, CyberShake, Epigenomics, and LIGO workflows, respectively.  
Figure 8: Comparison of average makespan for different algorithms.  
Figure 7: Comparison of Degree of Imbalance (\(DI\)) of tasks among VMs.