## 4 Experimental Study
### Results and Discussion  
We report the median and interquartile range (IQR) of the indicator values corresponding to the 20 independent runs per configuration in Tables 4-6. The best and second-best results are highlighted with a dark and light grey background, respectively. We applied the Wilkinson rank-sum test [109] (at a 5% level of significance) to determine whether the differences were statistically significant. The results of the test are shown in Tables 7-9. In each table, each symbol in the cells represent a problem instance, and there are different symbols: a dark up triangle means that the algorithm in the row performed better than the one in the column with confidence; a white down triangle stands for the opposite; a \"-\" symbol means that no significant differences were found. From the results of the tables, we can observe that NSGA-II was the best-performing algorithm, achieving the lowest indicator values in most of the cases. According to the EP indicator (Table 4), which measures the diversity of the Pareto front approximations found by the solvers, NSGA-II achieved the best values in seven out of nine benchmark problems, while SMS-EMOA and MOEA/D yielded the fronts with the highest degree of convergence in one instance. The Wilcoxon rank-sum test (Table 7) showed that the differences between NSGA-II and SMS-EMOA and between NSGA-II and MOEA/D were statistically significant in six problems and eight problems, respectively. The data reported in Table 5 show that NSGA-II obtained the lowest NHV values in seven instances. Here, we must note that values of 1.0 in the NHV indicator mean that the evaluated fronts were dominated by the reference point, and consequently, their HV was 0.0. This occurred, in particular, in all the algorithms when solving the dmu40 and ta77 problems. As these are the most complex JSP instances, it is clear that the algorithms would need more than 300,000 function evaluations as their stopping condition to find more accurate Pareto front approximations. The Wilcoxon rank-sum test (Table 8) confirmed that the differences in these two problems were not significant between the solvers in the comparison. Finally, the results of the IGD+ quality indicator, included in Table 6, confirmed again the best performance of NSGA-II against the other two algorithms in the conducted study with statistical confidence in most of the cases (see Table 9). We can observe that MOEA/D achieved the best IGD+ value in instance ta77, so this algorithm performed the best, with confidence, when solving the most complex JSP instance; this is coherent with its EP values (see Tables 4 and 7). To illustrate the fronts that were obtained, we selected the ta12 instance, consisting of 15 machines and 20 jobs. The reference front is shown in Figure 8, where we used different symbols to represent the points contributed by each algorithm: triangles (NSGA-II), circles (MOEA/D), and squares (SMS-EMOA). It can be seen how the best energy solution was obtained by SMS-EMOA, while the best makespan solution corresponded to NSGA-II. This brings an interesting observation: the quality indicators revealed that NSGA-II obtained the best values, but in the particular case of ta12, if a decision maker is interested in the energy, then SMS-EMOA could be the best choice. If we choose the extreme points of this reference front, we obtain a solution with the lowest energy consumption, but highest makespan and another one with the opposite. Figure 9 shows these two extreme solutions: the one with the best (minimum) energy consumption, above in the figure, with an energy of 603,908 units of energy and a makespan of 2378 units of time, and below the solution with 610,910 units of energy and a makespan of 1796 units of time. Including machine velocity and idle/stop energy consumption to the usual makespan problems has a big impact in the makespan and the energy consumption.