# An Improved MOEA/D with Optimal DE Schemes for Many-Objective Optimization Problems
## 4 Experimental Study
### Experimental Results and Analysis
#### 4.4.2 MOEA/D-oDE Compared with MOEA/D and NSGA-II
Apart from this instance, for the other test instances, MOEA/D-oDE performs better or much better than MOEA/D and NSGA-II. Meanwhile, NSGA-II performs better than MOEA/D on the DTLZ function family with 4-5 objectives. However, for F1 and F2, MOEA/D always performs better than NSGA-II, no matter with many objectives. With the number of objectives increasing, MOEA/D compared to NSGA-II becomes better on DTLZ1 and DTLZ3 with six objectives. For this, we can see that MOEA/D with DE operators in dealing with problems with complicated PS shapes has its unique advantages, but for problems such as the DTLZ functions' family, its superiority is not very obvious. Nevertheless, we have used the DE operators for crossover in MOEA/D-oDE for all of the test problems. In terms of the IGD metric values, the results obtained by MOEA/D-oDE are much better than those obtained by MOEA/D and NSGA-II for almost all of these test problems. Its advantages are very clear, especially for F1 and F2. Then, we can conclude that MOEA/D-oDE can handle the many-objective optimization problems well, whether the problems have complicated PS shapes or not. This also indicates that MOEA/D-oDE is robust for solving these problems. On the other hand, Table 5 shows the GD metric values obtained by three algorithms, including the mean, the best and the standard values of the 20 IGD metric results, as well. From the table, we can see that the mean GD metrics of MOEA/D-oDE are superior to MOEA/D and NSGA-II on most test instances, as well as almost all of the best values of the GD metric belong to MOEA/D-oDE. However, the stability of MOEA/D-oDE is not very good. Through this table, we can obtain the same conclusions as Table 4.  
The evolution of the mean IGD metric values based on 20 independent runs for three algorithms is plotted in Figures 3, 4 and 5, respectively. As shown in these figures, we can find that the convergence speed of the MOEA/D-oDE is faster than the other two algorithms. However, there are many final IGD metric values obtained by NSGA-II that are lower than those obtained by MOEA/D. The convergence speed of NSGA-II is not faster than the other two algorithms in most instances. Meanwhile, the runtime of NSGA-II is the longest of the three algorithms. For NSGA-II, on F1 and F2 with 5-6 objectives, in the later stage of evolution, the obtained IGD metric values do not decrease, but increase. Hence, this indicates that NSGA-II is not good at solving the many objective problems with complicated PS shapes in another way. As described in Figures 6, 7 and 8, box plots of the IGD metric values among these three algorithms for the continuous problems with 4-6 objectives based on 20 independent runs are visualized. The box diagram can reflect the stability of a set of data. From Table 4 and those three box diagrams, we can conclude that MOEA/D-oDE performs more stably, especially for the F1 and F2 test instances. On the contrary, though NSGA-II has superior performance with respect to MOEA/D on some instances, its stability is usually not very good. The reason may be that the original NSGA-II could not solve the problems with multi-modality and bias very well. Additionally, the stability of MOEA/D does well in some test instances. In some of these box figures, such as Figures 6 and 8, there are some '\(+\)' on some test instances, which means the outliers. We can see that MOEA/D and NSGA-II have some larger outliers, which means that the two algorithms may perform worse in one of 20 runs. In other words, to some extent, MOEA/D and NSGA-II do not behave stably on some instances, such as DTLZ4 with four objectives. However, compared to MOEA/D and NSGA-II, the performance of MOEA/D-oDE is more stable.