# Decision-Maker's Preference-Driven Dynamic Multi-Objective Optimization
## 3 Experimental Setup
### Statistical Analysis  
A statistical analysis of the performance measure values was conducted in accordance with the wins-losses\({}_{B}\) algorithm proposed in [1]. The wins-losses\({}_{B}\) algorithm was implemented in R [51], and the Kruskal-Wallis and Mann-Whitney U statistical functions in R were used as stipulated in [1]. The calculation of wins and losses by the wins-losses\({}_{B}\) algorithm is presented in Algorithm 9 [52]. A win or loss is only recorded if there was a statistical significant difference in performance of the two algorithms that are being compared with the pair-wise Mann-Whitney U test. Therefore, \(Diff>0\) indicates a good performance, since the DMOA obtained more wins than losses. On the other hand, \(Diff<0\) indicates a poor performance, since the DMOA was awarded more losses than wins. ```
1:for each benchmark do
2:for each \(n_{t}\)-\(\tau_{t}\) combination do
3:for each performance measure, \(p_{m}\)do
4:for each algorithm \(alg\)do
5: Calculate the average value \(pm_{avg}\) for each of the 30 runs
6: Perform Kruskal-Wallis test on the average values, \(pm_{avg}\)
7:if statistically significant difference then
8:for each pair of algorithms do
9: Perform Mann-Whitney U test
10:if statistically significant difference then
11:for each environment \(env\)do
12: Assign a win to algorithm with best average over all
13:\(pm_{avg}\) for \(env\)
14: Assign a loss to algorithm with worst average over all
15:\(pm_{avg}\) for \(env\)
16: Calculate \(Diff=\#wins-\#losses\)\(\triangleright\) calculate \(Diff\) for each parameter (benchmark, \(n_{t}\)-\(\tau_{t}\) combination, performance measure, algorithm) as required for analysis
```  
**Algorithm 9** wins-losses\({}_{B}\) algorithm for wins and losses calculation [52]  
Table 4 presents the results for six experimental configurations, for various \(n_{t}-\tau_{t}\) combinations, i.e., different types of environment changes. It highlights the total number of wins and losses obtained by each constraint managing approach (algorithm) over all benchmarks and measures for each of the environment types. The death-penalty algorithm (DPA) performed the best for four types of environments (\(n_{t}-\tau_{t}\) combinations), while the proportionate-penalty algorithm (PPA) outperformed the other DMOAs in the other two environment types (\(n_{t}=10,\tau=2\) and \(n_{t}=10,\tau=4\)). In those six \(n_{t}-\tau_{t}\) combinations, the RSTFRA never outperformed DPA but it performed better than PPA in two types of environments (\(n_{t}=1,\tau=4\) and \(n_{t}=1,\tau=2\)). DPA was the only DMOA that obtained more wins than losses for all environment types. The Restrict-search-to-feasible-region algorithm (RSTFRA) obtained more losses than wins for all environment types, except for \(n_{t}=10,\tau=2\). On the other hand, PPA obtained more wins than losses for all environments, except \(n_{t}=1,\tau=4\) and \(n_{t}=1,\tau=2\). Figure 2: Decisions found by DPA (above) and RSTFRA (below) for FDA5 with Sphere Spec = \((1.7808,2.9185\times 10^{-59},0.0,1.5)\), \(n_{t}=10\) and \(\tau_{t}=5\). Figure 3: Decisions found by DPA (above) and PPA (below) for FDA5 with Sphere Spec = \((1.0276\times 10^{-16},4.4955\times 10^{-62},1.68781,1.5)\), \(n_{t}=10\) and \(\tau_{t}=2\). Table 5 presents the performance of the proposed algorithms with respect to the performance measures discussed in Section 3.4. It highlights the total number of wins and losses obtained by each algorithm for all the benchmarks and environment types for each of the performance measures. DPA performed the best for five of the eight measures and second-best for the rest. Two (react and dVD) of its five wins were ties with RSTFRA. Results for the first four measures in Section 3.4 indicated that DPA performed the best for three (acc, hvr, react) out of the four measures. It won with the least number of losses for the accuracy measure, acc, making it the most accurate of the proposed algorithms. For those four measures, RSTFRA won once (stab), but obtained the same number of losses as PPA for the win.