# Multi-Objective BiLevel Optimization by Bayesian Optimization
## 4 Experiments
### Noise-Free Experiments
#### 4.1.3 Test Problems
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Proposed Algorithm**} & \multicolumn{3}{c}{**DBMA**} & \multicolumn{3}{c}{**H-BLEMO**} \\ \cline{2-10}  & \multicolumn{1}{c}{_(q)_} & **UL** & **LL** & **UL** & **LL** & **UL** & **LL** \\ \hline \multirow{4}{*}{Problem 1} & 1 & 4500 & 27,000 & & & & & \\  & 2 & 6500 & 54,000 & & & & & \\  & 4 & 11,500 & 108,000 & & & & & \\  & 8 & 18,500 & 216,000 & & & & & \\ \hline \multirow{4}{*}{Problem 2} & 1 & 2700 & 14,580 & & & & & \\  & 2 & 4300 & 28,620 & & & & & \\  & 4 & 7200 & 38,880 & & & & & \\  & 8 & 9900 & 48,060 & & & & & \\ \hline \multirow{4}{*}{Problem 3} & 1 & 2675 & 57,780 & & & & & \\  & 2 & 1575 & 34,020 & & N/A & N/A & 18,733 & 319,499 \\ \cline{1-1}  & 4 & 6575 & 142,020 & & & & & \\ \cline{1-1}  & 8 & 3125 & 67,500 & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Function evaluation comparison table for different batch sizes (\(q=1,q=2,q=4,q=8\)). \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Proposed Algorithm**} & \multicolumn{3}{c}{**DBMA**} & \multicolumn{3}{c}{**OMOPSO-BL**} & \multicolumn{3}{c}{**H-BLEMO**} \\ \cline{2-10}  & \multicolumn{1}{c}{_(q)_} & **HV** & **IGD** & **HV** & **IGD** & **HV** & **IGD** & **HV** & **IGD** \\ \hline \multirow{4}{*}{Problem 1} & 1 & 0.3524 & 0.0143 & & & & & & \\  & 1 & \(\pm\)0.0003 & \(\pm\)0.0005 & & & & & & \\  & 0.3632 & 0.0138 & & & & & & & \\  & 2 & \(\pm\)0.005 & \(\pm\)0.002 & & & & & & \\  & 0.3621 & 0.0126 & & & & & & & \\  & 4 & \(\pm\)0.012 & \(\pm\)0.005 & & & & & & \\  & 0.3773 & 0.0119 & & & & & & & \\  & 8 & \(\pm\)0.019 & \(\pm\)0.009 & & & & & & \\ \hline \multirow{4}{*}{Problem 2} & 1 & 0.2067 & 0.0105 & & & & & & \\  & 1 & \(\pm\)0.005 & \(\pm\) 0.006 & & & & & & \\  & 2 & \(\pm\)0.009 & \(\pm\)0.0068 & & & & & & \\  & 2 & \(\pm\)0.0005 & \(\pm\)0.002 & & & & & & \\  & 0.2125 & 0.0045 & 0.2069 & N/A & 0.2074 & 0.0102 & 0.2067 & 0.0063 \\  & 4 & \(\pm\)0.002 & \(\pm\)0.0005 & & & & & & \\  & 8 & \(\pm\)0.004 & \(\pm\)0.002 & & & & & & \\ \hline \multirow{4}{*}{Problem 3} & 1 & 0.2026 & 0.0105 & & & & & & \\  & 1 & \(\pm\)0.004 & \(\pm\)0.006 & & & & & & \\  & 0.1915 & 0.0147 & & & & & & & \\  & 2 & \(\pm\)0.023 & \(\pm\) 0.002 & & & & & & \\  & 4 & \(\pm\)0.009 & \(\pm\)0.001 & & & & & & \\  & 0.2241 & 0.0138 & & & & & & & \\  & 8 & \(\pm\)0.042 & \(\pm\) 0.0087 & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Medium hypervolume (HV) and inverted generational distance (IGD) comparison table for different batch sizes (\(q=1,q=2,q=4,q=8\)) with the standard deviation. taken from [25]. The IGD results in Table 3 show that the proposed algorithm approximates successfully to the Pareto-optimal solutions. Table 2 shows that the proposed algorithm significantly decreases function evaluations during the optimization while approximating the Pareto-optimum solutions for Problem 3. We compare our results with DBMA and H-BLEMO algorithms, but not OMOPSO-BL because they did not share this information (as mentioned above). The HV value for batch size 1 is better than OMOPSO-BL but not H-BLEMO, as we can see in Table 3. However, the proposed algorithm reached better HV for batch size 8. IGD values are shown in Table 3 and we can observe that the approximation solution sets are not as close as the other algorithms. However, we think that considering the significant decrease in function evaluations, it shows good performance. Practical Case: CEO Problem  
For the CEO problem, they use a weighted sum method to obtain a single optimal solution \(\mathbf{x_{u}}=(146.2955,28.9394)^{\top}\) and \(\mathbf{x_{l}}=(0,67.9318,0)^{\top}\) in [24]. They use a hybrid particle swarm optimization algorithm with a crossover operator in [24]. Ref. [43] present Pareto-front solutions and extreme points using CMODE/D algorithm. We present the extreme points of our results in Table 4. The proposed algorithm can obtain the Pareto-front containing the single optimal solution obtained in [43].