# Multi-Objective BiLevel Optimization by Bayesian Optimization
## 4 Experiments
### Noisy Experiments
#### 4.2.3 Test Problems
\begin{table}
\begin{tabular}{l l} \hline \hline
**Problem** & **Formulation** \\ \hline \multirow{4}{*}{Example 1} & \(\underset{(x_{u},\mathbf{x_{l}})}{\text{Min}}\ \mathbf{F}(x_{u},\mathbf{x_{l}})= \begin{pmatrix}(x_{l_{1}}-1)^{2}+\sum_{l=2}^{K}x_{l_{i}}+(x_{u})^{2}+\  
and approximated decisions algorithm in Figure 5. Again we tried different batch sizes for the experiment, and it can be seen that a batch number of 8 is best for this specific example at both dimensions. We could not compare the 20-dimensional version of the problem with the selected algorithms because of a lack of information in [45]. Example 2  
Table 7 shows that our proposed algorithm obtains the best IGD results compared to the other algorithms. In terms of FE, it significantly improves the state of the art, with an \(\approx\)81% improvement for 10 variables and an \(\approx\)88% improvement for 20 variables compared to m-BLEAQ. We also show the HV difference in Figure 5, and we can observe that, for this specific example, the batch number of 8 is the best selection for both 10 and 20 dimensional versions. Example 3  
Table 7 shows that our algorithm obtains the best IGD value compared to m-BLEAQ and H-BLEMO while improving efficiency in terms of FE: \(\approx\)84% and \(\approx\)97% with 10 variables, and \(\approx\)89% and \(\approx\)98% with 20 variables. Figure 5 shows that a batch size \(q=8\) gives the best results. In summary, our proposed algorithm is successful on the selected test problems while handling noisy objectives with less computational cost. Noise in the leader's objective makes the problem harder to solve but more realistic for modelling practical problems, because of real-world uncertainty. We show the proposed algorithm works well on these test benchmark problems. Practical Case: Gold Mining in Kuusamo  
In this section, we present the results obtained using the proposed algorithm on the analytical model of the problem proposed in Table 6. Figure 4 shows the Pareto-optimal front obtained using our approach. The plot gives the idea to the authority how to consider the trade-off between its own objectives. We used 50 iterations for upper-level optimization using a batch Bayesian approach with a batch size of 4. For lower-level optimization, the NSGA-II algorithm is implemented with the same parameters specified in Section 4.2.2. We can observe that with the proposed method, the obtained results are distributed around the true Pareto-optimal frontier and approximate to it successfully. We also run the experiments for the batch size \(q=1\), \(q=2\) and \(q=4\) for testing if the algorithm converges to the Pareto-optimal front. The IGD value is 0.0494 for \(q=1\), 0.0033 for \(q=2\) and 0.0021 for \(q=4\). We can see from the decreasing IGD values that, as we increase the batch number, the selected points more closely approximate the true Pareto-optimal front. The increasing batch size provides us with parallel evaluations during upper-level optimization, which decreases the needed execution time. It is also important to handle environmental uncertainty, represented by \(\xi\) in Table 6, which may be an uncontrollable parameter, such as inflation, during the time period of taxation or \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{1}{c}{**Number of**} & \multicolumn{1}{c}{**Proposed Algorithm**} & \multicolumn{2}{c}{**m-BLEAQ**} & \multicolumn{2}{c}{**H-BLEMO**} \\ \cline{3-8}  & **Variables** & **IGD** & **FE** & **IGD** & **FE** & **IGD** & **FE** \\ \hline Example 1 & 15 & 0.0044 & 4032 & 0.0013 & 6464 & 0.0046 & 39,818 \\ \hline Example 2 & 10 & 0.0051 & 4022 & 0.0069 & 22,223 & 0.0134 & 106,003 \\ \hline Example 3 & 10 & 0.0076 & 4022 & 0.0079 & 25,364 & 0.0134 & 132,907 \\ \hline Example 1 & 20 & 0.0105 & 4042 & - & - & - & - \\ \hline Example 2 & 20 & 0.1032 & 4042 & 0.0435 & 34,110 & 0.1106 & 191,357 \\ \hline Example 3 & 20 & 0.0924 & 4042 & 0.0623 & 36,439 & 0.1321 & 216,083 \\ \hline \hline \end{tabular}
\end{table}