# Multi-Objective BiLevel Optimization by Bayesian Optimization
## 4 Experiments
### Noise-Free Experiments
#### 4.1.3 Test Problems  
We discuss the numerical experiments using the proposed algorithm on three different benchmark problems from [25], including quadratic unconstrained and linear constrained problems. Furthermore, we applied the proposed algorithm to the popular CEO problem from [24]. Problem 1  
The problem has a total of three variables and is linear and constrained at both levels. The formulation of the problem is given in Table 1. Both upper- and lower-level optimization tasks have two objectives and the reference point for the problem is \(x_{ref}=(-1,0)\). The linear constraint in the upper-level optimization task does not allow the entire quarter circle to be feasible for some \(y\). Thus, at most a couple of points from the quarter circle belong to the Pareto-optimal set of the overall problem. Reported Pareto-optimal solutions for the problem are in the third column of Table 1. Figure 3 on the left graph shows the Pareto-front solutions for Problem 1. Note that in this problem there exists more than one lower-level Pareto-optimal solution such as A in Figure 3. Therefore, in this problem finding the right Pareto-optimal lower-level solution is important for finding the correct bilevel decision set. Finding a solution like A makes the lower-level task useless, and this is the challenging part of the problem. Dashed lines represent the lower-level Pareto-fronts and B and C points as the two most Pareto-optimal solutions for the upper-level decision \(y=0.9\). Problem 2  
Problem 3  
We increase the dimension of the variable vector of Problem 2. The formulation of the problem is given in Table 1. The reference point for the problem is \(x_{ref}=(1,0.5)\). The  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Problem** & **Formulation** & **Pareto-Optimal Solutions** \\ \hline Problem  
upper-level problem has one variable and the lower-level is scalable with \(K\) variables. For this work, \(K\) is set to 14 and the total number of variables is 15 in this problem. For a fixed \(x_{u}\), the Pareto-optimal solutions of the lower-level optimization problem are given as follows: \(\{x_{l}\in\mathbb{R}^{K}|x_{u}\in[0,y],x_{l_{i}}=0,\) for \(i=2,...,K\}\). For the upper-level problem, the Pareto-optimal solutions are formulated in Table 1.  
Practical Case: CEO Problem  
This case is taken from [24]. In a company, the CEO has the leader problem of optimizing net profit and the quality of products. The branch head has the follower problem of optimizing its own profit and worker satisfaction. According to the scenario above, the problem itself can be modelled as a multi-objective bilevel optimization problem. The deterministic version of the problem can be formulated as in Equation (6). Let the CEO's decision variable be \(\mathbf{x_{u}}=(x_{u_{1}},x_{u_{2}})\) and the branch head's decision variable be \(\mathbf{x_{l}}=(x_{l_{1}},x_{l_{2}},x_{l_{3}})\). The constraints model requirements, such as material, marking cost, labour cost and working hours. \[\underset{(\mathbf{x_{u}},\mathbf{x_{l}})}{\text{Maximize}}\ \mathbf{F}( \mathbf{x_{u}},\mathbf{x_{l}})=\begin{pmatrix}x_{u_{1}}+9x_{u_{2}}+10x_{l_{1}}+ x_{l_{2}}+3x_{l_{3}}\\ 9x_{u_{1}}  
specific problem with batch sizes \(2,4,8\), as we can observe from Table 3. We think that as the problem is linear, and because of limited iteration numbers, the algorithm could not get as close as the others. There were no reported IGD values for the DBMA algorithm on this comparison. We can observe from Table 2 for Problem 2 that the proposed algorithm improves computational performance and decreases the function evaluations significantly compared to DBMA and H-BLEMO. The HV values are much better than DBMA, OMOPSO-BL and H-BLEMO algorithms as we can see from Table 3. Note that because of the unavailability of OMOPSO-BL and H-BLEMO experiment data, the results shown for these algorithms are