## 2 Related Work
**An acquisition function** is a function by which the posterior is used to select the next alternative (sample) from the search space. The acquisition function is used to optimize the conditional probability of locations in the search space that are most likely to generate good alternatives. Example acquisition functions that can be used are: expected improvement [33], predicted improvement [34], and upper confidence bound [35]. The acquisition function operates on top of the mean and standard deviation of the \(SM\)'s output. We tested several acquisition functions, and their influence on the overall performance was minimal. The final version of BAG-DSM uses the expected improvement (\(EI\)) as an acquisition function:  
\[EI(\overline{x},x_{b})=\left\{ \begin{array}{rl}(\mu(SM(\overline{x}))-S_{b}- \xi)*&\Phi(Z)+\sigma(SM(\overline{x}))*\Phi(Z),\;\;if\;\;\sigma(SM(\overline{x}) )>0\\ &0,\;if\;\sigma(SM(\overline{x}))=0\end{array} \right. \tag{8}\]  
where,  
\[Z=\left\{ \begin{array}{rl}(\mu(SM(\overline{x}))-S_{b}-\xi)/\sigma(SM( \overline{x})),\;\;if\;\;\;\sigma(SM(\overline{x}))>0\\ 0,\;\;if\;\;\;\sigma(SM(\overline{x}))=0\end{array} \right. \tag{9}\]  
where \(SM(\overline{x})\) is the output of all base models in the \(SM\) for the alternative \(\overline{x}\); \(S_{b}\) is the maximum predicted value by the \(SM\) from all previous observations; \(\Phi(\cdot)\) is the normal cumulative distribution function, and \(\xi\), is a constant that controls the tradeoff between global search and local optimization (exploration/exploitation). In our experiments, we used \(\xi=0.01\) as proposed by [PERSON] [36]. In simpler terms, this acquisition function checks the improvement that each candidate alternative brings with respect to the maximum known value (\(\mu(SM(\overline{x}))-x_{b}\)), and scales those improvements with respect to the uncertainty. The uncertainty is calculated empirically from the standard deviation of the predictions of the based models available in the \(SM\). If two alternatives have a similar mean value, the one with higher uncertainty (\(\sigma(SM(\overline{x}))\) will be preferred by the acquisition function. Finally, we need to define the generator of alternatives. BAG-DSM uses two generators of alternatives: a neighborhood generator and a random generator. Based on the distance function \(d\), a neighborhood relation can be defined. Two alternatives \(\overline{x_{1}}\) and \(\overline{x_{2}}\) are considered as neighbors with a degree \(k\), if \(d(\overline{x_{1}},\overline{x_{2}})=k\). For example, \(\overline{x_{1}}\) and \(\overline{x_{2}}\) are considered neighbors with a degree \(k=1\), if the distance between \(\overline{x_{1}}\) and \(\overline{x_{2}}\) is 1. They are considered as neighbors with a degree \(k=2\) if the distance between \(\overline{x_{1}}\) and \(\overline{x_{2}}\) is 2. The distance function \(d\) is defined in Equation (3). In scenarios where the desired outcome is an improvement of the current alternative, \(d\) calculates how many attributes in \(\overline{x_{2}}\) have a bigger value than the corresponding attributes in \(\overline{x_{1}}\). In the opposite scenarios, i.e., one would like to decrease the output of the decision model, \(d\) calculates how many attributes in \(\overline{x_{2}}\) have a smaller value than the corresponding attributes in \(\overline{x_{1}}\). Consequently, for a given alternative \(\overline{x}\), we can generate a set of neighbors with degree \(k\), by alternating the value of \(k\) different attributes in \(\overline{x}\). The degree \(k\) was set experimentally. The relation between the computing time and \(k\) is exponential and is mostly affected by the number of input attributes. For the models with a depth of 3, we used \(k=5\); for the models with a depth of 4, we used \(k=4\); and for models with a depth of 5, we used \(k=3\).