# Integrated Simulation-Based Optimization of Operational Decisions at Container Terminals
## 2 Materials and Methods
### Employed Meta-Heuristics
#### 2.2.1 Tree-Structured Parzen Estimator  
TPE was developed to automate the search for a sufficiently well-performing configuration of a Deep Belief Network [41]. For a Deep Belief Network, parameters are either categorical variables (e.g., the decision whether to use pre-processed or raw data) or continuous variables (e.g., the learning rate). Integer values can be modeled as either categorical variables (the numbers are mere labels) or continuous variables that are rounded before further use. In addition, dependencies between variables exist: One variable determines the number of network layers and then each layer is configured on its own. Having the configuration of the third layer as part of a parameter configuration is only reasonable if the variable encoding the number of layers is set to at least three. Such a parameter is called a conditional parameter. To reflect the dependencies, this type of parameter configuration space is adequately represented as a tree. This requires specific meta-heuristics that support such a tree structure. The TPE approach has produced good benchmark results [42; 49], and the initial paper is among the most cited publications on hyper-parameter optimization; at the time of writing, Scopus indicates that there are 939 citations. The TPE models \(p(y<y*)\), \(p(x|y<y*)\), and \(p(x|y\geq y*)\), where \(p\) denotes a probability density function (short: density), \(y\) is a point evaluation of the model, and \(x\) is a parameter configuration. The first density \(p(y<y*)=\gamma\) is a fixed value (e.g., 0.15 was used in the first publication) set by the experimenter, and \(y*\) is altered to fit the set value of \(\gamma\) for each iteration. The other two densities can be summarized as \(p(x|y)\): The probability that a certain parameter configuration has been used, given a desired point evaluation value. Typically, TPE is formulated to find a minimum and, therefore, \(p(x|y<y*)\) describes the density of parameters that have shown better results, whereas \(p(x|y\geq y*)\) describes the density of the parameters that have led to poorer performance. As the true densities are unknown, they need to be estimated based on the obtained evaluations in each iteration. For each categorical parameter, two probability vectors are maintained and updated: Given the prior vector of probabilities \(p=(p_{1},...,p_{N})\), with each probability \(p_{i}\) for \(i\in 1,...,N\) representing one category, the posterior vector elements are proportional to \(N\cdot p_{i}+C_{i}\), where \(C_{i}\) counts the occurrences of choice \(i\) in the recorded evaluations so far. An example is depicted in Figure 4a. The estimator for the better-performing parameters (the top 15%) and that for the worse-performing parameters (the bottom 85%) are calculated based on the observations already recorded. For each continuous parameter, two adaptive Parzen estimators are used. Given a prior probability density distribution determined by the experimenter, with each point evaluation of the parameter configuration space with the help of the simulation model and the objective function, the densities are further approximated. An example is depicted in Figure 4b. The parameter choices of the better- and worse-performing estimators are shown in Figure 4c. The results are shown in Figure 4d. The results are shown in Figure 4e. performing models are used to create the respective densities. In each iteration, the model consisting of the two densities is used to select the next parameter configuration \(x\) to evaluate. To achieve this, several parameters \(x\) are sampled from the promising distribution \(p(x|y<y*)\). The parameter configuration \(x\) with the greatest expected improvement is chosen. This criterion is positively correlated with the ratio \(p(x|y<y*)/p(x|y\geq y*)\)[41]. In Figure 4, this is referred to as ratio.