# A Literature Survey on Offline Automatic Algorithm Configuration
## 6 Multi-Objective META-EAs
### Model-Based Methods
#### 2.4.1 SPO, SPO+, and TB-SPO  
The last extension of the EGO algorithm is the sequential parameter optimization (SPO) suggested by [PERSON] et al. [85]. The SPO procedure starts with defining design point sets (initial configuration) by using Latin Hypercube Sampling (LHS) (available at [[https://cran.r-project.org/web/packages/lhs/index.html](https://cran.r-project.org/web/packages/lhs/index.html)]([https://cran.r-project.org/web/packages/lhs/index.html](https://cran.r-project.org/web/packages/lhs/index.html)) (accessed on 10 June 2022)) such as SKO. To select m design points, the parameter value interval must be divided into m equal intervals, and then a random number is chosen in each interval. The performance of each parameter vector is determined after a number of executions due to the stochasticity of algorithms, and the best-performing point is selected as the initial solution. Then, a stochastic Gaussian model is constructed to represent the relation between the design points and the results and to estimate the algorithm performance. Then new design points that will be used in the proceeding iterations are selected and tested through the intensification mechanism. These new design points consist of the best points found so far and a set of expected good designs (which have the highest expected improvement) according to a model created in previous stages. This model is updated after each iteration based on the best found configuration and is used in the subsequent iteration. The process terminates when the stopping criterion has been reached, and gives one best performing configuration as a result. They evaluated the SPO method in three different cases and \begin{table}
\begin{tabular}{c c c} \hline \hline
**Methods** & **Advantages** & **Disadvantages** \\ \hline \multirow{6}{*}{CALIBRA} & \(\bullet\) & Because CALIBRA uses both the experimental \\  & designs and local search, it can quickly & \(\bullet\) \\  & approximate the promising region [13]. & \(\bullet\) \\  & It is expected that, when parameter values of \\  & the tuned algorithm have a major effect on its \\  & performance, the results of the CALIBRA & \(\bullet\) \\  & process will be more satisfactory [48]. & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: _Cont._all three scenarios; parameter vectors tuned by SPO increased the performance of the algorithm. They summarized some drawbacks of SPO, such as that it is constrained to decimal and integer values, so it needs the specification of some parameter values. After researching the key design components of the SPO algorithm, such as choosing the initial design, whether to fit models to raw or log-transformed performance data, the expected improvement criterion, and the intensification criterion, [PERSON] et al. [86] suggested an improved version called SPO+. In the first part of the study, they compared SPO and SKO, and SPO showed better performance than SKO. For details on the SKO method and implementation differences between the SKO and SPO methods, we refer to [86]. Then they investigated key design components of the SPO method, and, consequently, after evaluating these four, they showed that the log-transformation and the intensification criterion essentially affect its performance. Then they compared the performance of SPO and SPO+ to the ParamILS and demonstrated that SPO+ demonstrated better performance in their experiment. [PERSON] et al. [87] used random permutation tests to decide whether a candidate configuration must be tested again in the next iteration as often as the current best configuration or it can be eliminated fairly. Additionally, they gave recommendations for an adequate budget allocation for SPO. Furthermore, their study analyzed the interaction between global and local search in sequential tuning procedures. [PERSON] [88] applied [PERSON]'s _optimal computing budget allocation_ (OCBA) [89] to the SPO Toolbox (SPOT). The SPOT package is implemented in the R programming language and it can be achieved on the website [90].