# A Literature Survey on Offline Automatic Algorithm Configuration
## 6 Multi-Objective META-EAs  
As algorithm configuration procedures improved, researchers focused on the multi-objective algorithm configuration problem instead of optimizing a single performance objective, because in many cases, multiple competing performance objectives have importance for algorithm configuration problem, such as running time, memory consumption,solution quality, etc. A multi-objective automatic algorithm configuration problem also aims to explore the trade-offs between multiple performance objectives (the initial studies on this topic are [34; 35; 36; 37]). We will summarize the multi-objective algorithm configuration methods that take inspiration from multi-objective evolutionary algorithms (MOEAs) in this section. As mentioned in the study [31], when more than one problem or performance metric is used, an easier method is to aggregate them into one measure for optimizing all of them. However, according to the results of this study, such an approach leads to several problems. An alternative method to aggregation is regarding each performance measure and test function in the test set as one objective [38], which means creating a parameter Pareto front (non-dominated parameter sets, satisfying more than one objective). In the method of [PERSON] [39], they used multiple performance measures (algorithm speed and algorithm accuracy) of four different algorithms at the same time without specifying weights for those two objectives beforehand. However, they handle only one parameter at a time, and the test suite is a unique problem. They used the multiple runs (fixed number) and averaged them to estimate the utility. A multi-objective optimization algorithm, NSGA-II, has been used as a tuner. Although using performance fronts is costly, their results prove that it is very suitable for the parameter configuration problem of metaheuristics. Additionally, [PERSON] et al. [38] extended the study of [39] to a multi-function optimization problem, where each fitness function in the problem set represents one objective (so the number of objectives is the same as the number of fitness functions). They introduced the Multi-Function Evolutionary Tuning Algorithm (M-FETA), which is based on a Multi-Objective Evolutionary Algorithm (MOEA) method and can approximate the parameter Pareto front. The parameter Pareto set includes non-dominated parameter vectors, which means that all other parameter vectors perform significantly worse. In such configuration problems, the utility of parameter vectors is defined by the performance of the algorithm (with these parameter values) on a collection of functions. Because heuristic optimization algorithms are stochastic, the performance of a parameter configuration can only be estimated due to the noise in the results. A common way of improving these estimates is by repeating the measurement. However, it means doing more executions, and so it is a costly way of gaining more confidence. M-FETA has a special technique for assessing the quality of candidate parameter vectors. In their technique, the number of expensive tests is reduced (or confidence improved) by judging the utilities of neighboring parameter vectors evaluated before. Their results showed that Pareto fronts allow for the investigation of interactions between parameter stings, fitness functions, and the optimization algorithm [38]. When compared to the single objective meta-EA, these approaches provide an important added value. Following the same idea, BONESA [40] is suggested as an iterative model-based approach to find a robust parameter set for multiple test instances. According to their experimental results, their method can provide a good estimation of the utility values of parameters and collects a lot of useful information on EA robustness.