## 4 Multi-Objective Parameter Tuning  
The tuning process described thus far requires that some objective measure be defined that quantifies the performance of an algorithm given some set of control parameters. Thus far, four different performance objectives have been evaluated, each having different sets of optimal parameter configurations. But optimal performance based on a single objective is not always what is required from a meta-heuristic. If computational budgets are tight, the efficiency of the algorithm might be more important than its accuracy, or an algorithm might be required to be stable across multiple runs, consistently achieving an acceptable level  
\begin{table}
\begin{tabular}{l c c c c c c} \hline
**Objective** & \(w\) & \(c_{1}\) & \(c_{2}\) & \(N\) & **Evaluation Metric** & **Objective Function Value** \\ \hline Accuracy & \(0.8496\) & \(1.7091\) & \(0.7145\) & \(28\) & \(0.8528\) & \(0.8528\)\(\pm\) 0.6987 \\ Efficiency & \(0.7732\) & \(1.8594\) & \(0.8876\) & \(29\) & \(101.56\) & \(4.9794\)\(\pm\) 0.1637 \\ Stability & \(-0.1509\) & \(2.0264\) & \(0.0003\) & \(12\) & \(0.1933\) & \(20.3353\)\(\pm\) 0.1933 \\ Success rate & \(0.7951\) & \(2.1917\) & \(0.7019\) & \(31\) & \(100\%\) & \(1.9301\)\(\pm\) 0.1210 \\ \hline \end{tabular}
\end{table}
Table 4: Tuning results on Ackley.  
\begin{table}
\begin{tabular}{l c c} \hline
**Function** & **Lower Bound** & **Upper Bound** \\ \hline Rastrigin & \(-5.12\) & \(5.12\) \\ Ackley & \(-32.768\) & \(32.768\) \\ Styblinski and Tang & \(-5\) & \(5\) \\ \hline \end{tabular}
\end{table}
Table 2: Lower and upper bounds of the hypercubes defining the search spaces.  
\begin{table}
\begin{tabular}{l c c c c c} \hline
**Objective** & \(w\) & \(c_{1}\) & \(c_{2}\) & \(N\) & **Evaluation Metric** & **Objective Function Value** \\ \hline Accuracy & \(0.8052\) & \(2.3068\) & \(0.0515\) & \(29\) & \(0.3518\) & \(-782.8680\)\(\pm\) 0.2958 \\ Efficiency & \(0.6388\) & \(1.3306\) & \(0.7774\) & \(25\) & \(26.68\) & \(-594.9386\)\(\pm\) 11.7164 \\ Stability & \(0.8363\) & \(1.9570\) & \(0.0068\) & \(27\) & \(8.9029\) & \(-728.3246\)\(\pm\) 8.9029 \\ Success rate & \(0.7805\) & \(2.3361\) & \(0.0571\) & \(27\) & \(100\%\) & \(-780.4903\)\(\pm\) 0.5695 \\ \hline \end{tabular}
\end{table}
Table 5: Tuning results on Styblinski and Tang.  
of accuracy, rather than sporadically achieving a very accurate result. More importantly, different objectives could be relevant at the same time. Achieving good levels of accuracy might be desired while also having an efficient algorithm.  
This requires the use of a multi-objective tuning algorithm. Such an algorithm evaluates control parameter configurations based on multiple performance objectives. However, such a shift in evaluation no longer allows for a single optimal solution. A particular parameter configuration might have good performance based on one objective, but poor performance based on another, which makes it impossible to determine a single solution that optimises both objectives. A multi-objective tuning algorithm must be able to handle such competing objectives and present the experimenter with a set of solutions, each with varying levels of trade-off between objectives. The experimenter can then choose a parameter configuration that is optimal for their needs.  
This section introduces M-race, a multi-objective tuning algorithm based on the single-objective F-race. It follows similar principles as that of F-race, but serves as a generalisation to a multi-objective setting.