## 3 Single-Objective Parameter Tuning
### The Tuning Algorithm
#### 3.1.3 Iterated F-Race  
F-race has a major restriction. It only evaluates the initial set of candidate configurations \(\Theta_{1}\). The parameter space is not explored beyond this initial sampling of configurations. Thus, a large sample must be taken to ensure sufficient coverage of the search space. Iterated F-race (IF-race) aims to solve this problem [6].  
IF-race starts with \(t=0\) and an initial set of randomly sampled candidate configurations. F-race is then used to eliminate the worst-performing candidates. However, F-race does not search for only one optimal solution. Instead, it is terminated when \(|\Theta_{k}|\leq n_{min}\) for some \(n_{min}\). When F-race terminates with surviving candidate set \(\Theta_{s}\), \(n_{s}=|\Theta_{s}|\leq n_{min}\) candidates remain. From this surviving set, \(n_{g}=n-n_{s}\) new candidates will be generated. Here, \(n\) is the total number of configurations to evaluate with F-race at each iteration.  
For each configuration \(\theta_{z}\in\Theta_{s}\), a weight is calculated as  
\[w_{z}=\frac{n_{s}-r_{z}+1}{n_{s}\frac{n_{s}+1}{2}} \tag{8}\]  
where \(r_{z}\) is the rank of \(\theta_{z}\). A single elite configuration, \(\theta_{e}\in\Theta_{s}\), is then chosen with probability proportional to its weight. This elite configuration serves as the seed for generating a new set of configurations \(\Theta_{g}\). Suppose \(\theta_{e}\) has numerical parameter values \(x_{i}\), where \(i=1,2,\ldots,d\). Each new parameter value is then sampled from a normal distribution with mean \(\mu_{i}=x_{i}\) and standard deviation  
\[\sigma_{i}=v_{i}\left(\frac{1}{n}\right)^{\frac{1}{2}} \tag{9}\]where \(v_{i}=x_{i}^{max}-x_{i}^{min}\) is the difference between the maximum and minimum values for parameter value \(i\). The candidate that will then be evaluated at iteration \(t+1\) is then \(\Theta_{t+1}=\Theta_{s}\cup\Theta_{g}\). The inclusion of the iteration counter \(t\) in Equation (9) causes the standard deviation to decrease as time progresses, causing F-race to focus more on values around the elite configuration. The iterated F-race algorithm is described in Algorithm 1.  
```
Sample \(N\) configurations uniformly with \(d\) control parameters each. Let \(\alpha\) be the chosen significance level. \(t\gets 0\) while\(t<T\)do\(\triangleright\) Iterated F-race \(k\gets 0\)  Let \(\Theta_{k}\) be the surviving configs at iteration \(k\)  Let \(P\) be the history of \(p\)-values while\(|\Theta_{k}|>n_{min}\)do\(\triangleright\) F-race  Evaluate the performance of each \(\theta\in\Theta_{k}\)  Perform Friedman test on \(\Theta_{k}\)  Let \(p\) be the \(p\)-value of the test \(P\gets P\cup p\) if\(p<\alpha\)then  Remove worst configs from \(\Theta_{k}\) \(P\leftarrow\emptyset\) else  Perform least-squares regression on P  Let \(m\) be the gradient of regression line if\(m\geq 0\)thenreturn\(\Theta_{k}\) endif \(k\gets k+1\) endwhile  Calculate weights \(W\) of \(\Theta_{k}\)  Choose elite config \(\theta_{2}\) based on \(W\)  Sample \(N_{s}\) new configs \(\Theta_{new}\) from \(\theta_{2}\) \(\Theta_{0}\leftarrow\Theta_{k}\cup\Theta_{new}\) endwhile return\(\Theta_{0}\)
```  
**Algorithm 1** Iterated F-race.