# An Analysis of Differential Evolution Population Size
## 3 Differential Evolution
### Functional Analysis of Variance  
The functional analysis of variance (fANOVA) [100] is a powerful statistical technique used to decompose the observed variation in a response function into additive components associated with each subset of its input variables. The decomposition of variance in the response is used to analyze the extent to which performance variability is caused by a single control parameter or combinations of control parameters. The mathematical formulation for fANOVA is outlined below: Given an algorithm A that has \(n\) control parameters with domains \(\Theta_{1},\ldots,\Theta_{n}\) and configuration space \(\Theta=\Theta_{1}\times\cdots\times\Theta_{n}\). Let  
* the control parameters set be \(N=\{1,\ldots,n\}\);
* a control parameters instantiation be denoted as \(\theta=\langle\theta_{1},\ldots,\theta_{n}\rangle\) with \(\theta_{i}\in\Theta_{i}\);
* a control parameters subset be denoted as \(\Phi=\{\theta_{i},\ldots,\theta_{j}\}\), and the associated partial parameters instantiation is given \(\theta_{\Phi}=\langle\theta_{\phi_{i}},\ldots,\theta_{\phi_{j}}\rangle\) with a subset \(\Phi\subseteq N\);
* \(X(\theta_{\Phi})\) be the extension set of \(\theta_{\Phi}\), defined as the set of control parameters configurations that form a complete instantiation when combined with \(\theta_{\Phi}\), denoted by \(\theta_{T}\) where \(T=N\setminus\Phi\);
* the performance metric to quantify the performance of A denoted by \(m(\theta_{i},\pi_{j})\) over a set of \(k\) problems, \(\pi_{1},\ldots,\pi_{k}\).  
Then, the marginal performance of \(\theta_{\Phi}\), denoted as \(\hat{m}\), is defined as the expected variance in performance associated with \(\theta_{\Phi}\) with respect to all instantiations of the remaining control parameters \(T\), given by  
\[\hat{m}(\hat{\boldsymbol{\cdot}}_{\Phi})=\frac{1}{||\Theta_{T}||}\int\hat{y}( \theta_{T})d\theta_{T}, \tag{3}\]  
where \(\hat{y}:\mapsto\mathbb{R}\), and \(||\Theta_{T}||=\prod_{l=1}^{k}||\Theta_{T}||\). In short, given a subset \(\Phi\subset N\), \(\hat{m}(\theta\Phi)\) provides an estimation of the average performance across all instantiations in the parameter space \(N\setminus\Phi\) when the parameters of \(\Phi\) are fixed at \(\theta_{\Phi}\).  
The basic concept of the prediction technique in the context of fANOVA involves the construction of a random forest, where each regression tree defines a partition of the configuration space \(\Theta\), such that the marginal prediction of the entire forest is obtained by averaging the prediction over each tree [100].  
The fANOVA approach uses the predicted marginal performance \(\hat{m}\) to decompose the observed variation of a response value \(\mathbb{V}\) into additive components associated with each subset of its input variables \(\Phi\subset N\) as follows:  
\[\mathbb{V}=\sum_{\Phi\subset N}\mathbb{V}_{\Phi}, \tag{4}\]  
where \(\mathbb{V}_{\Phi}\) is the contribution of subset \(\Phi\) to the total variance. The importance of each subset can be computed as the fraction of the variance of the subset associated with the total performance variance. A larger fraction of variance associated with a particular control parameter indicates a higher sensitivity to the values of that control parameter. Therefore, control parameters with higher values for the variance are more influential and thus should have a higher priority when tuning control parameter values.