# Kriging-Based Variable Screening Method for Aircraft Optimization Problems with Expensive Functions
## 2 Review of the Variable Screening Methods  
### Variance Based Sensitivity Analysis  
Analysis of Variance (ANOVA) is a set of statistical models utilized to investigate the differences between inputs and corresponding processes. In ANOVA, the variance observed in a certain input can be classified according to different causes of variation.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Abbreviation** & **Meaning** \\ \hline CFD & Computational Fluid Dynamics \\ KRG-VSM & Kriging-based Variable Screening Method \\ GA & Genetic Algorithm \\ DR & Dimensionality reduction \\ PCA & Principal Component Analysis \\ MVU & Maximum Variance Unfolding \\ LLE & Local Linear Embedding \\ SA & Sensitivity Analysis \\ GSA & Global Sensitivity Analysis \\ quasi-OTEE & Quasi-optimized Trajectory Based Elementary Effects \\ SAFE & Sensitivity Analysis for Everybody \\ KRG-VSM & Kriging-based Variable Screening Method \\ MLE & Maximum Likelihood Estimator \\ ANOVA & Analysis of Variance \\ KRG-VSM & Kriging-based Variable Screening Method \\ GA & Genetic Algorithm \\ LHS & Latin Hypercube Sampling \\ Nfe & Number of function evaluations \\ \hline \hline \end{tabular}
\end{table}
Table 1: List of abbreviations.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Symbol** & **Meaning** \\ \hline \(N\) & Sample size \\ \(k\) & Number of inputs \\ \(X_{i}\) & Generic input \\ \(Y\) & Generic scalar model output expressed by \\ \(X\) & \(N\times k\) matrices of input factors \\ \(A\), \(B\) & \(N\times k\) matrices of inputs \\ \(X_{\sim i}\) & \(N\times(k-1)\) matrices of all inputs except \(X_{i}\). \\ \(A_{B}^{(i)}\) & Matrix, whose \(i\)th column comes from matrix \(B\) while the \\ \(V_{X_{i}}(\cdot),E_{X_{i}}(\cdot)\) & Variance and mean of variable \((\cdot\ )\) taken over \(X_{i}\) \\ \(V_{X_{\sim i}}(\cdot),E_{X_{\sim i}}(\cdot)\) & Variance and mean of variable \((\cdot\ )\) taken over all factors \\  & except \(X_{i}\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Notations description.  
The total sensitivity indicator is given by  
\[S_{T_{i}}=\frac{E_{X_{\sim i}}\big{(}V_{X_{i}}(Y\mid X_{\sim i})\big{)}}{V(Y)}=1- \frac{V_{X_{\sim i}}\big{(}E_{X_{i}}(Y\mid X_{\sim i})\big{)}}{V(Y)} \tag{1}\]  
where \(V_{X_{\sim i}}\big{(}E_{X_{i}}(Y\mid X_{\sim i})\big{)}\) is obtained as  
\[V_{X_{\sim i}}(E_{X_{i}}(Y\mid X_{\sim i}))=\frac{1}{N}\sum_{j=1}^{N}f(A)_{j}f \Big{(}A_{B}^{(i)}\Big{)}_{j}-f_{0}^{2} \tag{2}\]  
where the matrices \(A\) and \(B\) are \(N\times k\) matrices of input factors. And \(f_{0}\) denotes the objective function, i.e., \(f(X_{1},X_{2},\dots,X_{k})\), with zero inputs. As mentioned above, the sample size \(N\) is usually large for complex optimization problems; thus, the metamodel is leveraged to implement the variance-based sensitivity analysis. For Equation (9), the active inputs are \(x_{12},x_{4},x_{20},x_{19},x_{1}\), and \(x_{5}\). The Latin hypercube sampling (LHS) method is used to get 50 samples. Then, the total sensitivity indices of the inputs for the metamodel constructed by 50 expensive samples can be achieved. The metamodel is evaluated 5000 times to collect data for ANOVA-based sensitivity analysis. The result is shown in Table 3. The active inputs \(x_{12},x_{1}\), and \(x_{20}\) are selected, while the active inputs \(x_{4}\) and \(x_{19}\) are ignored. It is difficult to construct an acceptable metamodel for the variance-based sensitivity analysis using a few samples.  
As the sample size is added to 500, the metamodel constructed by these samples could identify all the active inputs through ANOVA-based sensitivity analysis. The result is shown in Table 4. Due to the stochastic nature of LHS, ANOVA-based SA may occasionally fail to identify all active inputs.