# A Model-Driven Approach for Conducting Simulation Experiments
## 5 Evaluation
### Increasing Productivity and Quality for Complex Experiments  
Besides sharing information about the base experiments, our approach is designed to share meta models for diverse, complex experiment types. Table 3, for instance, shows a meta model for global SA, which could be added to any of the above-described base meta models. It comprises various important ingredients for the specification of global SAs, such as factor ranges and factor distributions, as well as a choice of different sampling strategies: Monte Carlo (\(M\)C) and quasi-Monte Carlo (\(QMC\)) [58], orthogonal Latin hypercube (\(OLHC\)) [59], and polynomial chaos expansion (\(PC\)) [60]. For the first three strategies, the samples are used directly to calculate the indices. With a PC, on the other hand, first, a surrogate model is constructed based on which indices are computed. As the index type, e.g., Sobol indices [61; 62] can be used, which are variance-based measures. The first-order Sobol index of factor \(x_{i}\) describes the individual contribution of this factor to the overall variance in the output \(V(y)\):  
\[S_{i}=\frac{V_{x_{i}}[E_{x_{\sim i}}(y|x_{i})]}{V(y)}.\]  
The total-order sensitivity index, \(T_{i}\), accounts for all the contributions to the output variation due to factor \(x_{i}\) (i.e., first-order index plus higher-order interactions):  
\[T_{i}=\frac{E_{x_{\sim i}}[V_{x_{i}}(y|x_{\sim i})]}{V(y)}.\]  
Having made explicit the ingredients of global SA as a meta model, we can easily specify simulation experiments to calculate Sobol indices for various simulation models. We show this for three different models by composing the respective base meta model with the SA meta model. This has the potential to increase productivity during the simulation study since guidance is provided via a specialized GUI or CLI and input validation. Moreover, the complicated details of the code are abstracted away by the model-driven approach. Thus, the modeler does not have to worry about how to combine the simulation tools and analysis tools in a complex experiment. Figure 4 shows three different ways of performing a Sobol analysis; they all can be generated using the same meta model. \begin{table}
\begin{tabular}{l l l l l l l} \hline \hline  & **Name** & **Description** & **Type** & **Choices** & **Def.** & **Req.** \\ \hline \multirow{5}{*}{**S**} & factors & Information about the model factors & Map & – & – & yes \\  & \(\bullet\)factorName & Name of the factor (key) & String & – & – & yes \\  & \(\bullet\)factorMinimumValue & Lower bound on the factor value & Real & – & – & yes \\  & \(\bullet\)factorMaximumValue & Upper bound on the factor value & Real & – & – & yes \\  & \(\bullet\)factorDistribution & Assumed distribution of the factor & String & Uniform, Normal, \(\ldots\) & Uniform & yes \\  & \(\bullet\)factorDistributionParameters & Parameterize the distribution & Map & – & – & no \\  & \(\bullet\)distributionParameterName & Parameter of the distribution (key) & String & – & – & no \\  & \(\bullet\)distributionParameterValue & Initialize the distribution parameter & Real & – & – & no \\  & samplingStrategy & Choose the sampling strategy & String & MC, QMC, OLHC, PC, & MC & yes \\  & sampleSize & Number of samples & Integer & – & – & no \\ \hline \multirow{2}{*}{**S**} & indexType & Choose type of sensitivity index & String & Sobol, \(\ldots\) & – & yes \\  & bootstrapCI & Calculate confidence interval with bootstrapping & Boolean & – & false & no \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Meta model for the experiment type “global sensitivity analysis” (including information about default values (Def.) and the required status (Req.)). To create a valid sensitivity analysis experiment, various information about the model factors has to be provided, as well as information about the sampling procedure (experiment design) and instructions for calculating the sensitivity indices. The rows describe the different input properties. Sub-properties are denoted by “\(\bullet\)”.