## 8 Conclusions
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline  & **Reproducibility** & **Design Space Exploration** & **Robustness** & **Scalability** \\ \hline Computational resource need & SPC & exploring more solutions requires resources & **PC**: assessing robustness requires resources & \\ \hline Expected solution quality & PC & exploration is required to obtain different high quality solutions & **NC**: maximizing solution qualities can yield highly sensitive solutions & \\ \hline Parameter tuning & & & & **SNC** a large required effort for parameter selection, e.g., by tuning methods, decreases scalability \\ \hline Level of complexity of stochastic operators, can reduce reproducibility & PC & increase in complexity, i.e., use of stochastic operators, can increase the design space exploration & **PC**: increase in complexity, e.g., surrogates, parallel approaches, aims at achieving scalability \\ \hline Reproducibility & NC & high exploration has a high chance of producing different high quality solutions, i.e., a low reproducibility & \\ \hline Design space exploration & & & **NC**: high exploration can decrease the scalability (i.e., producing high dimensions or requiring an exponential increase in computational resources to obtain high quality solutions) & \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Part 2 of correlations between the quality criteria for the Decision Matrix. The correlations are: **SPC**, strong positive correlation; **PC**, positive correlation; **NC**, negative correlation; **SNC**, strong negative correlation. \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline  & **Computational Resource Need** & **Expected Solution Quality** & **Any-Time Performance** & **Parameter Tuning** \\ \hline Convergence to optimum value & **SR** more resources are required to find optimum value with high accuracy & **MR**: some, but not necessarily all, high quality solutions are located close to the optimum value & \\ \hline Low computational time & **SR** computational time depends on the used computational resources & **MR**: low computational time requirements to obtain high quality algorithms to obtain high quality algorithms & **MR**: any-time algorithms can provide improved solutions, compared to the initial solution, in short time & **MR**: parameter tuning effort requires time \\ \hline Generation of new design alternatives & **MR**: resources are required to create and evaluate (many) alternative designs & **WR**: new design alternatives can have an improved solution quality & & \\ \hline \multicolumn{3}{p{113.8 pt}}{} \\ \hline Ease of use & & & **SR**: parameter selection involves additional methods or special knowledge \\ \hline Algorithm & & & **MR**: ever knowledge/tuning effort is required to apply the algorithm to different problem or to change part of the algorithm & **MR**: parameter tuning effort is required to apply the algorithm to different problem or to change part of the algorithm \\ \hline Applicability to whole product range & & & **WR**: parameters may have to be changed for each product type \\ \hline Low effort for integrating algorithm in operational system & **WR**: demanding algorithms require allocation & & \\ \hline Comprehensibility & & & **MR**: the effect of the choice of parameter values on the obtained solution quality is not always be clear \\ \hline Trade-off decision support & **SR**: investigations of several characteristics requires more resources & **WR**: different characteristics may requires different parameter choices (and respective tuning) \\ \hline \end{tabular}
\end{table} 
Table 10: Part 1 of relationships between end-user requirements and comparison criteria for the Decision Matrix. The used relationships are: **SR**, strong relationship; **MR**, moderate relationship; **WR**, weak relationship.