# Efficient Multi-Objective Simulation Metamodeling for Researchers
## 4 Results
### Continuous News Vendor Simulation Case Study  
As mentioned in Section 3.1.3, the goal of this case study is to optimize a news vendor's stock quantity and selling price, considering random daily demand. In both the hypervolume evaluation plotted in Figure 12 and the Hausdorff distance evaluation plotted in Figure 13, the NN-GA pair is a consistent worst-performer when compared to the other three MO pairs. Figure 11: Telecom experiments—crowding distance evaluation—time series. Figure 12: Continuous News Vendor experiments—hypervolume evaluation—time series. Figure 10: Telecom experiments—Hausdorff distance evaluation—time series. Table 3 summarises the results from the Continuous News Vendor experiments based on the mean normalized hypervolume performance (\(\mu_{norm}\)s) of each MO pair with the best-performing one at each time limit. For any given time limit, the GR-B pair outperforms the rest of the methods. At the 100 s time limit, all methods deliver a similar performance to GR-B, although their \(\mu_{norm}\)s are slightly worse. This observation persists for the time limit of 150 s, while NN-GA performs significantly worse than GR-B. As the time limit increases above 200 s, the performance differences between GR-B and each of the other MO pairs become statistically significant with \(p\)-values less than 0.05, and GR-GA ranks the second best-performing method among all. The \(\mu_{norm}\)s of GR-B increases from 0.836 to 0.939 as the time limit increases by a factor of 2.5. One feature of the Continuous News Vendor optimization problem is that relatively few sets of inputs will correspond to a point on the Pareto front: an exhaustive search encompasses 651 potential input combinations of which only eleven make up the optimal Pareto front. Solution Pareto fronts had between four to fifteen points across all the experiments. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Time Limit (s)** & **MO Pair** & \(\mathbf{\mu_{norm}}\) & \(\mathbf{\sigma}\) & \(\mathbf{p}\)**-Value** \\ \hline
100 & GR-B & 0.835878 & 9.33 \(\times\) 10\({}^{-3}\) & Best \\  & GR-GA & 0.811036 & 1.17 \(\times\) 10\({}^{-2}\) & 1.45 \(\times\) 10\({}^{-1}\) \\  & NN-B & 0.820163 & 8.09 \(\times\) 10\({}^{-3}\) & 1.32 \(\times\) 10\({}^{-1}\) \\  & NN-GA & 0.807869 & 1.20 \(\times\) 10\({}^{-2}\) & 7.89 \(\times\) 10\({}^{-2}\) \\ \hline
150 & GR-B & 0.920727 & 5.03 \(\times\) 10\({}^{-3}\) & Best \\  & GR-GA & 0.918869 & 4.00 \(\times\) 10\({}^{-3}\) & 4.96 \(\times\) 10\({}^{-1}\) \\  & NN-B & 0.899821 & 7.03 \(\times\) 10\({}^{-3}\) & 1.18 \(\times\) 10\({}^{-2}\) \\  & NN-GA & 0.836305 & 9.49 \(\times\) 10\({}^{-3}\) & 7.18 \(\times\) 10\({}^{-8}\) \\ \hline
200 & GR-B & 0.935907 & 3.00 \(\times\) 10\({}^{-3}\) & Best \\  & GR-GA & 0.921874 & 3.95 \(\times\) 10\({}^{-3}\) & 9.02 \(\times\) 10\({}^{-3}\) \\  & NN-B & 0.914207 & 5.74 \(\times\) 10\({}^{-3}\) & 2.30 \(\times\) 10\({}^{-3}\) \\  & NN-GA & 0.847465 & 8.43 \(\times\) 10\({}^{-3}\) & 2.91 \(\times\) 10\({}^{-9}\) \\ \hline
250 & GR-B & 0.939361 & 2.22 \(\times\) 10\({}^{-3}\) & Best \\  & GR-GA & 0.915885 & 4.24 \(\times\) 10\({}^{-3}\) & 9.44 \(\times\) 10\({}^{-5}\) \\  & NN-B & 0.907325 & 9.17 \(\times\) 10\({}^{-3}\) & 1.21 \(\times\) 10\({}^{-3}\) \\  & NN-GA & 0.850925 & 9.84 \(\times\) 10\({}^{-3}\) & 8.26 \(\times\) 10\({}^{-9}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Continuous News Vendor experiments—normalized hypervolume performance comparison of each MO pair with the best-performing one at each time limit. Figure 13: Continuous News Vendor experiments—Hausdorff distance evaluation—time series.