## 4 Simulation Results and Comparisons
### Comparisons of Large Problem  
To illustrate the computing efficiency of the proposed method, three heuristic methods, PSO, GA and ES, were used to solve the large problem with precise evaluation. The parameters used in the PSO [28] were: both social factor and cognitive factor were 2; the maximum permissible velocity was 0.5; inertia factor was 1; and size of population was 50. The parameters used in the GA [29] were: size of population was 50; crossover rate was 0.8; and mutation rate was 0.03. A real-value coding, single point crossover and roulette wheel selection were employed in the GA. The parameters used in the ES [30] were: size of population was 50, number of offspring was 100, and mutation strength constant was \(1/\sqrt{12}\). It should be noted that the precise evaluation was used to estimate the objective value for each method. The large problem was simulated over 30 independent runs. Because the three heuristic methods spend a long period obtaining the optimal solutions, we terminated their executions when they had spent 100 min of CPU time. Since the case \(N=20\) takes the longest consumed CPU time in large problem, progression result obtained using the OOTSA for case \(N=20\) was compared with three heuristic methods. Figure 7 shows the solution qualities and computational efficiencies obtained by four methods for 30 independent runs. The \"\(*\)\" point with coordinates (1.61, 61.55) shown in Figure 7 represents the pair of (consumed CPU time, average obtained objective value \(F(\mathbf{x})\)) obtained using the OOTSA for case \(N=20\). The progressions of the average best-so-far objective value and the corresponding consumed CPU times at the end of every iteration of three competing methods are also shown in Figure 7. The progressions of these values associated with PSO, GA and ES are plotted as solid lines with triangles \" - - \", solid line with open circles \" - - \", and solid line with crosses \" - \", respectively. Table 3 demonstrates that the average best-so-far objective values that were obtained by PSO, GA and ES were 10.74%, 18.47% and 16.21% larger than that obtained by OOTSA. The average best-so-far objective values obtained by the three heuristic methods were worse than not only that determined by the OOTSA with \(N=20\), but also those obtained using the three other values of \(N\). Test results reveal that the OOTSA approach frequently determines an excellent solution in a limited amount of time and significantly outperforms three heuristic methods with precise evaluation. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \(N\) & \(\mathbf{x}^{*}\) & \(E[f(\mathbf{x})]\) & \(P(g(\mathbf{x})\geq b)\) & \(F(\mathbf{x})\) & **CPU Times (s)** \\ \hline
20 & [51,49,48,47,48,30,29,20,20,20,19,19]T & 68.39 & 0.9002 & 61.56 & 96.35 \\ \hline
15 & [52,47,48,49,49,29,29,20,20,19,19,19]T & 69.75 & 0.9013 & 62.77 & 93.41 \\ \hline
10 & [53,48,49,48,47,29,30,20,20,19,19,18]T & 70.94 & 0.9024 & 63.84 & 90.94 \\ \hline
5 & [54,49,47,48,48,30,29,20,19,18,20,18]T & 71.39 & 0.9019 & 64.25 & 89.17 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The excellent solution \(\mathbf{x}^{*}\), the corresponding \(E[f(\mathbf{x})]\), \(\mathrm{P}(g(\mathbf{x})\geq b)\), \(F(\mathbf{x})\) and the consumed CPU times for four values of \(N\) in large problem. Subsequently, the solution quality of the OOTSA is appreciable. To judge the global optimality of the obtained solution, the rankings of the solution resulted from four methods were compared. It is impossible to compare the rankings of all solutions in the entire search space. Accordingly, a representative subset, \(\Theta\), was constructed to substitute the large search space. We arbitrarily selected 16,641 solutions from search space to construct the representative subset \(\Theta\) and utilizes the precise evaluation to estimate their objective values. The size \(|\Theta|\) = 16,641 was calculated by the sample size formula using a confidence level of 99% and a confidence interval of 1% [27].