## 3 Analysis of the DQL Method
### Benchmark for Step Strategies
#### 3.2.2 Performance Benchmark  
We used the 59 test functions from Section 2 of [22] and [23]. These problems were transformed to the sum of square problems to fit into our code environment. The dimensions of these problems range from 2 to 20. A large portion (26%) of the problems are in \(\mathbb{R}^{5}\), which is identical to the first solid-tank design problem discussed in Section 5. We note that Problem 2.13 and 2.17 from [22] were omitted due to scaling problems.  
The benchmarking and analysis followed the processes recommend in [24].  
We first solved all the problems using the same accuracy and stability requirement by the mincon function from Matlab. We used these solutions as a reference to the quality of our solutions. Then, we solved the problems by each strategy combination and recorded their number of function calls and the stop_flag.  
Since the direct step strategy uses a random rotation, we performed each method multiple times to obtain its average performance. We denoted the function calls used by strategy combination \(s\) for problem \(p\) at trial \(r\) as \(t_{s,p,r}\) and the average performance of strategy combination \(s\) for problem \(p\) as \(t_{s,p}\). If a method failed at some trial, we proceeded with the next trial until a successful trial or until the evaluation budget was exhausted. If the method found a solution, then we considered the function calls it used as the summation among all the previously failed trials plus this successful trial. Therefore, the average performance of strategy combination \(s\) for problem \(p\) was defined as  
\[t_{s,p}=\frac{\sum_{r}t_{s,p,r}}{r_{\text{total}}-r_{\text{fail}}}, \tag{31}\]  
where \(r_{\text{total}}\) is the number of total trials and \(r_{\text{fail}}\) is the number of failed trials.  
If \(t_{s,p}\) was larger than max_search, then we said that the strategy combination could not find the target solution within the evaluation budget and reset \(t_{s,p}=\infty\).  
We used the performance profile described in [25] to compare the performance among the strategy combinations. The performance profile first evaluated the performance ratio,  
\[r_{s,p}=\frac{t_{s,p}}{\min{\{t_{s,p}:s\in S\}}}, \tag{32}\]  
where \(S\) is the set of all strategy combinations. This ratio told us how the performance of strategy \(s\) at problem \(p\) compared to the best performance of the strategy at the problem. Then, we plotted the performance profile of strategy \(s\) as  
\[\rho_{s}(\tau)=\frac{1}{|P|}|\{p\in P:r_{s,p}\leq\tau\}|, \tag{33}\]  
where \(P\) is the set of all problems and \(|\cdot|\) denotes the number of elements in a set. The performance profile told us the portion of the problems solved by strategy \(s\) when the performance ratio was not greater than a factor \(\tau\in\mathbb{R}\). In all results, we validated the performance profile by also creating profiles with fewer strategies to check if the switching effect occurred [26]. The switching effect never occurred.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline \(\epsilon_{\
abla}\) & \(10^{-6}\) \\ \(\epsilon_{\text{MAX\_STEP}}\) & \(10^{-3}\) \\ \(\epsilon_{\text{MIN\_STEP}}\) & \(10^{-12}\) \\ max\_search & \(10,000\) \\ \hline \(\delta_{0}\) & \(10\) \\ \(\gamma\) & \(0.3\) \\ \(\beta\) & \(3\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Parameters for the Performance Benchmark