# Self-Tuning Lam Annealing: Learning Hyperparameters While Problem Solving
## 2 Methods
### Self-Tuning Lam
#### 2.3.5 Putting It All Together
Figure 13: Minimize \(f_{2,10}(x)\): Self-Tuning Lam, Modified Lam, and target acceptance rates for (**a**) 1000 iterations, (**b**) 10,000 iterations, (**c**) 100,000 iterations, and (**d**) 1 million iterations. #### 3.2.2 Single Global Minimum and Large Number of Local Minimums

We now consider a problem with many local minimums, but only a single global minimum, as illustrated in Figure 16, where we must minimize the function [41]:

\[g(x)=\frac{\sin(10\pi x)}{2x}+(x-1)^{4}, \tag{37}\]

for \(x\in[0.5,2.5]\). The minimum occurs at \(x\approx 0.548563\), where \(g(x)\approx-0.869011\). We introduce a scale parameter to explore the impact of cost scale on the behavior of the annealing schedules. Thus, we define the variation for \(\phi\in\{1,10,100,1000\}\):

\[g_{\phi}(x)=\phi\bigg{(}\frac{\sin(10\pi x)}{2x}+(x-1)^{4}\bigg{)}. \tag{38}\]

Figure 16: Graph of \(g(x)\). Figure 15: Minimize \(f_{2,1000}(x)\): Self-Tuning Lam, Modified Lam, and target acceptance rates for (a) 1000 iterations, (b) 10,000 iterations, (c) 100,000 iterations, and (d) 1 million iterations. Tables 15-18 summarize the results. At a high-level, we find a similar pattern to the previous continuous problems. For the original function scale (Table 15), runs with \(N\geq\) 10,000 produced no statistically significant difference, but for the shortest runs (\(N=1000\)), the Modified Lam found better quality solutions. Once we scale the cost function (Tables 16-18), the Self-Tuning Lam finds near-optimal solutions an order of magnitude faster than the Modified Lam (at \(N=10\),000 for the Self-Tuning Lam vs. \(N=100\),000 for the Modified Lam). Furthermore, for each of those three scales, the differences in solution quality for short runs (\(N\leq\) 10,000) are very statistically significant in favor of the Self-Tuning Lam. The differences in solution quality for the longest runs (\(N\geq\) 100,000) are not statistically significant. Both algorithms find solutions that are on average near-optimal for runs of those lengths. Figure 17: Minimize \(g_{1}(x)\): Self-Tuning Lam, Modified Lam, and target acceptance rates for (a) 1000 iterations, (b) 10,000 iterations, (c) 100,000 iterations, and (d) 1 million iterations. Figure 18: Minimize \(g_{10}(x)\): Self-Tuning Lam, Modified Lam, and target acceptance rates for (a) 1000 iterations, (b) 10,000 iterations, (c) 100,000 iterations, and (d) 1 million iterations. Figure 19: Minimize \(g_{100}(x)\): Self-Tuning Lam, Modified Lam, and target acceptance rates for (a) 1000 iterations, (b) 10,000 iterations, (c) 100,000 iterations, and (d) 1 million iterations. Figure 14:0 Minimize \(g_{1000}(x)\): Self-Tuning Lam, Modified Lam, and target acceptance rates for (a) 1000 iterations, (b) 10,000 iterations, (c) 100,000 iterations, and (d) 1 million iterations. ### An NP-Hard Problem: The Traveling Salesperson

We now examine experiments with the Traveling Salesperson Problem (TSP), a classic NP-Hard [1] problem. We generate test problem instances randomly, with city locations distributed uniformly. To explore the effects of cost function scale, consider two problem variations with cities arranged within a: (a) unit square, and (b) 100 by 100 square. We average the results over 100 runs, using 100 random instances, whereas the earlier benchmark problems define a single instance per problem. We use 100 instances to ensure that we do not rely on an instance that is either especially easy or especially hard. The cost function is the Euclidean distance of the tour of the cities. The mutation operator is the classic two-change [43], which removes two edges from the tour, replacing them with two edges that create a different valid tour. Tables 19 and 20 compare average tour costs of the two algorithms for the case where the 1000 cities are randomly distributed over the unit square, and over a 100 by 100 square, respectively. To statistically validate the results, we use T-tests with paired samples since both algorithms solve the same 100 random instances.