# Self-Tuning Lam Annealing: Learning Hyperparameters While Problem Solving
## 2 Methods
### Self-Tuning Lam
#### 2.3.5 Putting It All Together
Figures 21 and 22 show the acceptance rates of the two algorithms relative to the target Lam rate. In the first case (Table 19 and Figure 21), when cities are distributed over the unit square, we find that for runs of length \(N=1000\) and \(N=10,000\), the Self-Tuning Lam finds better quality solutions at extremely statistically significant levels (\(p<10^{-52}\) and \(p=0.0001\), respectively); while at the longest run lengths, differences are not statistically significant. When the Self-Tuning Lam performs better it corresponds to when it better matches the target Lam rate as seen in Figure 21a,b. For the longest runs when there is no statistically significant difference in the cost function, we also find that there is no statistically significant difference in the MSE of the two algorithms relative to the target Lam rate, which can be seen in Figure 21c,d. When we distribute cities over a 100 by 100 square (Table 20 and Figure 22), we first find for very short runs (\(N=1000\) iterations) that the Modified Lam produces slightly better solutions on average at a statistically significant level (\(p=0.003\)). However, in this case there was no significant difference in the MSE of the acceptance rates of the two algorithms with respect to the target rate (Figure 22a). When we increase the run length to \(N=10,000\) iterations, the Self-Tuning Lam produces better quality solutions at an extremely statistically significant level (\(p<10^{-8}\)), in line with the statistically significant lower MSE of its acceptance rate from the target Lam rate (Figure 22b). As before, we did not find a significant difference in the cost function for the longest run lengths, where we also see approximately equivalent MSE for the acceptance rates (Figures 22c,d). Note that although the MSE difference is significant (\(p=0.01\)) for the \(N=100,000\) iteration runs (Figures 22c), visual inspection shows it is due very early in the run and that the Modified Lam otherwise well approximates the target rate. Figure 14:1 TSP with cities in unit square: Self-Tuning Lam, Modified Lam, and target acceptance rates for (**a**) 1000 iterations, (**b**) 10,000 iterations, (**c**) 100,000 iterations, and (**d**) 1 million iterations. Figure 14:2 TSP (cities in 100 by 100 square): Self-Tuning Lam, Modified Lam, and target acceptance rates for (**a**) 1000 iterations, (**b**) 10,000 iterations, (**c**) 100,000 iterations, and (**d**) 1 million iterations. ### Time Comparison of the Annealing Schedules

In this subsection, we examine whether the Self-Tuning Lam impacts the runtime of SA. We posit that the time required to tune the hyperparameters is negligible. Computing the tuning phase length \(M\) and \(\alpha\) involve a small number of basic arithmetic operations as seen earlier in Equations (8) and (10); and \(\text{AcceptRate}(0)\) is defined as one of two constants. All three of these are computed once. Tuning the initial temperature \(T_{0}\) and the rate of temperature change \(\beta\) are more involved. Each of these include some one-time computation (Equations (19) and (24)), which require computing a couple logarithms and an \(m\)-th root. However, they also depend on \(\Delta C\), which is computed across the \(M\) tuning iterations. However, during the \(M\) tuning iterations, the Self-Tuning Lam simply accepts all neighbors, without using the Boltzmann distribution for the decision. So calculating \(\Delta C\) is instead of calculating the Boltzmann distribution during those iterations, which saves SA from the need to compute any exponentiations during the tuning phase. These savings should offset the time needed to tune \(T_{0}\) and \(\beta\). To confirm that the Self-Tuning Lam's tuning process does not increase runtime compared to the Modified Lam, we isolate the annealing computation from SA, independent of any specific optimization problem, timing the operations that are strictly due to the annealing schedule. We did this as follows. For a given run length \(N\), we initialize the annealing schedule for that run length.