# Self-Tuning Lam Annealing: Learning Hyperparameters While Problem Solving
## 2 Methods
### Self-Tuning Lam
#### 2.3.4 Tuning the Rate of Temperature Change \(\beta\)  
During each iteration, the temperature is either cooled by multiplying by \(\beta\) or heated by dividing by \(\beta\) depending upon whether the acceptance rate is above or below the target Lam rate. The original Modified Lam sets \(\beta\) to a constant, independent of run length and cost function scale. However, the value of \(\beta\) in relation to run length and cost function scale may impact behavior. For example, a \(\beta\) that is too near to 1.0 and may prevent the annealing schedule from achieving the target Lam acceptance rate, but if \(\beta\) is too low it may cause large oscillations above and below the target Lam rate.  
In the Self-Tuning Lam, using the tuning phase data, we compute a value for \(\beta\) that will enable it to keep pace with the exponential drop that occurs in the target Lam rate. Under the assumption that each of the next \(M\) iterations after the tuning phase will cool the temperature, we compute a value for \(\beta\) that will drop the initial temperature \(T_{0}\) to the temperature \(T_{1}\) that leads to the target Lam rate at the end of that \(M\) iterations. Define \(R\) to be the target Lam rate after these \(M\) iterations as follows:  
\[R=\begin{cases}\lambda_{0.002}&\text{if }N\geq 10,000\\ \lambda_{0.02}&\text{if }N<10,000\end{cases} \tag{20}\]  
We can now follow a similar approach to Equation (16) to compute the necessary temperature \(T_{1}\) to achieve the above \(R\), setting up the following equation:  
\[\gamma+(1-\gamma)\cdot e^{-\Delta C/T_{1}}=R. \tag{21}\]We can then solve this for \(T_{1}\), making a similar assumption for the case when \(\gamma\geq R\) as we did in the previous section when \(\gamma\geq\text{AcceptRate}(0)\), to derive the following:  
\[T_{1}=\begin{cases}\frac{-\Delta C}{\ln\left(\frac{R-\gamma}{1-\gamma}\right)}& \text{ if }\gamma<R\\ \Delta C\cdot\zeta_{0.002}&\text{ if }\gamma\geq R\text{ and }N\geq 10,000\\ \Delta C\cdot\zeta_{0.02}&\text{ if }\gamma\geq R\text{ and }N<10,000,\end{cases} \tag{22}\]  
where \(\zeta_{0.002}\) and \(\zeta_{0.02}\) are as previously listed in Table 1.  
We can use the two temperatures, \(T_{0}\) and \(T_{1}\), to compute the \(\beta\) sufficient to cool \(T_{0}\) to \(T_{1}\) over \(M\) iterations of SA. Specifically, we must solve the following for \(\beta\):  
\[T_{0}\cdot\beta^{M}=T_{1}. \tag{23}\]  
Using our prior derivations of \(T_{0}\) and \(T_{1}\) (Equations (19) and (22)), we obtain:  
\[\beta=\begin{cases}\sqrt{\frac{\ln\left(\frac{\Delta\text{AcceptRate}(0)- \gamma}{1-\gamma}\right)}{\ln\left(\frac{\xi-\gamma}{1-\gamma}\right)}}&\text { if }\gamma<R\\ \sqrt{-\zeta_{0.002}\cdot\ln\left(\frac{\text{AcceptRate}(0)-\gamma}{1-\gamma }\right)}&\text{ if }R\leq\gamma<\text{AcceptRate}(0)\text{ and }N\geq 10,000\\ \sqrt{-\zeta_{0.002}\cdot\ln\left(\frac{\text{AcceptRate}(0)-\gamma}{1-\gamma }\right)}&\text{ if }R\leq\gamma<\text{AcceptRate}(0)\text{ and }N<10,000\\ \sqrt{\zeta_{0.002}/\zeta_{0.001}}&\text{ if }\gamma\geq\text{AcceptRate}(0)\text{ and }N\geq 10,000\\ \sqrt{\zeta_{0.02}/\zeta_{0.01}}&\text{ if }\gamma\geq\text{AcceptRate}(0)\text{ and }N<10,000.\end{cases} \tag{24}\]