# Self-Tuning Lam Annealing: Learning Hyperparameters While Problem Solving
## 1 Introduction
However, in reality, these constants should really be treated as algorithm hyperparameters [34], whose values can affect either quality of final solution or convergence speed or both. For example, the Modified Lam is sensitive to the scale of cost differences between neighboring solutions in the search space. If the run length is sufficiently long, the Modified Lam will eventually adapt its behavior to the scale of the cost function, but it can be slow in doing so. Our new Self-Tuning Lam annealing schedule uses feedback from the early portion of the search to self-learn the algorithm hyperparameters, adjusting to the scale of cost differences between neighboring solutions. Using a variety of discrete and continuous optimization problems, we show that the effects of our approach is an adaptive annealing schedule that nearly instantaneously converges to the target behavior (i.e., [PERSON] and [PERSON]'s idealized acceptance rate trajectory) and maintains that target behavior throughout the run. Additionally, we have contributed our Java implementation of the Self-Tuning Lam to Chips-n-Salsa. Chips-n-Salsa is an existing open-source library of stochastic local search algorithms whose key features include self-adaptive as well as parallel search [35]. The source code for Chips-n-Salsa is hosted on GitHub [[https://github.com/cicirello/Chips-n-Salsa](https://github.com/cicirello/Chips-n-Salsa)]([https://github.com/cicirello/Chips-n-Salsa](https://github.com/cicirello/Chips-n-Salsa)) (accessed on 16 September 2021); and regular releases are deployed to the Maven Central repository [[https://search.maven.org/artifact/org.cicirello/chips-n-salsa](https://search.maven.org/artifact/org.cicirello/chips-n-salsa)]([https://search.maven.org/artifact/org.cicirello/chips-n-salsa](https://search.maven.org/artifact/org.cicirello/chips-n-salsa)) (accessed on 16 September 2021), from which practitioners can easily import the library using popular build tools. More details about the library, including API documentation, are available via the Chips-n-Salsa website [[https://chips-n-salsa.cicirello.org/](https://chips-n-salsa.cicirello.org/)]([https://chips-n-salsa.cicirello.org/](https://chips-n-salsa.cicirello.org/)) (accessed on 16 September 2021). By integrating our new annealing schedule into an existing library, we increase the potential impact of our research, enabling others to easily build upon our work. To enable reproducibility [36], we released all of the source code of our experiments, the raw data of our experiments, as well as the source code implemented to analyze the data and to generate all of the figures of this paper. This is all available on GitHub [[https://github.com/cicirello/self-tuning-lam-experiments](https://github.com/cicirello/self-tuning-lam-experiments)]([https://github.com/cicirello/self-tuning-lam-experiments](https://github.com/cicirello/self-tuning-lam-experiments)) (accessed on 13 October 2021). The remainder of this paper is organized as follows. We present our approach in Section 2, in which we begin by detailing the original Modified Lam annealing schedule,and then providing the algorithmic details of our new Self-Tuning Lam. Then, in Section 3, we empirically compare the behavior of the Self-Tuning Lam to the original Modified Lam on a variety of benchmarking problems, including both discrete optimization and continuous function optimization, as well as an NP-Hard problem. The aim of our experiments is not to demonstrate that our approach is superior to the original Modified Lam. After all, the _No Free Lunch Theorem_[37] indicates that any two optimization algorithms are equivalent if performance is measured over all possible problems. Rather, the aim of our experiments is to demonstrate that the Self-Tuning Lam, independent of run length and independent of cost function scale, consistently achieves and maintains the target behavior of [PERSON] and [PERSON]'s idealized acceptance rate trajectory.