## 2 Materials and Methods
### Simulated Annealing as a Metaheuristic Used to Enhance SVM Model Accuracy  
SA is one of the oldest metaheuristics with an explicit strategy to avoid the problem of the local minima [26]. The subjacent idea is to allow the acceptance of some solutions that can eventually deteriorate the objective function, in order to keep away from local minima. The current solution is replaced by a new solution in a random neighborhood, and the extent of the search is based on a probability distribution with a scale that is proportional to the \"temperature\". An annealing schedule is used to decrease the temperature as the algorithm proceeds. As the temperature decreases, the extent of its search is reduced to enable an approximation to a minimum. However, it is difficult to conclude whether this is the global minimum or an undesired local minimum. Thus, a reannealing process is often required. This reannealing process can be viewed as a supplementary strategy to a global search because the temperature in each dimension (for each variable) is raised, depending on temperature sensitivity, giving a wider opportunity to accept solutions that are distant from the aforementioned ones.  
For this study, the SA approach to improve SVM models had some variations compared to the SVM models. Firstly, in order to be able to properly normalize the training and test subsets, only the hours in which heating and cooling systems were actually working were considered for the predictions; secondly, instead of a single combination of SVM parameters for all hours being predicted, each individual hour had its own combination, in order to allow SVM models to adapt to the time of the day at which prediction were made; and thirdly, also associated with the previous assumption, each hour being predicted is normalized using the minimum and maximum values registered in the training subset (unlike SVM, based on overall maximum and minimum).  
For the SA technique, thirty cycles (iterations) were used for each hour. In each cycle, fifteen trials, i.e., fifteen random initial combinations of SVM parameters were tested, and at the end of each cycle, the temperature would drop to narrow the range of options of each SVM parameter. The cost parameter C was tested in a range between 0 and 700, epsilon was tested between 0.0001 and 3, and gamma was tested between 0.0001 and 10. Other Simulated Annealing parameters were set as:  
* Probability to accept a worst solution in the beginning (p\({}_{1}\)) = 0.5;
* Probability to accept a worst solution in the last cycle (p\({}_{30}\)) = 0.001;
* Initial temperature (t\({}_{1}\)) = \(-1.0/\log\) (p\({}_{1}\));
* Final temperature (t\({}_{30}\)) = \(-1.0/\log\) (p\({}_{30}\));
* Temperature for a new cycle (t\({}_{n}\)) = (t\({}_{30}/\)t\({}_{1}\))\({}^{[(1/(n-1)]}\).t\({}_{n-1}\), n being the number of the cycle.  
The assumed objective function was based on the Root Mean Square Error (described in Section 2.5) verified in the validation subset (20% of the data, similar to the approach followed for ANN). An example of the optimization model being evaluated during thirty iterations is shown in Figure 4, where the objective function being minimized can be evaluated, while the acceptance of worse solutions and the exploitation of different parameters' combination can also be appreciated.  
Figure 3 shows the flowchart followed in this study when using SA to forecast hourly heating and cooling energy demand.  
Figure 4: SVM parameters’ optimization during the SA process.  
Figure 3: Simulated annealing (SA) flowchart depicting approach used to enhance support vector machines (SVM) models.