# Sales Forecasting with LSTM, Custom Loss Function, and Hyperparameter Optimization: A Case Study
## 3 Results  
For the development of this paper and the experimentation realized, we used a MacBook Pro M1 with 512 GB SSD, 16 GB RAM, and 10 CPUs. For the programming of the proposal, we used Python 3.11 and the following modules: Tensorflow 2.15.0, Keras 2.15.0, Numpy 2.1.1, Pandas 2.2.0, and Matplotlib 3.8.2.  
We performed the hyperparameter optimization of the LSTM network using a genetic algorithm based on [26]. Genetic algorithms were proposed by [PERSON] (1998) [27] and are one of the most popular evolutionary algorithms. Diverse authors have used the genetic algorithms [28; 29; 30]. These algorithms have several advantages, one of the most important being that genetic algorithms start with randomly selected initial values, making their implementation easier as they do not require much effort to choose good initial values [31]. According to [PERSON] and [PERSON] (2004) [32], a genetic algorithm is an optimization technique based on genetics and natural selection principles. A genetic algorithm allows a population of many individuals to evolve under specific selection rules toward a state that maximizes their fitness value.  
In our genetic algorithm implementation, the population consisted of a matrix of \(n\) solutions, each representing a configuration of the nine hyperparameters for the LSTM network. The first five chromosome values corresponded to discrete values and the rest to continuous values. Table 4 shows the structure of a chromosome.  
Since a genetic algorithm measures population fitness through the objective function [33], to measure the performance of a given hyperparameter set, we decided to use our custom loss function cost in the results obtained from the trained LSTM network.  
For the implementation of the genetic algorithm, we considered the following operators found in [26; 34]:  
* Selection: Binary Tournament.
* Uniform. In discrete parameters;
- SBX. In continuous parameters.
* Uniform. In discrete parameters;
- Polynomial. In continuous parameters.  
Algorithm 2 presents the genetic algorithm designed and implemented in this paper.  
```
0:\(Population\_size\), \(Problem\_size\), \(P_{\text{crossover}}\), \(P_{\text{mutation}}\)
0:\(S_{best}\)
1:\(Population\leftarrow\text{InitializePopulation}(Population\_size, Problem\_size)\)
2:EvalatePopulation(\(Population\))
3:\(S_{best}\leftarrow\text{GetBestSolution}(Population)\)
4:while not StopCondition() do
5:\(Parents\leftarrow\text{SelectParents}(Population,Population\_size)\)
6:\(Children\leftarrow\emptyset\)
7:for all\(Parent_{1}\), \(Parent_{2}\in Parents\)do
8:\(Child_{1},Child_{2}\leftarrow\text{Crossover}(Parent_{1}\), \(Parent_{2}\), \(P_{\text{crossover}})\)
9:\(Children\leftarrow\text{Mutate}(Child_{1}\), \(P_{\text{mutation}})\)
10:\(Children\leftarrow\text{Mutate}(Child_{2}\), \(P_{\text{mutation}})\)
11:endfor
12: EvaluatePopulation(\(Children\))
13:\(S_{best}\leftarrow\text{GetBestSolution}(Children)\)
14:\(Population\leftarrow\text{Replace}(Population,Children)\)
15:endwhile
16:return\(S_{best}\)
```  
**Algorithm 2** Pseudocode for the Genetic Algorithm  
In order to perform an adequate exploration of the solution space with the genetic algorithm, we made a sensitivity analysis of the algorithm's parameters to determine the configuration that provided the best results for optimizing the LSTM hyperparameters with the algorithm. Table 5 shows the range of values that we considered for the sensitivity analysis.  
\begin{table}
\begin{tabular}{c c c} \hline
**Hyperparameter** & **Description** & **Type** \\ \hline
0 & LSTM 1 Units & Discrete \\
1 & LSTM 2 Units & Discrete \\
2 & LSTM 3 Units & Discrete \\
3 & Epochs & Discrete \\
4 & Batch size & Discrete \\
5 & Learning Rate & Continuous \\
6 & Dropout 1 & Continuous \\
7 & Dropout 2 & Continuous \\
8 & Dropout 3 & Continuous \\ \hline \end{tabular}
\end{table}
Table 4: Structure of a chromosome.