## 2 Materials and Methods
### Module Development
#### 2.2.3 The Learning Algorithm  
Optimization is carried out by retraining the model with different combinations of hyperparameters (optimizer, batch size, number of layers, number of neurons per layer) and evaluating its performance (energy consumption and prediction accuracy). The training steps of the model are listed in the Algorithm 1. Optimizers  
Optimizers play a crucial role in improving performance and training speed. Selecting an appropriate optimizer is a non-trivial task. The choice of optimizer depends on the specific problem and the tradeoffs between computation time, convergence speed, and accuracy. In our study, we opted for the three most commonly used optimizers in sequential data problems; Adam (Adaptive Moment Estimation), RMSprop (Root Mean Square Propagation), and Nadam (Nesterov-accelerated Adaptive Moment Estimation). Adam is a first-order gradient-based optimization method for stochastic objective functions based on lower-order adaptive moment estimates. The advantages of using Adam include ease of implementation, efficient computation, and low memory requirements. RMSprop divides the weight learning rate by a moving average of magnitude. The magnitude sizes of the gradients vary by weight and evolve over time, hence the difficulty of selecting a single global learning rate. This aspect of issue is handled by RMSProp by maintaining a moving average of the squared gradient and changing the updates of the weights by this magnitude. Nadam is an extension of the Adam algorithm that incorporates the Nesterov momentum and can lead to better performance of the optimization algorithm. Dropout  
The dropout layer was added to improve the predictive performance of the model and prevent overfitting [32]. In the dropout layer, the model randomly deactivates the neurons in a layer with a certain probability. When the dropout value is added to a layer, the neural network will ignore selected neurons during training, and the training time will be faster. In general, the dropout rate can range from 0.1 to 0.5. However, it is important to note that choosing the optimal dropout rate is typically a trial-and-error process, and can depend on factors such as the complexity of the model, the amount of training data, and the task being performed. In our case, we chose rates of 0.1 and 0.2 based on the complexity of the model and our database. We considered various factors to find the best dropout value for our model, including the effects of regularization on learning and the overall performance of the model. One of the main hyperparameters to adjust before starting the training process is the batch size. The batch size is a parameter that specifies the number of training samples that will be used during training to perform an update of the network parameters [33]. Todetermine the impact of batch size on model consumption, we set the batch size between 32 and 512 with a step of 32, because our model begins to stabilize from 512. Hidden layers consist of recurrent cells whose states are affected by both past states and current inputs with feedback connections. Recurrent layers can be organized in various architectures to form different LSTMs. Different cells and internal connections, therefore, allow the LSTMs to have different capabilities. To investigate the impact of the hidden layers on the power consumption and performance of the model, we considered their number ranging from 1 to 4, because beyond 5 layers, the model began to exhibit overfitting due to the increased complexity of the network compared to our dataset. The units indicate how many neurons are in a specific layer. We carefully considered the number of neurons per layer for our model architecture, and ultimately decided on values ranging from 8 to 256. Our decision was based on several factors, including the size and complexity of our dataset, as well as the nature of the task at hand. We found that beyond 256 neurons, the model became increasingly complex and prone to overfitting.