## 4 Working
### Analysis of Consumption Forecast Using 11 Different Optimizer Methods on Four Different Models  
Using four different deep learning models, LSTM, GRU, SimpleRNN, and BiLSTM, 11 different optimization methods were tested for each model. The results of this study were measured with RMSE, MAE, and R\({}^{2}\) metrics and are given in Table 4. In the table, the models are listed from top to bottom according to their success status. In these experiments, the epoch value was taken as 50. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Model** & **Optimizer** & **test\_mse** & **test\_mse** & **test\_R\({}^{2}\)** & **Model** & **Optimizer** & **test\_mse** & **test\_R\({}^{2}\)** \\ \hline \hline \multirow{9}{*}{LSTM(50, activation = \"relu\" & \multirow{2}{*}{SGD} & Adam & 339.33 & 242.54 & 0.9972 & \multirow{9}{*}{SMD} & \multirow{2}{*}{SMD} & \multirow{2}{*}{SMD} & \multirow{2}{*}{SMD} \\  & Nadam & 428.91 & 321.56 & 0.9954 & & Adam & 425.05 & 307.47 & 0.9966 \\  & RMSprop & 420.57 & 288.89 & 0.9957 & & RMSprop & 486.2 & 357.39 & 0.9942 \\  & RMSprop\({}^{*}\) & 500.28 & 376.88 & 0.9938 & & Adamax & 567.34 & 440.71 & 0.9921 \\  & adamax & 517.6 & 406.57 & 0.9935 & & & SGDTrue & 667.51 & 503.7 & 0.9889 \\  & SGDTrue & 768.55 & 586.43 & 0.9853 & & & RMSprop\({}^{*}\) & 812.6 & 664.71 & 0.9832 \\  & SGD & 1602.39 & 1284.1 & 0.9294 & & & SGD & 904.34 & 684.67 & 0.9792 \\  & adagrad & 2278.76 & 1838.13 & 0.8518 & & & & adagrad & 1857.1 & 1488.04 & 0.9031 \\  & Prtl & 5526.37 & 4603.25 & -5.2028 & & & Ftrl & 3173.83 & 2492.14 & 0.4742 \\  & adadelta & 6749.2 & 5649.35 & -5.44 & & & & addalelta & 5071.94 & 4243.33 & -0.5113 \\  & SGDFalse & 13,125.06 & 11,938.01 & -1.8 101 & & & SGDFalse & 1757.57 & 156,466.97 & -2.8 1012 \\ \hline \multirow{9}{*}{GRU(50, activation = \"relu\" & \multirow{9}{*}{SGD} & Adam & 313.04 & 223.66 & & & & \\  & Nadam & 404.5 & 296.4 & 0.996 & & & & \\  & adamax & 481.16 & 366.41 & 0.9945 & & & & \\  & RMSprop & 474.57 & 347.39 & & & & & \\  & RMSprop & 469.23 & 537.33 & & & & & \\  & SGDTrue & 961.56 & 721.47 & 0.9754 & & & & \\  & SGD & 1336.02 & 1082.17 & & & & & \\  & adagrad & 1935.03 & 1553.2 & & & & \\  & Prtl & 4800.64 & 3921.81 & -1.7884 & & & & \\  & adadelta & 6497.18 & 5390.8 & -5.6841 & & & & \\  & SGDFalse & 12,246.21 & 10,396.46 & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Four model-11 optimizer study results. The background of values with an R\({}^{2}\) value of 0.99 and above is painted green, and the background of values between 0.98 and 0.99 is painted pink. The background of values with an R\({}^{2}\) value of 0 or negative is painted brown. \begin{table}
\begin{tabular}{l l} \hline \hline
**Method** & **How to Use in the Program** \\ \hline RMSprop\({}^{*}\) & optimizer = tf.keras.optimizers.RMSprop (learning\_rate = 0.01, rho = 0.9) \\ SGDTrue & optimizer = tf.keras.optimizers.SGD (lr = 0.01, decay = \(1\times 10^{-5}\), momentum = 0.9, nesterov = True) \\ SGDFalse & optimizer = tf.keras.optimizers.SGD (lr = 0.001, decay = \(1\times 10^{-5}\), momentum = 1.0, nesterov = False) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Details of the methods used. Analysis of Consumption Forecast Using 10 Epochs, 50 Epochs, and 100 Epochs Using 11 Different Optimizer Methods on Four Different Models  
Using four different deep learning models, LSTM, GRU, SimpleRNN, and BiLSTM, 11 different optimization methods were tested with 10 epochs, 50 epochs, and 100 epochs. However, in the study conducted with the BiLSTM model, the computer gave an error before reaching the 100-epoch value. Therefore, experiments were conducted with 70 epochs instead of 100 epochs for the BiLSTM model. This study was measured with RMSE, MAE, and R\({}^{2}\) metrics, and the results are given in Table 5. The values in the table are listed from top to bottom according to their success status for each model. In all experiments in this section, ReLU was used as the activation function. Table 5 includes Table 4 for comparisons.