## 4 Working
### Analysis of Consumption Forecast Using 11 Different Optimizer Methods on Four Different Models
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \multicolumn{2}{c}{**Epochs**} & \multicolumn{4}{c}{**10 Epochs**} & \multicolumn{4}{c}{**50 Epochs**} & \multicolumn{4}{c}{**100 Epochs**} \\ \hline
**Model** & **Optimizer** & **test** & **test\_R2** & **test\_R2** & **test\_R2** & **test\_R2** & **test\_R2** & **test\_R2** & **test\_R2** \\ \hline \multirow{9}{*}{LSTM(50, activation = \"relu\" & **mean** & **mean** & **mean** & **mean** & **mean** & **mean** & **mean** & **mean** \\  & Adam & 571 & 448 & 0.9918 & 339 & 243 & 0.9972 & 289 & 208 & 0.9979 \\  & Nadam & 660 & 515 & 0.9885 & 429 & 322 & 0.9954 & 311 & 229 & 0.9976 \\  & RMSprop & 810 & 653 & 0.9838 & 421 & 289 & 0.9957 & 536 & 430 & 0.9931 \\  & RMSprop+ & 828 & 691 & 0.9819 & 500 & 377 & 0.9938 & 2602 & 2198 & 0.7719 \\  & adamax & 1282 & 985 & 0.9542 & 518 & 407 & 0.9933 & 345 & 256 & 0.9971 \\  & SGDTrue & 1727 & 1348 & 0.9095 & 769 & 586 & 0.9853 & 607 & 448 & 0.9909 \\  & SGD & 2219 & 1792 & 0.8494 & 1602 & 1284 & 0.9294 & 965 & 753 & 0.9761 \\  & adagrad & 4293 & 3464 & -0.1808 & 2279 & 1838 & 0.8518 & 2076 & 1678 & 0.8796 \\  & Frtl & 6401 & 5722 & -8.2777 & 5526 & 4603 & -8.2028 & 4754 & 3896 & -1.4893 \\  & adadelta & 9462 & 7775 & -2.8307 & 6749 & 5649 & -5.44 & 4214 & 3415 & -0.5040 \\  & SGDFalse & 6851 & 6365 & -0.2154 & 13,125 & 11,938 & -1 \(\times\) 1011 & 33,342 & 32,704 & -7.4604 \\ \hline \multirow{9}{*}{GRU(50, activation = \"relu\" & **mean** & 601 & 472 & 0.991 & 412 & 296 & 0.996 & 339 & 237 & 0.9972 \\  & Nadam & 643 & 488 & 0.9986 & 405 & 296 & 0.996 & 358 & 266 & 0.9969 \\  & adamax & 908 & 707 & 0.9787 & 481 & 366 & 0.9945 & 418 & 308 & 0.9959 \\  & RMSprop & 729 & 587 & 0.9868 & 475 & 347 & 0.9944 & 538 & 431 & 0.993 \\  & RMSprop+ & 826 & 625 & 0.9809 & 659 & 537 & 0.9887 & 690 & 564 & 0.9873 \\  & SGDTrue & 1328 & 1033 & 0.9534 & 962 & 721 & 0.9754 & 733 & 546 & 0.9862 \\  & SGD & 1725 & 1378 & 0.9207 & 1336 & 1082 & 0.9546 & 1173 & 942 & 0.965 \\  & adagrad & 3868 & 3063 & -0.1815 & 1935 & 1535 & 0.8824 & 1713 & 1386 & 0.9248 \\  & Frtl & 6174 & 5109 & -9.9445 & 4801 & 3922 & -1.7584 & 3567 & 2837 & 0.1859 \\  & adadelta & 7176 & 5724 & -4.2308 & 6497 & 5391 & -5.6841 & 4605 & 3768 & -1.4608 \\  & SGDFalse & 12,883 & 11,131 & -4 \(\times\) 1018 & 12,246 & 10,396 & -4 \(\times\) 1018 & 47,910 & 47,468 & -3 \(\times\) 1014 \\ \hline \multirow{9}{*}{SimpleRNN(50, activation = \"relu\" & **mean** & 583 & 444 & 0.9912 & 368 & 266 & 0.9986 & 397 & 300 & 0.9962 \\  & Adam & 765 & 592 & 0.985 & 425 & 307 & 0.9956 & 396 & 275 & 0.9962 \\  & RMSprop & 926 & 786 & 0.9783 & 486 & 357 & 0.9942 & 440 & 306 & 0.9953 \\  & adamax & 789 & 602 & 0.9844 & 567 & 441 & 0.9921 & 416 & 310 & 0.9958 \\  & SGDTrue & 930 & 716 & 0.9779 & 668 & 504 & 0.9889 & 615 & 457 & 0.9906 \\  & RMSprop+ & 832 & 615 & 0.9814 & 813 & 665 & 0.9832 & 1150 & 941 & 0.963 \\  & SGD & 1640 & 1300 & 0.9215 & 904 & 685 & 0.9792 & 744 & 555 & 0.9861 \\  & adagrad & 2566 & 2035 & 0.7921 & 1857 & 1488 & 0.9031 & 1550 & 1242 & 0.9355 \\  & Frtl & 5403 & 4469 & -3.4172 & 3174 & 2492 & 0.4742 & 1673 & 1350 & 0.9184 \\  & adadelta & 9941 & 8009 & -0.7133 & 5072 & 4243 & -0.5113 & 2686 & 2132 & 0.7625 \\  & SGDFalse & 14,965 & 13,796 & -4.1018 & 17,577 & 16,467 & -2.8 1014 & 19,404 & 18,287 & 0 \\ \hline \multicolumn{2}{c}{**Epochs**} & \multicolumn{4}{c}{**50 Epochs**} & \multicolumn{4}{c}{**70 Epochs**} \\ \hline
**Eochs** & Adam & 532 & 404 & 0.9929 & 313 & 224 & 0.9976 & 321 & 228 & 0.9975 \\  & Nadam & 563 & 438 & 0.992 & 345 & 256 & 0.9971 & 337 & 253 & 0.9972 \\  & adamax & 749 & 614 & 0.9855 & 458 & 342 & 0.9948 & 432 & 320 & 0.9954 \\  & RMSprop & 748 & 597 & 0.9857 & 564 & 458 & 0.9923 & 486 & 344 & 0.9943 \\  & RMSprop+ & 893 & 754 & 0.9796 & 656 & 519 & 0.989 & error error error error error error \\  & SGDTrue & 1381 & 1071 & 0.9479 & 832 & 636 & 0.9829 & 718 & 544 & 0.9873 \\  & SGD & 2036 & 1568 & 0.8873 & 1481 & 1165 & 0.9419 & 1357 & 1072 & 0.9515 \\  & adagrad & 359 Analysis of Consumption Forecast Using Four Different Models, Six Different Activation Functions, and 11 Different Optimization Methods  
In our study, four different models (LSTM, GRU, Simple RNN, and BiLSTM), six different activation functions (ReLU, tanh, ELU, LeakyReLU, sigmoid, softmax), and 11 different optimization methods were tested.