## 2 Literature Review
However, researchers were not satisfied with this performance and tried to increase the performance by playing with the LSTM model, forecast data, training method, hyper parameters and other variables. In our work, we use a total of four different architectures, including SimpleRNN and LSTM, as well as GRU and BiLSTM. We were not satisfied with the architectures we chose, but attempted to see the effects of these variables on the system and increase the system performance by playing with hyper parameters and other variables. Briefly, we examined the studies conducted to increase performance over the LSTM architecture and use different architectures:  
[PERSON] et al. used a two-stage attention-based LSTM network for short-term regional load probability forecasting, taking into account feature correlation and temporal dependencies. In the first stage, a feature attention-based encoder was built to calculate the correlation of input features with the electric charge at each time step. The most relevant input features were adaptively selected. In the second phase, a temporal attention-based decoder was developed to investigate time dependencies. Then, an LSTM model integrated these attention results, and probabilistic predictions could be made using a pinball loss function [34]. [PERSON] et al. proposed an evolutionary dual attention-based long short-term memory model and introduced binary features using feature combination. This study compared the prediction performance of eight prediction methods. The results show that an attention mechanism can improve the efficiency of the LSTM algorithm when the model uses input time series data [25]. [PERSON] et al. proposed a new approach based on an LSTM network to predict periodic energy consumption. First, hidden features were extracted by an autocorrelation plot among real industrial data. Correlation analysis and mechanism analysis contributed to finding appropriate secondary variables as model inputs. In addition, the time variable was complemented to fully capture periodicity. Experimental results on a specific cooling system show that the proposed method has higher prediction performance compared with back propagation neural network (BPNN), autoregressive and moving average (ARMA), and autoregressive fractionally integrated moving average (ARFIMA) [35]. A different model is gate recurrent unit networks (GRU). In some studies, GRU is considered to provide more accuracy, improve prediction, produce better performance results, and be faster than LSTM [36; 37; 38; 39]. Another method to be evaluated is bidirectional LSTM (Bi-LSTM). Bi-LSTMs provide additional training by moving the input data twice (i.e., 1 left to right and 2 right to left). [PERSON] et al. show that Bi-LSTM-based models provide better predictions than normal LSTM-based models. More specifically, Bi-LSTM models provide better forecasts than ARIMA and LSTM models. It has also been observed that Bi-LSTM models reach equilibrium much slower than LSTM-based models [40]. In another short-term energy consumption study, it was seen that the BiLSTM method is more successful than the SVR, CNN, and GRU methods [41]. There are also some studies on short-term electricity load consumption forecasting that use four of the LSTM, GRU, Simple RNN, and Bi-LSTM models. However, it can be seen that similar studies are mostly conducted in the fields of pandemics, production forecasting, and stock markets. In [42], a study was conducted on deaths and recoveries in ten major countries affected by COVID-19, and the performance of the models was measured using root mean squared error (RMSE), mean absolute error (MAE), and R\({}^{2}\)_score indices. In most cases, the Bi-LSTM model outperforms in terms of validated indices. The models ranked from best to worst performance in all scenarios were Bi-LSTM, LSTM, GRU, SVR, and ARIMA. In [43], a price forecasting system was developed by comparative analysis between LSTM, GRU, and Bi-LSTM models.