# A Novel NODE Approach Combined with LSTM for Short-Term Electricity Load Forecasting
## 2 Related Work
### Deep Learning Method  
Deep learning methods, such as LSTM and its variants, have been used widely in electricity load forecasting because LSTM can capture long-term temporal dependency. Based on LSTM, bidirectional LSTM, stacked LSTM, and CNN-LSTM also have been proven effective. [PERSON] et al. [17] compared the accuracy of using stacked unidirectional and BiLSTM networks to handle electricity load forecasting in Switzerland. Their results indicated that a single-layer BiLSTM model performs well in a reasonable time compared with the original LSTM model, which makes it significant in exploring other LSTM-based forecasting approaches. [PERSON] et al. [18] proposed a multiscale CNN-LSTM hybrid neural network. The convolutional neural network (CNN) was used to extract the shallower and deeper feature from four different scales. Feature vectors of different scales were mixed as the input of LSTM, and the output of LSTM was used as the result of short-term load forecasting. The experiment results indicated that CNN-LSTM performs better than the standard LSTM model, SVM model, random forest (RF) model, and ARIMA model in the data collected from a city in Liaoning Province, China. Therefore, hybrid CNN-LSTM structures are adopted in our paper, and a comparison of forecasting accuracy between CNN-LSTM models and the improved models with NODE are given. [PERSON] et al. [19] introduced a multilayered self-normalizing gated recurrent unit (GRU) network to process STLF. They adopted scaled exponential linear units (SELU) to squash the hidden states to calculate the output of the model, which can solve the exploding and vanishing gradient problem in a stacked GRU network. The self-normalizing GRU network shows higher accuracy in STLF mission when compared with other RNN models. However, the RNN-based model GRU and LSTM require discretizing observation and emission intervals, while NODE can produce a continuous latent trajectory. Therefore, NODE can be used to solve the problem of the RNN-based model. As shown above, the LSTM-based model shows high accuracy in electricity load forecasting. Different LSTM-based models adopt different methods to capture long-term temporal patterns. However, a critical problem of recurrent neural networks is that LSTM has a discrete observation interval. Due to the discrete observation interval, LSTM ignores potential information hidden in missing observation space, which cause the loss of complex nonperiodic patterns. In addition, it is hard for the LSTM-based model to adapt acute changes of time series, because discrete observation has hysteresis. In contrast, NODE can learn differential equation relations of sequences to continuously describe the relations of features, i.e., learn the law of internal evolution equation. In this way, NODE builds continuous hidden layers to model time series. Therefore, NODE can capture continuous temporal information and learn rich dynamics of series, which addresses the essential problem of the LSTM-based model. Except for LSTM and its variants, other neural networks also used in the STLF field. [PERSON] et al. [20] proposed a new task-based load forecasting model LfEdNet for load forecasting, which includes two high-level layers, Lf layer and Ed layer. The Lf layer is an artificial neural network and the Ed layer is a stochastic economic dispatch (SED) model. This framework combines neural network and real-life SED to find the relationship between SED cost and load forecasting accuracy and find a method that leads to lower SED cost at the expense of slight reduction in forecasting accuracy. In our previous publications in the electricity load forecasting field, we proposed several methods to improve prediction accuracy. In [21], a novel Monte-Carlo-based neural network model was proposed. In [22], we proposed a novel deep learning model based on the multiple LSTM neural networks.