# Predicting the Gap in the Day-Ahead and Real-Time Market Prices Leveraging Exogenous Weather Data
###### Abstract
In [14], a Random Forest regression is leveraged to predict DAM prices. The proposed approach outperformed the ARIMA model. However, this paper does not consider the impacts of exogenous features such as temperature and solar irradiance to predict prices. Moreover, the price gap between the DAM and RTM is not predicted in the literature. Even though statistical models perform well at identifying patterns and indicators that will influence the price of electricity, they struggle to predict prices accurately in the presence of spikes, which is particularly important for predicting the gap price across a two-settlement market [15]. The electricity market will be cleared based on the net demand, which in turn depends on many characteristics such as weather, temperature, wind speed, and precipitation. Thus, the LMP tends to fluctuate over an operational horizon. In [16], the authors compared the ARIMA model with Artificial Neural Networks (ANNs) to forecast an electricity price. To handle the complexity of the electricity market, ANNs are used in [17]. The increase in the number of computation layers increases the feature abstraction capability of the networks, which makes them better at identifying non-linear trends [18]. An ensemble of CNN-LSTM and an ARMA model is utilized for financial time series data in [19]. In [20], a Temporal Convolutional Neural Network (TCNN) model is utilized for the analysis of financial time series data, specifically focusing on applications in Forex markets. This approach is contrasted with Recurrent Neural Networks and other deep learning models, as well as some of the top-performing Machine Learning methods, to demonstrate its effectiveness in handling financial data. The ARMA model captures the linear dependencies of features and target variables, while CNN-LSTM models the nonlinear spatial connections in data features between adjacent time intervals and also accounts for long-term time-based patterns in the data. The ensemble of CNN-LSTM and the ARMA model achieved a 0.8837 MAE score for The European Union Emission Trading System (EU ETS) dataset. In [21], the LSTM network and a variation of the deep Recurrent Neural Network (RNN) are used to forecast electricity load, and the outputs of the models are compared to those of statistical models. The electricity consumption of the past 10 days is used to predict the electricity consumption of the next day. The LSTM-based network significantly outperformed the Seasonal-ARIMA and Support Vector Regression (SVR)models. A similar model is used in [22] to predict electricity load, but in this case, in addition to the historical load data, weather datasets are also utilized. However, no change in the model performance is observed when weather-related features are removed and only the time lags are used as inputs. In [23], the LSTM network is used to predict the next 24 h of electricity prices for Australian and Singapore markets. The mean absolute percentage error (MAPE) was used to evaluate the model, and up to a 47.3% improvement was observed compared to a multi-layer ANN. According to [24], the prediction of real-time LMP is even more challenging, and most of the approaches adopted from previous studies generate an MAPE of around 10-20%. In [25], a homogeneous Markov chain representation of the RTM LMP is used to predict the RTM LMP for the prediction horizon of 6-8 h. Future prices are computed based on state transition matrices using the Monte Carlo method. Although the mean average error (MAE) metric of the model was 11.75 USD/MWh, it has a huge computation burden. In [26], the authors proposed a deep LSTM (D-LSTM) network to estimate short- and medium-term demand as well as the LMP. The D-LSTM network turned out to have a flat trend without the validation set. However, once the network was tuned, it outperformed the nonlinear auto-regressive network with exogenous variables (NARX) and Extreme Learning Machine (ELM) models in terms of accuracy.