# Short-Term Demand Forecasting Method in Power Markets Based on the KSVM–TCN–GBRT

Authors: Guang Yang; Songhuai Du; Qingling Duan; Juan Su
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

With the consumption of new energy and the variability of user activity, accurate and fast demand forecasting plays a crucial role in modern power markets. This paper considers the correlation between temperature, wind speed, and real-time electricity demand and proposes a novel method for forecasting short-term demand in the power market. Kernel Support Vector Machine is first used to classify real-time demand in combination with temperature and wind speed, and then the temporal convolutional network (TCN) is used to extract the temporal relationships and implied information of day-ahead demand. Finally, the Gradient Boosting Regression Tree is used to forecast daily and weekly real-time demand based on electrical, meteorological, and data characteristics. The validity of the method was verified using a dataset from the ISO-NE (New England Electricity Market). Comparative experiments with existing methods showed that the method could provide more accurate demand forecasting results.


---

## Body

## 1. Introduction

The inherent volatility and uncertainty of renewable energy sources, such as wind power and photovoltaics, may lead to large deviations between the bid output and the actual output when renewable energy sources participate in the market. Also, customers change their electricity load when they receive a price change or incentive signal from the supply side, taking into account their own production or consumption. Hence, timely and accurate information on the demand changes has imposed higher requirements on the accuracy of demand forecasting [1].In current electricity forecasting, it is necessary to fully integrate economic, meteorological, and other multidimensional information; use advanced data-driven processing means; and deeply analyze the change patterns and laws of renewable energy and demand [2]. This creates the need to study demand response characteristics under multidimensional variables in the electricity market environment to improve forecasting accuracy [3]. For power system operation, short-term demand forecasts can predict demands from 1 hour to 168 hours in advance [4]. To obtain accurate forecasting results, different forecasting models such as regression, statistical, and state-space methods are used. Expert system-based, evolutionary process, fuzzy systems, and artificial intelligence algorithms have also been introduced. Wang et al. [5] proposed a method that decomposes the demand into the trend series and the fluctuating series and then builds the corresponding forecasting models separately. To improve the internal clustering performance, Dongyeon et al. [6] proposed the logistic mixed vector autoregression model, which combines clustering and prediction into one model through an expectation-maximization algorithm. Dong et al. [7] proposed an indicator variable to capture the abnormal information on special days, such as national holidays, proximity days, and bridging days. In Yin et al. [8], the multispatiotemporal convolutional network was applied to short-term demand prediction tasks, which reduced the noise error of demand data and enhanced the time-series characteristics of demand data. In [9], the demand was decomposed into different frequency components using an integrated empirical modal decomposition algorithm; the low-frequency components were then predicted using multivariate linear regression, the high-frequency components were predicted using a long short-term memory neural network (LSTM), and the components were finally combined to obtain the demand forecast. Han et al. [10] presented a short-term individual residential demand forecasting model based on a combination of deep learning and k-means clustering, which is capable of effectively extracting the similarity of residential demand and performing residential demand forecasting accurately at the individual level. It first makes full use of k-means clustering to extract the similarity among residential demand and then employs deep learning to extract complicated patterns of residential demand. Lv et al. [11] designed a LightGBM-optimized LSTM to realize short-term stock prices. To improve the demand prediction accuracy in the case of single sample data, Mei et al. [12] proposed a model based on multiscale temporal features for LSTM. First, wavelet decomposition decomposes historical data into stable components, trend demand, and periodic series such as the response peak-valley magnitude and duration, highlighting different time-scale features. Second, LSTM is used to achieve further extraction of time-series characteristics and data fitting. Finally, the model directly outputs predicted values for multiple moments. M. Hadi Amini et al. [13] proposed an autoregressive integrated moving average method for forecasting conventional electrical loads and electric vehicle parking charging demand. Kianoosh et al. [14] proposed to model the nonseasonal and seasonal cycles of load data using regression (AR) and moving average (MA) components, which have been used to forecast electricity demand at different time scales. Moreover, Zhang et al. [15] used Singular Spectrum Analysis (SSA) to preprocess the data and then used a support vector machine (SVM) optimized by the cuckoo search (CS) algorithm to model the resulting sequence with different prediction strategies. Salah et al. [16] used wrapper and embedding feature selection methods to select the optimal features and a genetic algorithm (GA) to find the optimal time lag and number of layers to optimize the predictive performance of the LSTM model, which was used to construct a short-to-medium-term cumulative load forecasting model. LU et al. [17] proposed a hybrid model short-term load forecasting method based on the convolutional neural network (CNN) and LSTM network. The CNN was first used to extract the feature vector, and the feature vector was constructed in a time-series sequence and used as the input data for the LSTM network. In [18], a convolutional long- and short-term memory network (Conv-LSTM) was proposed for electrical load data, and it achieved better accuracy than the traditional prediction algorithm based on linear regression. CHEN et al. [19] derived the respective predictions based on LSTM and LightGBM training. The optimal weighting combination method was used to determine the weighting coefficients and derive the prediction values of the combined model to improve the accuracy of load prediction. In [20], predictions were extrapolated by calculating correlations between potential variables and outputs and predicting the future consumption of high performance. In [21], a hybrid method combining empirical mode decomposition (EMD), particle swarm optimization (PSO), and a fuzzy inference system based on the adaptive network (ANFISs) for short-term load prediction of microgrids was proposed. In [22], sample entropy was used to identify the nonlinearity and uncertainty of the original time series, and the optimal mode of the original series and the optimal input form of the model were determined by the feature selection method. Finally, the least square SVM adjusted by the multiobjective sines and cosines optimization algorithm was used to predict the power demand sequence. In [23], the parameters of the LSTM were first optimized using the Sparrow Search Algorithm (SSA); then, the dataset was preprocessed, and finally, the processed data were used for residential load training and prediction. Li et al. [24] combined LSTM with quantile regression to generate multiple quantile results and introduced a combinatorial layer that considers the constraint relationships between quantile prediction values to ensure that the quantile prediction values are reasonable.The TCN was proposed in 2018 to offer greater performance advantages over recurrent neural networks (RNNs) in temporal data processing tasks [25]. Following the motivation above, we propose a novel method based on Kernel Support Vector Machine (KSVM)–TCN–Gradient Boosting Regression Tree (GBRT) for improving the short-term demand forecasting accuracy of power markets. The contributions of this paper are as follows:(1)
The KSVM is used to extract and train classification features for real-time electricity demand on historical data by the features of temperature and wind speed, and a numerical calculation method is used to automatically select the parameters of the KSVM to derive a classification sequence of real-time electricity demand for future days as the feature sequence.(2)
A multivariable TCN method is used to capture the fluctuation trend of day-ahead demand in the day-ahead market to predict the real-time demand series in the real-time market.(3)
The TCN–GBRT method integrating time-domain processing, integrated learning, and parallel feature processing is proposed. TCN is able to extract features and temporal relationships owing to its residual network and convolutional structure, avoiding gradient disappearance and gradient explosion in deep learning. GBRT can combine multiple weak classifiers into a single strong classifier that takes the best of all the weak classifiers and achieves optimal performance.


## 2. Materials and Methods

### 2.1. KSVM Model

In this paper, a Gaussian Kernel SVM is used in data feature extraction to solve the classification problem of the presence of nonlinear separability. That is, given a training sample setD=x1,y1,.x2,y2,…,xm,ym, yi=−1+1, the basic type of Kernel SVM is defined as in the following equation:(1)minw,b12ω2,s.t.yiωTϕxi+b≥1,i=1,2,…,m,γ=ω22,where γ is the sum of the distances from the support vector to the hyperplane, called the “margin.” Finding the maximizing interval is equivalent to finding the minimum ||x||2. ω = (ω1; ω2; …; ωd) is the normal vector of the hyperplane, b is the displacement term, and ϕx denotes the feature vector after mapping x.The dual problem is defined in the following equation:(2)max∂∑i=1m∂i−12∑i=1m∑j=1m∂i∂jyiyjϕxiTϕxj,s.t.∑i=1m∂iyi=0,∂i≥0,i=1,2,…,m.Here,κ.,. is the kernel function given in(3)κxi,xj=ϕxiTϕxj.The kernel function maps the data from the original space into a high-dimensional Hilbert space, where a more efficient classification hyperplane exists than in the original space.Suppose thatxjij=1⋯Ni⊂Rd is the set of training samples in class i, where Ni represents the training samples in class ii=1,2,…,L. The Gaussian Radial Basis Function (RBF) kernel is defined as in the following equation:(4)κx,z,σ=exp−x−z22σ2,σ∈0,∞,where x, z∈Rd, and σ∈R−0 are the corresponding parameters.


### 2.2. TCN Model

TCN is a one-dimensional full convolution network, which combines the structure of causal convolution, extended convolution, and the residual network. Causal convolution means that the output time is convolved only with the elements of the previous layer of time, which ensures that there will be no information leakage in the future. Dilated convolution is designed to capture a sufficiently long history of information, and the depth of the network model increases dramatically.Define the one-dimensional sequence inputXx1,x2…xs, where F:⋅0,1,…,k−1 is the function to the dilatative convolution, and the TCN convolution operation is defined as in the following equation:(5)Fs=XCondfs=∑i=0k−1fixs−d·i,where Con is the convolution operation; d is the dilatative convolution parameter; s is the current number of sequences; k is Con Size; s-d·i is past directions; and a convolution with parameter d=1,2,4 and size k=3 is shown in Figure 1.Figure 1
A dilated causal convolution with dilation factorsd=1,2,4 and filter size k=3.As shown in Figure2, the mezzanine mapping is within the residual connection, as shown in equation (6). The residual network fits several nonlinear layers between the input and output data. The more the features that are extracted, the closer the residual Fx is to 0. Hence, when the network reaches an optimal structure, Fx is pushed to 0 as the network layers deepen, leaving only the identity mapping x. This overcomes the problem of TCN degradation due to increasing network layers. Using the residuals, each order of derivative plus the constant term 1 is as in equations (6) and (7). The error can still be effectively backpropagated at this point, even if the derivatives ∂f/∂x are small.(6)o=Fx+x,(7)∂h∂x=∂f+x∂x=1+∂f∂x,where F(x) is a residual function; x is a constant.Figure 2
Residual block.


### 2.3. GBRT Model

The GBRT algorithm is able to compensate for the tendency of the cart algorithm to overfit small sample data or produce instability and low prediction accuracy. The algorithm is an iterative decision tree algorithm, which consists of three parts: cart, Gradient Boosting algorithm, and reduction idea.The basic idea of the algorithm is to use the Boosting method to iterate multiple weak learners with low prediction accuracy to form a strong learner with high prediction accuracy, that is, to reduce the residuals of the previous model by learning again so that the next generated model has a smaller error. The gradient iteration makes the combined model continuously improved, which is a kind of decision tree integrated learning algorithm designed to improve the model learning rate and prevent overfitting. That is, it does not fully trust each residual tree and uses gradual approximation to learn through multiple tree residuals.For continuous data types, the loss function is the classical loss function in Boosting, that is, the sum of squared errors, which is calculated as shown in (4). After M-th iterations, the prediction is shown in equations (8) and (9).(8)Ly,fx=∑i=1nyi−fxi2,(9)fx=∑i=0nfix,where x is the input variables, y is the output variables, and i is the iteration count.The key to improving the prediction accuracy of the GBRT model is the calculation of the residuals, and this paper uses the method proposed by Friedman [26] to calculate the residuals based on the negative gradient of the loss function.In this paper, the negative gradient of the loss function is used as an approximation to the residuals in the boosted tree algorithm. Hence, thei-th sample of the m-th round gim is calculated as shown in the following equation:(10)gim=−∂Lyi,fxi∂fxifxi=fm−1xi.fmx can be calculated from βm and hm, where βm denotes the optimal step for each iteration and is calculated as shown in (13). Hmx is the decision tree created in the m-th iteration. The prediction is shown in (11) and (12).(11)βm=argminβ∑i=1nLyi,fm−1x+βmhmx,(12)fmx=fm−1x+βmhmx.


### 2.4. Short-Term Demand Forecasting Method Based on KSVM–TCN–GBRT

This paper takes into account day-ahead demand, real-time demand, date features, and meteorological features. An RBF kernel-SVM method is used to get the relationship between temperature, wind speed, and real-time power demand.The TCN–GBRT method is also proposed to forecast the next day’s real-time demand and the next week’s real-time demand. Processing datasets are shown in Table1.Table 1
Processing datasets.
Type of dataInput VARIABLE tUnitsProcessing methodOutput variabletElectrical featurest − 24/168 day-ahead cleared demandMWhttps://www.iso-ne.com/ [27]t + 24/168 real-time demandt − 24/168 day-ahead locational marginal priceMW$/MWht + 24/168 day-ahead cleared demandMWFeature extraction based on TCNMeteorological featuret − 24/168 temperature°Fhttps://www.wunderground.com/ [28]t − 24/168 dew point°Ft − 24/168 humidity°Ft − 24/168 wind speedMPHt − 24/168 conditionOne hott + 24/168 temperature°Ft + 24/168 dew point°Ft + 24/168 humidity°Ft + 24/168 wind speedMPHt + 24/168 conditionOne hotDate featurest − 24/168 WeekDayDatet − 24/168 Montht − 24/168 holidayt + 24/168 WeekDayt + 24/168 Montht + 24/168 holidayClassification featurest + 24/168 classification IDFeature extraction based on KSVMThe KSVM–TCN–GBRT method processing steps for power markets are shown in Figure3. The short-term demand forecasting framework based on the KSVM–TCN–GBRT method is shown in Figure 4.Figure 3
The KSVM–TCN–GBRT model processing steps for Power Markets.Figure 4
The short-term demand forecasting framework based on KSVM–TCN–GBRT model.In this paper, parameters are selected automatically for RBFkernel-SVM by the means of temperature and wind speed. Temperature and wind speed are the main influencing factors in the new energy power market, including photovoltaic and wind power. In this paper, temperature and wind speed are proposed for measuring real-time demand class dissociative in the feature space. The same classes’ temperatures and wind speeds are as close as possible. For the different classes’ temperatures and wind speeds, the greater the distance that can be created between them, the better. Hence, the mean of values applied by the normal kernel function on the samples in the same class is as shown in equation (13):(13)wγ=1∑i=1LΩi2∑i=1L∑x=Ωi∑x=Ωiκx,z,γ,where Ωi is the samples in class i. β is wγ, which is close to 1 [29]. The RBF kernel function is as in the following equation:(14)bγ=1∑i=1L∑j=1j≠iLΩiΩi∑i=1L∑j=1j≠iL∑x=Ωi∑x=Ωiκx,z,γ.Hence,γ should be determined such that it is closest to b(γ). The same classes are closest to 0. The different classes are closest to 1.It can be seen that0≤wγ≤1 and 0≤bγ≤1 if kx,y,γ≥t, which is equivalent to the optimization problem, as in the following equation:(15)minβJγ=1−wγ+bγ−0=1−wγ+bγ.In this paper, the KSVM–TCN–GBRT parameter settings are shown in Table2.Table 2
the KSVM–TCN–GBRT parameter settings.
ModelParametersValueKernel SVMKernelRBFGammaγ (automatic selecting the parameter for RBFkernel-SVM to consider temperature and wind speed)TCNinput_channels2layers_channels[32, 16, 8, 4, 2]kernel_size3OptimizerAdamlearning_rate0.001Lossmean_squared_error,MetricsmseGBRTn_estimators400Subsample1min_samples_split2min_samples_leaf1max_depth3Alpha0.7learning_rate0.2LosslsVerbose0Consider the impact of meteorological factors such as temperature and wind on new energy sources such as photovoltaics and wind power. In this paper, hourly datasets are extracted to create quarterly and similar day datasets based on actual electricity spot market data and weather data. This paper classifies real-time electricity demand by numerical intervals, using wind speed and temperature as the main characteristics and using 100 as the unit of measurement. A classification method using RBF SVMs with hyperparameters is used to find the optimum using numerical optimization. The wind speed and temperature are divided into groups that are particularly close to each other; the greater the difference between the different groups, the better. This master feature classification sequence is fed as a feature column into the succeeding deep learning neural network. KSVM can effectively solve machine learning problems with small samples and has good generalization ability; it can compensate for the problems of neural network structure selection and local minima.In the electricity market, a real-time electricity demand forecast is a multivariate time series consisting of the day-ahead demand and day-ahead price. In this paper, a multivariate TCN model is developed for supervised learning, and the dynamic relationships between its variables are extracted. We take theT time series of the day-ahead demand and day-ahead price of the day-ahead market as the cause and the time series of real-time demand series T + 24 and T + 168 of the real-time market as the effect. By increasing the number of layers, changing the expansion coefficients, revising the filters, and adjusting the length of the historical sequence, we can avoid the gradient dispersion and gradient explosion problems in the RNN model prediction, and longer-term memory and dynamic analysis capabilities can be obtained. Also, as a convolutional structure, TCNs can slide a one-dimensional convolutional kernel to receive inputs of an arbitrary length and can be massively processed in parallel for faster training and verification. This effectively guarantees the timeliness of power prediction.In this paper, one hot numerical treatment of meteorological features and date features is used to process the electricity market dataset into a non-high-dimensional nonsparse set of values suitable for GBRT forecasting. GBRT is integrated learning, using decision trees as weak classifiers and iterative learning based on the residuals of the decision tree predictions. It allows the GBRT model to be highly interpretative and robust, automatically discovering higher-order relationships between day-ahead market sequences and real-time demand characteristics.


### 2.5. Performance Evaluation

In this paper, absolute error (APE) is shown in equation (15), mean absolute percentage error (MAPE) is shown in equation (16), root mean square error (RMSE) is shown in equation (17), and mean absolute error (MAE) is shown in equation (18). The prediction errors and definitions are as follows:(16)APE=Pipre−PirealPireal,(17)MAPE=1N∑1NPipre−PirealPireal,(18)RMSE=∑1NPipre−Pireal2N,(19)MAE=∑1NPipre−PirealN,where N is the total number of test datasets, Pipre is the i-th demand prediction, and Pireal is the i-th demand prediction.


## 2.1. KSVM Model

In this paper, a Gaussian Kernel SVM is used in data feature extraction to solve the classification problem of the presence of nonlinear separability. That is, given a training sample setD=x1,y1,.x2,y2,…,xm,ym, yi=−1+1, the basic type of Kernel SVM is defined as in the following equation:(1)minw,b12ω2,s.t.yiωTϕxi+b≥1,i=1,2,…,m,γ=ω22,where γ is the sum of the distances from the support vector to the hyperplane, called the “margin.” Finding the maximizing interval is equivalent to finding the minimum ||x||2. ω = (ω1; ω2; …; ωd) is the normal vector of the hyperplane, b is the displacement term, and ϕx denotes the feature vector after mapping x.The dual problem is defined in the following equation:(2)max∂∑i=1m∂i−12∑i=1m∑j=1m∂i∂jyiyjϕxiTϕxj,s.t.∑i=1m∂iyi=0,∂i≥0,i=1,2,…,m.Here,κ.,. is the kernel function given in(3)κxi,xj=ϕxiTϕxj.The kernel function maps the data from the original space into a high-dimensional Hilbert space, where a more efficient classification hyperplane exists than in the original space.Suppose thatxjij=1⋯Ni⊂Rd is the set of training samples in class i, where Ni represents the training samples in class ii=1,2,…,L. The Gaussian Radial Basis Function (RBF) kernel is defined as in the following equation:(4)κx,z,σ=exp−x−z22σ2,σ∈0,∞,where x, z∈Rd, and σ∈R−0 are the corresponding parameters.


## 2.2. TCN Model

TCN is a one-dimensional full convolution network, which combines the structure of causal convolution, extended convolution, and the residual network. Causal convolution means that the output time is convolved only with the elements of the previous layer of time, which ensures that there will be no information leakage in the future. Dilated convolution is designed to capture a sufficiently long history of information, and the depth of the network model increases dramatically.Define the one-dimensional sequence inputXx1,x2…xs, where F:⋅0,1,…,k−1 is the function to the dilatative convolution, and the TCN convolution operation is defined as in the following equation:(5)Fs=XCondfs=∑i=0k−1fixs−d·i,where Con is the convolution operation; d is the dilatative convolution parameter; s is the current number of sequences; k is Con Size; s-d·i is past directions; and a convolution with parameter d=1,2,4 and size k=3 is shown in Figure 1.Figure 1
A dilated causal convolution with dilation factorsd=1,2,4 and filter size k=3.As shown in Figure2, the mezzanine mapping is within the residual connection, as shown in equation (6). The residual network fits several nonlinear layers between the input and output data. The more the features that are extracted, the closer the residual Fx is to 0. Hence, when the network reaches an optimal structure, Fx is pushed to 0 as the network layers deepen, leaving only the identity mapping x. This overcomes the problem of TCN degradation due to increasing network layers. Using the residuals, each order of derivative plus the constant term 1 is as in equations (6) and (7). The error can still be effectively backpropagated at this point, even if the derivatives ∂f/∂x are small.(6)o=Fx+x,(7)∂h∂x=∂f+x∂x=1+∂f∂x,where F(x) is a residual function; x is a constant.Figure 2
Residual block.


## 2.3. GBRT Model

The GBRT algorithm is able to compensate for the tendency of the cart algorithm to overfit small sample data or produce instability and low prediction accuracy. The algorithm is an iterative decision tree algorithm, which consists of three parts: cart, Gradient Boosting algorithm, and reduction idea.The basic idea of the algorithm is to use the Boosting method to iterate multiple weak learners with low prediction accuracy to form a strong learner with high prediction accuracy, that is, to reduce the residuals of the previous model by learning again so that the next generated model has a smaller error. The gradient iteration makes the combined model continuously improved, which is a kind of decision tree integrated learning algorithm designed to improve the model learning rate and prevent overfitting. That is, it does not fully trust each residual tree and uses gradual approximation to learn through multiple tree residuals.For continuous data types, the loss function is the classical loss function in Boosting, that is, the sum of squared errors, which is calculated as shown in (4). After M-th iterations, the prediction is shown in equations (8) and (9).(8)Ly,fx=∑i=1nyi−fxi2,(9)fx=∑i=0nfix,where x is the input variables, y is the output variables, and i is the iteration count.The key to improving the prediction accuracy of the GBRT model is the calculation of the residuals, and this paper uses the method proposed by Friedman [26] to calculate the residuals based on the negative gradient of the loss function.In this paper, the negative gradient of the loss function is used as an approximation to the residuals in the boosted tree algorithm. Hence, thei-th sample of the m-th round gim is calculated as shown in the following equation:(10)gim=−∂Lyi,fxi∂fxifxi=fm−1xi.fmx can be calculated from βm and hm, where βm denotes the optimal step for each iteration and is calculated as shown in (13). Hmx is the decision tree created in the m-th iteration. The prediction is shown in (11) and (12).(11)βm=argminβ∑i=1nLyi,fm−1x+βmhmx,(12)fmx=fm−1x+βmhmx.


## 2.4. Short-Term Demand Forecasting Method Based on KSVM–TCN–GBRT

This paper takes into account day-ahead demand, real-time demand, date features, and meteorological features. An RBF kernel-SVM method is used to get the relationship between temperature, wind speed, and real-time power demand.The TCN–GBRT method is also proposed to forecast the next day’s real-time demand and the next week’s real-time demand. Processing datasets are shown in Table1.Table 1
Processing datasets.
Type of dataInput VARIABLE tUnitsProcessing methodOutput variabletElectrical featurest − 24/168 day-ahead cleared demandMWhttps://www.iso-ne.com/ [27]t + 24/168 real-time demandt − 24/168 day-ahead locational marginal priceMW$/MWht + 24/168 day-ahead cleared demandMWFeature extraction based on TCNMeteorological featuret − 24/168 temperature°Fhttps://www.wunderground.com/ [28]t − 24/168 dew point°Ft − 24/168 humidity°Ft − 24/168 wind speedMPHt − 24/168 conditionOne hott + 24/168 temperature°Ft + 24/168 dew point°Ft + 24/168 humidity°Ft + 24/168 wind speedMPHt + 24/168 conditionOne hotDate featurest − 24/168 WeekDayDatet − 24/168 Montht − 24/168 holidayt + 24/168 WeekDayt + 24/168 Montht + 24/168 holidayClassification featurest + 24/168 classification IDFeature extraction based on KSVMThe KSVM–TCN–GBRT method processing steps for power markets are shown in Figure3. The short-term demand forecasting framework based on the KSVM–TCN–GBRT method is shown in Figure 4.Figure 3
The KSVM–TCN–GBRT model processing steps for Power Markets.Figure 4
The short-term demand forecasting framework based on KSVM–TCN–GBRT model.In this paper, parameters are selected automatically for RBFkernel-SVM by the means of temperature and wind speed. Temperature and wind speed are the main influencing factors in the new energy power market, including photovoltaic and wind power. In this paper, temperature and wind speed are proposed for measuring real-time demand class dissociative in the feature space. The same classes’ temperatures and wind speeds are as close as possible. For the different classes’ temperatures and wind speeds, the greater the distance that can be created between them, the better. Hence, the mean of values applied by the normal kernel function on the samples in the same class is as shown in equation (13):(13)wγ=1∑i=1LΩi2∑i=1L∑x=Ωi∑x=Ωiκx,z,γ,where Ωi is the samples in class i. β is wγ, which is close to 1 [29]. The RBF kernel function is as in the following equation:(14)bγ=1∑i=1L∑j=1j≠iLΩiΩi∑i=1L∑j=1j≠iL∑x=Ωi∑x=Ωiκx,z,γ.Hence,γ should be determined such that it is closest to b(γ). The same classes are closest to 0. The different classes are closest to 1.It can be seen that0≤wγ≤1 and 0≤bγ≤1 if kx,y,γ≥t, which is equivalent to the optimization problem, as in the following equation:(15)minβJγ=1−wγ+bγ−0=1−wγ+bγ.In this paper, the KSVM–TCN–GBRT parameter settings are shown in Table2.Table 2
the KSVM–TCN–GBRT parameter settings.
ModelParametersValueKernel SVMKernelRBFGammaγ (automatic selecting the parameter for RBFkernel-SVM to consider temperature and wind speed)TCNinput_channels2layers_channels[32, 16, 8, 4, 2]kernel_size3OptimizerAdamlearning_rate0.001Lossmean_squared_error,MetricsmseGBRTn_estimators400Subsample1min_samples_split2min_samples_leaf1max_depth3Alpha0.7learning_rate0.2LosslsVerbose0Consider the impact of meteorological factors such as temperature and wind on new energy sources such as photovoltaics and wind power. In this paper, hourly datasets are extracted to create quarterly and similar day datasets based on actual electricity spot market data and weather data. This paper classifies real-time electricity demand by numerical intervals, using wind speed and temperature as the main characteristics and using 100 as the unit of measurement. A classification method using RBF SVMs with hyperparameters is used to find the optimum using numerical optimization. The wind speed and temperature are divided into groups that are particularly close to each other; the greater the difference between the different groups, the better. This master feature classification sequence is fed as a feature column into the succeeding deep learning neural network. KSVM can effectively solve machine learning problems with small samples and has good generalization ability; it can compensate for the problems of neural network structure selection and local minima.In the electricity market, a real-time electricity demand forecast is a multivariate time series consisting of the day-ahead demand and day-ahead price. In this paper, a multivariate TCN model is developed for supervised learning, and the dynamic relationships between its variables are extracted. We take theT time series of the day-ahead demand and day-ahead price of the day-ahead market as the cause and the time series of real-time demand series T + 24 and T + 168 of the real-time market as the effect. By increasing the number of layers, changing the expansion coefficients, revising the filters, and adjusting the length of the historical sequence, we can avoid the gradient dispersion and gradient explosion problems in the RNN model prediction, and longer-term memory and dynamic analysis capabilities can be obtained. Also, as a convolutional structure, TCNs can slide a one-dimensional convolutional kernel to receive inputs of an arbitrary length and can be massively processed in parallel for faster training and verification. This effectively guarantees the timeliness of power prediction.In this paper, one hot numerical treatment of meteorological features and date features is used to process the electricity market dataset into a non-high-dimensional nonsparse set of values suitable for GBRT forecasting. GBRT is integrated learning, using decision trees as weak classifiers and iterative learning based on the residuals of the decision tree predictions. It allows the GBRT model to be highly interpretative and robust, automatically discovering higher-order relationships between day-ahead market sequences and real-time demand characteristics.


## 2.5. Performance Evaluation

In this paper, absolute error (APE) is shown in equation (15), mean absolute percentage error (MAPE) is shown in equation (16), root mean square error (RMSE) is shown in equation (17), and mean absolute error (MAE) is shown in equation (18). The prediction errors and definitions are as follows:(16)APE=Pipre−PirealPireal,(17)MAPE=1N∑1NPipre−PirealPireal,(18)RMSE=∑1NPipre−Pireal2N,(19)MAE=∑1NPipre−PirealN,where N is the total number of test datasets, Pipre is the i-th demand prediction, and Pireal is the i-th demand prediction.


## 3. Results and Discussion

### 3.1. Experimental Settings

The hardware environment was Intel(R) Core(TM) i5-7200U CPU @2.70 GHz, 64 bit, 8GBDDR4 RAM memory, NVIDIA GeForce 940MX. The software development environment was Python 3.7, Tensorflow2.2.The electricity spot market hourly dataset was obtained from ISO-NE, in the Connecticut region [27]. The meteorological data were derived from [28], and the names of weather stations were Windsor Locks and Bridgeport. The dataset spans January 1, 2016, to March 31, 2021, at an hourly frequency of 46009 moments.The real-time demand in 2019 is shown in Figure5. The four series of forecast dates are represented in order from top to bottom: original series, trend series, seasonal series, and residual series.Figure 5
Real-time electricity demand fluctuation chart of 24 hours per day in December 2019.The maximum real-time demand at different temperatures is shown in Figure6. The maximum real-time demand at different wind speeds is shown in Figure 7.Figure 6
Max real-time demand at different temperatures.Figure 7
Max real-time demand at different wind speeds.In this paper, the comparison method parameter settings are shown in Table3.Table 3
The comparison method parameter settings.
NameLayer (type)Output shapeParam #Connected toCNN- LSTMInput (InputLayer)[(None, 5, 2)]0conv1d (Conv1D)(None, 5, 16)48input_1[0][0]max_pooling1d (MaxPooling1D)(None, 1, 16)0conv1d[0][0]Dropout (dropout)(None, 1, 16)0max_pooling1d[0][0]Dense (dense)(None, 1，1)17multiply[0][0]CNN-LSTM-attentioninput (InputLayer)[(None, 5, 2)]0conv1d (Conv1D)(None, 5, 16)48input[0][0]max_pooling1d (MaxPooling1D)(None, 1, 16)0conv1d[0][0]Dropout (dropout)(None, 1, 16)0max_pooling1d[0][0]attention_vec (dense)(None, 32)1056bilstm[0][0]Multiply (multiply)(None, 32)0bilstm[0][0] attention_vec[0][0]Dense (dense)(None, 1)33multiply[0][0]CNN-Bi-LSTM-attentioninput(InputLayer)[(None, 5, 2)]0conv1d (Conv1D)(None, 5, 16)48input[0][0]max_pooling1d (MaxPooling1D)(None, 1, 16)0conv1d[0][0]Dropout (dropout)(None, 1, 16)0max_pooling1d[0][0]Bilstm (bidirectional)(None, 32)4224dropout[0][0]attention_vec (dense)(None, 32)1056bilstm[0][0]Multiply (multiply)(None, 32)0bilstm[0][0] attention_vec[0][0]Dense (dense)(None, 1)33multiply[0][0]


### 3.2. Experiment 1: Real-Time Demand Forecast for the Next Day

In this experiment, the real-time power demand was predicted for a day on March 31, 2021, at an hourly frequency of 24 moments. The training dataset includes data from January 1, 2016, to March 30, 2021. The test set has data from March 31, 2020.Real-time demand projections from March 1, 2021, to March 31, 2021, are shown in Table4. The result of the comparison method is shown in Table 5. Figure 8 shows the results of the KSVM-based real-time demand classification, considering the forecast results for temperature and wind speed for March 31, 2021. Figure 9 shows the comparison between the predicted and actual demand power for each model from March 30, 2021. It can be seen that the proposed models were able to match the actual demand power in their forecasts. In particular, the proposed models could capture occasional fluctuations.Table 4
Real-time demand projections on March 31, 2021.
Predicted timeActual value (MW)CNN-LSTMCNN-LSTM-attentionCNN-Bi-LSTM-attentionTCNProposed methodValue (MW)APE (MW)Value (MW)APE (MW)Value (MW)APE (MW)Value (MW)APE (MW)Value (MW)APE (MW)0: 0023402697357262728727794392750410234881: 00226325643012380117247120825272642232312: 002219246925022866723691502328109222123: 0022042404200228278231110723691652223194: 002266236094231044229125235286226605: 002453236984237578236093252067244766: 002766253922725791872571195273234276717: 0029962788208288910728271693025293035398: 0030902950140302664299991318292309339: 003111305259308130303774324513430367510: 00311832421243115331117325914130704811: 003090335526530933315464326317330385212: 0030763409333306214312953324516930136313: 00308333122293061223207124323615330166714: 003013326525232402273218205324022730311815: 003004323723331511473194190319419030292516: 0031033134313027763097631878430544917: 003215312590303617930871283253383215018: 00324331697431687531469733268332803719: 0032523207453218343206463343913254220: 00313832097131571931723433492113146821: 002922332039832393173256334325833629543222: 002663315449131444813127464286620327478423: 0024412837396280336228594182634193246423MAPE (%)0.18440.18060.18760.17520.0315MAX APE (MW)49148146441084Table 5
Real-time demand forecasting evaluation on the testing set.
TimeMarch 31, 2021Statistical metricsγmae (MW)γmape (%)γrmse (MW)CNN-LSTM206.187.61241.13Bi-LSTM125.564.66176.31ATT-LSTM154.95.8202.97TCN153.395.63178.63Proposed method28.87138.35Figure 8
KSVM-based real-time demand classification on March 31, 2021.Figure 9
Real-time demand forecast on March 31, 2021.


### 3.3. Experiment 2: Real-Time Demand Forecast for the Next Week

In this experiment, the real-time power demand was predicted for a week, from March 25, 2021, to March 31, 2021, with 168 moments. The training dataset includes data from January 1, 2016, to March 24, 2021. The test set includes data from March 25, 2021, to March 31, 2021.The result of comparison method is shown in Table6. Figure 10 shows the results of the KSVM-based real-time demand classification, considering the forecasted temperature and wind speed from March 25, 2021, to March 31, 2021. Figure 11 shows the comparison between the predicted and actual demand power for each model from March 31, 2021. It also can be seen that the proposed models were able to match the actual demand power better than other forecasts.Table 6
Real-time demand forecasting evaluation on the testing set.
TimeMarch 25, 2021, to March 31, 2021Statistical metricsγmae (MW)γmape (%)γrmse (MW)CNN-LSTM284.5811.05338.09Bi-LSTM216.668.28279.51ATT-LSTM236.339.09302.51TCN247.679.90272.13Proposed method40.471.4754.75Figure 10
KSVM-based real-time demand classification from March 25, 2021, to March 31, 2021.Figure 11
Real-time demand forecast from March 25, 2021, to March 31, 2021.


## 3.1. Experimental Settings

The hardware environment was Intel(R) Core(TM) i5-7200U CPU @2.70 GHz, 64 bit, 8GBDDR4 RAM memory, NVIDIA GeForce 940MX. The software development environment was Python 3.7, Tensorflow2.2.The electricity spot market hourly dataset was obtained from ISO-NE, in the Connecticut region [27]. The meteorological data were derived from [28], and the names of weather stations were Windsor Locks and Bridgeport. The dataset spans January 1, 2016, to March 31, 2021, at an hourly frequency of 46009 moments.The real-time demand in 2019 is shown in Figure5. The four series of forecast dates are represented in order from top to bottom: original series, trend series, seasonal series, and residual series.Figure 5
Real-time electricity demand fluctuation chart of 24 hours per day in December 2019.The maximum real-time demand at different temperatures is shown in Figure6. The maximum real-time demand at different wind speeds is shown in Figure 7.Figure 6
Max real-time demand at different temperatures.Figure 7
Max real-time demand at different wind speeds.In this paper, the comparison method parameter settings are shown in Table3.Table 3
The comparison method parameter settings.
NameLayer (type)Output shapeParam #Connected toCNN- LSTMInput (InputLayer)[(None, 5, 2)]0conv1d (Conv1D)(None, 5, 16)48input_1[0][0]max_pooling1d (MaxPooling1D)(None, 1, 16)0conv1d[0][0]Dropout (dropout)(None, 1, 16)0max_pooling1d[0][0]Dense (dense)(None, 1，1)17multiply[0][0]CNN-LSTM-attentioninput (InputLayer)[(None, 5, 2)]0conv1d (Conv1D)(None, 5, 16)48input[0][0]max_pooling1d (MaxPooling1D)(None, 1, 16)0conv1d[0][0]Dropout (dropout)(None, 1, 16)0max_pooling1d[0][0]attention_vec (dense)(None, 32)1056bilstm[0][0]Multiply (multiply)(None, 32)0bilstm[0][0] attention_vec[0][0]Dense (dense)(None, 1)33multiply[0][0]CNN-Bi-LSTM-attentioninput(InputLayer)[(None, 5, 2)]0conv1d (Conv1D)(None, 5, 16)48input[0][0]max_pooling1d (MaxPooling1D)(None, 1, 16)0conv1d[0][0]Dropout (dropout)(None, 1, 16)0max_pooling1d[0][0]Bilstm (bidirectional)(None, 32)4224dropout[0][0]attention_vec (dense)(None, 32)1056bilstm[0][0]Multiply (multiply)(None, 32)0bilstm[0][0] attention_vec[0][0]Dense (dense)(None, 1)33multiply[0][0]


## 3.2. Experiment 1: Real-Time Demand Forecast for the Next Day

In this experiment, the real-time power demand was predicted for a day on March 31, 2021, at an hourly frequency of 24 moments. The training dataset includes data from January 1, 2016, to March 30, 2021. The test set has data from March 31, 2020.Real-time demand projections from March 1, 2021, to March 31, 2021, are shown in Table4. The result of the comparison method is shown in Table 5. Figure 8 shows the results of the KSVM-based real-time demand classification, considering the forecast results for temperature and wind speed for March 31, 2021. Figure 9 shows the comparison between the predicted and actual demand power for each model from March 30, 2021. It can be seen that the proposed models were able to match the actual demand power in their forecasts. In particular, the proposed models could capture occasional fluctuations.Table 4
Real-time demand projections on March 31, 2021.
Predicted timeActual value (MW)CNN-LSTMCNN-LSTM-attentionCNN-Bi-LSTM-attentionTCNProposed methodValue (MW)APE (MW)Value (MW)APE (MW)Value (MW)APE (MW)Value (MW)APE (MW)Value (MW)APE (MW)0: 0023402697357262728727794392750410234881: 00226325643012380117247120825272642232312: 002219246925022866723691502328109222123: 0022042404200228278231110723691652223194: 002266236094231044229125235286226605: 002453236984237578236093252067244766: 002766253922725791872571195273234276717: 0029962788208288910728271693025293035398: 0030902950140302664299991318292309339: 003111305259308130303774324513430367510: 00311832421243115331117325914130704811: 003090335526530933315464326317330385212: 0030763409333306214312953324516930136313: 00308333122293061223207124323615330166714: 003013326525232402273218205324022730311815: 003004323723331511473194190319419030292516: 0031033134313027763097631878430544917: 003215312590303617930871283253383215018: 00324331697431687531469733268332803719: 0032523207453218343206463343913254220: 00313832097131571931723433492113146821: 002922332039832393173256334325833629543222: 002663315449131444813127464286620327478423: 0024412837396280336228594182634193246423MAPE (%)0.18440.18060.18760.17520.0315MAX APE (MW)49148146441084Table 5
Real-time demand forecasting evaluation on the testing set.
TimeMarch 31, 2021Statistical metricsγmae (MW)γmape (%)γrmse (MW)CNN-LSTM206.187.61241.13Bi-LSTM125.564.66176.31ATT-LSTM154.95.8202.97TCN153.395.63178.63Proposed method28.87138.35Figure 8
KSVM-based real-time demand classification on March 31, 2021.Figure 9
Real-time demand forecast on March 31, 2021.


## 3.3. Experiment 2: Real-Time Demand Forecast for the Next Week

In this experiment, the real-time power demand was predicted for a week, from March 25, 2021, to March 31, 2021, with 168 moments. The training dataset includes data from January 1, 2016, to March 24, 2021. The test set includes data from March 25, 2021, to March 31, 2021.The result of comparison method is shown in Table6. Figure 10 shows the results of the KSVM-based real-time demand classification, considering the forecasted temperature and wind speed from March 25, 2021, to March 31, 2021. Figure 11 shows the comparison between the predicted and actual demand power for each model from March 31, 2021. It also can be seen that the proposed models were able to match the actual demand power better than other forecasts.Table 6
Real-time demand forecasting evaluation on the testing set.
TimeMarch 25, 2021, to March 31, 2021Statistical metricsγmae (MW)γmape (%)γrmse (MW)CNN-LSTM284.5811.05338.09Bi-LSTM216.668.28279.51ATT-LSTM236.339.09302.51TCN247.679.90272.13Proposed method40.471.4754.75Figure 10
KSVM-based real-time demand classification from March 25, 2021, to March 31, 2021.Figure 11
Real-time demand forecast from March 25, 2021, to March 31, 2021.


## 4. Conclusions

This paper proposed a novel method for short-term demand forecasting in power markets based on KSVM–TCN–GBRT. The advantages of this method over previous methods are as follows:(1)
A data-driven method for short-term demand forecasting based on KSVM–TCN–GBRT was designed and improved, and the temperature and the wind speed were proposed for measuring the real-time demand to improve accuracy in forecasting market demand.(2)
We adopted a model structure consisting of data classification, a time-convolutional network, and an integrated forecasting model for daily and weekly forecasts. Our proposed model can do multistep forecasting and improve the accuracy by focusing on each feature differently.(3)
CNN-LSTM, LSTM with the attention mechanism, bidirectional LSTM, and TCN were used for forecasting and comparative analysis, and the operational results indicated that the proposed prediction method can reduce the prediction error and improve the prediction accuracy.



---

*Source: 6909558-2022-04-30.xml*