# Midterm Electricity Market Clearing Price Forecasting Using Two-Stage Multiple Support Vector Machine

Authors: Xing Yan; Nurul A. Chowdhury
Journal: Journal of Energy
 (2015)

---

## Abstract

Currently, there are many techniques available for short-term forecasting of the electricity market clearing price (MCP), but very little work has been done in the area of midterm forecasting of the electricity MCP. The midterm forecasting of the electricity MCP is essential for maintenance scheduling, planning, bilateral contracting, resources reallocation, and budgeting. A two-stage multiple support vector machine (SVM) based midterm forecasting model of the electricity MCP is proposed in this paper. The first stage is utilized to separate the input data into corresponding price zones by using a single SVM. Then, the second stage is applied utilizing four parallel designed SVMs to forecast the electricity price in four different price zones. Compared to the forecasting model using a single SVM, the proposed model showed improved forecasting accuracy in both peak prices and overall system. PJM interconnection data are used to test the proposed model.


---

## Body

## 1. Introduction

Since the generating companies as well as the bulk sellers want to maximize their profits under a deregulated electricity market, offering the appropriate amount of electricity at the right time with the right bidding price is of paramount importance. The forecasting of the electricity price is a prediction of future electricity price based on given forecast electricity demand, temperature, sunshine, fuel cost, precipitation, and other related factors. The electricity market clearing price (MCP) also called the equilibrium price exists when an electricity market is clear of shortage and surplus. Once the electricity MCP is determined, every supplier whose offering price is below or equal to the electricity MCP will be picked up to supply electricity at that hour. To ensure fairness of the market and to avoid market manipulation, the picked up suppliers will be paid at the electricity MCP, not the price they offered.Currently, in the electricity price forecasting studies, short-term forecasting of the electricity MCP is the most focused research area. It is also commonly known as the 24-hour day-ahead electricity price forecasting. Unlike the short-term forecasting, very little work has been done to forecast electricity MCP on a midterm basis [1–5]. The midterm forecasting of the electricity MCP focuses on a time frame from one month to six months. It is essential for decision making and midterm planning purposes such as generation plant expansion and maintenance schedule, reallocation of resources, bilateral contracts, and hedging strategies [5]. The midterm forecasting of the electricity MCP is different from the short-term forecasting in many ways. First of all, midterm forecasting by its nature cannot utilize the trend from the immediate past while the short-term forecasting can. Future segment is not contiguous to the immediate past history for which electricity MCP data are available for the midterm forecasting of the electricity MCP. In order to accurately forecast the future electricity MCP with segmented input data, the midterm forecasting model must possess very strong adaptability in handling out-of-sample and segmented data during training phase. Secondly, because of the unavailability of immediate past data, forecasting technique such as time series cannot be utilized in midterm forecasting. In midterm electricity price forecasting, each input data at hour t has to be treated independently from its nearby data such as t
-
n or t
+
n, for n equals 1,2
,
3
,
…. Moreover, locating and predicating peak prices in the midterm forecasting of the electricity MCP are extremely difficult. As the short-term forecasting can utilize the trend from the immediate past, locating the peak prices in most cases is usually very accurate. The main challenge in short-term forecasting is how accurately the forecasting model can predict the values at each peak price spot. However, in the midterm forecasting of the electricity MCP, because of the unavailability of data from the immediate past, locating the peak prices becomes extremely difficult. As a result, accurately predicting the values at peak price spots becomes even more difficult. The fourth difference is the length of historical data that are needed to train the forecasting model. The short-term forecasting of the electricity price model usually only needs the data of the last couple of days to train the forecasting model. On the other hand, the midterm forecasting model requires one year of historical data in order to train the forecasting model. Finally, only nonlinear regression based forecasting models are capable of midterm forecasting. Linear regression based forecasting models can only be considered as an add-on module in the midterm forecasting.Before hybrid models utilized in the forecasting of the electricity MCP, autoregressive integrated moving average (ARIMA) [9], wavelet transform [10–12], Monte Carlo simulation [13], bid-based stochastic model [13], time series [14, 15], and dynamic regression [16] are the first generation of techniques utilized in the forecasting of the electricity MCP. Artificial neural network (ANN) was later on applied to forecast short-term electricity MCP [2–4, 17–22] because of its flexibility in handling highly nonlinear relationships and relatively easy implementation. PJM interconnection, Australian electricity market, England-Wales pool, and New England ISO [19] are currently utilizing ANN to forecast the electricity MCP.Support vector machine (SVM) is a new learning method based on structural risk minimization and has gained increased attention in recent electricity price forecasting [23–26] studies. The major advantages of SVM over ANN or any other forecasting models are that SVM can avoid problems such as data overfitting, local minimum, and unpredictably large out-of-sample data error while at the same time still achieving better results. SVM is also a very robust forecasting model and will always end up with the same acceptable result regardless of the initial values. On top of that, SVM has less adjustable parameters compared to ANN and is less complicated in control parameter selection. In electricity price forecasting, a traditional SVM can achieve around 3% [27] improved forecasting accuracy compared to a traditional ANN. Several algorithms including genetic algorithm (GA) [27–30], artificial fish swarm algorithm [31], independent component analysis (ICA) algorithm [32, 33], and rough sets algorithm [34, 35] are used to further optimize the training of SVM. Moreover, the least squares support vector machine (LSSVM) was later developed to improve the accuracy of the original SVM [28, 30, 36–38]. Although each method has shown some improvements, the overall system accuracy was still quite low.The new trend in recent short-term forecasting of the electricity price studies is using hybrid models combining several prediction methods. Hybrid models can compensate the weaknesses of utilizing any individual established method and achieve better overall system results. Torghaban et al. [5] proposed hybrid midterm electricity monthly average price forecasting models combined with SVM/SVM, SVM/NN, NN/SVM, and NN/NN. Swief et al. [26] utilized the principle component analysis (PCA) and k nearest neighbor (kNN) points technique to reduce the number of data entries to the SVM model. Wang and Qin [35] introduced an electricity price prediction model using rough sets and SVM where the rough sets theory was used to simplify the SVM input variables. Xie et al. [36] proposed a hybrid electricity price forecasting model that integrates clustering algorithm with LSSVM. Fan et al. [38] proposed a hybrid forecasting model that includes the Bayesian clustering by dynamics (BCD) and SVM where the BCD classifier is applied to cluster the input data set into several subsets in an unsupervised manner. Zhao et al. [39] forecasted the prediction interval of the electricity price utilizing SVM and nonlinear conditional heteroscedastic forecasting (NCHF) model. They also proposed a hybrid model [40, 41] using SVM and probability classifier to predict the price spike occurrence and values. Areekul et al. [42] proposed forecast techniques based on autoregressive integrated moving average (ARIMA) and ANN. Xie et al. [43] proposed a sensitivity analysis of electricity price utilizing SVM and regression model where several price-load elasticity equations are built based on the price pattern data sets classified by SVM classification. A similar work focused on price-load elasticity analysis is done by Wang in [44]. Pousinho et al. [45] proposed a hybrid short-term electricity price forecasting model combined with particle swarm optimization (PSO) and adaptive-network based fuzzy inference system (ANFIS).A new midterm forecasting model of the electricity MCP utilizing two-stage multiple SVM is proposed in this paper based on the studying of previous works in midterm forecasting of the electricity MCP shown in Table1. The proposed model can predict both patterns and values of electricity MCP. A single SVM is first used to separate the input data into corresponding price zones. Then, four parallel designed SVMs are used to forecast the price values inside each separated price zone. The main contributions of this paper include (
1
) addressing and resolving problems associated with midterm forecasting of the electricity MCP, (
2
) addressing and resolving the problems associated with using a single nonlinear forecasting model to forecast midterm electricity MCP, such as SVM, and (
3
) presenting a two-stage multiple SVM based forecasting model. Experimental results using historical data from the PJM interconnection system demonstrated that the proposed two-stage multiple SVM forecasting model can improve the price predicting accuracy in both low and peak price zones. The rest of the paper is organized as follows. Two-stage multiple SVM based midterm forecasting model of the electricity MCP is given in Section 2. Case studies are presented in Section 3. Conclusions are included in Section 4.Table 1
Comparison of previously published midterm forecasting techniques of electricity prices.
Authors
Forecasting objectives and techniques
Strengths
Weaknesses
Torbaghan et al. [1]
Midterm forecasting of the monthly average electricity spot prices using various techniques
Both single and hybrid models are tested
Only focused on the monthly average electricity spot prices (only 12 outputs)
Yan [2], Yan and Chowdhury [3, 4]
Midterm forecasting of the electricity hourly MCP utilizing ANN technique
Extended training data set
Poor performance in forecasting peak prices
Torghaban et al. [5]
Midterm forecasting of the monthly average electricity spot prices utilizing linear forecasting models
Dummy variables are considered in the model indicating seasonality during simulation
Only focused on the monthly average electricity spot prices (only 12 outputs)
Yan and Chowdhury [6]
Midterm forecasting of the electricity hourly MCP utilizing hybrid LSSVM and ARMAX techniques
The proposed hybrid model is capable of adjusting the errors from the previous module
Limited improvements in forecasting peak prices
Pedregal and Trapero [7]
Midterm forecasting of the electricity hourly price utilizing unobserved component models
Utilization of numbers framed short-term forecast results to serve in the midterm forecasting
The proposed model is compared with an ARIMA model which has quite weak performance in midterm forecasting
González et al. [8]
Midterm forecasting of electricity hour price utilizing hybrid approaches based on the analysis between market price and marginal costs
Forecast the real price based on adjusting the prediction values of the marginal price forecasting
Only price data is involved in the proposed study and the forecast of the marginal price is under an ideal electricity market


## 2. Two-Stage Multiple SVM Based Midterm Forecasting Model of the Electricity MCP

### 2.1. Support Vector Machine

SVM was first introduced by Vladimir Vapnik in 1979 based on the statistical learning and later on developed by Vladimir Vapnik and his coworkers at the AT&T Bell Laboratories in 1995. After the bookAn Introduction of Support Vector Machines and Other Kernel-Based Learning Methods by Nello Cristianini and John Shawe-Taylor published in 2000, it started getting more popularity and application in many fields. At the early stage, SVM was only used for classification purposes. Then, the regression computation of nonlinear function was added by solving a convex quadratic optimization problem [46].A brief SVM architecture is shown in Figure1. It is very similar to a 3-layer feed-forward ANN. It has the input and output layers containing either single or multiple input/output data. The only difference is inside the hidden layer where kernels replace the hidden neurons. The working principle of a SVM is also different from that of an ANN. A 3-layer feed-forward ANN contains two transfer functions connecting the input layer to the hidden layer and the hidden layer to the output layer. SVM, on the other hand, only has kernels acting like transfer functions inside the hidden layer connecting the input layer and the output layer. Kernels transfer low dimensional input data vector into a much higher dimensional vector (sometime can be infinite) and eventually transfer the highly nonlinear problem inside the input space into a linear problem inside the feature space. After the transformation is completed, optimization algorithms are then applied in order to perform the regression or classification computation. SVM uses a quadratic formulation as optimization algorithm.Figure 1
SVM architecture [39].Suppose that{
(
X
t
,
y
t
)
} for t
=
1 to N is a given set of data, where X
t
=
(
x
t
1
,
x
t
2
,
…
,
x
t
k
) is the input vector at time t with k elements and y
t is the corresponding price data at time t which could be defined as
(1)
y
t
=
f
X
t
=
W
,
φ
X
t
+
b
,
where , denotes the dot product, W is the weight vector, b is the bias, and φ is the mapping function transfers input vector X
t into a much higher dimensional feature space (could be infinite). The corresponding optimization problem is then
(2)
minimize
1
2
W
2
+
C
∑
t
=
1
N
ξ
t
+
ξ
t
*
,
subject
to
y
t
-
W
,
φ
X
t
-
b
≤
ε
+
ξ
t
,
subject
to
W
,
φ
X
t
+
b
-
y
t
≤
ε
+
ξ
t
*
,
ξ
t
,
ξ
t
*
≥
0
,
where C is the regularization constant with regard to the unit cost of errors. ξ
t and ξ
t
* are the slack variables measuring the cost of the errors both above and below the target value in the training points. The ε-insensitive loss function with 2ε bandwidth is defined as
(3)
y
t
-
f
X
t
ε
=
0
,
if
y
t
-
f
X
t
≤
ε
y
t
-
f
X
t
-
ε
,
otherwise
.
When adding the Lagrange multipliers to (2), the problem can be rewritten as
(4)
L
W
,
ξ
,
ξ
*
,
α
,
α
*
,
η
,
η
*
=
1
2
W
2
+
C
∑
t
=
1
N
ξ
t
+
ξ
t
*
-
∑
t
=
1
N
α
t
ε
+
ξ
t
-
y
t
+
W
,
φ
X
t
+
b
-
∑
t
=
1
N
α
t
*
ε
+
ξ
t
*
+
y
t
-
W
,
φ
X
t
-
b
-
∑
t
=
1
N
η
t
ξ
t
+
η
t
*
ξ
t
*
,
where α
t, α
t
*, η
t, and η
t
* are the Lagrange multipliers. The partial derivates of the Lagrange function with the primal variables (W, b, ξ
t, and ξ
t
*) are then
(5)
∂
L
SVM
∂
W
=
0
⟶
W
=
∑
t
=
1
N
α
t
-
α
t
*
φ
X
t
,
∂
L
SVM
∂
b
=
0
⟶
∑
t
=
1
N
α
t
-
α
t
*
=
0
,
∂
L
SVM
∂
ξ
t
=
0
⟶
C
-
α
t
-
η
t
=
0
,
∂
L
SVM
∂
ξ
t
*
=
0
⟶
C
-
α
t
*
-
η
t
*
=
0
.
The dual problem can be obtained by substituting the relations from (5):
(6)
maximize
-
1
2
∑
t
,
l
=
1
N
α
t
-
α
t
*
α
l
-
α
l
*
φ
X
t
,
φ
X
l
-
ε
∑
t
=
1
N
α
t
+
α
t
*
+
∑
t
=
1
N
y
t
α
t
-
α
t
*
,
subject
to
∑
t
=
1
N
α
t
-
α
t
*
=
0
,
subject
to
α
t
,
α
t
*
∈
0
,
C
.
According to the Karush-Kuhn-Tucker (KKT) conditions, the terms inside the equation containing the Lagrange multipliers will vanish at the optimal solution. This gives the following equations:
(7)
α
t
ε
+
ξ
t
-
y
t
+
W
,
φ
X
t
+
b
=
0
,
α
t
*
ε
+
ξ
t
*
+
y
t
-
W
,
φ
X
t
-
b
=
0
,
η
t
ξ
t
=
C
-
α
t
ξ
t
=
0
,
η
t
*
ξ
t
*
=
C
-
α
t
*
ξ
t
*
=
0
.
The above equations indicate that for all samples whose Lagrange multipliers are equal to zero are considered as nonsupport vectors. Samples with nonzero coefficients are considered as support vectors. Meanwhile, b can be calculated when the slack variables ξ
t and ξ
t
* are equal to zero
(8)
b
=
y
t
-
W
,
φ
X
t
±
ε
for
α
t
,
α
t
*
∈
0
,
C
.
The final SVM for nonlinear functions can be written as
(9)
y
t
=
f
X
t
=
∑
t
=
1
N
α
t
-
α
t
*
φ
X
,
φ
X
t
+
b
.
A modified version of (9) can be expressed as
(10)
y
t
=
f
X
t
=
∑
t
=
1
N
α
t
-
α
t
*
K
X
,
X
t
+
b
,
where K
(
X
,
X
t
)
=
φ
(
X
)
,
φ
(
X
t
) is called the kernel function. Gaussian radial basis kernel (11) is the most powerful one in nonlinear function estimation:
(11)
K
X
,
X
t
=
exp
⁡
-
X
-
X
t
2
2
σ
2
.


### 2.2. Price Zones Analysis

One of the special characteristics of electricity is that it has to be produced as used. Taking PJM interconnected electricity market as an example, electricity price can vary from as low as –$10/MWh to as high as $748/MWh [33]. The relationship between supply and demand under normal deregulated market is no longer the only dominant principle determining the electricity price. Other factors such as business competing strategy and unethical business behaviour are heavily involved and cannot be efficiently simulated. Therefore, the accuracy of the forecasted electricity MCP in the peak price zone is always very low. Although electricity MCP is very volatile, it is still normally distributed along its average value [23–25]. In the proposed work, the electricity MCP is separated into four price zones: low, medium, high, and peak price zones. Let μ and σ be the mean and standard deviation of the monthly historical electricity MCP; the four price zones of each month are determined based on the four criteria listed below:low:MCP
<
μ
-
σ;
medium:μ
-
σ
≤
MCP
<
μ
+
0.5
σ;
high:μ
+
0.5
σ
≤
MCP
<
μ
+
1.5
σ;
peak:μ
+
1.5
σ
≤
MCP.About 15%–25% of the price is in the low price zone. 50%–60% of the price is in the medium price zone. 15% −20% of the price is in the high price zone. Finally, the last 5%–10% of the price will be in the peak price zone. The standard deviation multipliers are chosen based on the price characteristics of the PJM interconnected electricity market and forecasting model parameter optimization. They can be modified when applying in a different electricity market. Inside the medium price zone, most generating companies are participating in supplying electricity. Every participating generating unit is running at its optimal output. There is plenty of reserved energy available in the market which can be supplied immediately. Congestion is low in most areas and congestion cost is limited. At this moment, the market is at its stable state. Sudden shortage or surplus can be easily recovered or offset by other generating companies in the same and nearby jurisdiction. Supply and demand elasticity plays a major role and electricity MCP is mainly determined by the demand and fuel cost. Inside the low price zone, the profit margin is small and, therefore, only companies with low production cost and mandatory all-time running power units such as nuclear power plants are participating. Sudden shortage or surplus could cause price spikes because they cannot be recovered or offset as easily as in the medium price zone. In the peak price zone, most power plants are running at their peak capacity. Transmission lines are under extreme load resulting in extremely high congestion cost in some areas. System reserved energy is very low and generating companies expect to take advantages and make huge profits from it. Electricity MCP is mainly determined by business competing strategy and even unethical business behaviour in the peak price zone. The high price zone is between the medium and peak price zones and, therefore, shows characteristics of both price zones. The supply and demand relationship and business competing strategy both dominate the final electricity MCP inside the high price zone.


### 2.3. Two-Stage Multiple SVM Forecasting Model Architecture

The architecture of the proposed two-stage multiple SVM based midterm forecasting model of the electricity MCP is shown in Figure2. It contains four layers: (
1
) input layer, (
2
) input data distribution layer, (
3
) SVM price prediction layer, and (
4
) output layer.Figure 2
Two-stage multiple SVM forecasting model architecture.Inside the input layer, there will be input data such as electricity hourly demand, electricity daily peak demand, and daily natural gas price. Data selection and preprocessing will be explained in detail in Section2.4. After all the data are preprocessed inside the input layer, a single SVM is first utilized to predict the initial values of the electricity prices. These price values are then separated into four predefined price zones: low, medium, high, and peak. The input data distribution layer can significantly reduce the range each price prediction SVM shall cover and improve the overall system accuracy when forecasting the midterm electricity MCP. After the input data distribution is completed, the SVM prediction module is utilized to forecast the midterm electricity MCP. There are four parallel connected SVMs inside the price prediction module to forecast the electricity MCP in four different price zones. Based on the different range of input data, each price prediction SVM optimizes its own control parameters during the training process. These different control parameters give each price prediction SVM the ability to capture the characteristics between input and output data within each price zone. Inside the output layer, four SVMs predicted electricity price values are combined to form the final forecast electricity MCP. Input data distribution module will assign each input data with serial numbers that will be used to regroup all forecast price values from the SVM price prediction layer inside the output layer. The training input data for both modules are from the input layer in Figure 2. The target data for both modules are the historical electricity MCP. Cross-validation technique is applied in data distribution and price prediction modules for the optimization of the control parameters.


### 2.4. Data Collection and Preprocessing

Electricity MCP can be forecasted by evaluating a variety of elements [34] such as electricity demand, supply, natural gas price, coal price, hydrocapacity, weather, and temperature. An ideal forecasting model of the electricity MCP should include all the possible elements that affect the final electricity MCP. In reality, however, it is impossible and unnecessary to do so. Since weather conditions including daily temperature are already considered in load forecasting, they do not have to be included in MCP forecasting process. Other elements such as business strategy and unethical competition behaviours cannot be easily represented mathematically. Finally, the current failure or operating status of a generator in many deregulated electric market is confidential information. Due to the factors mentioned above, a data selection process is presented in Figure 3 using cross-validation technique to finalize the input data that were taken into account for the proposed work.Figure 3
Data selection process.The data selection process starts with only the default input data (hourly electricity demand) inside the forecasting model. Then, the next element inside the potential input data is added to the initial forecasting model to test whether the added element improves the forecasting accuracy or not. Tested element will be added to the forecasting model if it improves the forecasting accuracy. Otherwise, the forecasting model maintains its current input data and the next element inside the potential input data will be tested. Due to the fact that sometimes adding more than one element at a time will achieve better forecasting results, different combinations of elements will also be tested once all the elements have been tested one at a time. By applying the data selection process shown in Figure3, it will guarantee to test not only every single element but also every possible combination of elements that would lead to the optimized forecasting results. The final selected input data for the proposed two-stage multiple SVM forecasting model at each hour t include (
1
) electricity hourly demand at hour t, (
2
) electricity daily peak demand, (
3
) electricity monthly average demand, (4) daily price of natural gas, (
5
) previous year’s monthly average electricity MCP, (
6
) month (1–12), and (
7
) hour of the day (1–24). The target data at hour t is the electricity MCP at hour t. Moreover, one year is the most optimized length of historical data to train the forecasting model based on the previous published works [1–3] regarding the selection of training data for midterm forecasting of the electricity MCP.Part of the training data is separated from the rest of the training data and served as the so-called the testing data. The testing data are used to optimize the control parameters of the proposed forecasting model and are treated unseen from the rest of the training data. For the rest of this paper, the remaining part of the training data is called the actual training data. In the proposed work, data from January 1, 2009, to December 31, 2009, excluding June 2009, are selected as the actual training data and can be viewed as a matrix with 8040 (24 × 335) rows and 8 columns. The 8040 rows are the number of hours from hour 1 (1:00 a.m.) on January 1, 2009, to hour 24 (12:00 a.m.) on December 31, 2009, excluding June 2009. The first 7 columns are the 7 input elements mentioned before. The last column is the training target data, electricity MCP from hour 1 (1:00 a.m.) on January 1, 2009, to hour 24 (12:00 a.m.) on December 31, 2009, excluding June 2009. Because the proposed two-stage multiple SVM based midterm forecasting model of the electricity MCP is designed to predict the 720 hourly electricity MCPs in June 2010, selecting data from hour 1 (1:00 a.m.) on June 1, 2009, to hour 24 (12:00 a.m.) on June 30, 2009, as the testing data could create the best scenarios on daily demand pattern, daily price pattern, weather, sunshine, and precipitation during training process to optimize the control parameters. For the same reason, if the proposed model is utilized to forecast the 744 hourly electricity MCPs in May 2010, data in May 2009 will be selected as the testing data during training process. The testing data contain a matrix with 720 rows (24 × 30) and 8 columns. The first 7 columns are the testing input data and the last column is the testing target data.Once the training procedure is done, the proposed two-stage multiple SVM forecasting model is used to forecast the midterm electricity MCP of every hour in June 2010 using historical data from January 2009 to December 2009. This work is based on the assumption that all forecasting input data have already been accurately predicted to limit the affects from inaccurate input data. The forecast output contains 720 (24 × 30) hourly electricity MCP which will insure the comprehensiveness of the test sample. The forecasting input data could be viewed as a matrix with 720 (24 × 30) rows and 8 columns. The 720 rows are the number of hours from hour 1 (1:00 a.m.) on June 1, 2010, to hour 24 (12:00 a.m.) on June 30, 2010. The first 7 columns are the forecasting input data and the last column is the electricity MCP from hour 1 (1:00 a.m.) on June 1, 2010, to hour 24 (12:00 a.m.) on June 30, 2010.In order to improve the regression accuracy and avoid the dominance of some extremely large values inside the data set, data preprocessing is needed in machine learning. For electricity MCP forecasting, the most common and recommended data preprocessing algorithms are the normalization. Normalization converts each datum inside its corresponding element into a value between –1 and +1 (sometimes between 0 and 1) with respect to each element’s maximum and minimum value. Depending on the calculation, the maximum and minimum values can be either global or local. Suppose thatX
t
=
(
x
t
1
,
x
t
2
,
…
,
x
t
k
) is a given set of vectors at time t with k multiple elements for t
=
1 to N. The normalization for datum x
t
k could be defined as
(12)
x
^
t
k
=
x
t
k
-
x
MAX
k
+
x
MIN
k
/
2
x
MAX
k
-
x
MAX
k
+
x
MIN
k
/
2
,
where x
MAX
k and x
MIN
k are the global or local maximum and minimum values inside the kth element. The normalization on forecasting input data is performed using each element’s corresponding global or local maximum and minimum values during the training process. The target data remain unchanged as there is only the electricity MCP data. The designed two-stage multiple SVM based midterm forecasting model of the electricity MCP can utilize different dimensions of input data for forecasting accuracy optimization. The full dimension of the input data is 7 as mentioned before. Recall the data selection process from Figure 3 and, using mean absolute error for the performance evaluation, the optimized final dimensions of input data for the proposed multiple SVM are 7 for the initial prediction SVM, 5 for the low price SVM, 7 for the medium price SVM, 7 for the high price SVM, and 7 for the peak price SVM.


### 2.5. Performance Evaluation

Mean absolute error (MAE), mean absolute percentage error (MAPE), and mean square root error (MSRE) are the three most widely used measurements for performance evaluation in forecasting electricity price values. GivenN historical electricity MCP data, y
t, and the corresponding forecasted price data y
^
t for t
=
1 to N, MAE, MAPE, and MSRE are defined as
(13)
MAE
=
1
N
∑
t
=
1
N
y
t
-
y
^
t
,
MAPE
=
1
N
∑
t
=
1
N
y
t
-
y
^
t
y
t
×
100
%
,
MSRE
=
1
N
∑
t
=
1
N
y
t
-
y
^
t
2
.
During the training process, two SVM control parameters, the coefficient of the Gaussian radial basis kernel σ and the regularization constant γ, are set at an initial value and updated using testing data during each iteration until the MAE of the SVM reaches its global minimum. Once all control parameters are determined, forecasting input data can be applied to forecast the midterm electricity MCP.


## 2.1. Support Vector Machine

SVM was first introduced by Vladimir Vapnik in 1979 based on the statistical learning and later on developed by Vladimir Vapnik and his coworkers at the AT&T Bell Laboratories in 1995. After the bookAn Introduction of Support Vector Machines and Other Kernel-Based Learning Methods by Nello Cristianini and John Shawe-Taylor published in 2000, it started getting more popularity and application in many fields. At the early stage, SVM was only used for classification purposes. Then, the regression computation of nonlinear function was added by solving a convex quadratic optimization problem [46].A brief SVM architecture is shown in Figure1. It is very similar to a 3-layer feed-forward ANN. It has the input and output layers containing either single or multiple input/output data. The only difference is inside the hidden layer where kernels replace the hidden neurons. The working principle of a SVM is also different from that of an ANN. A 3-layer feed-forward ANN contains two transfer functions connecting the input layer to the hidden layer and the hidden layer to the output layer. SVM, on the other hand, only has kernels acting like transfer functions inside the hidden layer connecting the input layer and the output layer. Kernels transfer low dimensional input data vector into a much higher dimensional vector (sometime can be infinite) and eventually transfer the highly nonlinear problem inside the input space into a linear problem inside the feature space. After the transformation is completed, optimization algorithms are then applied in order to perform the regression or classification computation. SVM uses a quadratic formulation as optimization algorithm.Figure 1
SVM architecture [39].Suppose that{
(
X
t
,
y
t
)
} for t
=
1 to N is a given set of data, where X
t
=
(
x
t
1
,
x
t
2
,
…
,
x
t
k
) is the input vector at time t with k elements and y
t is the corresponding price data at time t which could be defined as
(1)
y
t
=
f
X
t
=
W
,
φ
X
t
+
b
,
where , denotes the dot product, W is the weight vector, b is the bias, and φ is the mapping function transfers input vector X
t into a much higher dimensional feature space (could be infinite). The corresponding optimization problem is then
(2)
minimize
1
2
W
2
+
C
∑
t
=
1
N
ξ
t
+
ξ
t
*
,
subject
to
y
t
-
W
,
φ
X
t
-
b
≤
ε
+
ξ
t
,
subject
to
W
,
φ
X
t
+
b
-
y
t
≤
ε
+
ξ
t
*
,
ξ
t
,
ξ
t
*
≥
0
,
where C is the regularization constant with regard to the unit cost of errors. ξ
t and ξ
t
* are the slack variables measuring the cost of the errors both above and below the target value in the training points. The ε-insensitive loss function with 2ε bandwidth is defined as
(3)
y
t
-
f
X
t
ε
=
0
,
if
y
t
-
f
X
t
≤
ε
y
t
-
f
X
t
-
ε
,
otherwise
.
When adding the Lagrange multipliers to (2), the problem can be rewritten as
(4)
L
W
,
ξ
,
ξ
*
,
α
,
α
*
,
η
,
η
*
=
1
2
W
2
+
C
∑
t
=
1
N
ξ
t
+
ξ
t
*
-
∑
t
=
1
N
α
t
ε
+
ξ
t
-
y
t
+
W
,
φ
X
t
+
b
-
∑
t
=
1
N
α
t
*
ε
+
ξ
t
*
+
y
t
-
W
,
φ
X
t
-
b
-
∑
t
=
1
N
η
t
ξ
t
+
η
t
*
ξ
t
*
,
where α
t, α
t
*, η
t, and η
t
* are the Lagrange multipliers. The partial derivates of the Lagrange function with the primal variables (W, b, ξ
t, and ξ
t
*) are then
(5)
∂
L
SVM
∂
W
=
0
⟶
W
=
∑
t
=
1
N
α
t
-
α
t
*
φ
X
t
,
∂
L
SVM
∂
b
=
0
⟶
∑
t
=
1
N
α
t
-
α
t
*
=
0
,
∂
L
SVM
∂
ξ
t
=
0
⟶
C
-
α
t
-
η
t
=
0
,
∂
L
SVM
∂
ξ
t
*
=
0
⟶
C
-
α
t
*
-
η
t
*
=
0
.
The dual problem can be obtained by substituting the relations from (5):
(6)
maximize
-
1
2
∑
t
,
l
=
1
N
α
t
-
α
t
*
α
l
-
α
l
*
φ
X
t
,
φ
X
l
-
ε
∑
t
=
1
N
α
t
+
α
t
*
+
∑
t
=
1
N
y
t
α
t
-
α
t
*
,
subject
to
∑
t
=
1
N
α
t
-
α
t
*
=
0
,
subject
to
α
t
,
α
t
*
∈
0
,
C
.
According to the Karush-Kuhn-Tucker (KKT) conditions, the terms inside the equation containing the Lagrange multipliers will vanish at the optimal solution. This gives the following equations:
(7)
α
t
ε
+
ξ
t
-
y
t
+
W
,
φ
X
t
+
b
=
0
,
α
t
*
ε
+
ξ
t
*
+
y
t
-
W
,
φ
X
t
-
b
=
0
,
η
t
ξ
t
=
C
-
α
t
ξ
t
=
0
,
η
t
*
ξ
t
*
=
C
-
α
t
*
ξ
t
*
=
0
.
The above equations indicate that for all samples whose Lagrange multipliers are equal to zero are considered as nonsupport vectors. Samples with nonzero coefficients are considered as support vectors. Meanwhile, b can be calculated when the slack variables ξ
t and ξ
t
* are equal to zero
(8)
b
=
y
t
-
W
,
φ
X
t
±
ε
for
α
t
,
α
t
*
∈
0
,
C
.
The final SVM for nonlinear functions can be written as
(9)
y
t
=
f
X
t
=
∑
t
=
1
N
α
t
-
α
t
*
φ
X
,
φ
X
t
+
b
.
A modified version of (9) can be expressed as
(10)
y
t
=
f
X
t
=
∑
t
=
1
N
α
t
-
α
t
*
K
X
,
X
t
+
b
,
where K
(
X
,
X
t
)
=
φ
(
X
)
,
φ
(
X
t
) is called the kernel function. Gaussian radial basis kernel (11) is the most powerful one in nonlinear function estimation:
(11)
K
X
,
X
t
=
exp
⁡
-
X
-
X
t
2
2
σ
2
.


## 2.2. Price Zones Analysis

One of the special characteristics of electricity is that it has to be produced as used. Taking PJM interconnected electricity market as an example, electricity price can vary from as low as –$10/MWh to as high as $748/MWh [33]. The relationship between supply and demand under normal deregulated market is no longer the only dominant principle determining the electricity price. Other factors such as business competing strategy and unethical business behaviour are heavily involved and cannot be efficiently simulated. Therefore, the accuracy of the forecasted electricity MCP in the peak price zone is always very low. Although electricity MCP is very volatile, it is still normally distributed along its average value [23–25]. In the proposed work, the electricity MCP is separated into four price zones: low, medium, high, and peak price zones. Let μ and σ be the mean and standard deviation of the monthly historical electricity MCP; the four price zones of each month are determined based on the four criteria listed below:low:MCP
<
μ
-
σ;
medium:μ
-
σ
≤
MCP
<
μ
+
0.5
σ;
high:μ
+
0.5
σ
≤
MCP
<
μ
+
1.5
σ;
peak:μ
+
1.5
σ
≤
MCP.About 15%–25% of the price is in the low price zone. 50%–60% of the price is in the medium price zone. 15% −20% of the price is in the high price zone. Finally, the last 5%–10% of the price will be in the peak price zone. The standard deviation multipliers are chosen based on the price characteristics of the PJM interconnected electricity market and forecasting model parameter optimization. They can be modified when applying in a different electricity market. Inside the medium price zone, most generating companies are participating in supplying electricity. Every participating generating unit is running at its optimal output. There is plenty of reserved energy available in the market which can be supplied immediately. Congestion is low in most areas and congestion cost is limited. At this moment, the market is at its stable state. Sudden shortage or surplus can be easily recovered or offset by other generating companies in the same and nearby jurisdiction. Supply and demand elasticity plays a major role and electricity MCP is mainly determined by the demand and fuel cost. Inside the low price zone, the profit margin is small and, therefore, only companies with low production cost and mandatory all-time running power units such as nuclear power plants are participating. Sudden shortage or surplus could cause price spikes because they cannot be recovered or offset as easily as in the medium price zone. In the peak price zone, most power plants are running at their peak capacity. Transmission lines are under extreme load resulting in extremely high congestion cost in some areas. System reserved energy is very low and generating companies expect to take advantages and make huge profits from it. Electricity MCP is mainly determined by business competing strategy and even unethical business behaviour in the peak price zone. The high price zone is between the medium and peak price zones and, therefore, shows characteristics of both price zones. The supply and demand relationship and business competing strategy both dominate the final electricity MCP inside the high price zone.


## 2.3. Two-Stage Multiple SVM Forecasting Model Architecture

The architecture of the proposed two-stage multiple SVM based midterm forecasting model of the electricity MCP is shown in Figure2. It contains four layers: (
1
) input layer, (
2
) input data distribution layer, (
3
) SVM price prediction layer, and (
4
) output layer.Figure 2
Two-stage multiple SVM forecasting model architecture.Inside the input layer, there will be input data such as electricity hourly demand, electricity daily peak demand, and daily natural gas price. Data selection and preprocessing will be explained in detail in Section2.4. After all the data are preprocessed inside the input layer, a single SVM is first utilized to predict the initial values of the electricity prices. These price values are then separated into four predefined price zones: low, medium, high, and peak. The input data distribution layer can significantly reduce the range each price prediction SVM shall cover and improve the overall system accuracy when forecasting the midterm electricity MCP. After the input data distribution is completed, the SVM prediction module is utilized to forecast the midterm electricity MCP. There are four parallel connected SVMs inside the price prediction module to forecast the electricity MCP in four different price zones. Based on the different range of input data, each price prediction SVM optimizes its own control parameters during the training process. These different control parameters give each price prediction SVM the ability to capture the characteristics between input and output data within each price zone. Inside the output layer, four SVMs predicted electricity price values are combined to form the final forecast electricity MCP. Input data distribution module will assign each input data with serial numbers that will be used to regroup all forecast price values from the SVM price prediction layer inside the output layer. The training input data for both modules are from the input layer in Figure 2. The target data for both modules are the historical electricity MCP. Cross-validation technique is applied in data distribution and price prediction modules for the optimization of the control parameters.


## 2.4. Data Collection and Preprocessing

Electricity MCP can be forecasted by evaluating a variety of elements [34] such as electricity demand, supply, natural gas price, coal price, hydrocapacity, weather, and temperature. An ideal forecasting model of the electricity MCP should include all the possible elements that affect the final electricity MCP. In reality, however, it is impossible and unnecessary to do so. Since weather conditions including daily temperature are already considered in load forecasting, they do not have to be included in MCP forecasting process. Other elements such as business strategy and unethical competition behaviours cannot be easily represented mathematically. Finally, the current failure or operating status of a generator in many deregulated electric market is confidential information. Due to the factors mentioned above, a data selection process is presented in Figure 3 using cross-validation technique to finalize the input data that were taken into account for the proposed work.Figure 3
Data selection process.The data selection process starts with only the default input data (hourly electricity demand) inside the forecasting model. Then, the next element inside the potential input data is added to the initial forecasting model to test whether the added element improves the forecasting accuracy or not. Tested element will be added to the forecasting model if it improves the forecasting accuracy. Otherwise, the forecasting model maintains its current input data and the next element inside the potential input data will be tested. Due to the fact that sometimes adding more than one element at a time will achieve better forecasting results, different combinations of elements will also be tested once all the elements have been tested one at a time. By applying the data selection process shown in Figure3, it will guarantee to test not only every single element but also every possible combination of elements that would lead to the optimized forecasting results. The final selected input data for the proposed two-stage multiple SVM forecasting model at each hour t include (
1
) electricity hourly demand at hour t, (
2
) electricity daily peak demand, (
3
) electricity monthly average demand, (4) daily price of natural gas, (
5
) previous year’s monthly average electricity MCP, (
6
) month (1–12), and (
7
) hour of the day (1–24). The target data at hour t is the electricity MCP at hour t. Moreover, one year is the most optimized length of historical data to train the forecasting model based on the previous published works [1–3] regarding the selection of training data for midterm forecasting of the electricity MCP.Part of the training data is separated from the rest of the training data and served as the so-called the testing data. The testing data are used to optimize the control parameters of the proposed forecasting model and are treated unseen from the rest of the training data. For the rest of this paper, the remaining part of the training data is called the actual training data. In the proposed work, data from January 1, 2009, to December 31, 2009, excluding June 2009, are selected as the actual training data and can be viewed as a matrix with 8040 (24 × 335) rows and 8 columns. The 8040 rows are the number of hours from hour 1 (1:00 a.m.) on January 1, 2009, to hour 24 (12:00 a.m.) on December 31, 2009, excluding June 2009. The first 7 columns are the 7 input elements mentioned before. The last column is the training target data, electricity MCP from hour 1 (1:00 a.m.) on January 1, 2009, to hour 24 (12:00 a.m.) on December 31, 2009, excluding June 2009. Because the proposed two-stage multiple SVM based midterm forecasting model of the electricity MCP is designed to predict the 720 hourly electricity MCPs in June 2010, selecting data from hour 1 (1:00 a.m.) on June 1, 2009, to hour 24 (12:00 a.m.) on June 30, 2009, as the testing data could create the best scenarios on daily demand pattern, daily price pattern, weather, sunshine, and precipitation during training process to optimize the control parameters. For the same reason, if the proposed model is utilized to forecast the 744 hourly electricity MCPs in May 2010, data in May 2009 will be selected as the testing data during training process. The testing data contain a matrix with 720 rows (24 × 30) and 8 columns. The first 7 columns are the testing input data and the last column is the testing target data.Once the training procedure is done, the proposed two-stage multiple SVM forecasting model is used to forecast the midterm electricity MCP of every hour in June 2010 using historical data from January 2009 to December 2009. This work is based on the assumption that all forecasting input data have already been accurately predicted to limit the affects from inaccurate input data. The forecast output contains 720 (24 × 30) hourly electricity MCP which will insure the comprehensiveness of the test sample. The forecasting input data could be viewed as a matrix with 720 (24 × 30) rows and 8 columns. The 720 rows are the number of hours from hour 1 (1:00 a.m.) on June 1, 2010, to hour 24 (12:00 a.m.) on June 30, 2010. The first 7 columns are the forecasting input data and the last column is the electricity MCP from hour 1 (1:00 a.m.) on June 1, 2010, to hour 24 (12:00 a.m.) on June 30, 2010.In order to improve the regression accuracy and avoid the dominance of some extremely large values inside the data set, data preprocessing is needed in machine learning. For electricity MCP forecasting, the most common and recommended data preprocessing algorithms are the normalization. Normalization converts each datum inside its corresponding element into a value between –1 and +1 (sometimes between 0 and 1) with respect to each element’s maximum and minimum value. Depending on the calculation, the maximum and minimum values can be either global or local. Suppose thatX
t
=
(
x
t
1
,
x
t
2
,
…
,
x
t
k
) is a given set of vectors at time t with k multiple elements for t
=
1 to N. The normalization for datum x
t
k could be defined as
(12)
x
^
t
k
=
x
t
k
-
x
MAX
k
+
x
MIN
k
/
2
x
MAX
k
-
x
MAX
k
+
x
MIN
k
/
2
,
where x
MAX
k and x
MIN
k are the global or local maximum and minimum values inside the kth element. The normalization on forecasting input data is performed using each element’s corresponding global or local maximum and minimum values during the training process. The target data remain unchanged as there is only the electricity MCP data. The designed two-stage multiple SVM based midterm forecasting model of the electricity MCP can utilize different dimensions of input data for forecasting accuracy optimization. The full dimension of the input data is 7 as mentioned before. Recall the data selection process from Figure 3 and, using mean absolute error for the performance evaluation, the optimized final dimensions of input data for the proposed multiple SVM are 7 for the initial prediction SVM, 5 for the low price SVM, 7 for the medium price SVM, 7 for the high price SVM, and 7 for the peak price SVM.


## 2.5. Performance Evaluation

Mean absolute error (MAE), mean absolute percentage error (MAPE), and mean square root error (MSRE) are the three most widely used measurements for performance evaluation in forecasting electricity price values. GivenN historical electricity MCP data, y
t, and the corresponding forecasted price data y
^
t for t
=
1 to N, MAE, MAPE, and MSRE are defined as
(13)
MAE
=
1
N
∑
t
=
1
N
y
t
-
y
^
t
,
MAPE
=
1
N
∑
t
=
1
N
y
t
-
y
^
t
y
t
×
100
%
,
MSRE
=
1
N
∑
t
=
1
N
y
t
-
y
^
t
2
.
During the training process, two SVM control parameters, the coefficient of the Gaussian radial basis kernel σ and the regularization constant γ, are set at an initial value and updated using testing data during each iteration until the MAE of the SVM reaches its global minimum. Once all control parameters are determined, forecasting input data can be applied to forecast the midterm electricity MCP.


## 3. Case Studies

### 3.1. PJM Interconnected Electric Market

PJM interconnected electric market is the largest interconnected system in the world. It includes all or most of Delaware, District of Columbia, Maryland, New Jersey, Ohio, Pennsylvania, Virginia, and West Virginia. Parts of Indiana, Illinois, Kentucky, Michigan, North Carolina, and Tennessee are also included in the PJM interconnected electric market. The market serves more than 61 million people in the United States. As of December 31, 2012, it had an installed generating capacity of 183,604 megawatts and more than 800 participating members. The market covered 243,417 square miles of territory containing 62,556 miles of transmission lines and more than 1,376 electric generators [47]. Coal (74%) and natural gas (22%) are the two major types of fuel in the market. Locational marginal pricing (LMP) is used to reflect the value of the energy at the specific location and time it is delivered. The market consists of day-ahead and real-time markets. The day-ahead market is a forward market in which hourly LMP is calculated for the next operating day based on generation offers, demand bids, and scheduled bilateral transactions. The real-time market is a spot market in which current LMP is calculated at five-minute intervals based on actual grid operating conditions. PJM settles transactions hourly and issues invoices to market participants monthly [48]. The LMP inside the PJM interconnected electric market is used as electricity MCP in this work. Historical data from the day-ahead market of 2008 and 2009 are used in this work to forecast the electricity MCP in June 2010 and are available at [47].


### 3.2. Experimental Results

The testing results using both single SVM and the proposed two-stage multiple SVM are shown in Figure4. Data from January 2009 to December 2009 excluding June 2009 are used as the actual training data and data in June 2009 are used as the testing data.Figure 4
Forecasted electricity MCP in June 2009 using both single and two-stage multiple SVM.It can be seen that the major contribution of utilizing a two-stage multiple SVM based forecasting model is the improved forecasting results in the low and peak price zones. The regression computation performance evaluations are performed using MAE, MAPE, and MSRE analyses. The performance numbers are shown in Tables2, 3, and 4. Percentage improvement (PRIM) is used to compare the regression computation performance between the single SVM model and the proposed two-stage multiple SVM model. The PRIM is calculated as
(14)
PRIM
=

g
d
-
1
single
_
MAE
/
MSRE
/
MAPE
-
multiple
_
MAE
/
MSRE
/
MAPE
·
single
_
MAE
/
MSRE
/
MAPE
-
1
×
100
%
.
Positive PRIM values indicate that the two-stage multiple SVM model is better than the single SVM model in this price zone. Negative PRIM values indicate the opposite.Table 2
Performance evaluation results of MAE between the single and the two-stage multiple SVM in June 2009.
MAE
Low
Medium
High
Peak
System
Single
4.0094
1.8941
3.0837
8.2403
2.9743
Multiple
3.1985
1.8678
3.8336
5.7636
2.9625
PRIM
20.22%
1.39%
−24.32%
30.06%
0.40%Table 3
Performance evaluation results of MAPE between the single and the two-stage multiple SVM in June 2009.
MAPE
Low
Medium
High
Peak
System
Single
35.9821
6.1526
7.2207
14.5750
11.7491
Multiple
24.9639
5.6042
8.7352
11.2517
11.4083
PRIM
30.62%
8.91%
−20.97%
22.80%
2.90%Table 4
Performance evaluation results of MSRE between the single and the two-stage multiple SVM in June 2009.
MSRE
Low
Medium
High
Peak
System
Single
0.5200
0.1191
0.2933
1.2738
0.1564
Multiple
0.3381
0.1408
0.3590
1.0060
0.1536
PRIM
34.98%
−18.22%
−22.40%
21.02%
1.79%According to all three tables, MAE, MAPE, and MSRE values show that the two-stage multiple SVM model is more accurate than the single SVM model by more than at least 20% in the low and peak price zones. However, the results are opposite in the high price zones. This is caused by the characteristics of the high price zone as it is between the medium and peak price zones. The supply and demand relationship and business competing strategy both dominate the final electricity MCP inside the high price zone. It is very difficult for a single SVM to simulate these two very different characteristics at the same time. Significantly reduced data caused by the SVM classification process is also another reason; MAE, MAPE, and MSRE are low in the high price zone. As the MSRE values in the medium price zones are opposite compared to the MAE and MAPE values, it is hard to say which model has the better performance in the medium price zone. Usually, the conclusions obtained from the performance numbers of MAE, MAPE, and MSRE should be identical. However, in special cases, the conclusions would differ from each other due to the different distribution of results. For instance, suppose that the forecasting errors from the single SVM and two-stage multiple SVM are (6, 6) and (1, 10) and the forecasting percentage errors from the single SVM and two-stage multiple SVM are (6%, 6%) and (1%, 10%). The size of the data set is only 2. Therefore, the MAE of the single SVM and the two-stage multiple SVM are (6 + 6)/2 = 6 and (1 + 10)/2 = 5.5. According to the MAE values, the two-stage multiple SVM is better than the single SVM. The same conclusions apply to the MAPE evaluation where the MAPEs of the single SVM and the two-stage multiple SVM are (6% + 6%)/2 = 6% and (1% + 10%)/2 = 5.5%. However, when comparing the MSRE values, the conclusion is different. The MSREs of the single SVM and the two-stage multiple SVM are (62 + 62)0.5/2 = 4.243 and (12 + 102)0.5/2 = 5.025. According to the MSRE values, the single SVM is better than the two-stage multiple SVM. Under special situation like this, when the conclusions obtained by the MAE, MAPE, and MSRE values do not agree with each other in certain price zones (e.g., the medium zone), because all the control parameters are determined based on the global minimal MAE values, we will use the MAE values to determine which forecasting model is better.As the testing results supported the fact that the two-stage multiple SVM is better than the single SVM in midterm forecasting of the electricity MCP, the proposed forecasting model is used to forecast the midterm hourly electricity MCP in June 2010 using historical data from January 2009 to December 2009. Figure5 shows the forecast electricity hourly MCP in June 2010 using both single and the two-stage multiple SVM. The final midterm forecasting results of the electricity hourly MCP are obtained from the combination of predicted price from four price prediction SVMs. Detailed evaluation results of MAE, MAPE, and MSRE are shown in Tables 5, 6, and 7. The two-stage multiple SVM forecasting model shows an average 6% improvement on the system MAE, MAPE, and MSRE evaluations. The negative PRIM values in the high price zone are expected while the negative PRIM values in the medium price zone indicate that there is misdistribution during the forecast process. The forecasting results of the proposed multiple SVM model are also compared with other machine learning methods [6] shown in Table 8. Although the proposed two-stage multiple SVM is only slightly better in forecasting accuracy compared with other midterm electricity MCP forecasting methods such as the hybrid LSSVM and ARMAX during the forecasting process, the proposed two-stage multiple SVM model is way more flexible in adjusting input data in any SVM and therefore has much higher potential to improve in future research than other midterm electricity price forecasting methods. All of the study cases were run on a PC with 4 GB of RAM and a 2.4 GHz processor. MATLAB 7.7 is used. The average forecasting time is less than 1 minute.Table 5
Performance evaluation results of MAE between the single and the two-stage multiple SVM in June 2010.
MAE
Low
Medium
High
Peak
System
Single
5.7523
6.9711
5.0316
15.9496
7.2523
Multiple
2.7764
7.5398
7.6556
11.1233
6.8106
PRIM
51.73%
−8.16%
−52.15%
30.26%
6.09%Table 6
Performance evaluation results of MAPE between the single and the two-stage multiple SVM in June 2010.
MAPE
Low
Medium
High
Peak
System
Single
21.0094
16.3663
7.5146
16.7736
15.6454
Multiple
10.2069
18.9028
12.4708
12.0534
14.6567
PRIM
51.42%
−15.50%
−65.95%
28.14%
6.32%Table 7
Performance evaluation results of MSRE between the single and the two-stage multiple SVM in June 2010.
MSRE
Low
Medium
High
Peak
System
Single
0.4572
0.3350
0.5299
2.6890
0.3316
Multiple
0.2569
0.4872
0.6411
1.7935
0.3126
PRIM
43.81%
−45.43%
−20.99%
33.30%
5.73%Table 8
Performance evaluation results between the proposed two-stage multiple SVM model and other forecasting models.
June 2009
June 2010
MAE
MAPE
MSRE
MAE
MAPE
MSRE
Single SVM
2.9743
11.7491
0.1564
7.2523
15.6454
0.3316
Single LSSVM
2.8152
10.9722
0.1513
7.9003
16.2610
0.3964
Hybrid LSSVM and ARMAX
2.7630
10.6706
0.1495
7.0989
13.9709
0.3859
Multiple LSSVM
2.7811
10.7466
0.1463
7.0950
13.2850
0.3733
Two-stage multiple SVM
2.9625
11.4083
0.1536
6.8106
14.6567
0.3126Figure 5
Forecasted electricity MCP in June 2010 using both single and two-stage multiple SVM.


### 3.3. Discussions

The SVM intends to lose the top and the bottom peak values during training process because most of those values are considered as nonsupport vectors when utilizing theε-insensitive loss function with 2ε bandwidth. Only the support vectors are used to create the 2ε tube and only the values outside the 2ε tube are considered during simulation. As mentioned before, the electricity MCP in the peak price zone is mainly determined by business competing strategy and even unethical business behaviour and those elements cannot be easily represented mathematically. Therefore, when using a single SVM forecasting model, the forecast electricity MCP will lose most of its top and bottom peak values during the forecasting process. The proposed two-stage multiple SVM forecasting model can improve the compromise that is caused by using a single SVM by distributing the initially predicted price values into different price zones. By doing so, instead of having only two control parameters in a single SVM forecasting model, the proposed two-stage multiple SVM forecasting model has 10 control parameters that can be adjusted to optimize the system forecasting accuracy.


## 3.1. PJM Interconnected Electric Market

PJM interconnected electric market is the largest interconnected system in the world. It includes all or most of Delaware, District of Columbia, Maryland, New Jersey, Ohio, Pennsylvania, Virginia, and West Virginia. Parts of Indiana, Illinois, Kentucky, Michigan, North Carolina, and Tennessee are also included in the PJM interconnected electric market. The market serves more than 61 million people in the United States. As of December 31, 2012, it had an installed generating capacity of 183,604 megawatts and more than 800 participating members. The market covered 243,417 square miles of territory containing 62,556 miles of transmission lines and more than 1,376 electric generators [47]. Coal (74%) and natural gas (22%) are the two major types of fuel in the market. Locational marginal pricing (LMP) is used to reflect the value of the energy at the specific location and time it is delivered. The market consists of day-ahead and real-time markets. The day-ahead market is a forward market in which hourly LMP is calculated for the next operating day based on generation offers, demand bids, and scheduled bilateral transactions. The real-time market is a spot market in which current LMP is calculated at five-minute intervals based on actual grid operating conditions. PJM settles transactions hourly and issues invoices to market participants monthly [48]. The LMP inside the PJM interconnected electric market is used as electricity MCP in this work. Historical data from the day-ahead market of 2008 and 2009 are used in this work to forecast the electricity MCP in June 2010 and are available at [47].


## 3.2. Experimental Results

The testing results using both single SVM and the proposed two-stage multiple SVM are shown in Figure4. Data from January 2009 to December 2009 excluding June 2009 are used as the actual training data and data in June 2009 are used as the testing data.Figure 4
Forecasted electricity MCP in June 2009 using both single and two-stage multiple SVM.It can be seen that the major contribution of utilizing a two-stage multiple SVM based forecasting model is the improved forecasting results in the low and peak price zones. The regression computation performance evaluations are performed using MAE, MAPE, and MSRE analyses. The performance numbers are shown in Tables2, 3, and 4. Percentage improvement (PRIM) is used to compare the regression computation performance between the single SVM model and the proposed two-stage multiple SVM model. The PRIM is calculated as
(14)
PRIM
=

g
d
-
1
single
_
MAE
/
MSRE
/
MAPE
-
multiple
_
MAE
/
MSRE
/
MAPE
·
single
_
MAE
/
MSRE
/
MAPE
-
1
×
100
%
.
Positive PRIM values indicate that the two-stage multiple SVM model is better than the single SVM model in this price zone. Negative PRIM values indicate the opposite.Table 2
Performance evaluation results of MAE between the single and the two-stage multiple SVM in June 2009.
MAE
Low
Medium
High
Peak
System
Single
4.0094
1.8941
3.0837
8.2403
2.9743
Multiple
3.1985
1.8678
3.8336
5.7636
2.9625
PRIM
20.22%
1.39%
−24.32%
30.06%
0.40%Table 3
Performance evaluation results of MAPE between the single and the two-stage multiple SVM in June 2009.
MAPE
Low
Medium
High
Peak
System
Single
35.9821
6.1526
7.2207
14.5750
11.7491
Multiple
24.9639
5.6042
8.7352
11.2517
11.4083
PRIM
30.62%
8.91%
−20.97%
22.80%
2.90%Table 4
Performance evaluation results of MSRE between the single and the two-stage multiple SVM in June 2009.
MSRE
Low
Medium
High
Peak
System
Single
0.5200
0.1191
0.2933
1.2738
0.1564
Multiple
0.3381
0.1408
0.3590
1.0060
0.1536
PRIM
34.98%
−18.22%
−22.40%
21.02%
1.79%According to all three tables, MAE, MAPE, and MSRE values show that the two-stage multiple SVM model is more accurate than the single SVM model by more than at least 20% in the low and peak price zones. However, the results are opposite in the high price zones. This is caused by the characteristics of the high price zone as it is between the medium and peak price zones. The supply and demand relationship and business competing strategy both dominate the final electricity MCP inside the high price zone. It is very difficult for a single SVM to simulate these two very different characteristics at the same time. Significantly reduced data caused by the SVM classification process is also another reason; MAE, MAPE, and MSRE are low in the high price zone. As the MSRE values in the medium price zones are opposite compared to the MAE and MAPE values, it is hard to say which model has the better performance in the medium price zone. Usually, the conclusions obtained from the performance numbers of MAE, MAPE, and MSRE should be identical. However, in special cases, the conclusions would differ from each other due to the different distribution of results. For instance, suppose that the forecasting errors from the single SVM and two-stage multiple SVM are (6, 6) and (1, 10) and the forecasting percentage errors from the single SVM and two-stage multiple SVM are (6%, 6%) and (1%, 10%). The size of the data set is only 2. Therefore, the MAE of the single SVM and the two-stage multiple SVM are (6 + 6)/2 = 6 and (1 + 10)/2 = 5.5. According to the MAE values, the two-stage multiple SVM is better than the single SVM. The same conclusions apply to the MAPE evaluation where the MAPEs of the single SVM and the two-stage multiple SVM are (6% + 6%)/2 = 6% and (1% + 10%)/2 = 5.5%. However, when comparing the MSRE values, the conclusion is different. The MSREs of the single SVM and the two-stage multiple SVM are (62 + 62)0.5/2 = 4.243 and (12 + 102)0.5/2 = 5.025. According to the MSRE values, the single SVM is better than the two-stage multiple SVM. Under special situation like this, when the conclusions obtained by the MAE, MAPE, and MSRE values do not agree with each other in certain price zones (e.g., the medium zone), because all the control parameters are determined based on the global minimal MAE values, we will use the MAE values to determine which forecasting model is better.As the testing results supported the fact that the two-stage multiple SVM is better than the single SVM in midterm forecasting of the electricity MCP, the proposed forecasting model is used to forecast the midterm hourly electricity MCP in June 2010 using historical data from January 2009 to December 2009. Figure5 shows the forecast electricity hourly MCP in June 2010 using both single and the two-stage multiple SVM. The final midterm forecasting results of the electricity hourly MCP are obtained from the combination of predicted price from four price prediction SVMs. Detailed evaluation results of MAE, MAPE, and MSRE are shown in Tables 5, 6, and 7. The two-stage multiple SVM forecasting model shows an average 6% improvement on the system MAE, MAPE, and MSRE evaluations. The negative PRIM values in the high price zone are expected while the negative PRIM values in the medium price zone indicate that there is misdistribution during the forecast process. The forecasting results of the proposed multiple SVM model are also compared with other machine learning methods [6] shown in Table 8. Although the proposed two-stage multiple SVM is only slightly better in forecasting accuracy compared with other midterm electricity MCP forecasting methods such as the hybrid LSSVM and ARMAX during the forecasting process, the proposed two-stage multiple SVM model is way more flexible in adjusting input data in any SVM and therefore has much higher potential to improve in future research than other midterm electricity price forecasting methods. All of the study cases were run on a PC with 4 GB of RAM and a 2.4 GHz processor. MATLAB 7.7 is used. The average forecasting time is less than 1 minute.Table 5
Performance evaluation results of MAE between the single and the two-stage multiple SVM in June 2010.
MAE
Low
Medium
High
Peak
System
Single
5.7523
6.9711
5.0316
15.9496
7.2523
Multiple
2.7764
7.5398
7.6556
11.1233
6.8106
PRIM
51.73%
−8.16%
−52.15%
30.26%
6.09%Table 6
Performance evaluation results of MAPE between the single and the two-stage multiple SVM in June 2010.
MAPE
Low
Medium
High
Peak
System
Single
21.0094
16.3663
7.5146
16.7736
15.6454
Multiple
10.2069
18.9028
12.4708
12.0534
14.6567
PRIM
51.42%
−15.50%
−65.95%
28.14%
6.32%Table 7
Performance evaluation results of MSRE between the single and the two-stage multiple SVM in June 2010.
MSRE
Low
Medium
High
Peak
System
Single
0.4572
0.3350
0.5299
2.6890
0.3316
Multiple
0.2569
0.4872
0.6411
1.7935
0.3126
PRIM
43.81%
−45.43%
−20.99%
33.30%
5.73%Table 8
Performance evaluation results between the proposed two-stage multiple SVM model and other forecasting models.
June 2009
June 2010
MAE
MAPE
MSRE
MAE
MAPE
MSRE
Single SVM
2.9743
11.7491
0.1564
7.2523
15.6454
0.3316
Single LSSVM
2.8152
10.9722
0.1513
7.9003
16.2610
0.3964
Hybrid LSSVM and ARMAX
2.7630
10.6706
0.1495
7.0989
13.9709
0.3859
Multiple LSSVM
2.7811
10.7466
0.1463
7.0950
13.2850
0.3733
Two-stage multiple SVM
2.9625
11.4083
0.1536
6.8106
14.6567
0.3126Figure 5
Forecasted electricity MCP in June 2010 using both single and two-stage multiple SVM.


## 3.3. Discussions

The SVM intends to lose the top and the bottom peak values during training process because most of those values are considered as nonsupport vectors when utilizing theε-insensitive loss function with 2ε bandwidth. Only the support vectors are used to create the 2ε tube and only the values outside the 2ε tube are considered during simulation. As mentioned before, the electricity MCP in the peak price zone is mainly determined by business competing strategy and even unethical business behaviour and those elements cannot be easily represented mathematically. Therefore, when using a single SVM forecasting model, the forecast electricity MCP will lose most of its top and bottom peak values during the forecasting process. The proposed two-stage multiple SVM forecasting model can improve the compromise that is caused by using a single SVM by distributing the initially predicted price values into different price zones. By doing so, instead of having only two control parameters in a single SVM forecasting model, the proposed two-stage multiple SVM forecasting model has 10 control parameters that can be adjusted to optimize the system forecasting accuracy.


## 4. Conclusions

A two-stage multiple SVM based midterm forecasting model of the electricity MCP is proposed in this paper. Two modules, input data distribution module and price forecasting module, are designed to first preprocess the initially predicted price values into corresponding price zones and then forecast the electricity price in four parallel designed SVMs. The proposed two-stage multiple SVM model showed improved forecasting accuracy in the low and the peak price zones and thus improving the overall forecasting accuracy compared to the forecasting model utilizing a single SVM. The case studies also show that the performance of either a single or a two-stage multiple SVM is highly depending on the selection of the input data. Carefully selected training input data and correctly predicted subdata sets would significantly improve the accuracy of the forecasting model. The proposed forecasting model has ten adjustable control parameters compared to only two adjustable control parameters in a single SVM forecasting model. The four parallel SVMs in the price forecasting module also have the flexibility of using different dimensions of input data to achieve the optimal results. Future research will focus on possible modules that can be added to the distribution module to improve the system accuracy.



---

*Source: 384528-2015-01-29.xml*