# Deep Learning Enhanced Solar Energy Forecasting with AI-Driven IoT

Authors: Hangxia Zhou; Qian Liu; Ke Yan; Yang Du
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

Short-term photovoltaic (PV) energy generation forecasting models are important, stabilizing the power integration between the PV and the smart grid for artificial intelligence- (AI-) driven internet of things (IoT) modeling of smart cities. With the recent development of AI and IoT technologies, it is possible for deep learning techniques to achieve more accurate energy generation forecasting results for the PV systems. Difficulties exist for the traditional PV energy generation forecasting method considering external feature variables, such as the seasonality. In this study, we propose a hybrid deep learning method that combines the clustering techniques, convolutional neural network (CNN), long short-term memory (LSTM), and attention mechanism with the wireless sensor network to overcome the existing difficulties of the PV energy generation forecasting problem. The overall proposed method is divided into three stages, namely, clustering, training, and forecasting. In the clustering stage, correlation analysis and self-organizing mapping are employed to select the highest relevant factors in historical data. In the training stage, a convolutional neural network, long short-term memory neural network, and attention mechanism are combined to construct a hybrid deep learning model to perform the forecasting task. In the testing stage, the most appropriate training model is selected based on the month of the testing data. The experimental results showed significantly higher prediction accuracy rates for all time intervals compared to existing methods, including traditional artificial neural networks, long short-term memory neural networks, and an algorithm combining long short-term memory neural network and attention mechanism.


---

## Body

## 1. Introduction

Photovoltaic power generation has the advantages of low carbon consumption, adaptive to various applications, and low installation and maintenance costs, which is known as a sustainable energy source [1]. Because of different weather conditions, PV panels often cannot stably output electrical power from solar energy. While integrating the PV power to the power grid, the grid is seriously influenced. The stability of the entire grid will be greatly reduced. With the wireless sensor network, it is possible for deep learning techniques to forecast solar energy generation and consequently stabilize the smart grid systems [2]. Therefore, artificial intelligence- (AI-) driven internet of things (IoT) technology becomes one of the key technologies of solving this problem [3, 4]. Along with the fast development of IoT technology, the extended deep learning methods nowadays are capable of performing short-term time series data forecasting, such as PV power generation, anomaly detection, and energy consumption forecasting, with considerable high forecasting accuracy [5–7].In recent years, with the fast development of AI-driven IoT technology, the applications of deep learning technologies have been extended to various fields [8, 9], such as digital twinning [10], computer security [11], cyberphysical systems [12], transportation systems [13], and air quality forecasting [14]. Jiménez-Pérez and Mora-López [15] proposed a forecasting system simulating global solar irradiance forecasts every hour. The system can be separated into two phases. The first phase is clustering. The original dataset was divided into groups by the k-means clustering algorithm. Each group represents data of different weather types. Machine learning techniques, such as decision trees, artificial neural networks, and support vector machines, are employed to perform forecasting in the second phase. Yang et al. [16] proposed a hybrid deep learning method based on weather types for PV power output forecasting with a timestep at 1 hour. The proposed method has three steps: classification, training, and prediction. The classification step involves a self-organizing map (SOM) [17] and Learning Vector Quantization (LVQ) [18]. In the training and prediction step, the fuzzy training method is used to select the most appropriate deep learning model for prediction. Han et al. [19] proposed an alternative multimodel PV power interval prediction method that considers the seasonal property of the PV power and absolute power deviation in the prediction process. VanDeventer et al. [20] proposed a support vector machine model based on a genetic algorithm (GASVM). The GASVM model classifies the historical weather data using an SVM classifier initially, and later, it is optimized by the genetic algorithm using an ensemble technique. Malvoni et al. [21] proposed to use wavelet decomposition and principal component analysis to decompose meteorological data and treat it as predictive inputs. A time series forecasting method, named as GLSSVM (Group Least Square Support Vector Machine), which combines the Least Square Support Vector Machines (LS-SVM) and Group Method of Data Handling (GMDH), was applied to the measured weather data.More recently, deep learning techniques, such as the convolutional neural network (CNN), long short-term memory (LSTM), and deep belief networks (DBNs), are widely applied to the PV system energy generation forecasting problem. Srivastava and Lessmann [22] proposed to adopt the original LSTM neural network for PV energy generation forecasting, and the proposed algorithm is hardly generalizable. Jian et al. [23] proposed an online sequential extreme learning machine with a forgetting mechanism model for PV energy generation prediction. The normalized data is input into a two-layer CNN for feature extraction and then merged into LSTM by fusion layer for PV power prediction. Zhou et al. [24] proposed a LSTM neural network combining the attention mechanism (AM) for PV energy generation forecasting. The AM is used to adaptively perceive feature information from the time series data. Chen et al. [25] proposed a cloud shadow model using computer vision techniques to forecast real cloud coverage nature for PV systems and enhanced the PV energy generation forecasting results consequently. Chang et al. [26] introduced a virtual inertia control based on PV load forecasting results, showing real-world applications of the PV energy generation forecasting techniques.In this paper, we proposed a novel PV power forecasting framework combining clustering and deep learning technologies. The entire framework can be divided into three stages. In the first stage, according to the impact of the training data in the previous paragraph, we divide the training data into four clusters using the SOM algorithm, mimicking the four seasons. The reason for using the SOM algorithm as the experimental clustering algorithm is described in2.2.2, and the reason for simulating the data into four seasons for the experiment is described in 2.2.1. In the second stage, use AM, CNN, and LSTM to build the model. The third stage is the forecasting stage. According to the month of the testing data, the most suitable forecasting model is selected to predict the PV energy generation for the next time stamp. Experimental results show the superior performance of the proposed method over the existing works. Compared to the existing methods tackling on the same problem, the main contributions of the proposed method are summarized in the following points.


### 1.1. The Raw Data Is Processed by Clustering Techniques for More Accurate Forecasting Performance

Based on the data collected from the eastern region of China, the solar irradiance patterns vary between different seasons. The data is first clustered into four classes and trained using four different LSTM neural networks, respectively, to enhance the forecasting accuracy.


### 1.2. CNN and LSTM Are Combined for a More Sophisticated Deep Learning Framework

Both CNN and LSTM are popular deep learning techniques for forecasting problems. In this study, the two techniques are combined to produce better forecasting results for the PV energy generation forecasting problem.


### 1.3. The AM Technique Is Adopted to Further Enhance the Forecasting Performance

The AM is an important extension for the traditional LSTM neural network. It forces the LSTM neural networks to pay more attention to the features that are more relevant to the forecasting outputs. According to the experimental results, the AM technique greatly improves the forecasting results produced by LSTM neural networks.


## 1.1. The Raw Data Is Processed by Clustering Techniques for More Accurate Forecasting Performance

Based on the data collected from the eastern region of China, the solar irradiance patterns vary between different seasons. The data is first clustered into four classes and trained using four different LSTM neural networks, respectively, to enhance the forecasting accuracy.


## 1.2. CNN and LSTM Are Combined for a More Sophisticated Deep Learning Framework

Both CNN and LSTM are popular deep learning techniques for forecasting problems. In this study, the two techniques are combined to produce better forecasting results for the PV energy generation forecasting problem.


## 1.3. The AM Technique Is Adopted to Further Enhance the Forecasting Performance

The AM is an important extension for the traditional LSTM neural network. It forces the LSTM neural networks to pay more attention to the features that are more relevant to the forecasting outputs. According to the experimental results, the AM technique greatly improves the forecasting results produced by LSTM neural networks.


## 2. Methodology

The dataset employed in this study was collected by a PV power station located in Shaoxing city in the eastern part of China. This solar energy generation dataset was collected from October 2014 to September 2018, in a time interval of 7.5 min. The data from 2014 to 2016 is taken as the training dataset, and the data from 2017 to 2018 is used for testing. Remote sensors were utilized to record the PV module temperature, the current, voltage, frequency, phases, and PV power every 7.5 minutes. Since the power station uses three-phase inverter equipment, the data contains the PV module temperature, three AC currents (alternating current 1, alternating current 2, and alternating current 3), three AC voltages (AC voltage 1, AC voltage 2, and AC voltage 3), two DC currents (direct current 1 and direct current 2), two DC voltages (DC voltage 1 and DC voltage 1), frequency, phases, and PV power. The frequency and phase do not change with the PV power, so the frequency and phase data are not used in the experiments. It is very difficult to obtain huge amount of historical weather data with a time interval of 7.5 min at the same location from the weather bureau. We do not add weather data to the experimental data and find a data preprocessing method to effectively improve the predictive ability of the model.We calculated correlation coefficients between other factor data and PV power. The correlation of the multifactor combination on the PV power is shown in Table1. A combined correlation test method for calculating the modified RV coefficient is introduced here. Modified RV coefficient (RVmod) is a correlation analysis method based on matrix calculation. The equation for calculating the modified RV coefficient is
(1)RVmod=trJKtrJ2trK2,Table 1
Correlation coefficient of PV data factors.
PV data factor nameCorrelation coefficientDC voltage 1 (V)0.19DC voltage 2 (V)0.20Direct current 1 (A)0.13Direct current 2 (A)0.13AC voltage 1 (V)0.10AC voltage 2 (V)0.11AC voltage 3 (V)0.14Alternating current 1 (A)0.28Alternating current 2 (A)0.992Alternating current 3 (A)0.998Alternating current 2 (A) and alternating current 3 (A) combination0.73PV module temperature0.16whereJ=M·MT−diagM·MT; K=N·NT−diagN·NT; M represents the matrix of influencing factors, including alternating current 2 (AC2) and alternating current 3 (AC 3); N represents PV power output matrix; diag· is a function that takes out the diagonal elements of the matrix; tr· is a function that takes the sum of the elements on the diagonal of the matrix; the range of RVmod is (-1,1), when the RVmod is closer to -1 or 1; and the correlation is higher between the factor and the power output.For the coefficients AC2 and coefficients AC3 which are very close, we specially carry out experiments and compare the experimental results of the datasets of AC2 and AC3. The experimental results show that, although the coefficient difference is not large, the prediction accuracy of AC2 is significantly higher than AC3. In this study, we extract AC2 data as experiment data.


### 2.1. Clustering of the Original Data

In the clustering stage, the AC2 data from 2014 to 2016 is the training dataset and used for clustering. Considering the different numbers of days per month and the lack of PV data, we selected 15 days of complete data from each month and made each month AC2 data to a dataset asx1,x2,⋯,x15; the dataset indicates that a month’s AC2 data, x, contains 192 AC2 real-time data for one day. The clustering experiment uses two-year AC2 data. We merged the datasets with the same month label. The SOM algorithm is used to perform the clustering task. The raw data was clustered into 4 clusters.The SOM model structure is shown in Figure1. The learning steps of the SOM algorithm are as follows.Figure 1
Structure of the SOM model.In the first step, the weight vector corresponding to each neuron in the competition layer was initialized, normalizing the current input mode vectorX and the weight vector corresponding to the neuron. Secondly, the neuron node d with the smallest Euclidean distance between Xc (input vector) and ѡj (connection weight vector) is selected as the winning neuron node. As a result, the weights were updated accordingly. Following Equation (3), the weight of the winning neuron node d and other nodes in its domain is updated, where ηt is the learning rate of the t step in the range of (0, 1) and hd，jt is the neighborhood function. hd，jt generally uses a Gaussian function, as shown in Equation (4). The specific adjustment rules are as shown in Equations (5) and (6).
(2)Xc−wj=mindjc,(3)Wijt+1=Wijt+ηthd,jtXi−Wijt,(4)hd,jt=exp−ddj22r2t,(5)rt+1=INTrt−1×1−tT+1,(6)ηt+1=ηt−η0T,where the distance between the neurond and the neuron j is ddj2, rt is the neighborhood radius, INT is the rounding function, and T is the learning frequency.


### 2.2. Deep Learning Models

In the training stage, the 4 clusters of data are inputted into a model for training, and 4 forecasting models are trained. The training process follows the training method of time series. We display the training data of each category asy1,y2,y3,⋯,yn, where y represents AC2 data and n represents the number of AC2 data contained in each category. We set timestep i, y1,y2,⋯,yi, as a new training set, and yi+1 as the target value; then, we put the new training set into the training model, and the model parameters are adjusted during training so that the predicted output is constantly close to yi+1. Next, y2,y3,⋯,yi+1 is inputted into the training model and adjusts the model parameters so that the predicted output is continuously close to yi+2, and so on, until the target value is yn; end the model training and get the forecasting model.The training model structure of this experiment consists of two convolutional kernel layers of CNN, one layer of LSTM with attention mechanism, and a fully connected (FC) layer. The overall flowchart of the proposed framework is depicted in Figure2.Figure 2
The overall flowchart of the proposed algorithm.This proposed deep learning model (CNN-ALSTM) is a hybrid deep learning model to extract features from the raw data and perform foresting using the LSTM neural network. The CNN layer is employed to extract the useful features from the time series data, representing additional latent information of the data, which has the potential to improve the prediction accuracy. The experimental results show that the CNN layer contains one 163×1 convolution kernel layer and one 32 3×1 convolution kernel convolution layer that optimizes the prediction performance.The feature vector obtained from the second layer of CNN was inputted into the LSTM layer for prediction. Each element of the feature vector has correspondence to one of the 32 units in the LSTM layer. The attention mechanism puts higher weight to the feature quantities that are significantly related to the current output. Last, the output vector of the attention mechanism is processed by a FC layer using the unfolding operation. The predicted value of the AC2 at the next moment is output.LSTM is very suitable for prediction experiments of time series data. Existing works show superior forecasting performance combining CNN and LSTM for various applications [23, 27–29]. CNN helps LSTM to better extract the characteristics of experimental data. The attention mechanism is a process of assigning weights. Using the attention mechanism, more accurate weightage values are assigned to the LSTM output vector to improve the prediction capability of the model.The predictive power is obtained by the inverse normalization according to Equation (7).
(7)PrP=PrIac2∗Pmax−Pmin+Pmin,wherePrp is the predicted value of power and PrIac2 is the predicted value of AC2.


#### 2.2.1. Long Short-Term Memory Neural Network

The LSTM model structure is shown in Figure3.Figure 3
Structure of LSTM.The appearance of LSTM cell structure effectively resolves the gradient explosion/vanishing problems. There are four important elements in the flowchart of the LSTM model: cell status, input gate, forget gate, and output gate. The input, forget, and output gates are used to control the update, maintenance, and deletion of information contained in cell status. The forward computation process can be denoted as(8)ft=σWf·ht−1,xt+bf,it=σWi·ht−1,xt+bi,Ot=σWO·ht−1,xt+bo,C~t=tanhWC·ht−1,xt+bC,Ct=ft·Ct−1+it·C~t,ht=Ot·tanhCt,whereWf, Wi, and Wo are the weight matrix of the forgetting gate, input gate, and output gate, respectively; bf, bi, and bo are the offset items of the forgetting gate, the input gate, and the output gate, respectively; σ is the sigmoid activation function; tanh is the hyperbolic tangent activation function.


#### 2.2.2. The Attention Mechanism

The attention mechanism is a brain signal processing mechanism peculiar to human vision. Human vision quickly scans the global image to obtain the target area that needs attention and ignores other areas of useless information. The attention mechanism algorithm has been successfully implemented and applied to model training [26] and other related fields.The model proposed in this paper uses the LSTM hidden layer output vectorH=h1,h2,⋯,ht as the input of the attention mechanism, and the attention mechanism will find the attention weight αi of hi, which can be calculated as shown in
(9)ei=tanhWhhi+bh,αi=expei∑i=1texpei,whereWh is the weight matrix of hi and bh is the bias. The values of Wh and bh vary during the ALSTM training process. The range of ei is (-1,1). The attention vector H′=h1′,h2′,⋯,ht′ can be obtained by multiplying attention weight αi and hi:
(10)hi′=αi·hi.The attention mechanism is implemented as a custom layer where the parameters are optimized using RMSProp backpropagation [33, 34].


### 2.3. The Overall Forecasting Framework

There are four deep learning models trained in the proposed framework (Figure4). In the testing stage, an appropriate forecasting model is selected based on the grouping result of the testing data sample. The proposed hybrid deep learning model structure is shown in Figure 5. As a necessary preprocessing step, we used correlation coefficient to determine inputs for prediction. The weather data was employed by the SOM algorithm to cluster inputs into four categories. Each category is trained by the proposed hybrid deep learning framework. In particular, the CNN is used to extract the time series features of AC2 data; the LSTM neural network further extracts high latitude features in the data. The attention mechanism is used to assign different attention weights to the output elements of the LSTM hidden layer. In the testing phase, according to the month of the test set, we chose the appropriate model to predict the testing result (Figures 4 and 5).Figure 4
Structure of prediction stage. The numbers under test data and training data represent months.Figure 5
Structure of the proposed hybrid model.The process of data classification method determination and hyperparameter determination is also described, including the reason for simulating data in four seasons, the selection of clustering algorithm, the settings of SOM competition layer, and the selection of SOM classified result.


#### 2.3.1. The Selection of Clustering Algorithm

The original idea of the clustering is to map the raw data into four different groups, representing the four seasons. These four groups of data are trained and tested using the proposed CNN-ALSTM structure separately. Different clustering algorithms are tested in the section. In the training phase, we analyze clustering algorithms, includingk-means, FCM, and SOM (Figure 6), for clustering the training dataset into four clusters. Based on the experimental results on the training data, SOM is more accurate than most of the above-mentioned traditional clustering algorithms (Figure 6) according to the original season labels of the raw data.Figure 6
Accuracy of different clustering algorithms in the training phase.The size of the SOM competition layer has been evaluated through a series of experiments as shown in Figure7. The X-axis represents the size of the competition layer, and the Y-axis represents the number of output neurons. The result shows that the most suitable competition layer for SOM is 2×2.Figure 7
Horizontally, theX-axis represents the size of the competition layer, and vertically, the Y-axis represents the number of output neurons.The clustering results of the SOM algorithm have a potential relationship with the prediction effect of the forecasting model. The number of iterations of the SOM algorithm is set between 500 and 1000, and the number of times is debugged in hundred. The batch size is set to 15 or 30.


## 2.1. Clustering of the Original Data

In the clustering stage, the AC2 data from 2014 to 2016 is the training dataset and used for clustering. Considering the different numbers of days per month and the lack of PV data, we selected 15 days of complete data from each month and made each month AC2 data to a dataset asx1,x2,⋯,x15; the dataset indicates that a month’s AC2 data, x, contains 192 AC2 real-time data for one day. The clustering experiment uses two-year AC2 data. We merged the datasets with the same month label. The SOM algorithm is used to perform the clustering task. The raw data was clustered into 4 clusters.The SOM model structure is shown in Figure1. The learning steps of the SOM algorithm are as follows.Figure 1
Structure of the SOM model.In the first step, the weight vector corresponding to each neuron in the competition layer was initialized, normalizing the current input mode vectorX and the weight vector corresponding to the neuron. Secondly, the neuron node d with the smallest Euclidean distance between Xc (input vector) and ѡj (connection weight vector) is selected as the winning neuron node. As a result, the weights were updated accordingly. Following Equation (3), the weight of the winning neuron node d and other nodes in its domain is updated, where ηt is the learning rate of the t step in the range of (0, 1) and hd，jt is the neighborhood function. hd，jt generally uses a Gaussian function, as shown in Equation (4). The specific adjustment rules are as shown in Equations (5) and (6).
(2)Xc−wj=mindjc,(3)Wijt+1=Wijt+ηthd,jtXi−Wijt,(4)hd,jt=exp−ddj22r2t,(5)rt+1=INTrt−1×1−tT+1,(6)ηt+1=ηt−η0T,where the distance between the neurond and the neuron j is ddj2, rt is the neighborhood radius, INT is the rounding function, and T is the learning frequency.


## 2.2. Deep Learning Models

In the training stage, the 4 clusters of data are inputted into a model for training, and 4 forecasting models are trained. The training process follows the training method of time series. We display the training data of each category asy1,y2,y3,⋯,yn, where y represents AC2 data and n represents the number of AC2 data contained in each category. We set timestep i, y1,y2,⋯,yi, as a new training set, and yi+1 as the target value; then, we put the new training set into the training model, and the model parameters are adjusted during training so that the predicted output is constantly close to yi+1. Next, y2,y3,⋯,yi+1 is inputted into the training model and adjusts the model parameters so that the predicted output is continuously close to yi+2, and so on, until the target value is yn; end the model training and get the forecasting model.The training model structure of this experiment consists of two convolutional kernel layers of CNN, one layer of LSTM with attention mechanism, and a fully connected (FC) layer. The overall flowchart of the proposed framework is depicted in Figure2.Figure 2
The overall flowchart of the proposed algorithm.This proposed deep learning model (CNN-ALSTM) is a hybrid deep learning model to extract features from the raw data and perform foresting using the LSTM neural network. The CNN layer is employed to extract the useful features from the time series data, representing additional latent information of the data, which has the potential to improve the prediction accuracy. The experimental results show that the CNN layer contains one 163×1 convolution kernel layer and one 32 3×1 convolution kernel convolution layer that optimizes the prediction performance.The feature vector obtained from the second layer of CNN was inputted into the LSTM layer for prediction. Each element of the feature vector has correspondence to one of the 32 units in the LSTM layer. The attention mechanism puts higher weight to the feature quantities that are significantly related to the current output. Last, the output vector of the attention mechanism is processed by a FC layer using the unfolding operation. The predicted value of the AC2 at the next moment is output.LSTM is very suitable for prediction experiments of time series data. Existing works show superior forecasting performance combining CNN and LSTM for various applications [23, 27–29]. CNN helps LSTM to better extract the characteristics of experimental data. The attention mechanism is a process of assigning weights. Using the attention mechanism, more accurate weightage values are assigned to the LSTM output vector to improve the prediction capability of the model.The predictive power is obtained by the inverse normalization according to Equation (7).
(7)PrP=PrIac2∗Pmax−Pmin+Pmin,wherePrp is the predicted value of power and PrIac2 is the predicted value of AC2.


### 2.2.1. Long Short-Term Memory Neural Network

The LSTM model structure is shown in Figure3.Figure 3
Structure of LSTM.The appearance of LSTM cell structure effectively resolves the gradient explosion/vanishing problems. There are four important elements in the flowchart of the LSTM model: cell status, input gate, forget gate, and output gate. The input, forget, and output gates are used to control the update, maintenance, and deletion of information contained in cell status. The forward computation process can be denoted as(8)ft=σWf·ht−1,xt+bf,it=σWi·ht−1,xt+bi,Ot=σWO·ht−1,xt+bo,C~t=tanhWC·ht−1,xt+bC,Ct=ft·Ct−1+it·C~t,ht=Ot·tanhCt,whereWf, Wi, and Wo are the weight matrix of the forgetting gate, input gate, and output gate, respectively; bf, bi, and bo are the offset items of the forgetting gate, the input gate, and the output gate, respectively; σ is the sigmoid activation function; tanh is the hyperbolic tangent activation function.


### 2.2.2. The Attention Mechanism

The attention mechanism is a brain signal processing mechanism peculiar to human vision. Human vision quickly scans the global image to obtain the target area that needs attention and ignores other areas of useless information. The attention mechanism algorithm has been successfully implemented and applied to model training [26] and other related fields.The model proposed in this paper uses the LSTM hidden layer output vectorH=h1,h2,⋯,ht as the input of the attention mechanism, and the attention mechanism will find the attention weight αi of hi, which can be calculated as shown in
(9)ei=tanhWhhi+bh,αi=expei∑i=1texpei,whereWh is the weight matrix of hi and bh is the bias. The values of Wh and bh vary during the ALSTM training process. The range of ei is (-1,1). The attention vector H′=h1′,h2′,⋯,ht′ can be obtained by multiplying attention weight αi and hi:
(10)hi′=αi·hi.The attention mechanism is implemented as a custom layer where the parameters are optimized using RMSProp backpropagation [33, 34].


## 2.2.1. Long Short-Term Memory Neural Network

The LSTM model structure is shown in Figure3.Figure 3
Structure of LSTM.The appearance of LSTM cell structure effectively resolves the gradient explosion/vanishing problems. There are four important elements in the flowchart of the LSTM model: cell status, input gate, forget gate, and output gate. The input, forget, and output gates are used to control the update, maintenance, and deletion of information contained in cell status. The forward computation process can be denoted as(8)ft=σWf·ht−1,xt+bf,it=σWi·ht−1,xt+bi,Ot=σWO·ht−1,xt+bo,C~t=tanhWC·ht−1,xt+bC,Ct=ft·Ct−1+it·C~t,ht=Ot·tanhCt,whereWf, Wi, and Wo are the weight matrix of the forgetting gate, input gate, and output gate, respectively; bf, bi, and bo are the offset items of the forgetting gate, the input gate, and the output gate, respectively; σ is the sigmoid activation function; tanh is the hyperbolic tangent activation function.


## 2.2.2. The Attention Mechanism

The attention mechanism is a brain signal processing mechanism peculiar to human vision. Human vision quickly scans the global image to obtain the target area that needs attention and ignores other areas of useless information. The attention mechanism algorithm has been successfully implemented and applied to model training [26] and other related fields.The model proposed in this paper uses the LSTM hidden layer output vectorH=h1,h2,⋯,ht as the input of the attention mechanism, and the attention mechanism will find the attention weight αi of hi, which can be calculated as shown in
(9)ei=tanhWhhi+bh,αi=expei∑i=1texpei,whereWh is the weight matrix of hi and bh is the bias. The values of Wh and bh vary during the ALSTM training process. The range of ei is (-1,1). The attention vector H′=h1′,h2′,⋯,ht′ can be obtained by multiplying attention weight αi and hi:
(10)hi′=αi·hi.The attention mechanism is implemented as a custom layer where the parameters are optimized using RMSProp backpropagation [33, 34].


## 2.3. The Overall Forecasting Framework

There are four deep learning models trained in the proposed framework (Figure4). In the testing stage, an appropriate forecasting model is selected based on the grouping result of the testing data sample. The proposed hybrid deep learning model structure is shown in Figure 5. As a necessary preprocessing step, we used correlation coefficient to determine inputs for prediction. The weather data was employed by the SOM algorithm to cluster inputs into four categories. Each category is trained by the proposed hybrid deep learning framework. In particular, the CNN is used to extract the time series features of AC2 data; the LSTM neural network further extracts high latitude features in the data. The attention mechanism is used to assign different attention weights to the output elements of the LSTM hidden layer. In the testing phase, according to the month of the test set, we chose the appropriate model to predict the testing result (Figures 4 and 5).Figure 4
Structure of prediction stage. The numbers under test data and training data represent months.Figure 5
Structure of the proposed hybrid model.The process of data classification method determination and hyperparameter determination is also described, including the reason for simulating data in four seasons, the selection of clustering algorithm, the settings of SOM competition layer, and the selection of SOM classified result.


### 2.3.1. The Selection of Clustering Algorithm

The original idea of the clustering is to map the raw data into four different groups, representing the four seasons. These four groups of data are trained and tested using the proposed CNN-ALSTM structure separately. Different clustering algorithms are tested in the section. In the training phase, we analyze clustering algorithms, includingk-means, FCM, and SOM (Figure 6), for clustering the training dataset into four clusters. Based on the experimental results on the training data, SOM is more accurate than most of the above-mentioned traditional clustering algorithms (Figure 6) according to the original season labels of the raw data.Figure 6
Accuracy of different clustering algorithms in the training phase.The size of the SOM competition layer has been evaluated through a series of experiments as shown in Figure7. The X-axis represents the size of the competition layer, and the Y-axis represents the number of output neurons. The result shows that the most suitable competition layer for SOM is 2×2.Figure 7
Horizontally, theX-axis represents the size of the competition layer, and vertically, the Y-axis represents the number of output neurons.The clustering results of the SOM algorithm have a potential relationship with the prediction effect of the forecasting model. The number of iterations of the SOM algorithm is set between 500 and 1000, and the number of times is debugged in hundred. The batch size is set to 15 or 30.


## 2.3.1. The Selection of Clustering Algorithm

The original idea of the clustering is to map the raw data into four different groups, representing the four seasons. These four groups of data are trained and tested using the proposed CNN-ALSTM structure separately. Different clustering algorithms are tested in the section. In the training phase, we analyze clustering algorithms, includingk-means, FCM, and SOM (Figure 6), for clustering the training dataset into four clusters. Based on the experimental results on the training data, SOM is more accurate than most of the above-mentioned traditional clustering algorithms (Figure 6) according to the original season labels of the raw data.Figure 6
Accuracy of different clustering algorithms in the training phase.The size of the SOM competition layer has been evaluated through a series of experiments as shown in Figure7. The X-axis represents the size of the competition layer, and the Y-axis represents the number of output neurons. The result shows that the most suitable competition layer for SOM is 2×2.Figure 7
Horizontally, theX-axis represents the size of the competition layer, and vertically, the Y-axis represents the number of output neurons.The clustering results of the SOM algorithm have a potential relationship with the prediction effect of the forecasting model. The number of iterations of the SOM algorithm is set between 500 and 1000, and the number of times is debugged in hundred. The batch size is set to 15 or 30.


## 3. Experimental Process and Results

In this study, the average absolute percentage error MAPE, the mean absolute error MAE, and the root mean square error RMSE are used to evaluate the prediction ability of the model. The detailed equations of three error metrics are formulated in(11)MAPE=1n∑i=1n∣xmodel,i−xactual,i∣xactual,i,RMSE=1n∑i=1nxmodel,i−xactual,i2,MAE=1n∑i=1nxmodel,i−xactual,i.The experiment compares the proposed model, our previous work, that combines the LSTM and attention, together with the MLP and LSTM model alone. The most relevant factor obtained in this paper is AC2 as input to the training model. The resulting MAPE results are shown in Table2. The resulting RMSE results are shown in Table 3. The resulting MAE results are shown in Table 4.Table 2
Predicting MAPE results for one year of data.
2017.102017.112017.122018.12018.22018.32018.42018.52018.62018.72018.82018.9Overall7.5 minMLP22.1520.0723.0525.6218.1024.9726.0023.1928.3925.0822.1820.0323.11LSTM23.8721.4220.9628.7616.6328.9620.9931.1940.0220.2122.1821.7324.61ALSTM21.6422.4421.4428.1623.5224.5518.4317.8232.7925.4629.0519.7823.32Prop.19.9516.7620.7925.0116.9019.2915.8416.8317.2122.2019.8818.6918.8215 minMLP30.1924.4126.4228.5820.6337.3847.4040.8053.7038.3931.3027.1034.17LSTM23.4221.4723.7427.5117.7423.6222.6321.2128.9528.7219.0018.6422.75ALSTM20.3023.0822.4028.3820.4029.9629.6920.5368.6447.6024.0622.4928.82Prop.22.6920.3226.5128.0421.05528.1728.6125.1218.2325.4423.8720.8923.7930 minMLP33.0935.1536.9837.6828.4933.4428.8528.9230.1735.9232.7830.6532.57LSTM37.7541.6238.8638.4241.4536.0736.3430.0637.9039.0132.5940.9637.39ALSTM25.6729.8637.7642.7733.8930.4157.0128.6430.0930.7137.5035.7934.18Prop.32.6229.5929.3232.8823.5033.3834.8731.3445.6230.1931.8432.6232.39Table 3
Predicting RMSE results for one year of data.
2017.102017.112017.122018.12018.22018.32018.42018.52018.62018.72018.82018.9Overall7.5 minMLP1.391.582.092.161.231.070.630.720.691.251.101.451.32LSTM1.451.632.162.201.291.100.630.810.781.301.151.501.37ALSTM1.601.752.162.361.331.100.690.810.711.341.171.631.42Prop.1.441.522.082.321.231.100.630.650.701.361.241.401.3015 minMLP1.421.602.122.191.221.120.770.820.821.291.161.471.37LSTM1.631.942.512.441.451.160.690.830.731.561.171.611.53ALSTM1.541.902.382.491.741.190.710.770.961.611.251.691.55Prop.1.571.781.471.861.471.210.750.900.871.451.291.621.4030 minMLP2.232.742.913.291.841.560.941.221.101.961.832.032.02LSTM3.103.954.133.723.522.011.461.641.192.322.523.512.89ALSTM1.802.953.783.912.661.190.940.990.762.061.463.182.22Prop.2.262.292.953.121.711.921.000.940.961.711.692.712.04Table 4
Predicting MAE results for one year of data.
2017.102017.112017.122018.12018.22018.32018.42018.52018.62018.72018.82018.9Overall7.5 minMLP0.8290.881.151.360.700.610.390.420.390.720.670.830.72LSTM0.890.951.211.470.740.670.350.520.530.740.690.910.78ALSTM1.0491.121.221.560.840.620.390.450.440.830.771.050.83Prop.0.850.841.251.550.780.570.320.400.330.810.800.880.7015 minMLP0.910.921.171.380.690.730.580.580.610.840.750.910.82LSTM1.051.251.611.610.970.670.410.490.420.970.701.020.90ALSTM0.951.241.421.651.250.760.470.440.781.160.821.120.95Prop.1.0221.201.531.551.160.740.490.540.420.810.830.910.8530 minMLP1.582.022.092.291.351.030.550.730.571.261.301.471.32LSTM2.182.983.002.732.781.270.860.970.671.511.732.601.90ALSTM1.192.242.993.162.230.760.770.660.461.391.122.431.47Prop.1.641.662.072.291.301.390.670.640.681.101.232.131.38In the case of less training data, using the clustering phase and prediction phase mentioned in this paper will reduce the prediction accuracy. When the time interval is 30 min, the amount of data is reduced to a quarter of the data volume with a time interval of 7.5 min. From the comparison of different time intervals, when the interval is 30 min, the prediction accuracy drops. On the contrary, in the case of more training data, the prediction accuracy can be effectively improved.The experiment randomly selects two days of the month as a display of the predicted results. As shown in Figures8–10, according to the law of PV power generation, the power generation in the early evening to the early morning of the next day is 0, which is not shown in Figures 8–10.Figure 8
PV power forecast result in 7.5 min ahead. (a-1) and (a-2) are the January 5th and January 6th, (a-3) and (a-4) are the March 1st and March 3rd, (a-5) and (a-6) are the June 6th and June 8th, and (a-7) and (a-8) are the October 3rd and October 4th.In Figures8–10, (a-1) and (a-2), (b-1) and (b-2), and (c-1) and (c-2) are forecasting results of two days in winter; (a-3) and (a-4), (b-3) and (b-4), and (c-3) and (c-4) are forecasting results of two days in spring; (a-5) and (a-6), (b-5) and (b-6), and (c-5) and (c-6) are forecasting results of two days in summer; (a-7) and (a-8), (b-7) and (b-8), and (c-7) and (c-8) are forecasting results for two days in autumn. For winter, in Figure 10, deviations of the predicted power curve are found for 30 min ahead predictions. Satisfactory prediction performances in the winter season for 7.5 min ahead and 15 min ahead forecasts are achieved according to Figures 8 and 9.Figure 9
PV power forecast result in 15 min ahead. (b-1) and (b-2) are the January 14th and January 15th, (b-3) and (b-4) are the May 1st and May 2nd, (b-5) and (b-6) are the July 1st and July 2nd, and (b-7) and (b-8) are the October 3rd and October 4th.Figure 10
PV power forecast result in 30 min ahead. (c-1) and (c-2) are the January 9th and January 11th, (c-3) and (c-4) are the May 30th and May 31th, (c-5) and (c-6) are the August 2nd and July 9th, and (c-7) and (c-8) are the October 6th and October 12th.Through the experimental clustering, training, and prediction stage, the accuracy of the forecasting model has a significant outperformance over the compared models when the amount of training data is sufficient. The clustering stage divides the original PV data into four different clusters. And each clustered data is trained by a hybrid deep learning framework combining CNN and LSTM. The sophisticated deep learning framework obviously enhances the forecasting performance over singular methods, such as MLP and LSTM. The AM technique already shows a great influence of the forecasting results in the experiments with ALSTM. The performance improvement is further enlarged for the proposed methods. According to Tables2–4, the proposed model has a good performance on the RMSE indicator. Compared with the RMSE indicator, the MAE indicator reflects the actual situation of the prediction value error. The proposed model has the better performance than other models in the MAE indicator for intervals within 7.5 min. It is noted that the proposed model has significantly higher accuracy in the prediction of intervals within 7.5 min and has no obviously high accuracy compared with the other individual model in the prediction of intervals within 15 min and 30 min.


## 4. Conclusion and Discussion

Photovoltaic power generation prediction is of great significance for maintaining grid security and coordinating resource utilization. In the era of big data, it is possible for AI-driven IoT technology to perform accurate solar energy generation forecasting based on historical solar energy data [24, 30–32]. This paper proposes a hybrid deep learning method based on weather categories. Unlike traditional models, this experiment includes correlation analysis and clustering calculation of data, which effectively improves the generalization ability of the model and improves the prediction accuracy. The training algorithm employs the CNN algorithm, which can extract the data features more effectively and get more potential information in the data. Secondly, the attention mechanism is applied to the LSTM model, focusing on the extracted important features. In the prediction stage, the month of test set is used to determine the forecasting model, which improves the accuracy of the prediction. This forecasting model is superior to the traditional algorithm model in predicting performance, and it also shows outstanding results in predicting other data types.The experimental results shown in Section3 show that the proposed method outperforms the traditional time series data prediction methods, such as MLP, LSTM, and ALSTM for PV system energy generation forecasting. Three evaluation metrics are used to show the superior performance, including RMSE, MAPE, and MAE. According to the results collected in Tables 2–4, the proposed method achieves higher forecasting accuracy for short-term and very short-term forecasting, e.g., the 7.5 min advanced forecasting. For longer-term forecasting, such as time intervals of 15 and 30 min, the forecasting performance advantage decreases.One of the future works of this study is to extend the existing work for more generalized datasets, i.e., achieving acceptable forecasting results for longer-term forecasting of the PV system energy generation. Another future work direction is to apply the proposed framework towards a broader range of time series data applications in other fields, such as air quality forecasting [14, 35] and energy consumption forecasting [36].



---

*Source: 9249387-2021-06-19.xml*