## 2 Improved Deep Learning Model for Day-Ahead Solar Irradiance Forecasting
### Convolutional Neural Networks Based Local Feature Extractor
\[s_{ii+m-1}=s_{i}\oplus s_{i+1}\oplus s_{i+2}\oplus\cdots\oplus s_{i+m-1} \tag{6}\]  
in which \(\oplus\) is the concatenation operator, and \(s_{ii+m-1}\) denotes a window of \({}_{m}\) continuous time steps starting from the \(i\)-th time step. Moreover, the bias term \(b\in R\) should also be considered into the convolution operation. Thus, the final calculation equation is written as follows. \[c_{i}=f\big{(}\mathbf{w}_{j}{}^{\mathrm{T}}s_{ii+m-1}+b\big{)} \tag{7}\]  
in which \(\mathbf{w}_{j}{}^{\mathrm{T}}\) represents the transpose of a filter matrix \(\mathbf{w}_{j}\), and \(f\) is a nonlinear activation function. In addition, index \(i\) denotes the \(i\)-th time step, and index \(j\) is the \(j\)-th filter. The application of activation function aims to enhance the ability of models to learn more complex functions, which can further improve forecasting performance. Applying suitable activation function can not only accelerate the convergence rate but also improve the expression ability of model. Here, Rectified Linear Units (ReLu) are adopted in our model due to their superiority over other kinds of activation functions [51]. Pooling layer  
In the above subsection, the given example only introduces the detailed convolution operation process between one filter and the input sequence. In actual application, one filter can only generate one feature map. Generally, multiple filters are set in the convolution layer in order to better excavate the key features of input data. Just as assumed above, there are \(K\) filters with a window size of \(m\) in the convolutional layer. In Equations (5) and (7), each vector \(\mathbf{w}_{j}\) represents a filter, and the sing value \(c_{i}\) denotes the activation of the window. The convolution operation over the whole sequential input is implemented via sliding a filtering window from the beginning time step to the ending time step. So the feature map corresponding to that filter can be denoted in the form of a vector as follows. \[\mathbf{F}_{j}=[c_{1},c_{2},c_{3},\cdots,c_{L-m+1}] \tag{8}\]  
in which index \(j\) is the \(j\)-th filter, and the elements in \(\mathbf{F}_{j}\) corresponds to the multi-windows as \(\{s_{1:m},s_{2:m},\cdots,s_{l-m+1:L}\}\). The function of pooling is equal to subsampling as it subsamples the output of convolutional layer based on the definite pooling size \(p\). That means the pooling layer can effectively compress the length of feature map so as to further reduce the number of model parameters. Based on the max-pooling applied in our model, the compressed feature vector \(\mathbf{F}_{j-compress}\) can be obtained as follows. In addition, the max operation takes a max function over the \(p\) consecutive values in feature map \(\mathbf{F}_{j}\). \[\mathbf{F}_{j-compress}=[h_{1},h_{2},h_{3},\cdots,h_{\frac{L-m}{p}+1}] \tag{9}\]  
in which \(h_{j}=\max\Big{(}c_{(j-1)p},c_{(j-1)p+1},\cdots,c_{jp-1}\Big{)}\). In the application in our solar irradiance forecasting, the solar irradiance sequence input is a vector with only one dimension. The subsequences that are decomposed from the solar irradiance sequence are also a vector with only one dimension. Therefore, the size of the input subsequences in the convolution layer is \(n\times L\times 1\). \(n\) is the number of data samples and \(L\) is the length of the subsequences. The size of the corresponding outputs after the pooling layer is \(n\times((L-m)/\mathrm{p}+1)\times K\). It can be obviously noted that the length of the input sequence is compressed from \(L\) to \(((L-m)/p+1)\). In sum, the CNN based feature extractor can provide more representative and relevant information than the raw sequential input. Moreover, the compression of the input sequence's length also increases the capability of the subsequent LSTM models to capture temporal information. To give a brief illustration, the framework for the CNN-based local feature extractor is shown in Figure 4. Additionally, in the actual application, some important parameters need to be set according to the specific circumstances.