## 3 Case Study
### Model Training and Hyperparameters Selection  
In the DL based forecasting models, the mean square error (MSE) is chosen as loss function, and Adam Optimization is selected as an optimizer. During the deep learning training process, weight initialization and bias initialization play a vital role. Therefore, we choose the data from truncated normal distribution with 0 mean and 0.05 standard deviation as weight initialization method of CNN and fully connected layer. This method is the recommended initializer for neural network weights and filters. Orthogonal method, a popular initialization way, is selected as weight initializer for LSTM block. The bias for all hidden layers is set as 0.1. The learning rate is 0.001, the batch size is 24 and the epoch is 200.  
In addition, for two dataset, the numbers of training set and the testing set are different under four general weather types. The training set is used for training forecasting model, the testing set for evaluating forecasting result. All the above mentioned details of the division of training and testing sets, as well as parameter setting of DWT-CNN-LSTM model, are listed in Tables 1 and 2.  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Weather Types** & **Elizabeth City** & **Desert Rock Station** \\ \hline \multirow{3}{*}{Sunny type} & The number of training set: [288] & The number of training set: [412] \\  & The number of validation set: [32] & The number of validation set [46] \\  & The number of testing set: [80] & The number of testing set: [115] \\ \hline \multirow{3}{*}{Cloudy type} & The number of training set: [504] & The number of training set: [230] \\  & The number of validation set [56] & The number of validation set[25] \\  & The number of testing set: [140] & The number of testing set:[65] \\ \hline \multirow{3}{*}{Rainy type} & The number of training set: [366] & The number of training set: [147] \\  & The number of validation set [40] & The number of validation set [14] \\  & The number of testing set: [100] & The number of testing set: [40] \\ \hline \multirow{3}{*}{Heavy Rainy type} & The number of training set: [153] & The number of training set: [72] \\  & The number of validation set [16] & The number of validation set [10] \\ \cline{1-1}  & The number of testing set: [42] & The number of testing set: [20] \\ \hline \hline \end{tabular}
\end{table}
Table 1: The division detail of samples sets under four general weather types.  
We set the split proportion of training set, validation set and testing set as 0.7:0.1:0.2. The training set is used to train the solar irradiance forecasting models. The validation set is used to adjust the hyper-parameters of these DL forecasting models. The testing set is used to verify the model performance.  
For the proposed model, we first design two CNN layers with 64 filters, and the filter size and pooling size are both set to 3. Then, two LSTM layers are connected to CNN output with 100 neurons. The outputs of LSTM are fed into two fully connected layers with linear activation function. The Relu activation function is applied to CNN and LSTM layers. To overcome the overfitting problems in models, dropout method with 0.2 parameter is applied after CNN and LSTM layers. In addition, early stopping method is also applied. In addition, the output data format of the input layer, each intermediate layer, and the output layer are accordingly shown in Table 3. Additionally, Table 4 illustrates the structure of the other forecasting models used as benchmarks.