# A Deep Learning Model to Forecast Solar Irradiance Using a Sky Camera
## 3 Methods
### Deep Learning Model  
In this study, we used LSTM as a deep learning model to forecast the cloud cover up to several minutes ahead. LSTM was chosen as it provides satisfactory results in handling time series data. [PERSON] et al. [7] identified that this model performs better than other deep learning models, such as RNN and GRU. Furthermore, this deep learning model demonstrates good performance in solar irradiance forecasting in both short-term and long-term forecast horizons. LSTM was developed to overcome the problems with vanishing and explosion gradients that often occur in other deep learning models. For instance, in the case of the RNN, when these problems occurred in the learning process, the learning performance failed to increase [22]. The LSTM model was proposed by [PERSON] and [PERSON] to adapt to the long-term dependence on the information [23]. The unit, as has been illustrated in Figure 6, consists of forget gate, input gate, output gate, and cell state. For simplicity's sake, the structure can be formulated as follows:  
\[f_{t}=\sigma\Big{(}W_{f}\times x_{t}+U_{f}\times h_{t-1}+b_{f}\Big{)} \tag{11}\]  
\[i_{t}=\sigma(W_{i}\times x_{t}+U_{i}\times h_{t-1}+b_{i}) \tag{12}\]  
\[S_{t}=\tanh(W_{c}\times x_{t}+U_{c}\times h_{t-1}+b_{c}) \tag{13}\]  
\[C_{t}=i_{t}\odot S_{t}+f_{t}\odot C_{t-1} \tag{14}\]  
\[o_{t}=\sigma(W_{o}\times x_{t}+U_{o}\times h_{t-1}+V_{o}\times C_{t}+b_{o}) \tag{15}\]  
\[h_{t}=o_{t}\odot\tanh(C_{t}) \tag{16}\]  
where \(f_{t}\) is the forget gate, \(i\) is the input gate, \(o_{t}\) is the output gate, \(x_{t}\) is input data, \(b\) is bias, \(W\),\(U\),\(V\) are weight matrices, \(h_{t}\) is the value of memory cell, \(S_{t}\) is candidate state of the memory cell, \(\tanh\) and \(\sigma\) are the activation functions, and \(C_{t}\) is the state of the memory cell. In Equations (11), (12) and (15), the sigmoid function is used to calculate the amount of information that passes through the gate with values from 0 to 1. The candidate state of the memory cell in Equation (13) contains the \(\tanh\) function that makes the value ranging from \(-1\) to \(1\) in order to calculate the new information. The input and forgot gates in Equation (14) are operated with the Hadamard product \((\odot)\) to calculate the state memory of the cell. The final output of the memory cell in Equation (16) was obtained after multiplying with the output gate. A total of 12,000 images were processed to obtain minutely cloud cover information in this study. These calculated data provided the input for LSTM to forecast the future  
Figure 6: The architecture of the long short-term memory (LSTM). cloud cover. To illustrate the model design, the details of these hyperparameters are listed in Table 2.  
The optimum hyperparameters such as epochs, batch size, learning rate, optimization algorithm, and activation functions for deep learning models depend on the datasets. The epochs describe the number of complete passes (forward and backward) through the neural network. The batch size denotes the number of training examples in one pass (forward and backward). The learning rate controls the weight in a neural network with reference to loss gradient. The optimization algorithm is used to find the attributes, such as determining the weights neural network to reduce the losses. Along with this structure, this LSTM model is also equipped with an activation function known as a rectified linear unit (ReLU). ReLu is a non-linear function that oppresses a value below 0 to become exactly 0 but still inherits some linear property for cases above 0. Because this function has a linear characteristic, it can easily train by a deep network of neurons and also solve a case of gradient problem by ignoring the negative values. It should be noted that there is no fixed value in deep learning models to explain the optimum design for each model because the networks inside deep learning models are trained iteratively.