# A Review on Neural Network Based Models for Short Term Solar Irradiance Forecasting
## 3 The Architecture of Deep Learning Algorithms
### Artificial Neural Network (ANN)  
The structure of the neural network is modeled on the human brain, and it has made significant contributions to the evolution of machine learning technologies. It is considered a simple mathematical model that can be used to address a variety of non-linear issues. Previous researchers reviewed ANN models in forecasting solar energy, such as Ref. [46] for solar radiation forecasting, Ref. [92] for PV applications, and Ref. [93] for solar energy forecasting. They concluded that the ANN model can offer higher accuracy for solar irradiation forecasting than other conventional models. Figure 3 shows the neural network architecture that consists of a multi-layer perceptron [94]. This multi-layer perceptron has aspects known as the input, hidden, and output layers. There are also auxiliary components that are labeled as neurons, weight, bias, and activation functions. These layers communicate with each other such that the input layer receives input values, which are then analyzed via the hidden layer. Next, the output layer gathers data from the hidden layer and decides how to use it. The training process of the neural network is iterative, and it modifies the structure during the learning process to achieve the exact reference or set point as the supervisor [95]. The network's multiple hidden layers allow it to forecast exact feature extraction and the non-linear structure of a model [96]. The findings of the ANN model are compared to those of the ARMA model (Auto Regressive Moving Average) and demonstrate how an ANN model can estimate global solar irradiation.  
Equation (9) is a basic mathematical function for ANN:  
\[An=\sum_{i=1}^{n}\left(w_{i}\;*\;I_{i}\right)+b \tag{9}\]  
where \(w_{i}\) represents the weight, \(I_{i}\) represents the input, \(A_{n}\) represents the result, \(b\) represents the bias, and \(n\) represents the number of inputs.  
The activation function is a mathematical equation that governs how neurons produce information, and it is interchangeably known as the transfer function. Hypothetically, it can influence the output value. The activation function types are linear and non-linear. For both input and output layers, a linear activation function produces the exact linear results. However, such a linear connection is insufficient for actual implementations, since problems involving complex information and several elements, which include image, video text, and sound, require more processing. The limits of the linear activation functions can be circumvented using a neural network with a non-linear activation function. Table 1 lists the most used activation functions of neural networks. The leaky ReLU and Rectified Linear Unit (ReLU) are non-linear activation functions, since their slope is not constant for all values. Positive values have a slope of 1, whereas negative values have a slope of 0 [94].  
\begin{table}
\begin{tabular}{c c c} \hline
**Activation Function** & **Equation** & **Plot** \\ \hline Linear & \(f(x)=x\) & \\ \hline Relu & \(f(x)=\max\left(0,x\right)\) & \\ \hline Leaky Relu & \(f(x)=\max(0.1\cdots x,x)\) & \\ \hline Tanh & \(f(x)=\tanh(x)\) & \\ \hline \end{tabular}
\end{table}
Table 1: The activation function of the neural network [94].  
Figure 3: The architecture of a neural network [94].