# ANN Sizing Procedure for the Day-Ahead Output Power Forecast of a PV Plant
## 3 Methodology
### Artificial Neural Network Size  
The ANN sizing problem primarily means to choose how many neurons are in the layers, and their layout. The neurons (or units) in the input and output layers are fixed by the problem to be solved, while characterizing the neurons in the hidden layers is controversial, since the hidden neurons are regarded as the processing neurons in the network. In addition, having a small number of hidden layers might increase the speed of the training process, whereas a large number of hidden layers could make it longer. Furthermore, the nonlinearity capability of an ANN increases with the number of layers; i.e., the more complicated (nonlinear) the input/output relationship is, the more layers an ANN will need to model it. In addition, the greater the number of neurons per layer, the more accurately can ANNs identify input/output relationships. In order to guarantee a sufficient generalization capability for the ANN, the number of neurons is reasonably bound to the training set size. Though, from the literature, only some general conditions and intuitive rules could be inferred. In [PERSON]'s review [21], it is stated that the number of weights \(N_{w}\) existing within a layered ANN is bound to the number of patterns \(N_{p}\) in the training set and outputs \(N_{o}\) by the following condition:  
\[N_{p}>>N_{w}/N_{o}. \tag{2}\]  
Condition (2) means that a given ANN should have an amount of neurons that is sufficiently smaller than the training samples. In fact, if the degrees of freedom (i.e., \(N_{w}\)) are larger than the constraints associated with the desired response function (i.e., \(N_{o}\) and \(N_{p}\)), the training procedure will be unable to completely constrain the weights in the network. In the mid-1960s, [PERSON] studied the capacity of an FFNN with an arbitrary number of layers and a single output element [22; 23]. In the same line, [PERSON] and [PERSON] [21] addressed the question of the net size, which provides a valid generalization stating, with a desired accuracy of 90%, that at least 10 times as many training examples as there are weights in the ANN should be considered. In [24], the authors also provide theoretical lower and upper bounds on the sample size as a function of the net size, such that a valid generalization can be expected. However, they limited their study under the assumption that the node functions are linear threshold functions (or at least Boolean valued), leaving open the problem for classes of real valued functions (such as sigmoid functions), and multiple hidden layers networks. To characterize the number of neurons in the layers, there are different rules suggested in the literature, starting from two incremental algorithms ([25] and the reference therein), passing through a trial and error procedure [26], to some thumb rules. For example, [PERSON] et al. [27] applied--in the same forecasting context of this work--a thumb rule found in [28] in the \"backpropagation architecture--standard connections\" section. There, it is stated that the default number of hidden neurons \(N_{n}\) for a three layer network is computed with the following formula:  
\[N_{n}=\frac{(i+o)}{2}+\sqrt{N_{p}} \tag{3}\]  
where \(i\) is the number of inputs; and \(o\) is the number of outputs. For more layers, it is stated that \(N_{n}\) should be divided by the number of hidden layers. Finally, there is a symmetrical technique from the already mentioned tiling algorithm [29]. In the first, a small number of neurons is selected. Then, they are increased gradually. The symmetrical approach is referred to as the \"pruning\" algorithm [30]. This method, starting from a large network, carries on a gradual removal of the neurons by erasing the less significant units, and it requires in advance the largest size of the network [31].