## 5 Forecasting
### Forecasting Model  
We model the forecasting problem based on multi-layer perceptrons (MLP). An MLP is a feed-forward artificial neural network (ANN), fully connected between layers. A learning algorithm adjusts the weights and biases of the network's neurons using a loss function and a training algorithm. The learning algorithm achieves these adjustments by exposing the network to every sample feature vector and its target. ANN are computing systems vaguely inspired by the biological neural networks that constitute animal brains [24]. Figure 11 shows the basic structure of an artificial neuron. A neuron \(n\) can receive multiple inputs \([x_{1},x_{2},\ldots,x_{m}]\) which are weighed individually \([w_{1n},w_{2n},\ldots,w_{mn}]\) to make an intermediate output \(a_{n}\). That is  
\[a_{n}=\sum_{i=1}^{m}w_{i}x_{i}+b_{n} \tag{10}\]  
where \(b_{n}\) represents a bias, which acts as a weight. This bias acts as an independent term in an inherently non-linear model. The output \(a_{n}\) is passed to a \(\varphi(\cdot)\) activation function which yields the output \(y_{n}\) of the neuron. The neuron computes its output as  
\[y_{n}=\varphi(a_{n}) \tag{11}\]  
By piling many neurons operating in parallel, it is possible to form a layer. ANNs usually consist of one input layer of length \(m\), one output layer (with one or more outputs), and several hidden layers. Those hidden layers can also have a varying number of neurons [25]. Figure 12 shows a typical ANN topology. Figure 11: Artificial neuron. Generally speaking, an ANN is a universal approximator. Theorem 1 states the universal approximation theorem [26]. Theorem 1: _Let \(\varphi(\cdot)\) be a non-constant, bounded, and monotonically increasing continuous function. Let \(I_{m_{0}}\) denote the \(m_{0}\)-dimensional unit hypercube \([0,1]^{m_{0}}\). The space of continuous functions on \(I_{m_{0}}\) is denoted by \(C(I_{m_{0}})\). Then, given any function \(f\
i C(I_{m_{0}})\) and \(\epsilon>0\), there exists an integer \(m_{1}\) and sets of real constants \(\alpha_{i}\), \(b_{i}\), and \(w_{ij}\), where \(i=1,\ldots,m_{1}\) and \(j=1,\ldots,m_{0}\) such that we may define_  
\[F(x_{1},\ldots,x_{m_{0}})=\sum_{i=0}^{m_{1}}\alpha_{i}\varphi\left(\sum_{j=1}^ {m_{0}}w_{ij}x_{j}+b_{i}\right) \tag{12}\]  
_as an approximate realization of the function \(f(\cdot)\); that is,_  
\[\mid F(x_{1},\ldots,x_{m_{0}})-f(x_{1},\ldots,x_{m_{0}})\mid<\epsilon\]  
_for all \(x_{1},x_{2},\ldots,x_{m_{0}}\) that lie in the input space._  
In time-series forecasting, an ANN can identify patterns and trends in the data and adjust the weights of each neuron to match the desired output. ANNs are successful models used in time-series forecasting; research work exhibits their forecasting capabilities. Those forecasting models include applications on forecasting water demand [27], foreign exchange markets [28], market volatility [29], and more. For this work, we designed a simple MLP; Table 3 describes the components of the MLP architecture. \begin{table}
\begin{tabular}{l r} \hline
**Component** & **Value** \\ \hline Input length & \(50\) \\ Hidden layers length & \(60,60\) \\ Output layer length & \(168\) \\ Neuron activation function & \(f(x)=\max(0,x)\) \\ Weight optimizer & Adam \\ Maximum number of epochs & \(1000\) \\ Early stopping & True \\ \hline \end{tabular}
\end{table} 
Table 3: MLP Architecture. Figure 12: Artificial neural network. To produce the forecasting models, we took the last 5000 values of every time series, representing a little more than six months of data. This value was set since, after preprocessing all the time series, we found that many types of equipment had regime changes around that time. To minimize the modeling time, we implemented a simple parallel computing technique. The models were trained on a cluster server with 40 computing threads. Each thread was in charge of producing, for a given PQM, the models of the individual variables.