## 4 Brief Literature Survey on ANNs Applied to Time Series Modeling
### Combining Feedforward Neural Networks and Particle Swarm Optimization for Time Series Forecasting  
As we mentioned already briefly in Section 3.6, it is possible to train ANN models via evolutionary algorithms. The objective of evolutionary algorithms is to discover global solutions that are optimal and low cost. Evolutionary algorithms are usually based on various agents, such as chromosomes, particles, bees, ants, etc., searching iteratively to discover the global optimum or the local optimum (population-based algorithms) [36]. In 1975, [PERSON] [98] introduced the genetic algorithm (GA), which is considered the first evolutionary algorithm. As with any evolutionary algorithm, the GA is based on a metaphor from the theory of evolution. In the field of evolutionary computing, good solutions to a problem can be seen as individuals well-adapted to their environment. Although the GA has had many applications, it has been surpassed by other evolutionary algorithms, such as the PSO algorithm [99]. Today, due to its simplicity and ability to be used in a wide range of applications, the PSO algorithm has become one of the most well-known swarm intelligence algorithms [100]. [PERSON] and [PERSON] performed the first experiment using PSO to train ANN weights instead of using the more traditional backpropagation algorithm [101]. Several approaches have been proposed to apply PSO in ANN, such as the works published by [PERSON] and [PERSON] [102], [PERSON] and [PERSON] [103], and [PERSON], [PERSON], and [PERSON] [104]. In this section, we explore in a little more detail some possible ways we can pair an ANN model with the PSO algorithm to produce time series forecasts, but first, we will briefly describe how PSO works. In the words of its creators...  
_PSO is an optimization algorithm inspired by the motion of a bird flock; any member of the flock is called a \"particle\"_. ([PERSON] and [PERSON] [101], 1995)  
In the PSO algorithm, a particle moves through a real-valued dimensionality search space \(D\), guided by three attributes at each time (iteration) \(t\): position \(\vec{x}_{t}\), velocity \(\vec{v}_{t}\), and memory \(\vec{P}_{Best}\). In the beginning, the position of the particle is generated by a random variable with a uniform distribution, delimited in each dimension by the search space \(\vec{x}_{0}=U(lower,upper)\); thereby, its best visited position is set as equal to its initial position \(\vec{P}_{Best}=\vec{x}_{0}\), with an initial velocity \(\vec{v}_{0}=0\). After the first iteration, the attribute \(\vec{P}_{Best}\) remembers the best position visited by the particle based on an objective function \(f\); the other two attributes, \(\vec{v}_{t}\) and \(\vec{x}_{t}\), are updated according to Equations (15) and (16), respectively. The best of all the best particle positions \(\vec{P}_{Best}\) is called the global best \(\vec{G}_{Best}\). In each iteration of the algorithm, the swarm is inspected to update the best member. Whenever a member is found to improve the objective function of the current leader, that member becomes the new global best [105]. The objective of the PSO algorithm is to minimize function \(f:\mathbb{R}^{D}\rightarrow\mathbb{R}\); i.e., find \(\vec{a}\in\mathbb{R}^{D}\) such that \(f(\vec{a})\leq f(\vec{x})\) for all \(\vec{x}\) in the search space. In PSO, _variation_ (diversity) comes from two sources. The first is the difference between the current position of particle \(x_{t}\) and its memory \(P_{Best}\). The second is the current position of particle \(x_{t}\) and the global best \(G_{Best}\) (see Equation (15)). \[v_{t+1} = \omega\times v_{t}+c_{1}\times U(0,1)\times(P_{Best}-x_{t}) \tag{15}\] \[+ c_{2}\times U(0,1)\times(G_{Best}-x_{t})\] \[x_{t+1} = x_{t}+v_{t+1} \tag{16}\]  
Equation (15) reflects the three main elements of the PSO algorithm: the inertia path, local interaction, and neighborhood influence [106]. The inertial path is the previous velocity \(\omega*v_{t}\), where \(\omega\) is the inertial weight.