## 4 Brief Literature Survey on ANNs Applied to Time Series Modeling
### Combining Feedforward Neural Networks and Particle Swarm Optimization for Time Series Forecasting
#### 4.1.2 Particle Swarm Optimization Convergence
As a result of this study, the velocity equation changes to  
\[v_{t+1}=\chi*\{v_{t}+c_{1}*U(0,1)*(P_{Best}-x_{t})+c_{2}*U(0,1)*(G_{Best}-x_{t})\} \tag{18}\]  
where \(\chi\) is the constriction coefficient calculated as  
\[\chi=\frac{2*\kappa}{|2-\phi-\sqrt{\phi^{2}-4\phi}|} \tag{19}\]  
with \(\phi\geq 4\) and \(0\leq\kappa\leq 1\). The constant \(\kappa\) controls the rate of convergence. For \(\kappa\approx 0\), faster convergence to a stable point is achieved, and for \(\kappa\approx 1\), slow convergence to a stable point is obtained [107]. In 2010, [PERSON] and [PERSON] [135] formally demonstrated that the original PSO is not a local or global optimizer. They identified an imperfection in PSO and addressed it in their approach called guaranteed convergence PSO (GCPSO). The goal of the GCPSO is to update only the speed of the best particle in the swarm (\(\tau\)) to  
\[v_{\tau,t+1}=-x_{\tau,t}+G_{Best}+\omega*v_{\tau,t}+\rho_{t}*(1-2*\gamma_{t}) \tag{20}\]  
Substituting Equation (20) into Equation (16), we obtain  
\[x_{\tau,t+1}=G_{Best}+\omega*v_{\tau,t}+\rho_{t}*(1-2*\gamma_{t}) \tag{21}\]  
Equation (21) has three terms: the first term introduces a direct relationship with the current global best position, the second term conveys the inertia of the global best particle, and the third term shows a point of a uniform distribution in a hypercube with side lengths \(2*\rho\); \(\rho\) is strictly greater than zero. The authors indicate that their proposal ensures that the best global particle never stops moving completely. [PERSON] and [PERSON] [135] showed that the GCPSO is a local optimizer. In 2012, [PERSON] and [PERSON] [136] demonstrated the existence and uniqueness of the convergence position in PSO, using the theorem of Banach space and the contraction mapping principle. They gave the parameter condition that influences the stability of PSO and showed that, if the parameter satisfies this condition, the probability that the particle swarm optimization converges to the best position is one. In 2018, [PERSON] and [PERSON] [137] proposed an improved PSO (IPSO) algorithm according to the following strategy. They introduced a Gaussian perturbation in the \(P_{Best}\) position to guarantee that IPSO converges to the \(\epsilon\)-optimum solution with probability one for any \(\epsilon\). Also in 2018, [PERSON] and [PERSON] [138] defined the swarm state sequence and examined its Markov properties according to the theory of PSO. Subsequently, from the evolutionary sequence of the particle swarm with the best fitness value, the authors derived a supermartingale. Based on this result, the authors applied the supermartingale convergence theorem to analyze the convergence of the PSO. The results show that PSO reaches the global optimum in probability. Recently, [PERSON], [PERSON], and [PERSON] [139] established PSO convergence to a global minimizer based on continuous time modeling for a non-convex and non-smooth objective function of particle dynamics through a system of stochastic differential equations. In summary, there are several works that prove the convergence of PSO. However, it may be interesting to carry out a study of the convergence properties of PSO to optimize ANNs. A Study on the Ability of Support Vector Regression and Feedforward Neural Networks to Forecast Basic Time Series Patterns  
According to [PERSON], [PERSON], and [PERSON] (2018), \"Support Vector Regression (SVR) and Feed Forward Neural Networks (FFNNs) have found increasing consideration in forecasting theory, leading to successful applications in time series forecasting for various domains, often outperforming conventional statistical approaches of ARIMA -or exponential smoothing- methods. Despite their theoretical and practical capabilities, FFNN and SVR models are not established forecasting methods. Substantial theoretical criticism on FFNNs has raised skepticism regarding their ability to forecast even simple time series patterns of seasonality or trends without prior data preprocessing ([PERSON] et al., 2018)\".