Time series forecasting using dynamic particle swarm optimizer trained neural networks

Time series forecasting is a very important research area because of its practical application in many fields. Due to the importance of time series forecasting, much research effort has gone into the development of forecasting models and in improving prediction accuracies. The interest in using artificial neural networks (NNs) to model and forecast time series has been growing. The most popular type of NN is arguably the feedforward NN (FNN). FNNs have structures capable of learning static input-output mappings, suitable for prediction of non-linear stationary time series. To model nonstationary time series, recurrent NNs (RNNs) are often used. The recurrent/delayed connections in RNNs give the network dynamic properties to effectively handle temporal sequences. These recurent/delayed connections, however, increase the number of weights that are required to be optimized during training of the NN. Particle swarm optimization (PSO) is an efficient population based search algorithm based on the social dynamics of group interactions in bird flocks. Several studies have applied PSO to train NNs for time series forecasting, and the results indicated good performance on stationary time series, and poor performance on non-stationary and highly noisy time series. These studies have assumed static environments, making the original PSO, which was designed for static environments, unsuitable for training NNs for forecasting many real-world time series generated by non-stationary processes. In dealing with non-stationary data, modified versions of PSOs for optimization in dynamic environments are used. These dynamic PSOs are yet to be applied to train NNs on forecasting problems. The first part of this thesis formulates training of a FNN forecaster as a dynamic optimization problem, to investigate the application of a dynamic PSO algorithm to train FNNs in forecasting time series in non-stationary environments. For this purpose, a set of experiments were conducted on ten forecasting problems under nine different dynamic scenarios. Results obtained are compared to the results of FNNs trained using a standard PSO and resilient backpropagation (RPROP). The results show that the dynamic PSO algorithm outperform the PSO and RPROP algorithms. These findings highlight the potential of using dynamic PSO in training FNNs for real-world forecasting applications. The second part of the thesis tests the hypothesis that recurrent/delayed connections are not necessary if a dynamic PSO is used as the training algorithm. For this purpose, set of experiments were carried out on the same problems and under the same dynamic scenarios. Each experiment involves training a FNN using a dynamic PSO algorithm, and comparing the result to that obtained from four different types of RNNs (i.e. Elman NN, Jordan NN, Multi-Recurrent NN and Time Delay NN), each trained separately using RPROP, standard PSO and the dynamic PSO algorithm. The results show that the FNNs trained with the dynamic PSO significantly outperform all the RNNs trained using any of the algorithms considered. These findings show that recurrent/delayed connections are not necessary in NNs used for time series forecasting (for the time series considered in this study) as long as a dynamic PSO algorithm is used as the training method.