A robust parameters self-tuning learning algorithm for multilayer feedforward neural network
In this paper, a new and efficient adaptive-learning algorithm for multilayer feedforward neural networks is proposed. The main characteristic of this new algorithm is that learning parameters such as learning rate (?) and momentum (?) can be automatically adjusted according to the learning trajectory. Originally, the proposed algorithm was inspired by the use of the 1st order Taylor series expansion to approximate ?EP, the variation of the error function. Two conditions, ?EP 0 are considered first to ensure effective learning. To increase the accuracy of the ?EP approximation, we further developed a more robust procedure, namely the robust parameters self-tuning learning (RSTL). The key functions of the RSTL are: (1) ?wi are included in the performance index, (2) relationship between ? and ? is determined by the geometric approach, and (3) optimal ? and ? are obtained by using the optimizing technique. Computer simulations show that the proposed RSTL outperforms other algorithms both in converging speed and computing time. Additional advantages such as being insensitive to the initial weights and easy programming can also be illustrated during simulations.
parameters self-tuning learning multilayer feedforward neural network