Another training parameter, momentum, helps prevent oscillation when higher learning rates are used to find solutions similar to those found with lower learning rates, without extending the training time. With a unique error surface for each application, learning rates must be adjusted during training to avoid oscillation while optimizing the tradeoff between training time and overall error reduction. Many neural network tools offer other training parameters that can be adjusted during the training process including temperature, gain, and noise. Due to the number of training parameters, preprocessing methods, and architectural configurations that can be explored, an automated training and testing regime that integrates testing with training is absolutely necessary to effectively apply this technology to financial market analysis. Tools including genetic algorithms and simulated annealing can be used to expedite searching these parameter spaces.
Genetic algorithms are effective for many parameter optimization tasks.
Simulated annealing automates learning rate adjustments during training by including a variable temperature term that affects the learning rate. When the temperature is high, learning is rapid. When the temperature drops, learning slows down as the network settles upon a solution. Figure 4 shows an example of simulated annealing, in which the step size is reduced to avoid oscillation while finding a minimum point on the error surface. Overtraining, analogous to curve fitting of rule-based trading systems, is one of the major pitfalls that must be avoided when developing neural networks.