For related reports by Shavlik and Towell, follow this link . This paper concerns starting with rules to produce an initial backprop network, training it and then extracting a new set of rules that in this report turned out to be better than the original set of rules. The Ghostscript 2.6.1 interpreter conked out on page 44 and I have not yet tried any of the newer Ghostscripts to see if they can handle the situation. Other related papers are available via Rich Sutton's Publications page . This paper shows a series of experiments to try to improve backprop and finishes with quickprop which may be one of the best ways to speed up the training of a network.
Cascade correlation builds a network one hidden unit at a time and it is extremely fast.
It does not work very well with function approximation and there have been reports that a better version is under development that will work well on function approximation problems and be simpler as well. This is a version of cascade correlation designed to work with recurrent networks. This article contains a multitude of methods for improving training and generalization. One of them I approve of is their method for averaging outputs when you need to identify a time series with a recurrent network. However they used tanh in the context of a different error function and I have never tried exactly that.