See net-gd.doc for details. Net-gd does not try to determine relative stepsizes itself, but stepsizes for groups of parameters can be set individually. The above command takes about 10 minutes on our machine. While waiting, you can monitor progress using net-plt. Once training has finished, we can make predictions based on the last network, which should have the lowest training error.
Performance is substantially worse than that obtained using Bayesian training.
Of course, we can't stop training based on the test error plotted above, since we don't know the test targets when training. Gradient descent with early stopping for the binary response problem. We can try to prevent overfitting by choosing one of the networks found during training according to performance on a subset of the available training cases that we have excluded from the set used for the gradient descent training. Note that to save computer time, one might wish to actually stop the training once it becomes apparent that further training is unlikely to find a better network, but that is not attempted here. Three net-gd commands are used above so that networks can be saved to the log file more frequently early in training.