It might therefore indicate that a neural network has a larger effective capacity compared to the classical classifiers. Another factor which influences the generalisation behaviour, is the architecture of a network. In this thesis the architecture is determined by the number of hidden units. If the number of hidden units is larger than needed, but not too large, the network will learn faster. This is due to the redundancy present in the network.
Redundancy causes the neurons to cluster.
At the start of a learning cycle, neurons fullfil approximately the same functions. As the training progresses, the neurons start to specialise. This specialisation, known as symmetry breaking, can be visualised using a projection technique which maps the high dimensional space in which the hidden units move onto a two dimensional plane. In this two dimensional plane the trajectories of the hidden units during training can be depicted in order to understand the training behaviour. Finally, the reliability of a neural network classification was studied.