The latter will be considered in this chapter. The basic idea of pruning is to start training a network that is considered sufficiently big to ensure convergence. When convergence is reached certain weights neurons are removed, and the network is retrained. A Survey . The importance of the VC dimension is the source of a huge amount of research that is devoted to the evolution of appropriately sized networks.
All approaches consider a training set that has to be known in advance.
This problem does not occur in biological brains. The training growing cycle is repeated until a certain stopping criterion is met. The pruning approach starts with training a sufficiently large network. After the network has learned the given task, certain connections and or hidden units are removed. Most methods assign a relevance or salience to each connection unit and pruning is done with respect to these parameters.