Henceforth we will consider the case of a single input and output layer. The goal of learning is to adjust the weights on the connections among the units so as to allow the network to produce the target output pattern in response to the given input pattern. Weights changes are based on calculating the derivative of the error in the network's output with respect to each weight. Training is organized into a series of epochs. In each epoch, the network is exposed to a set of events, often the entire set of events that comprise the training environment.
Training begins by initializing the weights and biases to small random values.
The process of learning then begins, and continues for some number of epochs or until a performance criterion is reached. We now consider in each of the three phases involved in processing each pattern.