the network to learn the bigger the gradient the bigger the adjustments and vice versa here's where the problem lies when doing back propagation each node in a layer calculates its gradient with respect to the effects of the gradients and the layer before so the adjustments in the layer before it is small then the adjustments in the current layer will be even smaller this cost gradients to exponentially shrink as it back propagates down the earlier layers failed to do any learning as the internal weights are barely being