Momentum is a technique used to speed up the training process. It adds a fraction of the previous gradient to the current gradient, so that the model can move more smoothly towards the minimum of the loss function.