The learning rates now decrease.