Decrease workers because we hit the memory quota