A method for programming a set of non-volatile storage elements using a multi-phase programming operation which includes one phase (FOGGY) and a next phase (FINE), the method comprising, for non-volatile storage elements of a target word line (WLn) in the set of non-volatile storage elements: performing the one phase (FOGGY), including: (a) distinguishing faster-programming non-volatile storage elements from slower-programming non-volatile storage elements, and (b) responsive to the distinguishing: (i) continuing programming of at least some of the faster-programming non-volatile storage elements, while setting an associated bit line voltage to a first non-zero bit line level to impose a programming speed-based slow down measure, (ii) continuing programming of at least some of the slower-programming non-volatile storage elements, while setting an associated bit line voltage to a second non-zero bit line level which is less than the first non-zero bit line level, and (iii) setting a common control gate voltage (Vsgd) of drain side select gates associated with the non-volatile storage elements at a first control gate level; raising an associated bit line voltage to a lockout level, which is greater than the first bit line level, for the at least some of the faster-programming non-volatile storage elements and the at least some of the slower-programming non-volatile storage elements which reach an associated verify level in the one phase; subsequently performing the next phase (FINE), including continuing programming of the faster and slower non-volatile storage elements while setting associated bit line voltages to a level which is less than the first bit line level, and setting the common control gate voltage of the select gates at a second control gate level, below the first control gate level; and raising an associated bit line voltage to the lockout level for the at least some of the faster-programming non-volatile storage elements and the at least some of the slower-programming non-volatile storage elements which reach an associated verify level in the next phase. performing the one phase (FOGGY), including: (a) distinguishing faster-programming non-volatile storage elements from slower-programming non-volatile storage elements, and (b) responsive to the distinguishing: (i) continuing programming of at least some of the faster-programming non-volatile storage elements, while setting an associated bit line voltage to a first non-zero bit line level to impose a programming speed-based slow down measure, (ii) continuing programming of at least some of the slower-programming non-volatile storage elements, while setting an associated bit line voltage to a second non-zero bit line level which is less than the first non-zero bit line level, and (iii) setting a common control gate voltage (Vsgd) of drain side select gates associated with the non-volatile storage elements at a first control gate level; raising an associated bit line voltage to a lockout level, which is greater than the first bit line level, for the at least some of the faster-programming non-volatile storage elements and the at least some of the slower-programming non-volatile storage elements which reach an associated verify level in the one phase; subsequently performing the next phase (FINE), including continuing programming of the faster and slower non-volatile storage elements while setting associated bit line voltages to a level which is less than the first bit line level, and setting the common control gate voltage of the select gates at a second control gate level, below the first control gate level; and raising an associated bit line voltage to the lockout level for the at least some of the faster-programming non-volatile storage elements and the at least some of the slower-programming non-volatile storage elements which reach an associated verify level in the next phase.