Fix #15862 remove dirs in keep list when walk path

Does python complain of the list being changed while iterating?