Only refine the selected landmark