Don't set the destination if you're already there