As you visit the points-of-interest, make sure you have the quest open and visible on your phone so that it can register your arrival. In order for it to check off in your list of visited points, the quest must be open at the moment you visit that point. When the quest is open, you will see a message "You have arrived."  If you move off the quest in order to look at something else on your phone and are not on it when you visit that point, it will not register that you've been to that location.