refactor the large scanUrl method into some smaller ones