now we only crawl each page once =^-^=