A simple crawl helper that will help connect an api with page links with the crawler of the actual pages, like for using with Kimono API