Updates the crawler to return absolute urls