Fixed a bug that stopped the crawl function from indexing linked pages