Crawler and pages.json added.