intergrate the crawler