A crawler used to download specify resources of a website. It also can test all resources are ok in a website.