Removed overcloked rules for regular scraping. Removed unused code to check page response status. More refactoring for spider utils.