Separated url methods from Crawler. Refactored code in all classes.