modified webcrawler method