For crawling website, given a list of domains