The crawl function takes an optional allowed_domains parameter. If not provided, it defaults to the domains of the URLs in url_list. When the crawler goes through the pages of example.com, it follows links to discover pages. If it finds pages on help.exmaple.com it won’t crawl them (it’s a different domain). The solution, therefore, is to provide a list of domains to the allowed_domains parameter. Make sure you also include the original domain, in this case example.com.