it lxml.html.iterlinks is not good for us, what do you think about basing the new link extractor on top of scrapy selectors. Additionally it avoids the switch of the default parser, now lxml.etree.HtmlParser, to lxml.html.HtmlParser.