scrapy/scrapy — pull request #386 by redapple (2013-09-14 13:14:46+00:00)
Title: Add "method" and "with_tail" parameters to XPathSelector(List).extract()

When extacting text content, `hxs.select("//element/text()").extract()` usually works,
but I find myself preferring using something like `lxml.etree.tostring(element, method="text", encoding=unicode)`, especially when the text is nested in various `...` or `...` HTML elements.

I also think the `.../text()` step in XPath expressions (and in Scrapy doc examples) misleads users in thinking it will extract the flattened text content of a node, and not only the direct text child nodes (therefore omitting nested text() nodes)

Here I'm adding the `method` and `with_tail` parameters from `lxml.etree.tostring()` to `.extract()`

Note: I haven't handled them in `libxml2sel.py`