scrapy/scrapy — pull request #471 by redapple (2013-11-23 14:53:02+00:00)
Title: Support compiled XPath expressions and CSS selectors

`lxml` supports compiled XPath expressions (http://lxml.de/xpathxslt.html#the-xpath-class)

> The XPath class compiles an XPath expression into a callable function (...)
> The compilation takes as much time as in the xpath() method, but it is done only once per class instantiation. This makes it especially efficient for repeated evaluation of the same XPath expression.

This PR adds support for compiled selectors in `Selector` through new `XPath` and `CSS` classes.

`XPath` and `CSS` instances are therefore also suitable in `ItemLoader`'s `.add/replace/get_xpath()` and `.add/replace/get_css()`

There's also a refactoring of `FormRequest` to using `lxml.etree.XPath` where it makes sense.

I haven't done any performance comparison tests.
Some benchmarking results: http://lxml.de/performance.html#xpath

There are probably other code areas where compiled selectors make sense. Have you spotted any?