mozilla/dxr — pull request #314 by erikrose (2014-08-20 18:09:54+00:00)
Title: [bug 1030311] Port query machinery to ES. Add a basic TextFilter to prov...

...e it works.

You can now run dxr-serve and do find-as-you-type string queries. Direct search is yet to come.

* Remove SQL profiling support from query machinery. Add support for path highlighting.
* Add TextFilter. Works for basic text searches. Still needs case sensitivity and negation.
* Delete a lot of dead code from extents module.
* Change highlight() (neé highlight_line()) to operate on unicode (which everything is now) and take overlapping and unordered extents (which happens when they're drawn from disparate plugins).
* Change the 'type' key of query terms to 'name' for consistency with the 'name' attr of Filter.
* Add description attr to Filter contract.
* Add field arg to Filter.highlight() to save redundant computation and make query module code shorter.
* Add abstract Filter class.

@willkg: Here's how to review it. Sorry you had to jump in at the deep end, and thanks for trading reviews with me!

Most of the tests are still supposed to fail, though `nosetests -s tests/test_strings.py:test_highlight` and the new `nosetests -s tests/test_core_plugin.py` should pass. (Most of the tests test filters that have yet to be rewritten for ES.)

Basically, I'm just looking to avert disaster in this review: did I add any gaping security holes? We don't have any PII or anything in DXR, but if somebody could bust through and read arbitrary files off the box, that would be bad. I don't see how they could, but you never know! :-)

I'm not happy with how I handled the ES pool (creating redundant ones and throwing them away) in the test harness and plan to fix that down the line. But right now, this patch blocks Marcell integration-testing his own code.

Thanks again, and sorry for the size of this commit.