alphagov/need-o-tron — pull request #8 by jystewart (2013-01-18 12:47:46+00:00)
Title: Elasticsearch

Ports need-o-tron from solr to elasticsearch so that we're no longer dependent on solr and it's easier to run tests in the development environment.

We've been having trouble maintaining this app because tests didn't run consistently in the standard dev environment. A move to elasticsearch solves that because the search configuration can be contained (almost) entirely within the app, and we're supporting elasticsearch better across the board. If this is merged and deployed we can remove solr from our stack.

The code left after these changes is far from pretty. Some of the abstractions put in place to interface with solr don't make much sense in the elasticsearch world but I wanted to focus on getting something working and tests passing before we consider whether to spend more time tidying up the codebase. I've also used the tire gem which does the job and let me focus on the task at hand. When considering further work on need-o-tron we should think about whether it makes more sense to bring this in-line with everything else and use rummager and/or whether better abstractions in the app itself will remove the need for tire.

I did end up doing a little extra work supporting some gem changes which came in when I ran some other updates and seemed best to work with rather than separate out.

**This pull request is not ready to merge** Sorting is not working properly yet. When I enable it there are errors of the form

```500 : {"error":"SearchPhaseExecutionException[Failed to execute phase [query_fetch], total failure; shardFailures {[TIrT1TeEQvGByF2aKePQeQ][test-needs][0]: QueryPhaseExecutionException[[test-needs][0]: query[ConstantScore(NotDeleted(*:*))],from[0],size[50],sort[]: Query Failed [Failed to execute main query]]; nested: IOException[Can't sort on string types with more than one value per doc, or more than one token per field]; }]","status":500}```

I intend to work on resolving that before this is reviewed, but wanted to get the changes shared asap as this has lived on my laptop for far too long.