Whitespace.
Provide metasearch (best_bets) mappings for tests.

Removes unused `elasticsearch_schema.fixture.yml`.
Add debugger for integration tests.
Pass type to search to Index.raw_search

Allow the document type to be passed to raw_search, so that we can just
search for, say, documents of type best_bet.
Add metasearch_index_name to the config
Add BestBetsChecker to look up best bets

This fetches best bets from the metasearch index, and merges multiple
best bets appropriately.
Tell test helper about metasearch helper
Raise a clearer error for invalid index names
Look up and use best + worst bets at search time

Add metasearch index to app.  Pass it through UnifiedSearcher to
UnifiedSearchBuilder.  Use a BestBetsChecker to look up bets from
UnifiedSearchBuilder, and build a query which obeys the bets if there
are any.
Add disable_best_bets debug option

This adds a generic debug parameter, which accepts a comma separated
list of options. There are plenty of options we can add in future, such
as providing access to elasticsearches explain API so we can understand
the weighting better.  For now though, just add an option
"disable_best_bets" which disables the use of best bets for query
ranking.
Use a custom analyzer for the stemmed best bets

We want an analyzer that does stemming, but not stop wording or
synonyms.
Fix stemmed best-bet matching.

Somewhat annoyingly, I can't find any way to build an elasticsearch
index + query to perform the best bet checking entirely in elasticsearch
(at least, not without involving large and quadratically growing numbers
of terms in the query, or other nasties).

The problem is that we want to have best bets which are returned
whenever the user's query contains (after stemming and other
elasticsearch analyzer normalization of words) an "stemmed" style best
bet as an exact substring.  For example, the best bet "car tax" should
be matches if the user's query is "car tax" or "car taxes" or "paying
car tax", but not if the user's query is "car" or "tax" or "tax car" or
"car rental tax".

We implement this by introducing a new field "stemmed_query_as_term",
which is populated at index time for best bet documents of type
"stemmed".

If a document is a best bet, and is using the stemmed_query field, we
populate the stemmed_query_as_term field with a processed version of the
stemmed_query.  This produces a representation of the best-bet query
with all words stemmed and lowercased, and joined with a single space.
We use a call to elasticsearch's analyzer to do this processing.

At search time, we process the query similarly, and all best bets with
at least one word in common with the user's query are fetched.  For each
of these the stemmed_query_as_term field of each is checked to see if it
is a substring match for the (similarly normalised) user's query.  If
so, the best bet is used.

There's some tech debt building up in the index.rb file; we should build
a more generic way to modify documents as they're being indexed, since
we now have two special cases there: one for adding the popularity
information, and one for this best-bet modification.  I think this
modification-at-index-time is fine, but we should pull it out
separately.  To do this cleanly requires passing more configuration
about index structures into the indexing code, and I'd like to do it in
a separate pull request given that other branches (eg, the best bet
deletion branch) are already backing up waiting for this one to be
completed.
Add best-bet integration test
Tidy up index creation and cleanup in tests

Use create_test_indexes and clean_test_indexes instead of a variety of
calls to create or delete individual indexes.

This fixes some occasional fails; some tests were only passing if other
tests ran first and left "metasearch-test" indexes around.  After this
commit, the tests should all create the indexes they need, and clean up
properly after themselves, so failures should be more deterministic.
Rename stub_best_bets to stub_zero_best_bets

This makes it clearer that this sets up the stubs as if there are no
best bets in action.
Rename variables in tests for clarity
Add unit test of search builder without best bets
Merge pull request #233 from alphagov/use_best_bets

Use best bets