Separate index names into content and auxiliary.

The page-traffic index is an example of an auxiliary index; it doesn't
hold content, but is used outside the main query (in this case, at
indexing time). We'll shortly be adding another such index to hold
best-bets.

Sitemaps should be made only from content indexes.  The search server
needs to know all the allowed names for its validation of index names
when opening indexes.

Conflicts:
	elasticsearch.yml
Add debugger in test & dev.
Fix tests for auxiliary indexes.
Whitespace.
Add `metasearch` index.

Holds information about search such as "best bets",
a mechanism for altering the results returned by
search.
Merge pull request #220 from alphagov/add-metasearch-index

Add metasearch index
Add mappings for the page_traffic index
Add a popularity field to the mainstream, government, and detailed guide schemas
Add popularity field to documents when indexing

This field is populated from the page-traffic index, which in turn is
populated from a dump file produced by the python scripts in the
search-analytics repository.

The popularity is currently defined here as 1.0 / the rank of the page
in terms of traffic.  ie, Our highest traffic page gets 1.0, the next
highest gets 0.5, the next highest gets 0.333.  This is a nice and
stable measure.
Use the popularity field when searching
Describe how to include popularity information in the search index.
If there's no query or sort, sort by popularity

Without a query, there's no relevance score, so unless an explicit order
is given the sort order is essentially undefined.  Use the popularity
field in this case to return the most popular items first, because this
is probably useful and almost free to do.
Separate index names into content and auxiliary.

The page-traffic index is an example of an auxiliary index; it doesn't
hold content, but is used outside the main query (in this case, at
indexing time). We'll shortly be adding another such index to hold
best-bets.

Sitemaps should be made only from content indexes.  The search server
needs to know all the allowed names for its validation of index names
when opening indexes.
Update Elasticsearch::Index tests to support popularity
Update the base query in the UnifiedSearcherTest
Update integration helper to support auxiliary indexes
Configure page_traffic in the test environment

This renames "page-traffic" to "page_traffic", so that we can adopt the 
convention of appending "_test" to the index in the test environment.

This also adds configuration for "page_traffic_test", and changes 
Elasticsearch::Index to automatically infer the traffic index based on 
matching against the auxiliary indexes that are present.
Pre-fill popularity data in integration tests

This changes the multi-index and bulk-load integration tests so that, 
before we insert or update any documents in the search index, we seed 
the page_traffic index with mock popularity data for each URL.
Clean page_traffic_test index before running tests
Don't blow up when no popularity data is present

When indexing, if popularity data is missing for the path we wish to 
index, make sure we return zero rather than trying to divide by it.