alphagov/rummager — pull request #226 by JordanHatch (2014-05-09 12:48:27+00:00)
Title: Use popularity data to improve search result weightings

This relates to https://www.pivotaltracker.com/story/show/66422732

This adds a new index, `page-traffic`, into which page traffic data about each URL on GOV.UK can be imported. For now, we have an initial dump of data we will import, but in a subsequent piece of work we will automate this to pull in data regularly.

Once the page traffic is present, we use it at index time to add a popularity score to each document. This is done for all documents in the index as part of the regular `rummager:migrate_index` Rake task. If a document has no page traffic data, we give it a default popularity score of zero.

When searching, popularity is now included as one of the weighting factors in sorting results returned. We're finding that this significantly improves the quality of search results.