Wikia/app — pull request #254 by relwell (2012-12-11 18:43:57+00:00)
Title: Update Backlinks extension to utilize atomic updates in Solr

This is some planning ahead for the upgrade to Solr 4.0. Once this is introduced, activating this extension will enable the follow workflow during parse:

* Any time a link is rendered in an article, that link's text, source, and target are stored
* When a page has completed parsing, it registers a job responsible for using the backlinks aggregating during parse to create Solarium_Document_AtomicUpdate instances
* These documents will have dynamic fields set that are uniquely identified by the source article's ID
* All such dynamic fields are copied into a general "backlink text" field, allowing us to use this for generic searching
* Because these dynamic fields use the document ID in its naming, we can actually use [Solr's JOIN capabilities](http://wiki.apache.org/solr/Join) to get some interesting information out of the index

An update to the backlinks maintenance script should allow us to drop all of the tables previously created for this project, as we are pushing this data directly into Solr via our Job application backend.

Before we can activate these changes, we need to be fully upgraded to Solr 4.0. We will also need to update our backend indexing component to use atomic updates, or we risk overwriting this data any time the page is reindexed.

This update also includes adding in unit tests to the Backlinks extension, where possible.