Indexer changes to reduce memory footprint and improve performance

These include:

  * Store batches of documents on the filesystem before sending to Solr

  * Don't send both 'fullrecord' and 'json'.  Use a Solr copyField to derive
    one from the other.

  * Convert data cleanup functions to use in-place string functions.

  * Use multiple indexing threads