Add DEFAULT_MAX_BULK to regulate the maximum number of documents that may be sent at a time
in bulk through a DocManager. pysolr does not break large numbers of documents into separate
add requests, and pushing many documents through it at once may result in MemoryError.
fix: avoid StopIteration in bulk_upsert in solr_doc_manager.py
Use a separate collection to store namespace and timestamp metadata when using Mongo DocManager
Avoid string concatenation in order to support unicode namespace names in Python 2
Uniform exception handling across all DocManagers
Make exit status more meaningful in mongo_connector.connector
add DocManagerBase

DocManagers inherit from DocManagerBase

add tests for new DocManager update method
OplogThread support for update operations using the update spec provided in oplog records
Remove tests for obsolete retrieve_doc, add test for filter_oplog_entry
Elasticsearch test refactorings

Changes:
 abstract connection behavior to ElasticsearchTestCase
 move _remove and _search methods to ElasticsearchTestCase
 remove index in tearDown to avoid test dependency

Related fixes:
 get_last_doc now runs on empty index correctly with latest ES
 test suite now always connects to the configured ES cluster
Avoid doing scroll queries when only interested in count
Fix test_stress method to actually test docs are inserted
Use the scan helper in _stream_search
Use streaming_bulk to avoid loading huge response lists into memory
Use a filter instead of query
Better support for update operations on sub-documents and arrays
Add integration tests for update
Support update operations that replace documents

Do a Solr commit if documents aren't available yet for update
added ns to entry for delete operations to fix elasticsearch urls
fix --unique-key throughout DocManagers
the mongo_doc_manager no longer supports --unique-key, since the unique key is always _id, and _id is an immutable field
Added the DocumentFormatter class to provide an extensible way to deal
with converting BSON types and modifying document structure, so that
external drivers are able to ingest documents.