Package: historical-collection
historical_collectionEasily keep track of changes to Mongo document changes with a collection.ConceptsHistoricalCollectionAnHistoricalCollectionbehaves just like a regular MongoDBCollection, but adds additional fields and methods to apply
patching.PatchingThis is most likely something you will not need to worry about since the
logic is taken care of. However…APatchis a set of changes. A patch is associated with aDocumentof anHistoricalCollection.RevisionARevisionis a state of a document with a patch applied. Only the
baseRevisionis stored in the document table. Note (forhistorical_collectiondevelopers):Revisionis more of a concept
than an actual object.You won’t find yourself actively creatingRevisions (remember,
you’re creatingPatches). They’re only retrieved.ChangeJust like a patch, this is most likely something you don’t need to worry
about. Changes are calculated for you.A change consists of a dict of actions. The actions are one of:INITIAL(character keyI)ADD(character keyA)REMOVE(character keyR)UPDATE(character keyU)INITIALactions do not not have any attributes. TheDocumentstored in the associatedCollectionis the all the information
needed. TheINITIALis stored as a revision for any metadata
associated with the patch.ADDandUPDATEactions take a Pythondictof keys mapped to
values. Obviously any of thePK_FIELDs will not be included in
this.REMOVEtakes a list of document keys to be removed. Just likeADDandUPDATE, noPK_FIELDs are in this list.RequirementsBasic RequirementsPython 3.6 or higherMongoDBpymongoOptionally, you may wantpipand to run this in a virtual
environment.Requirements For Development or TestingFakerInstallationTo install from PIPuser@host~# pip3 install -U historical_collectionOr clone the repostitory and execute:user@host~# python3 setup.py installUsageExtend the CollectionIn order to keep track of changes to a document, extend
HistoricalCollection.from historical_collection.historical import HistoricalCollection
from pymongo import MongoClient
class Users(HistoricalCollection):
    PK_FIELDS = ['username', ]  # <<= This is the only requirementThe only requirement is thePK_FIELDSattribute that specifies the
primary keys of the document. If omitted, Python will complain. This is
so that any incoming document can be seen as “the same.”It’s recommended tonotuse the_idfield unless you have a valid
reason, and if you have a mechanism in place to keep track of the_id. Otherwise, the changes will most likely be ignored.Perhaps an example of how to patch will make it more clear.Connect to a Mongo Database InstanceCLIENT_URL = "mongodb://localhost:27017/"
DATABASE = "historical_collection_example"
mongo = MongoClient(CLIENT_URL)
db = mongo[DATABASE]

mongo.drop_database(db)

users = Users(database=db)There’s a lot going on under the hood with the lineusers = Users(database=db). We’re also creating adeltascollection with the format__deltas_User. Usually you will not need
to access the deltas collection, but if you do, then you can always
access it with<collection_instance>._deltas_collection:users._deltas_collectionCollection(Database(MongoClient(host=['localhost:27017'], document_class=dict, tz_aware=False, connect=True), 'historical_collection_example'), '__deltas_Users')Patching DocumentsLet’s add an initial user to your document. You’re probably already
familiar withCollection.insert_one()andCollection.insert_many(). Well,HistoricalCollectionhas 2
additional methods for inserting:HistoricalCollection.patch_one()HistoricalCollection.patch_many()They behave similarly toinsert_oneandinsert_manywith one
major difference: Only the firstDocumentis inserted. Additional
documents have deltas generated and stored in the_deltas_collectioncollection.Let’s patch our first document.users.patch_one({"email": "darth_later@example2.com"})---------------------------------------------------------------------------

KeyError                                  Traceback (most recent call last)

~/projects/historical_collection/historical_collection/historical.py in _document_filter(self, document)
    134         try:
--> 135             return dict([(k, document[k]) for k in self.PK_FIELDS])
    136         except KeyError as e:


~/projects/historical_collection/historical_collection/historical.py in <listcomp>(.0)
    134         try:
--> 135             return dict([(k, document[k]) for k in self.PK_FIELDS])
    136         except KeyError as e:


KeyError: 'username'


During handling of the above exception, another exception occurred:


KeyError                                  Traceback (most recent call last)

<ipython-input-4-03f5bb018ec2> in <module>
----> 1 users.patch_one({"email": "darth_later@example2.com"})


~/projects/historical_collection/historical_collection/historical.py in patch_one(self, *args, **kwargs)
    258         doc = args[0]
    259         metadata = kwargs.pop("metadata", None)
--> 260         fltr = self._document_filter(doc)
    261         latest = self.latest(fltr)
    262         insert_result = None


~/projects/historical_collection/historical_collection/historical.py in _document_filter(self, document)
    138                 raise KeyError(
    139                     "Perhaps you forgot to include {} in projection?".format(
--> 140                         self.PK_FIELDS
    141                     )
    142                 )


KeyError: "Perhaps you forgot to include ['username'] in projection?"Whoopsie! That’s right! We need to include theusernamefield!users.patch_one({"username": "darth_later", "email": "darthlater@example.com"})
users.find_one({"username": "darth_later"}){'_id': ObjectId('5d98c3385d8edadaf0bb845b'),
 'username': 'darth_later',
 'email': 'darthlater@example.com'}Okay, now let’s patch it! For starters let’s simply add a field.users.patch_one({"username": "darth_later", "email": "darthlater@example.com", "laser_sword_color": "red"})[]One Important Thing to Note:We need to keep everything from thepreviousexample, in that, we must include theusernamefield
(otherwise, theUserscollection will not finddarth_vader) and
theemail(otherwise, this will be seen as aREMOVE-al).users.find_one({"username": "darth_later"}){'_id': ObjectId('5d98c3385d8edadaf0bb845b'),
 'username': 'darth_later',
 'email': 'darthlater@example.com'}What? What happened? We patcheddarth_vader, didn’t we?Yes, we did. So the first (and only)Documentstored in theUsersDocumentis the first one. But we do have several
revisions. These can be retrieved with therevisions()function.
This behaves just likefind_all()for a standardCollection.list(users.revisions({"username": "darth_later"}))[{'_id': ObjectId('5d98c3385d8edadaf0bb845b'),
  'username': 'darth_later',
  'email': 'darthlater@example.com',
  '_revision_metadata': None},
 {'_id': ObjectId('5d98c3385d8edadaf0bb845b'),
  'username': 'darth_later',
  'email': 'darthlater@example.com',
  '_revision_metadata': None,
  'laser_sword_color': 'red'}]There we go! There’s the revision we were looking for! This may be
annoying, though to get all revisions when you most likely just want the
latest one. That’s why there’s alatest()method to make it easy.users.latest({"username": "darth_later"}){'_id': ObjectId('5d98c3385d8edadaf0bb845b'),
 'username': 'darth_later',
 'email': 'darthlater@example.com',
 '_revision_metadata': None,
 'laser_sword_color': 'red'}Note that this assumes one document. If you want the latest revision ofseveraldocuments, usefind_latest()list(users.find_latest({"username": "darth_later"}))[{'_id': ObjectId('5d98c3385d8edadaf0bb845b'),
  'username': 'darth_later',
  'email': 'darthlater@example.com',
  '_revision_metadata': None,
  'laser_sword_color': 'red'}]Those curious may have noticed a_revision_metadataelement in the
document. That’s added byHistoricalCollectionin the_deltas_collectionfor any additional data that you want to
associate with the document. Timestamps are an excellent usage case.Let’s start over with no users to show an example.mongo.drop_database(DATABASE)from datetime import datetime
from time import sleep
import random

SWORD_COLORS='red blue orange green transparent'.split(' ')

for i in range(0, 5):
    timestamp = datetime.now()
    laser_sword_color = random.choice(SWORD_COLORS)
    document = {"username": "darth_later", "laser_sword_color": laser_sword_color}
    metadata = {"timestamp": timestamp}
    users.patch_one(document, metadata=metadata)
    sleep(1)

list(users.revisions({"username": "darth_later"}))[{'_id': ObjectId('5d98c3435d8edadaf0bb845e'),
  'username': 'darth_later',
  'laser_sword_color': 'green',
  '_revision_metadata': {'timestamp': datetime.datetime(2019, 10, 5, 9, 22, 27, 994000)}},
 {'_id': ObjectId('5d98c3435d8edadaf0bb845e'),
  'username': 'darth_later',
  'laser_sword_color': 'orange',
  '_revision_metadata': {'timestamp': datetime.datetime(2019, 10, 5, 9, 22, 29, 26000)}},
 {'_id': ObjectId('5d98c3435d8edadaf0bb845e'),
  'username': 'darth_later',
  'laser_sword_color': 'blue',
  '_revision_metadata': {'timestamp': datetime.datetime(2019, 10, 5, 9, 22, 30, 29000)}},
 {'_id': ObjectId('5d98c3435d8edadaf0bb845e'),
  'username': 'darth_later',
  'laser_sword_color': 'blue',
  '_revision_metadata': {'timestamp': datetime.datetime(2019, 10, 5, 9, 22, 31, 31000)}},
 {'_id': ObjectId('5d98c3435d8edadaf0bb845e'),
  'username': 'darth_later',
  'laser_sword_color': 'green',
  '_revision_metadata': {'timestamp': datetime.datetime(2019, 10, 5, 9, 22, 32, 33000)}}]