Package: django-synchro
Aim & purposeThis app is for synchronization of django objects between databases.It logs information about objects’ manipulations (additions, changes, deletions).
When synchronization is launched, all objects logged from the last checkpoint are synced to another database.Important note: This app doesn’t log detailed information about changes (e.g. which fields were updated),
just that such manipulation occured. When the synchronization is performed, the objects are synced with their newest, actual values.
(however, you can specify some fields to beskippedduring synchronization, seebelow).Example 1Consider scenario:there is one production project deployed on the weband the same project is deployed on some office computer in case of main server failureAssuming that the local database is regularly synced (eg. once a day the main database is exported and imported into the local system),
in case of a long main server downtime the staff may use the local project (inserting objects etc.).After the server is up again, the local changes (from the point of the last checkpoint) can be painlessly synchronized to the remote server.Example 2You can also synchronize databases both ways, not only in the slave-master model like in the previous example.However, it is probably better (if possible) to have a common database rather than to have
one for every project deployment and to perform synchronization between them.RequirementsThe app is tested to work with Django 1.7 - 1.11. If you want to use app in older versions of Django,
use the 0.6 release.The app needsdjango-dbsettingsto store the time of last synchronization.InstallationInstall app (note:django-dbsettingsis required and please view its install notes,
such ascache backendimportant remarks):$ pip install django-synchroor download it manually along with dependencies and put in python path.ConfigureDATABASES.AddsynchroanddbsettingstoINSTALLED_APPS.Specify in yoursettings.pywhat isremote databasename and which models should be watched and synchronized:SYNCHRO_REMOTE = 'remote'
SYNCHRO_MODELS = (
    'my_first_app', # all models from my_first_app
    ('my_second_app', 'model1', 'model2'), # only listed models (letter case doesn't matter)
    'my_third_app', # all models again
    'django.contrib.sites', # you may specify fully qualified name...
    'auth',                 # or just app label
)Later,REMOTEwill meanremote database.UsageSynchronizationJust invokesynchronizemanagement command:$ ./manage.py synchronizeAdmin synchro viewIn order to allow performing synchronization without shell access, you can use special admin view.Include in your urls:url(r'^synchro/', include('synchro.urls', 'synchro', 'synchro')),Then the view will be available at reversed url:synchro:synchro.The view provides two buttons: one to perform synchronization, and the other toreset checkpoint. If you would like to disable the reset button, setSYNCHRO_ALLOW_RESET = Falsein yoursettings.py.DebuggingIn order to track a cause of exception during synchronization, setSYNCHRO_DEBUG = True(andDEBUG = Trueas well) in yoursettings.pyand try to perform synchronization by admin view.SYNCHRO_REMOTEsettingGenerally,SYNCHRO_REMOTEsetting can behave in 3 different ways:The most naturally: it holds name ofREMOTEdatabase. Whensynchronizeis called,sychrowill
sync objects fromLOCALdatabase toREMOTEone.WhenSYNCHRO_REMOTEisNone: it means that noREMOTEis needed assynchrowill only store
logs (seebelow). It’s useful onREMOTEitself.WhenSYNCHRO_REMOTEis not specified at all, it behaves just like above (as if it wasNone), but
will show a RuntimeWarning.Remarks and featuresQuerySetupdateissueDjango-synchro logs information about objects modifications and later use it when asked for synchronization.The logging take place using thepost_saveandpost_deletesignal handlers.That means that actions which don’t emmit those signals (likeobjects.updatemethod) would result
in no log stored, hence no synchronization of actions’ objects.So, please remind: objects modified viaobjects.updatewon’t be synchronized unless some special code is prepared
(eg. callingsaveon all updated objects or manually invokingpost_savesignal).Natural keysFor efficient objects finding, it ishighly suggestedto providenatural_keyobject method
andget_by_natural_keymanager method.
This will allow easy finding whether the synchronized object exists inREMOTEand to prevent duplicating.Although addingnatural_keyto model definition is relatively quick, extending a manager may
require extra work in cases when the default manager is used:class MyManager(models.Manager):
    def get_by_natural_key(self, code, day):
        return self.get(code=code, day=day)

class MyModel(models.Model):
    ...
    objects = MyManager()
    def natural_key(self):
        return self.code, self.dayTo minimalize the effort of implementing a custom manager, a shortcut is provided:from synchro.core import NaturalManager

class MyModel(models.Model):
    ...
    objects = NaturalManager('code', 'day')
    def natural_key(self):
        return self.code, self.dayOr even easier (effect is exactly the same):from synchro.core import NaturalKeyModel

class MyModel(NaturalKeyModel):
    ...
    _natural_key = ('code', 'day')NaturalManagerextends the built-in Manager by default; you can change its superclass usingmanagerkeyword:from synchro.core import NaturalManager

class MyVeryCustomManager(models.Manager):
    ... # some mumbo-jumbo magic

class MyModel(models.Model):
    ...
    objects = NaturalManager('code', 'day', manager=MyVeryCustomManager)
    def natural_key(self):
        return self.code, self.dayWhen usingNaturalKeyModel,NaturalManagerwill extend the defined (objects) manager:from synchro.core import NaturalKeyModel

class MyVeryCustomManager(models.Manager):
    ... # some mumbo-jumbo magic

class MyModel(NaturalKeyModel):
    ...
    _natural_key = ('code', 'day')
    objects = MyVeryCustomManager()Side note: in fact invokingNaturalManagercreates a new class beingNaturalManager’s subclass.The purpose of a natural key is touniquelydistinguish among model instances;
however, there are situations where it is impossible. You can choose such fields that will causeget_by_natural_keyto find more than one object. In such a situation, it will raiseMultipleObjectsReturnedexception and the synchronization will fail.But you can tellNaturalManagerthat you are aware of such a situation and that it
should just take the first object found:class Person(models.Model):
    ...
    # combination of person name and city is not unique
    objects = NaturalManager('first_name', 'last_name', 'city', allow_many=True)
    def natural_key(self):
        return self.first_name, self.last_name, self.cityOr withNaturalKeyModel:class Person(NaturalKeyModel):
    ...
    # combination of person name and city is not unique
    _natural_key = ('first_name', 'last_name', 'city')
    _natural_manager_kwargs = {'allow_many': True}  # I know, it looks quite uglyDon’t useallow_manyunless you are completely sure what you are doing and what
you want to achieve.Side note: ifnatural_keyconsist of only one field, be sure to return a tuple anyway:class MyModel(models.Model):
    ...
    objects = NaturalManager('code')
    def natural_key(self):
        return self.code,  # comma makes it tupleOr to assign tuple inNaturalKeyModel:_natural_key = ('code',)Previously, there werenatural_managerfunction that was used instead ofNaturalManager- however, it’s deprecated.Skipping fieldsIf your model has some fields that should not be synchronized, like computed fields
(eg. field with payment balances, which is updated on every order save - inorder.post_savesignal),
you can exclude them from synchronization:class MyModel(models.Model):
    ...
    SYNCHRO_SKIP = ('balance',)When a new object is synchronized, all its skipped fields will be reset to default values onREMOTE.
Of course, theLOCALobject will stay untouched.Temporary logging disablingIf you don’t want to log some actions:from synchro.core import DisableSynchroLog

with DisableSynchroLog():
    mymodel.name = foo
    mymodel.save()Or, in a less robust way, with a decorator:from synchro.core import disable_synchro_log

@disable_synchro_log
def foo(mymodel):
    mymodel.name = foo
    mymodel.save()SignalsThat’s a harder part.If your signal handlers modify other objects, such an action will be probably reproduced twice:first, when the model will be updated onREMOTE, then normalREMOTEsignal handler will launchsecond time, because the original signal handler’s action was logged, the whole modified object will be synchronized;
this is probably undesirable.Consider a bad scenario:Initially databases are synced. There is an objectAin each of the databases.A.fooandA.barvalues are both 1.OnREMOTE, we changeA.footo 42 and save.OnLOCAL, we save objectX. In someXsignal handler,A.baris incremented.We perform synchronization:Xis synced.Xsignal handler is invoked onREMOTE, resulting inREMOTE’sA.barincrementation.
So far so good.REMOTE’sA.bar == 2andA.foo == 42, just like it should.BecauseAchange (during step 3) was logged,Ais synced.Not good-REMOTEvalue ofA.foowill be overwritten with 1
(becauseLOCALversion is considered newer, as it was saved later).It happened because the signal handler actions were logged.To prevent this from happening, wrap handler withDisableSynchroLog:@receiver(models.signals.post_delete, sender=Parcel)
def update_agent_balance_delete(sender, instance, *args, **kwargs):
    with DisableSynchroLog():
        instance.agent.balance -= float(instance.payment_left))
        instance.agent.save()Or with the decorator:@receiver(models.signals.post_delete, sender=Parcel)
@disable_synchro_log
def update_agent_balance_delete(sender, instance, *args, **kwargs):
    instance.agent.balance -= float(instance.payment_left))
    instance.agent.save()If using the decorator, be sure to place it after connecting to the signal, not before - otherwise it won’t work.Updateissue againOne can benefit from the fact thatobjects.updateis not logged and use it in signal handlers instead ofDisableSynchroLog.Signal handlers for multi-dbJust a reminder note.When a synchronization is performed, signal handlers are invoked for created/updated/deletedREMOTEobjects.
And those signals are of course handled on theLOCALmachine.That means: signal handlers (and probably other part of project code) must be ready to handle bothLOCALandREMOTEobjects. It must useusing(...)clause ordb_manager(...)to ensure that the proper database
is used:def reset_specials(sender, instance, *args, **kwargs):
    Offer.objects.db_manager(instance._state.db).filter(date__lt=instance.date).update(special=False)Plainobjects, withoutdb_managerorusing, always use thedefaultdatabase (which meansLOCAL).But that is normal in multi-db projects.Synchro onREMOTEand time comparingIf you wish only to synchronize one-way (always fromLOCALtoREMOTE), you may be tempted not to includesynchroinREMOTEINSTALLED_APPS.Yes, you can do that and you will save some resources - logs won’t be stored.But keepingsynchroactive onREMOTEis a better idea. It will pay at synchonization: the synchro will look
at logs and determine which object is newer. If theLOCALone is older, it won’t be synced.You probably should setSYNCHRO_REMOTE = NoneonREMOTEif no synchronizations will be
performed there (alternatively, you can add some dummy sqlite database toDATABASES).CheckpointsIf you wish to reset sychronization status (that is - delete logs and set checkpoint):from synchro.core import reset_synchro

reset_synchro()Or raw way of manually changing synchro checkpoint:from synchro.models import options

options.last_check = datetime.datetime.now()  # or any time you wishChangelog0.7(12/11/2017)Support Django 1.8 - 1.11Dropped support for Django 1.6 and olderBackward incompatibility:
you need to refactor allfrom synchro import …intofrom synchro.core import …0.6(27/12/2014)Support Django 1.7Fixed deprecation warnings0.5.2(29/07/2014)Fixed dangerous typoAdded ‘reset’ button to synchro view and SYNCHRO_ALLOW_RESET settingPrepared all texts for translationAdded PL, DE, FR, ES translationsAddedSYNCHRO_DEBUGsetting0.5.1(28/02/2013)Fixed a few issues with 0.5 release0.5(27/02/2013)Refactored code to be compatible with Django 1.5Required Django version increased from 1.3 to 1.4 (the code was already using some
1.4-specific functions)Removed deprecated natural_manager function0.4.2(18/10/2012)Fixed issue with app loading (thanks to Alexander Todorov for reporting)Added 1 test regarding the issue above0.4.1(23/09/2012)Fixed symmetrical m2m synchronizationAdded 1 test regarding the issue above0.4(16/09/2012)Deprecation: natural_manager function is deprecated. Use NaturalManager insteadRefactored NaturalManager class so that it plays well with models involved in m2m relationsRefactored NaturalManager class so that natural_manager function is unnecessaryAdded NaturalKeyModel base classFixed bug with m2m user-defined intermediary table synchronizationFixed bugs with m2m changes synchronizationAdded 3 tests regarding m2m aspects0.3.1(12/09/2012)SYNCHRO_REMOTEsetting is not required anymore.
Its lack will only blocksynchronizecommandAdded 2 tests regarding the change aboveUpdated README0.3(04/09/2012)Backward incompatible: ChangedReferencefields type fromIntegertoCharin
order to store non-numeric keysIncluded 24 testsRefactored NaturalManager class so that it is accessible and importableException is raised if class passed to natural_manager is not Manager subclassSwitched to dbsettings-bundled DateTimeValueUpdated README0.2(10/06/2012)Initial PyPI release0.1Local developmentAuthor:Jacek TomaszewskiThanks:to my wife for text correction