Package: django-pdr
PDR (Pantograph Database Replicator)Flexible Database replication system compatible with most SQL databasesThis README is intended to be viewdon githubsome links and updates might not reflect correctly in the PyPI website.Table of ContentsAbout The ProjectFeaturesBuilt WithQuick StartRoadmapContributingLicenseContactAcknowledgementsAbout The ProjectPDR is a standalone Django app for establishing a reliable, high-performance, cross-platform, live mirror of any table[s] in an SQL database server to any other SQL database server.Pantograph database replication is event driven, not time driven, ie. it doesn’t update the whole mirror database periodically; Instead, it updates each individual group of affected records in the affected table in the mirror at the moment it's modified in the original database. Which results in a more speedy instant update, yet consumes less bandwidth and server resources.
Pantograph database replication is also perfectly reliable because the server listening for changes doesn’t has to be online at the moment changes occur. Instead, the host server creates notification channels where changes are stacked awaiting the listening server’s action to read notification, and update its replications in remote servers.A list of commonly used resources that I find helpful are listed in the acknowledgements.FeaturesCan replicate data to multiple tables in separate databases.Can replicate data into a new structure, yet, keep it live.Can be used with most RDBMS. Currently supports Microsoft SQL, PostgreSQL, MySQL, MariaSQL, SQLite, FirebirdSQL.Very reliable as PDR events are created by native SQL trigger functions inside source databases.High in performance as it’s event-driven.One-way data replication can be used to join data from multiple different tables in multiple different databases into one live table.Provides a simple easy to use GUI to configure replications.Distributed as a standalone Django app wrapped in a PyPi package so you can easily integrate it with any existing Django Project.Built WithThe project is based on Django, the popular python framework, SQLAlchemy, A wonderful open-source SQL abstraction layer for the Python programing language, and native SQL trigger functions.Quick startRunpip install django-pdrto install django_pdr's python package from the python package indexAdd "django_pdr" and "django_ace" to your INSTALLED_APPS setting like this:INSTALLED_APPS=[...'django_ace','django_pdr',]Include the admin URLs anywhere in your project urls.py like this:fromdjango.contribimportadminurlpatterns=[...path('admin/',admin.site.urls),]Runpython manage.py makemigrationsand thenpython manage.py migrateto create django_pdr's data models.Start the development server and head over to the Django admin website. You will find that Django_pdr's data models has been added.Start by adding all source and destination databases you want to replicate data from/to. To add a database, first select theSourceRDBMS type. Once you select any type, it will show a placeholder JSON connectionConfigedit this JSON code with the configuration of your database. And then fill in the rest of the form and save it.Once you add your databases, you can set any table from any database as a SourceTable. To define a table as a SourceTable, first select the database where the table exists, and Django_pdr will automatically list all tables in that database. Select one and save.Now that you have defined SourceTables, it's time to create Reflections. A Reflection defines how any changes in records in a source table should reflect in another table. Any SourceTable can have as many Reflections as you want, and Reflections are very flexible since they can reflect data into a whole new structure, in an entirely different database system, yet keep it up to date.
The first step to add a reflection is by defining the SourceTable and then the destination database, and finally the destination table. Once you do so, it automatically generates default Reflection configuration represented in theDestination FieldsJSON object and theReflection statement. Initially, theDestination FieldsJSON is the same as theSource FieldsJSON. Which defines the columns of the table and the primary key. However, you can customize the JSON of theDestination Fieldsand the SQLReflection statementto change which fields you want to be replicated and how exactly do you expect them to be replicated.In another terminal window, run the commandpython manage.py pdr-reflect -i 1to start the PDR background activity. The pdr-reflect management command checks if there are any pending changes in any source table if it finds any, it reflects them into their destination Reflection or Reflections. the-ioption is the interval in seconds, during which the server checks if there are any pending changes. Setting this interval to a small value can take more from the network bandwidth in case of setting too many reflections. Setting it to a great value can result in a delay in replication, however, all changes will be reflected sooner or later.RoadmapSee theopen issuesfor a list of proposed features (and known issues).ContributingContributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make aregreatly appreciated.Fork the ProjectCreate your Feature Branch (git checkout -b feature/AmazingFeature)Commit your Changes (git commit -m 'Add some AmazingFeature')Push to the Branch (git push origin feature/AmazingFeature)Open a Pull RequestLicenseDistributed under the Apache 2 License. SeeLICENSEfor more information.ContactMeena (Menas) Erian -hi@menas.proProject Link:https://github.com/meena-erian/pdrAcknowledgementsPythonDjangoSQLAlchemyDjango ACEpsycopg2pytzDjango pgcrypto