Package: xdump
XDumpXDump is a utility to make a consistent partial dump and load it into the database.The idea is to provide an ability to specify what to include in the dump via SQL queries.InstallationXDump can be obtained withpip:$ pip install xdumpUsage exampleMake a dump (on production replica for example):>>>fromxdump.postgresqlimportPostgreSQLBackend>>>>>>backend=PostgreSQLBackend(dbname='app_db',user='prod',password='pass',host='127.0.0.1',port='5432')>>>backend.dump('/path/to/dump.zip',full_tables=['groups'],partial_tables={'employees':'SELECT * FROM employees ORDER BY id DESC LIMIT 2'})Load a dump on your local machine:>>>backend=PostgreSQLBackend(dbname='app_db',user='local',password='pass',host='127.0.0.1',port='5432')# If you need a clear DB>>>backend.recreate_database()# or `backend.truncate()`>>>backend.load('/path/to/dump.zip')Dump is compressed by default. Compression level could be changed with passingcompressionargument todumpmethod.
Valid options arezipfile.ZIP_STORED,zipfile.ZIP_DEFLATED,zipfile.ZIP_BZIP2andzipfile.ZIP_LZMA.The verbosity of the output could be customized viaverbosity(with values 0, 1 or 2) argument of a backend class.There are two options to control the content of the dump:dump_schema- controls if the schema should be includeddump_data- controls if the data should be includedAutomatic selection of related objectsYou don’t have to specify all queries for related objects - XDump will load them for you automatically. It covers
both, recursive and non-recursive relations.
For example, if theemployeestable has foreign keysgroup_id(togroupstable) andmanager_id(toemployeestable) the resulting dump will have all objects related to selected employees
(as well as for objects related to related objects, recursively).Command Line Interfacexloadprovides an ability to create a dump.Signature:xdump[postgres|sqlite][OPTIONS]Common options:-o, --output TEXT               output file name  [required]
-f, --full TEXT                 table name to be fully dumped. Could be used
                                multiple times
-p, --partial TEXT              partial tables specification in a form
                                "table_name:select SQL". Could be used
                                multiple times
-c, --compression [deflated|stored|bzip2|lzma]
                                dump compression level
--schema / --no-schema          include / exclude the schema from the dump
--data / --no-data              include / exclude the data from the dump
-D, --dbname TEXT               database to work with  [required]
-v, --verbosity                 verbosity levelPostgreSQL-specific options:-U, --user TEXT                 connect as specified database user
                                [required]
-W, --password TEXT             password for the DB connection
-H, --host TEXT                 database server host or socket directory
-P, --port TEXT                 database server port numberxloadloads a dump into a database.Signature:xload[postgres|sqlite][OPTIONS]Common options:-i, --input TEXT                input file name  [required]
-m, --cleanup-method [recreate|truncate]
                                method of DB cleaning up
-D, --dbname TEXT               database to work with  [required]
-v, --verbosity                 verbosity levelPostgreSQL-specific options are the same as forxdump.RDBMS supportAt the moment only the following are supported:PostgreSQLSQLite >= 3.8.3Django supportAddxdump.extra.djangoto yourINSTALLED_APPSsettings:INSTALLED_APPS=[...,'xdump.extra.django',]AddXDUMPto your project settings file. It should contain minimum two entries:FULL_TABLES - a list of tables that should be fully dumped.PARTIAL_TABLES - a dictionary withtable_name:select SQLXDUMP={'FULL_TABLES':['groups'],'PARTIAL_TABLES':{'employees':'SELECT * FROM employees WHERE id > 100'}}Optionally you could use a custom backend:XDUMP={...,'BACKEND':'importable.string',}Runxdumpcommand:$ ./manage.py xdump dump.zipRunxloadcommand:$ ./manage.py xload dump.zipPossible options to both commands:-a/--alias- allows you to choose database config fromDATABASES, that is used during the execution;-b/--backend- importable string, that leads to custom dump backend class.Options forxdumpcommand:-s/--dump-schema- controls if the schema should be included;-d/--dump-data- controls if the data should be included.Options forxloadcommand:-m/--cleanup-method- optionally re-creates DB or truncates the data.NOTE. If the dump has no schema inside, DB won’t be re-created.The followingmakecommand could be useful to get a configured dump from production to your local machine:sync-production:ssh-t$(TARGET)"DJANGO_SETTINGS_MODULE=settings.production /path/to/manage.py xdump /tmp/dump.zip"scp$(TARGET):/tmp/dump.zip./dump.zipssh-t$(TARGET)"rm /tmp/dump.zip"DJANGO_SETTINGS_MODULE=settings.local$(PYTHON)manage.pyxload./dump.zipAnd the usage is:$makesync-productionTARGET=john@production.comPYTHON=/path/to/python/in/venvPython supportXDump supports Python 2.7, 3.4 - 3.7 and PyPy 2 & 3.