Package: django-pg-copy
Django PostgreSQL CopyThe packagedjango-pg-copyprovides Django management commands for backing up and restoring PostgreSQL databases. These were developed for copying production databases to development, to allow developers to share images with one another, or bring local development databases up to date. It can also be handy for creating different local databases for different branches, and for only creating one migration after tweaking models to get them the way they need to be.We also use it with Jenkins to automatically back up production, and restore to a staging database environment, so we can test new migrations repeatedly to ensure they'll work when we run them in production.Installationpip install django-pg-copyThen add'pg_copy',to yourINSTALLED_APPSlist. It is recommended that it is used in all environments (development, production) so that you can use it against different instances of your database.SettingsPG_COPY_BACKUP_PATH = 'db_backup'By default, PostgreSQL backups will be stored in a directory calleddb_backupwhere you run the command. This setting will override that location.It is also recommended to add this path to your.gitignorefile, if the path falls under your version control repository.Parameters--database: The database defined in the DATABASES settings to backup from or restore to.--db-override: A value to override the db argument sent to psql.--host-override: A value to override the host argument sent to psql.--pg-home: The path to the PostgreSQL installation, if it is not on your path.--file,-f: The filename to backup to, or restore from.--jobs,-j: The number of parallel jobs to run. This candrasticallyincrease the speed of the backup and restore. When backing up, this must use the--directoryoption.Be careful!This will also create multiple database connections and can slow your database down.--directory,-d: Restore from a directory. Overrides--filewhen used.Backup Only Parameters forpg_backup--ignore-table,-i: Excludes the table completely during the backup process. Can pass multiple tables:-i bigtable1 -i bigtable2--exclude-table-data,-e: Excludes the table data during the backup process. Can pass multiple tables:-e bigtable1 -e bigtable2Restore Only Parameters forpg_restore--no-confirm: Restores the database without confirmation: be careful! DON'T OVERWRITE YOUR PROD DATABASE! (Ask me how I know.)Example Commandspython manage.py pg_backupThis command will create a backup file in the directory./db_backup/(or the directory you specified withPG_COPY_BACKUP_PATH) called[timestamp].sqlcusing thedefaultsettings fromDATABASESusing the default Django settings file resolved bymanage.py.python manage.py pg_backup --settings=config.settings.production --database=default --filename=my_backup.sqlcThis command will create a backup in the same directory asmanage.pycalledmy_backup.sqlcusing thedefaultsettings fromDATABASESusing the Django settings file located atconfig/settings/production.py.python manage.py pg_backup --directory=/tmp/pg_backup --jobs=8This command will create a backup directory at/tmp/backupuse 8 parallel jobs and database connections.python manage.py pg_backup -i bigtable1 -i bigtable2This will do the same as the previous command, but omit the tables namedbigtable1andbigtable2.python manage.py pg_backup -e bigtable1 -e bigtable2This will do the same as the previous command, but include the table structure without any data for the tables namedbigtable1andbigtable2.python manage.py pg_restoreThis command will provide a list of backup files inPG_COPY_BACKUP_PATHthat can be restored. After selecting a backup file, it will confirm that the user wants to overwrite the destination database by showing which server and database will be overwritten from the settings. Here's what it will look like:(venv)[django-project]$./manage.pypg_restore
Thereare8backupfilesin'/var/dev/username/django-project/db_backup'.Whichwouldyouliketorestore?1:2020-05-27-13-33-38.sqlc2:2020-05-10-09-07-45.sqlc3:2020-05-11-13-05-49.sqlc4:2020-07-14-15-21-22.sqlc5:2020-05-15-08-31-59.sqlc6:2020-06-11-13-11-00.sqlc7:2020-06-02-13-28-09.sqlc8:2020-05-27-14-54-15.sqlc
Enternumberofthefiletorestore:4Abouttorestore'django_project_db'onhost'localhost'fromthefile:'/var/dev/username/django-project/db_backup/2020-07-14-15-21-22.sqlc'.
THISWILLOVERWRITETHEDATABASE.
Type"yes"tostarttherestore[no]:yespython manage.py pg_restore --filename=my_file.sqlc --no-confirmThis command will read the filemy_file.sqlcandskip confirmationthat the user wants to overwrite the destination database.python manage.py pg_restore --directory=/tmp/pg_backup --jobs=8This command will restore from the directory/tmp/pg_backupusing 8 parallel jobs and database connections.Known IssuesErrors Displayed Duringpg_restoreWhen restoring, PostgreSQL'spg_restorecommand may output some warnings and erorrs. I haven't figured out a command line option to make these warnings disappear, but they can likely be ignored if you read them. Here is an example of what these errors may look like:pg_restore:[archiver(db)]ErrorwhilePROCESSINGTOC:
pg_restore:[archiver(db)]ErrorfromTOCentry1;307913792EXTENSIONplpgsql
pg_restore:[archiver(db)]couldnotexecutequery:ERROR:mustbeownerofextensionplpgsqlCommandwas:DROPEXTENSIONIFEXISTSplpgsql;pg_restore:[archiver(db)]ErrorfromTOCentry6;26152200SCHEMApublicpostgres
pg_restore:[archiver(db)]couldnotexecutequery:ERROR:mustbeownerofschemapublicCommandwas:DROPSCHEMAIFEXISTSpublic;pg_restore:[archiver(db)]couldnotexecutequery:ERROR:schema"public"alreadyexistsCommandwas:CREATESCHEMApublic;These may be errors from extensions that aren't used, or warnings about default schemata that already exist.Release NotesRelease notes are available on GitHub.MaintainerTimothy AllenatThe Wharton SchoolThis package is maintained by the staff ofWharton Research Data Services. We are thrilled thatThe Wharton Schoolallows us a certain amount of time to contribute to open-source projects. We add features as they are necessary for our projects, and try to keep up with Issues and Pull Requests as best we can. Due to constraints of time (our full time jobs!), Feature Requests without a Pull Request may not be implemented, but we are always open to new ideas and grateful for contributions and our package users.ContributorsAlex Malek (https://github.com/amalek215)Noel Victor (https://github.com/noeldvictor)Ryan Sullivan (https://github.com/rgs258)