Package: django-migrations-ci
django-migrations-ciReuse database state on CI. Run migrations on CI tests only for changes.Migrations are slow, but you have to run it on CI for testing reasons, so avoid
to run them when the database state was already tested.InstallInstall the package with pip:pipinstalldjango-migrations-ciAdddjango_migrations_cito Django settingsINSTALLED_APPS.INSTALLED_APPS=[...,# other packages"django_migrations_ci",]How to useThe commandmigrateciexecute all migrations and save dump filesmigrateci-*.If these files already exist on disk, they are used to prepare the database
without running all migrations again.WorkflowThis is how the "run test" CI job should work../manage.pymigrateci
./manage.pytest--keepdbIt works withpytest-djangotoo as a plugin:pytest--migrateci--reuse-dbThe recommended way to work with it is configuring defaultpytestaddoptswith--migrateci --reuse-dbto run without recreating database. When you want to recreate, run pytest with--create-dbthat has precedence over--reuse-db.Parallel tests./manage.pymigrateci--parallel$(nproc)./manage.pytest--keepdb--parallel$(nproc)Parallel tests with pytest-djangopytest--migrateci--reuse-db--parallel$(nproc)Also checkdatabase names for parallel tests.SettingsMIGRATECI_STORAGE="django.core.files.storage.FileSystemStorage"File storage class. Thedjango-storagespackage has many backends implemented.Saving cache files to an external storage allow the lib to reuse partial migrations.
When you write a new migration, it will try to get a cache without this
last migration and load from it, running only the new migrations.An S3 example,, but it works with any custom backend:fromstorages.backends.s3boto3importS3Boto3StorageclassMigrateCIStorage(S3Boto3Storage):bucket_name="mybucket-migrateci-cache"region_name="us-east-1"object_parameters={"StorageClass":"REDUCED_REDUNDANCY",}MIGRATECI_LOCATION=""File storage APIhas a location arg that all backend use in some way.If no storage is defined, it defaults to~/.migratecito make it easy to work local.MIGRATECI_PYTEST=FalseThepytest-djangopackage use custom test database names.If you use it and don´t change their default fixtures, just useMIGRATECI_PYTEST=True.MIGRATECI_PARALLEL=NoneBefore tests, Django execute all migrations in one database and clone it to be able to run parallel tests.
UseMIGRATECI_PARALLEL="auto"to create one database per process or define the exact number of processes withMIGRATECI_PARALLEL=4.It supports how Django test and howpytest-xdistworks.MIGRATECI_DEPTH=1This is how we decide which migration cache to use.First, it'll try to find a cache with all migration files, but in some cases it's not possible,
like when you just pushed a new migration.ForMIGRATECI_DEPTH=1, it'll remove one migration a time for each Django app installed and check if some cached migration exists. It support the most common use case and it's reasonably fast.Bigger values cause a cost operation, it'll remove N migrations a time and check if some cached migration exists. It's a combination of every Django app. E.g. for 10 apps, it'll take at most 10^N checks, with some hashing operations.Command line settingsAll below settings can be defined through command line args.manage.py migrateci [-h] [-n PARALLEL] [--storage STORAGE_CLASS] [--location LOCATION]
[--pytest] [--depth DEPTH] [-v {0,1,2,3}]

options:
  -h, --help            show this help message and exit
  -n PARALLEL, --parallel PARALLEL
  --storage STORAGE_CLASS
  --location LOCATION
  --pytest
  --depth DEPTH
  -v {0,1,2,3}Local migration cachingAs a stretch of this package, it's possible to use the same strategy during local
development. It'll by default cache files at~/.migrateci../manage.pymigrateci--parallel$(nproc)./manage.pytest--keepdb--parallel$(nproc)Why migrations are slow?Django migrations are slow because of state recreation for every migration and other internal Django magic.In the past, I tried to optimize that on Django core, but learnt it's arunning issue.Supported databasesmysqlpostgresqlsqlite3Django default run sqlite3 tests as in memory database and does not work becausemigrateciruns in a different process. Add a test database name to settings,
likesqlite test settings.Django supports oracle, but the dump function is not implemented here.Database names for parallel testsDjango test framework has a--parallel Nflag to test with N parallel processes,
naming databases from 1 to N.On sqlite3, adb.sqlite3generatedb_N.sqlite3files.On other databases, adbgeneratetest_db_N.Pytestpytest-djangousepytest-xdistfor parallel support, naming databases
from 0 to N-1.On sqlite3, adb.sqlite3generatedb.sqlite3_gwNfiles.On other databases, adbgeneratetest_db_gwN.