Repository: cal-itp/benefits

Issue:
Move JSON fixtures to Django data migrations
From #785:

> The main problem with our current process is that we manually add entries to our `.po` files because not all our `msgids` are in our code (see https://github.com/cal-itp/benefits/issues/338 for the reasoning for that) so we can't leverage Django's `makemessages`.

One possible way to solve for this is to move away from JSON fixtures and use [Django data migrations](https://docs.djangoproject.com/en/3.2/topics/migrations/#data-migrations) instead.

#789 is a small example / POC built on top of the work in #785 to demonstrate this.

## Acceptance Criteria

<!-- Remember to consider edge cases -->

- [ ] The `fixtures/` directory and its contents are removed from the repo
- [ ] All data in former `fixtures/??_classname.json` files is represented in new data migrations
- [ ] The container startup process uses the new migration(s) instead of the old fixture files
- [ ] The deployed apps bring in their own migration files instead of fixture files as part of the app redeploy/restart process


Discussion:
@angela-tran asks:

> I'm definitely interested in exploring this idea more. I really like that it would allow us to just use `makemessages` to process all the entries.
> 
> Since this would replace our JSON fixture files with a migration file (or files), I'm wondering: how would we define and load in different configurations (`dev`, `test`, `prod`, and our own local configs)?
Well I don't know that I answered the question 😉 but I definitely have the same one you do - how will we make this work with different environments?

I think at a basic level, we're just swapping the JSON fixtures for the python data migrations. The same type of process that we use now for deployment -- environment-specific config saved in storage, mounting storage into the container at runtime -- should work mostly the same.

We'll have to adjust the [`bin/init.sh`](https://github.com/cal-itp/benefits/blob/dev/bin/init.sh#L11) script, maybe instead of `DJANGO_INIT_PATH` for a list of fixtures to load, we use something like `DJANGO_MIGRATIONS` for a list of migrations to run (defaulting to all)?


Very true that at a basic level, this is not too different from our current deployment process. The part I wasn't sure about was if it'd be just as straightforward to get the migration file into the right place before the app initializes. I think it is since ultimately we're just putting some file in some location, and we rebuild the database every time.
https://github.com/cal-itp/benefits/pull/814#discussion_r926113274

Resolving patch:
diff --git a/benefits/core/migrations/0002_sample_data.py b/benefits/core/migrations/0002_sample_data.py
--- a/benefits/core/migrations/0002_sample_data.py
+++ b/benefits/core/migrations/0002_sample_data.py
@@ -1,8 +1,16 @@
+"""Data migration which loads sample data.
+Set environment variable DJANGO_LOAD_SAMPLE_DATA to False to skip loading sample data.
+"""
+from django.conf import settings
 from django.db import migrations
 from django.utils.translation import gettext_lazy as _
 
 
-def load_initial_data(app, *args, **kwargs):
+def load_sample_data(app, *args, **kwargs):
+    if not settings.LOAD_SAMPLE_DATA:
+        print("  LOAD_SAMPLE_DATA is set to False, skipping sample data")
+        return
+
     EligibilityType = app.get_model("core", "EligibilityType")
 
     type1 = EligibilityType.objects.create(name="type1", label="Eligibility Type 1", group_id="group1")
@@ -220,5 +228,5 @@ class Migration(migrations.Migration):
     ]
 
     operations = [
-        migrations.RunPython(load_initial_data),
+        migrations.RunPython(load_sample_data),
     ]
diff --git a/benefits/core/migrations/0003_data_migration_order.py b/benefits/core/migrations/0003_data_migration_order.py
new file mode 100644
--- /dev/null
+++ b/benefits/core/migrations/0003_data_migration_order.py
@@ -0,0 +1,15 @@
+"""This migration simply ensures that data migrations (assumed to be files that start with 0002) will run sequentially.
+It does so by looking for all migration files starting with "0002" and declaring them in its dependency list.
+"""
+import os
+
+from django.conf import settings
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    migrations_directory = os.path.join(settings.BASE_DIR, "benefits", "core", "migrations")
+    dependencies = [("core", file.replace(".py", "")) for file in os.listdir(migrations_directory) if file.startswith("0002")]
+
+    operations = []
diff --git a/benefits/settings.py b/benefits/settings.py
--- a/benefits/settings.py
+++ b/benefits/settings.py
@@ -172,6 +172,8 @@ def _filter_empty(ls):
     }
 }
 
+LOAD_SAMPLE_DATA = os.environ.get("DJANGO_LOAD_SAMPLE_DATA", "true").lower() != "false"
+
 # Password validation
 
 AUTH_PASSWORD_VALIDATORS = []


Test patch: