Repository: pulp/pulpcore

Issue:
Need a way to directly migrate pulp_labels from 3.21 to 3.28
**Is your feature request related to a problem? Please describe.**
As RHUI developers, we are heavy users of pulp_labels on repository objects.  As the current 4.5 release is on pulpcore 3.21, we would like our next release (4.6) to be based on pulpcore 3.28, and we need to ensure a smooth upgrade path for our customers.  As it is currently laid out, the 3.21 to 3.28 migration path requires an intermediate step (<3.25) where `pulpcore-manager datarepair-labels` command is available.

**Describe the solution you'd like**
The ideal solution would be to make Django migrations take care of that automatically, without requiring an intermediate step/release.  Since we live in non-ideal world, a solution where an additional command like `datarepair-labels` is available would also suffice.

**Describe alternatives you've considered**
We have considered an intermediate release of RHUI 4.6 with pulpcore 3.22 followed by RHUI 4.7 built on 3.28, but it would create a lot of migration issues for our customers, with many of them still running RHUI 4.0.




Discussion:
Marked as bug/prio - this problem blocks upgrading from 3.21 to post-3.25, which is...Rude. Already impacting one of our stakeholders.
git show 3d67cc1be8a332cc4af9f36e9c52fbb10916ebd6

Tells the story behind this.

Resolving patch:
diff --git a/pulpcore/app/migrations/0104_delete_label.py b/pulpcore/app/migrations/0104_delete_label.py
--- a/pulpcore/app/migrations/0104_delete_label.py
+++ b/pulpcore/app/migrations/0104_delete_label.py
@@ -1,6 +1,45 @@
 # Generated by Django 3.2.18 on 2023-05-04 07:06
 
 from django.db import migrations
+from django.db.models.expressions import OuterRef, RawSQL
+
+
+def migrate_remaining_labels(apps, schema_editor):
+    """
+    This data migration handles the "but what about plugins" problem noted in the issue [0], with only two caveats:
+
+    Case 1: If there were to exist a plugin containing a Model whose model-name ended in (for example) "Repository",
+    that was NOT a detail-model of a Repository master-model, AND that plugin allowed Labels for such a model - then,
+    upon running this migration, those Labels would be lost.
+
+    Case 2: If there were to exist a plugin containing a Model that was a Detail of (for example) Repository,
+    but named something like "PluginRepositoryButWhy", and that plugin allowed Labels,
+    and instances of such a Model had Labels associated with them - then this migration would fail,
+    because the Labels would not be found, migrated, and deleted, and the old-Label table would not be able to be dropped.
+
+    And the plugins described above would have to have existed and been in use with pulpcore/3.21,only -
+    if they appeared with core/3.22, they'd be using new-Labels and all would be (already) well.
+
+    No such plugins/Models exist, to the best of our knowledge.
+
+    [0] https://github.com/pulp/pulpcore/issues/4319
+    """
+    Label = apps.get_model("core", "Label")
+    Repository = apps.get_model("core", "Repository")
+    Remote = apps.get_model("core", "Remote")
+    Distribution = apps.get_model("core", "Distribution")
+    ContentType = apps.get_model("contenttypes", "ContentType")
+
+    for master_model, model_name in [(Repository, "repository"), (Remote, "remote"), (Distribution, "distribution")]:
+        detail_ctypes = ContentType.objects.filter(app_label__ne="core", model__endswith=model_name)
+        affected_ids = Label.objects.filter(content_type__in=detail_ctypes).values("object_id").distinct()
+        label_subq = Label.objects.filter(
+            content_type__in=detail_ctypes, object_id=OuterRef("pulp_id")
+        ).annotate(
+            label_data=RawSQL("hstore(array_agg(key), array_agg(value))", [])
+        ).values("label_data")
+        master_model.objects.filter(pulp_id__in=affected_ids).update(pulp_labels=label_subq)
+        Label.objects.filter(content_type__in=detail_ctypes).delete()
 
 
 def check_no_existing_labels(apps, schema_editor):
@@ -18,11 +57,20 @@ class Migration(migrations.Migration):
     ]
 
     operations = [
+        migrations.RunPython(
+            code=migrate_remaining_labels,
+            reverse_code=migrations.RunPython.noop,
+            elidable=True,
+        ),
         migrations.RunPython(
             code=check_no_existing_labels,
             reverse_code=migrations.RunPython.noop,
             elidable=True,
         ),
+        migrations.RunSQL(
+            sql="SET CONSTRAINTS ALL IMMEDIATE;",
+            reverse_sql="",
+        ),
         migrations.DeleteModel(
             name='Label',
         ),


Test patch: