Repository: DefectDojo/django-DefectDojo
Pull request #12194: Tag: Update allowed characters for a unified format
Author: login: Maffooch
Labels: name: New Migration
description: Adding a new migration file. Take care when merging.
name: apiv2
name: docs
name: unittests
Created: 2025-04-08T01:44:40Z | Merged: 2025-04-16T21:00:27Z

Tags do not have consistent behavior between the UI and the API. This is caused by a handful of characters:
- Spaces
- Commas
- Quotes single/double

To create cohesion, we should no longer allow these characters as they tend to make global searches very tricky/nonfunctional anyways. The PR accomplishes the following
- Migrations to convert spaces to underscores, commas to hyphens, and removes singe/double quotes.
- Release notes explaining the previous point in further detail
- Common validation for the UI and API 
- Unit tests to reflect the change in behavior

Screenshots:
<img width="1528" alt="Screenshot 2025-04-07 at 7 16 14 PM" src="https://github.com/user-attachments/assets/cf6a8d6a-60d8-4de9-b1fd-7425d5279c9d" />
<img width="1169" alt="Screenshot 2025-04-07 at 7 25 21 PM" src="https://github.com/user-attachments/assets/e57a6b1a-8045-4010-95b8-1c2ed7491aee" />



[sc-10605]


Diff:
diff --git a/docs/content/en/open_source/upgrading/2.46.md b/docs/content/en/open_source/upgrading/2.46.md
index e80bbd580e3..4ff642dfd8c 100644
--- a/docs/content/en/open_source/upgrading/2.46.md
+++ b/docs/content/en/open_source/upgrading/2.46.md
@@ -2,7 +2,42 @@
 title: 'Upgrading to DefectDojo Version 2.46.x'
 toc_hide: true
 weight: -20250407
-description: Import Payload Changes
+description: Tag Formatting Changes + Import Payload Changes
+---
+
+### Tag Formatting Update
+
+Tags can no longer contain the following characters:
+
+- **Commas (,)**
+- **Quotations** (both single `'` and double `"`)
+- **Spaces**
+
+#### Automatic Migration
+
+To ensure a smooth transition, an automatic migration will be applied to existing tags as follows:
+
+- **Commas** → Replaced with **hyphens (`-`)**
+- **Quotations** (single and double) → **Removed**
+- **Spaces** → Replaced with **underscores (`_`)**
+
+#### Examples
+
+- `example,tag` → `example-tag`
+- `'SingleQuoted'` → `SingleQuoted`
+- `"DoubleQuoted"` → `DoubleQuoted`
+- `space separated tag` → `space_separated_tag`
+
+#### Why This Change?
+
+This update improves consistency, enhances search capabilities, and aligns with best practices for tag formatting.
+
+#### Recommended Actions
+
+We recommend reviewing your current tags, specifically CI/CD processes before upgrading to ensure they align with the new format. Performing a database backup is also suggested as this migration cannot be reverted.
+
+Following the deployment of these new behaviors, requests sent to the API or through the UI with any of the violations listed above will result in an error, with the details of the error raised in the response.
+
 ---
 
 ### Import / Reimport Statistics Delta
diff --git a/dojo/api_v2/serializers.py b/dojo/api_v2/serializers.py
index dc2b226baef..5f4f55fce95 100644
--- a/dojo/api_v2/serializers.py
+++ b/dojo/api_v2/serializers.py
@@ -118,7 +118,7 @@
     requires_tool_type,
 )
 from dojo.user.utils import get_configuration_permissions_codenames
-from dojo.utils import is_scan_file_too_large
+from dojo.utils import is_scan_file_too_large, tag_validator
 
 logger = logging.getLogger(__name__)
 deduplicationLogger = logging.getLogger("dojo.specific-loggers.deduplication")
@@ -225,6 +225,8 @@ def to_internal_value(self, data):
                 self.fail("not_a_str")
             # Run the children validation
             self.child.run_validation(s)
+            # Validate the tag to ensure it doesn't contain invalid characters
+            tag_validator(s, exception_class=RestFrameworkValidationError)
             substrings = re.findall(r'(?:"[^"]*"|[^",]+)', s)
             data_safe.extend(substrings)
 
diff --git a/dojo/db_migrations/0227_migrate_tags.py b/dojo/db_migrations/0227_migrate_tags.py
new file mode 100644
index 00000000000..349567659b8
--- /dev/null
+++ b/dojo/db_migrations/0227_migrate_tags.py
@@ -0,0 +1,122 @@
+# Generated by Django 5.0.8 on 2024-09-12 18:22
+
+import logging
+from django.db import migrations
+from django.db.models import Q
+
+logger = logging.getLogger(__name__)
+
+# Only apply the process to models that _could_ have tags
+model_names = [
+    "Product",
+    "Endpoint",
+    "Engagement",
+    "Test",
+    "Finding",
+    "Finding_Template",
+    "App_Analysis",
+    "Objects_Product",
+]
+
+
+def clean_tag_value(tag: str) -> str:
+    """
+    Clean each tag value by:
+    - Converting all commas to hyphens
+    - Converting all spaces to underscores
+    - Removing all single/double quotes
+    """
+    return tag.replace(",", "-").replace(" ", "_").replace('"', "").replace("'", "")
+
+
+def clean_all_tag_fields(apps, schema_editor):
+    """
+    Cleans tag values for all models in the `model_names` list, removing unwanted characters.
+    Updates both 'tags' and 'inherited_tags' fields where applicable.
+    """
+    updated_count = {}
+    for model_name in model_names:
+        TaggedModel = apps.get_model("dojo", model_name)
+        unique_tags_per_model = {}
+        count_per_model = 0
+        # Only fetch the objects with tags that contain a character in violation
+        queryset = (
+            TaggedModel.objects.filter(
+                Q(**{"tags__name__icontains": ","})
+                | Q(**{"tags__name__icontains": " "})
+                | Q(**{"tags__name__icontains": '"'})
+                | Q(**{"tags__name__icontains": "'"})
+            )
+            .distinct()
+            .prefetch_related("tags")
+        )
+        # Iterate over each instance to clean the tags. The iterator is used here
+        # to prevent loading the entire queryset into memory at once. Instead, we
+        # will only process 500 objects at a time
+        for instance in queryset.iterator(chunk_size=500):
+            # Get the current list of tags to work with
+            raw_tags = instance.tags.all()
+            # Clean each tag here while preserving the original value
+            cleaned_tags = {tag.name: clean_tag_value(tag.name) for tag in raw_tags}
+            # Quick check to avoid writing things without impact
+            if cleaned_tags:
+                instance.tags.set(list(cleaned_tags.values()), clear=True)
+                count_per_model += 1
+                # Update the running list of cleaned tags with the changes on this model
+                unique_tags_per_model.update(cleaned_tags)
+            # Add a quick logging statement every 100 objects cleaned
+            if count_per_model > 0 and count_per_model % 100 == 0:
+                logger.info(
+                    f"{TaggedModel.__name__}.tags: cleaned {count_per_model} tags..."
+                )
+        # Update the final count of the tags cleaned for the given model
+        if count_per_model:
+            updated_count[f"{TaggedModel.__name__}"] = (
+                count_per_model,
+                unique_tags_per_model,
+            )
+    """
+    Write a helpful statement about what tags were changed for each model in the list.
+    It looks something like this:
+
+    Product: 1 instances cleaned
+      "quoted string with spaces" -> quoted_string_with_spaces
+      "quoted with spaces, and also commas!" -> quoted_with_spaces-_and_also_commas!
+      "quoted,comma,tag" -> quoted-comma-tag
+    Engagement: 1 instances cleaned
+      "quoted string with spaces" -> quoted_string_with_spaces
+      "quoted with spaces, and also commas!" -> quoted_with_spaces-_and_also_commas!
+      "quoted,comma,tag" -> quoted-comma-tag
+    Test: 1 instances cleaned
+      "quoted string with spaces" -> quoted_string_with_spaces
+      "quoted with spaces, and also commas!" -> quoted_with_spaces-_and_also_commas!
+      "quoted,comma,tag" -> quoted-comma-tag
+    Finding: 1 instances cleaned
+      "quoted string with spaces" -> quoted_string_with_spaces
+      "quoted with spaces, and also commas!" -> quoted_with_spaces-_and_also_commas!
+      "quoted,comma,tag" -> quoted-comma-tag
+    """
+    for key, (count, tags) in updated_count.items():
+        logger.info(f"{key}: {count} instances cleaned")
+        for old, new in tags.items():
+            if old != new:
+                logger.info(f"  {old} -> {new}")
+
+
+def cannot_turn_back_time(apps, schema_editor):
+    """
+    We cannot possibly return to the original state without knowing
+    the original value at the time the migration is revoked. Instead
+    we will do nothing.
+    """
+    pass
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ("dojo", "0226_import_history_left_untouched_rename"),
+    ]
+
+    operations = [
+        migrations.RunPython(clean_all_tag_fields, cannot_turn_back_time),
+    ]
diff --git a/dojo/forms.py b/dojo/forms.py
index 6f63091fa00..a3e3da13187 100644
--- a/dojo/forms.py
+++ b/dojo/forms.py
@@ -110,6 +110,7 @@
     get_system_setting,
     is_finding_groups_enabled,
     is_scan_file_too_large,
+    tag_validator,
 )
 from dojo.widgets import TableCheckboxWidget
 
@@ -338,6 +339,9 @@ class Meta:
                 "business_criticality", "platform", "lifecycle", "origin", "user_records", "revenue", "external_audience", "enable_product_tag_inheritance",
                 "internet_accessible", "enable_simple_risk_acceptance", "enable_full_risk_acceptance", "disable_sla_breach_notifications"]
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
 
 class DeleteProductForm(forms.ModelForm):
     id = forms.IntegerField(required=True,
@@ -602,6 +606,9 @@ def clean(self):
 
         return cleaned_data
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
     # date can only be today or in the past, not the future
     def clean_scan_date(self):
         date = self.cleaned_data.get("scan_date", None)
@@ -708,6 +715,9 @@ def clean(self):
 
         return cleaned_data
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
     # date can only be today or in the past, not the future
     def clean_scan_date(self):
         date = self.cleaned_data.get("scan_date", None)
@@ -1021,6 +1031,9 @@ def is_valid(self):
             return False
         return True
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
     class Meta:
         model = Engagement
         exclude = ("first_contacted", "real_start", "engagement_type", "inherited_tags",
@@ -1076,6 +1089,9 @@ class Meta:
                   "environment", "percent_complete", "tags", "lead", "version", "branch_tag", "build_id", "commit_hash",
                   "api_scan_configuration"]
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
 
 class DeleteTestForm(forms.ModelForm):
     id = forms.IntegerField(required=True,
@@ -1172,6 +1188,9 @@ def clean(self):
 
         return cleaned_data
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
     class Meta:
         model = Finding
         exclude = ("reporter", "url", "numerical_severity", "under_review", "reviewers", "cve", "inherited_tags",
@@ -1249,6 +1268,9 @@ def clean(self):
 
         return cleaned_data
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
     class Meta:
         model = Finding
         exclude = ("reporter", "url", "numerical_severity", "under_review", "reviewers", "cve", "inherited_tags",
@@ -1306,6 +1328,9 @@ def clean(self):
 
         return cleaned_data
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
     class Meta:
         model = Finding
         exclude = ("reporter", "url", "numerical_severity", "active", "false_p", "verified", "endpoint_status", "cve", "inherited_tags",
@@ -1428,6 +1453,9 @@ def clean(self):
 
         return cleaned_data
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
     def _post_clean(self):
         super()._post_clean()
 
@@ -1504,6 +1532,9 @@ def clean(self):
 
         return cleaned_data
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
     class Meta:
         fields = ["title", "cwe", "vulnerability_ids", "cvssv3", "severity", "description", "mitigation", "impact", "references", "tags"]
         order = ("title", "cwe", "vulnerability_ids", "cvssv3", "severity", "description", "impact", "is_mitigated")
@@ -1534,6 +1565,9 @@ class Meta:
         order = ("title", "cwe", "vulnerability_ids", "cvssv3", "severity", "description", "impact")
         exclude = ("numerical_severity", "is_mitigated", "last_used", "endpoint_status", "cve")
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
 
 class DeleteFindingTemplateForm(forms.ModelForm):
     id = forms.IntegerField(required=True,
@@ -1587,6 +1621,9 @@ def clean(self):
             raise forms.ValidationError(msg)
         return cleaned_data
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
     class Meta:
         model = Finding
         fields = ("severity", "date", "planned_remediation_date", "active", "verified", "false_p", "duplicate", "out_of_scope",
@@ -1637,6 +1674,9 @@ def clean(self):
 
         return cleaned_data
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
 
 class AddEndpointForm(forms.Form):
     endpoint = forms.CharField(max_length=5000, required=True, label="Endpoint(s)",
@@ -1700,6 +1740,9 @@ def clean(self):
 
         return cleaned_data
 
+    def clean_tags(self):
+        tag_validator(self.cleaned_data.get("tags"))
+
 
 class DeleteEndpointForm(forms.ModelForm):
     id = forms.IntegerField(required=True,
diff --git a/dojo/utils.py b/dojo/utils.py
index ed8942258bc..2772abea5ed 100644
--- a/dojo/utils.py
+++ b/dojo/utils.py
@@ -25,6 +25,7 @@
 from django.conf import settings
 from django.contrib import messages
 from django.contrib.auth.signals import user_logged_in, user_logged_out, user_login_failed
+from django.core.exceptions import ValidationError
 from django.core.paginator import Paginator
 from django.db.models import Case, Count, IntegerField, Q, Sum, Value, When
 from django.db.models.query import QuerySet
@@ -2697,3 +2698,21 @@ def generate_file_response_from_file_path(
     response["Content-Disposition"] = f'attachment; filename="{full_file_name}"'
     response["Content-Length"] = file_size
     return response
+
+
+def tag_validator(value: str | list[str], exception_class: Callable = ValidationError) -> None:
+    TAG_PATTERN = re.compile(r'[ ,\'"]')
+    error_messages = []
+
+    if isinstance(value, list):
+        for tag in value:
+            if TAG_PATTERN.search(tag):
+                error_messages.append(f"Invalid tag: '{tag}'. Tags should not contain spaces, commas, or quotes.")
+    elif isinstance(value, str):
+        if TAG_PATTERN.search(value):
+            error_messages.append(f"Invalid tag: '{value}'. Tags should not contain spaces, commas, or quotes.")
+    else:
+        error_messages.append(f"Value must be a string or list of strings: {value} - {type(value)}.")
+
+    if error_messages:
+        raise exception_class(error_messages)
diff --git a/unittests/dojo_test_case.py b/unittests/dojo_test_case.py
index caa150ea857..f737947e53b 100644
--- a/unittests/dojo_test_case.py
+++ b/unittests/dojo_test_case.py
@@ -632,7 +632,7 @@ def get_finding_api(self, finding_id):
         self.assertEqual(200, response.status_code, response.content[:1000])
         return response.data
 
-    def post_new_finding_api(self, finding_details, push_to_jira=None):
+    def post_new_finding_api(self, finding_details: dict, push_to_jira=None, expected_status_code: int = 201):
         payload = copy.deepcopy(finding_details)
         if push_to_jira is not None:
             payload["push_to_jira"] = push_to_jira
@@ -640,7 +640,7 @@ def post_new_finding_api(self, finding_details, push_to_jira=None):
         # logger.debug('posting new finding push_to_jira: %s', payload.get('push_to_jira', None))
 
         response = self.client.post(reverse("finding-list"), payload, format="json")
-        self.assertEqual(201, response.status_code, response.content[:1000])
+        self.assertEqual(expected_status_code, response.status_code, response.content[:1000])
         return response.data
 
     def put_finding_api(self, finding_id, finding_details, push_to_jira=None):
diff --git a/unittests/test_tags.py b/unittests/test_tags.py
index cc0429668de..e431f7e94e6 100644
--- a/unittests/test_tags.py
+++ b/unittests/test_tags.py
@@ -18,7 +18,7 @@ def setUp(self, *args, **kwargs):
         self.scans_path = get_unit_tests_scans_path("zap")
         self.zap_sample5_filename = self.scans_path / "5_zap_sample_one.xml"
 
-    def create_finding_with_tags(self, tags):
+    def create_finding_with_tags(self, tags: list[str], expected_status_code: int = 201):
         finding_id = Finding.objects.all().first().id
         finding_details = self.get_finding_api(finding_id)
 
@@ -26,9 +26,9 @@ def create_finding_with_tags(self, tags):
 
         finding_details["title"] = "tags test " + str(random.randint(1, 9999))  # noqa: S311
         finding_details["tags"] = tags
-        response = self.post_new_finding_api(finding_details)
+        response = self.post_new_finding_api(finding_details, expected_status_code=expected_status_code)
 
-        return response["id"]
+        return response["id"] if expected_status_code == 201 else 0
 
     def test_finding_get_tags(self):
         tags = ["tag1", "tag2"]
@@ -165,55 +165,19 @@ def test_finding_patch_remove_tags_non_existent(self):
 
     def test_finding_create_tags_with_commas(self):
         tags = ["one,two"]
-        finding_id = self.create_finding_with_tags(tags)
-        response = self.get_finding_tags_api(finding_id)
-
-        self.assertEqual(2, len(response.get("tags")))
-        self.assertIn("one", str(response["tags"]))
-        self.assertIn("two", str(response["tags"]))
-
-    def test_finding_create_tags_with_commas_quoted(self):
-        tags = ['"one,two"']
-        finding_id = self.create_finding_with_tags(tags)
-        response = self.get_finding_tags_api(finding_id)
-
-        # no splitting due to quotes
-        self.assertEqual(len(tags), len(response.get("tags", None)))
-        for tag in tags:
-            logger.debug("looking for tag %s in tag list %s", tag, response["tags"])
-            # with django-tagging the quotes were stripped, with tagulous they remain
-            # self.assertIn(tag.strip('\"'), response['tags'])
-            self.assertIn(tag, response["tags"])
+        self.create_finding_with_tags(tags, expected_status_code=400)
 
     def test_finding_create_tags_with_spaces(self):
         tags = ["one two"]
-        finding_id = self.create_finding_with_tags(tags)
-        response = self.get_finding_tags_api(finding_id)
+        self.create_finding_with_tags(tags, expected_status_code=400)
 
-        # the old django-tagging library was splitting this tag into 2 tags
-        # with djangotagulous the tag does no longer get split up and we cannot modify tagulous
-        # to keep doing the old behaviour. so this is a small incompatibility, but only for
-        # tags with commas, so should be minor trouble
-        # self.assertEqual(2, len(response.get('tags')))
-        self.assertEqual(1, len(response.get("tags")))
-        self.assertIn("one", str(response["tags"]))
-        self.assertIn("two", str(response["tags"]))
-        # finding.tags: [<Tag: one>, <Tag: two>]
-
-    def test_finding_create_tags_with_spaces_quoted(self):
-        tags = ['"one two"']
-        finding_id = self.create_finding_with_tags(tags)
-        response = self.get_finding_tags_api(finding_id)
-
-        # no splitting due to quotes
-        self.assertEqual(len(tags), len(response.get("tags", None)))
-        for tag in tags:
-            logger.debug("looking for tag %s in tag list %s", tag, response["tags"])
-            # with django-tagging the quotes were stripped, with tagulous they remain
-            # self.assertIn(tag.strip('\"'), response['tags'])
-            self.assertIn(tag, response["tags"])
+    def test_finding_create_tags_with_double_quotes(self):
+        tags = ['"one-two"']
+        self.create_finding_with_tags(tags, expected_status_code=400)
 
-        # finding.tags: <QuerySet [<Tag: one two>]>
+    def test_finding_create_tags_with_single_quotes(self):
+        tags = ["'one-two'"]
+        self.create_finding_with_tags(tags, expected_status_code=400)
 
     def test_finding_create_tags_with_slashes(self):
         tags = ["a/b/c"]


Reviews:
user: login: mtesauro
body: Approved
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T16:46:13Z
user: login: cneill
body: A couple comments/questions here
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T16:49:14Z
user: login: valentijnscholten
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T18:03:30Z
user: login: dogboat
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T16:31:52Z
user: login: Maffooch
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T16:48:07Z
user: login: Maffooch
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T16:48:24Z
user: login: Maffooch
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T16:48:39Z
user: login: Maffooch
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-14T20:23:37Z
user: login: Maffooch
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-14T20:24:23Z
user: login: valentijnscholten
body: Looks good now, thanks.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-15T16:36:47Z
user: login: cneill
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-16T16:43:08Z

Review comments:
diff_hunk: @@ -2,6 +2,42 @@
 title: 'Upgrading to DefectDojo Version 2.46.x'
 toc_hide: true
 weight: -20250407
-description: No special instructions.
+description: Tag Formatting Changes
 ---
-There are no special instructions for upgrading to 2.46.x. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.46.0) for the contents of the release.
+
+### Tag Formatting Update
+
+Tags can no longer contain the following characters:
+
+- **Commas (,)**
+- **Quotations** (both single `'` and double `"`)
+- **Spaces**
+
+#### Automatic Migration
+
+To ensure a smooth transition, an automatic migration will be applied to existing tags as follows:
+
+- **Commas** → Replaced with **hyphens (`-`)**
+- **Quotations** (single and double) → **Removed**
+- **Spaces** → Replaced with **underscores (`_`)**
+
+#### Examples
+
+- `example,tag` → `example-tag`
+- `'SingleQuoted'` → `SingleQuoted`
+- `"DoubleQuoted"` → `DoubleQuoted`
+- `space separated tag` → `space_separated_tag`
+
+#### Why This Change?
+
+This update improves consistency, enhances search capabilities, and aligns with best practices for tag formatting.
+
+#### Recommended Actions
+
+We recommend reviewing your current tags, specifically CI/CD processes before upgrading to ensure they align with the new format.
path: docs/content/en/open_source/upgrading/2.46.md
user: login: cneill
body: We might also note that this migration cannot be rolled back once completed, since we can't know that e.g. a `_` *used to be* a space?
created_at: 2025-04-08T16:29:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,96 @@
+# Generated by Django 5.0.8 on 2024-09-12 18:22
+
+import logging
+from django.db import migrations
+from django.db.models import Q
+
+logger = logging.getLogger(__name__)
+
+model_names = [
+    "Product",
+    "Endpoint",
+    "Engagement",
+    "Test",
+    "Finding",
+    "Finding_Template",
+    "App_Analysis",
+    "Objects_Product",
+]
+
+
+def clean_tag_value(tag: str) -> str:
+    """
+    Clean each tag value by:
+    - Converting all commas to hyphens
+    - Converting all spaces to underscores
+    - Removing all single/double quotes
+    """
+    return (
+        tag.replace(",", "-")
+        .replace(" ", "_")
+        .replace('"', "")
+        .replace("'", "")
+    )
+
+
+def clean_all_tag_fields(apps, schema_editor):
+    updated_count = {}
+    # Iterate over each model in the app
+    for model_name in model_names:
+        TaggedModel = apps.get_model("dojo", model_name)
+        for field_name in ["tags", "inherited_tags"]:
+            # Make sure we have a field to work with here
+            try:
+                _field = TaggedModel._meta.get_field(field_name)
+            except Exception:
+                continue
+            # Only query objects with tags that are in violation and need to be changed
+            dirty_tag_q = (
+                Q(**{f"{field_name}__name__icontains": ","})
+                | Q(**{f"{field_name}__name__icontains": " "})
+                | Q(**{f"{field_name}__name__icontains": '"'})
+                | Q(**{f"{field_name}__name__icontains": "'"})
+            )
+            count = 0
+            # Iterate over each instance that meets the criteria
+            for instance in (
+                TaggedModel.objects.filter(dirty_tag_q).distinct().iterator()
+            ):
+                # Make sure have a tag field first. If not, move on
+                if (
+                    tag_manager := getattr(instance, field_name, None)
+                ) is None or not hasattr(tag_manager, "all"):
+                    continue
+                # Ensure we actually have some tags to work with here
+                if not (raw_tags := list(tag_manager.all().only("name"))):
+                    continue
+                # Clean the tags, and if we still have something, set it through the manager
+                if cleaned := [clean_tag_value(tag.name) for tag in raw_tags]:
+                    tag_manager.set(cleaned, clear=True)
+                    count += 1
path: dojo/db_migrations/0226_migrate_tags.py
user: login: cneill
body: This will count the number of model fields we cleaned, but might it be useful to count the number of unique *tags* that were modified and share that list at the end instead? Given that the tags we're modifying are potentially hard to search for before performing the migration, it seems like users will have a hard time anticipating which of their existing tags will be affected.
created_at: 2025-04-08T16:40:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,96 @@
+# Generated by Django 5.0.8 on 2024-09-12 18:22
+
+import logging
+from django.db import migrations
+from django.db.models import Q
+
+logger = logging.getLogger(__name__)
+
+model_names = [
+    "Product",
+    "Endpoint",
+    "Engagement",
+    "Test",
+    "Finding",
+    "Finding_Template",
+    "App_Analysis",
+    "Objects_Product",
+]
+
+
+def clean_tag_value(tag: str) -> str:
+    """
+    Clean each tag value by:
+    - Converting all commas to hyphens
+    - Converting all spaces to underscores
+    - Removing all single/double quotes
+    """
+    return (
+        tag.replace(",", "-")
+        .replace(" ", "_")
+        .replace('"', "")
+        .replace("'", "")
+    )
+
+
+def clean_all_tag_fields(apps, schema_editor):
+    updated_count = {}
+    # Iterate over each model in the app
+    for model_name in model_names:
+        TaggedModel = apps.get_model("dojo", model_name)
+        for field_name in ["tags", "inherited_tags"]:
+            # Make sure we have a field to work with here
+            try:
+                _field = TaggedModel._meta.get_field(field_name)
+            except Exception:
+                continue
+            # Only query objects with tags that are in violation and need to be changed
+            dirty_tag_q = (
+                Q(**{f"{field_name}__name__icontains": ","})
+                | Q(**{f"{field_name}__name__icontains": " "})
+                | Q(**{f"{field_name}__name__icontains": '"'})
+                | Q(**{f"{field_name}__name__icontains": "'"})
+            )
+            count = 0
+            # Iterate over each instance that meets the criteria
+            for instance in (
+                TaggedModel.objects.filter(dirty_tag_q).distinct().iterator()
+            ):
+                # Make sure have a tag field first. If not, move on
+                if (
+                    tag_manager := getattr(instance, field_name, None)
+                ) is None or not hasattr(tag_manager, "all"):
+                    continue
+                # Ensure we actually have some tags to work with here
+                if not (raw_tags := list(tag_manager.all().only("name"))):
+                    continue
+                # Clean the tags, and if we still have something, set it through the manager
+                if cleaned := [clean_tag_value(tag.name) for tag in raw_tags]:
path: dojo/db_migrations/0226_migrate_tags.py
user: login: cneill
body: I hope no one has a `""` or `     ` tag in their instance today, but if they do, they'll end up with empty tags here. Maybe worth removing those tags entirely instead of trying to set empty ones? And after that, we probably still want to save it even if the final list is empty, right?
created_at: 2025-04-08T16:44:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2,6 +2,42 @@
 title: 'Upgrading to DefectDojo Version 2.46.x'
 toc_hide: true
 weight: -20250407
-description: No special instructions.
+description: Tag Formatting Changes
 ---
-There are no special instructions for upgrading to 2.46.x. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.46.0) for the contents of the release.
+
+### Tag Formatting Update
+
+Tags can no longer contain the following characters:
+
+- **Commas (,)**
+- **Quotations** (both single `'` and double `"`)
+- **Spaces**
+
+#### Automatic Migration
+
+To ensure a smooth transition, an automatic migration will be applied to existing tags as follows:
+
+- **Commas** → Replaced with **hyphens (`-`)**
+- **Quotations** (single and double) → **Removed**
+- **Spaces** → Replaced with **underscores (`_`)**
+
+#### Examples
+
+- `example,tag` → `example-tag`
+- `'SingleQuoted'` → `SingleQuoted`
+- `"DoubleQuoted"` → `DoubleQuoted`
+- `space separated tag` → `space_separated_tag`
+
+#### Why This Change?
+
+This update improves consistency, enhances search capabilities, and aligns with best practices for tag formatting.
+
+#### Recommended Actions
+
+We recommend reviewing your current tags, specifically CI/CD processes before upgrading to ensure they align with the new format.
path: docs/content/en/open_source/upgrading/2.46.md
user: login: valentijnscholten
body: And mention that they should make a backup before upgrading. It's an open door, but it might still save a few users not running production yet.
created_at: 2025-04-08T18:03:29Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,96 @@
+# Generated by Django 5.0.8 on 2024-09-12 18:22
+
+import logging
+from django.db import migrations
+from django.db.models import Q
+
+logger = logging.getLogger(__name__)
+
+model_names = [
+    "Product",
+    "Endpoint",
+    "Engagement",
+    "Test",
+    "Finding",
+    "Finding_Template",
+    "App_Analysis",
+    "Objects_Product",
+]
+
+
+def clean_tag_value(tag: str) -> str:
+    """
+    Clean each tag value by:
+    - Converting all commas to hyphens
+    - Converting all spaces to underscores
+    - Removing all single/double quotes
+    """
+    return (
+        tag.replace(",", "-")
+        .replace(" ", "_")
+        .replace('"', "")
+        .replace("'", "")
+    )
+
+
+def clean_all_tag_fields(apps, schema_editor):
+    updated_count = {}
+    # Iterate over each model in the app
+    for model_name in model_names:
+        TaggedModel = apps.get_model("dojo", model_name)
+        for field_name in ["tags", "inherited_tags"]:
+            # Make sure we have a field to work with here
+            try:
+                _field = TaggedModel._meta.get_field(field_name)
+            except Exception:
+                continue
+            # Only query objects with tags that are in violation and need to be changed
+            dirty_tag_q = (
+                Q(**{f"{field_name}__name__icontains": ","})
+                | Q(**{f"{field_name}__name__icontains": " "})
+                | Q(**{f"{field_name}__name__icontains": '"'})
+                | Q(**{f"{field_name}__name__icontains": "'"})
+            )
+            count = 0
+            # Iterate over each instance that meets the criteria
+            for instance in (
+                TaggedModel.objects.filter(dirty_tag_q).distinct().iterator()
+            ):
+                # Make sure have a tag field first. If not, move on
+                if (
+                    tag_manager := getattr(instance, field_name, None)
+                ) is None or not hasattr(tag_manager, "all"):
+                    continue
+                # Ensure we actually have some tags to work with here
+                if not (raw_tags := list(tag_manager.all().only("name"))):
+                    continue
+                # Clean the tags, and if we still have something, set it through the manager
+                if cleaned := [clean_tag_value(tag.name) for tag in raw_tags]:
path: dojo/db_migrations/0226_migrate_tags.py
user: login: Maffooch
body: The tricky thing about this library is that there are individual tables for each model, so a table for finding tags, test tags, etc. I was struggling to find a way to programmatically access these tables to manage these tables directly, but I can take another stab at it. 
created_at: 2025-04-09T16:48:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,96 @@
+# Generated by Django 5.0.8 on 2024-09-12 18:22
+
+import logging
+from django.db import migrations
+from django.db.models import Q
+
+logger = logging.getLogger(__name__)
+
+model_names = [
+    "Product",
+    "Endpoint",
+    "Engagement",
+    "Test",
+    "Finding",
+    "Finding_Template",
+    "App_Analysis",
+    "Objects_Product",
+]
+
+
+def clean_tag_value(tag: str) -> str:
+    """
+    Clean each tag value by:
+    - Converting all commas to hyphens
+    - Converting all spaces to underscores
+    - Removing all single/double quotes
+    """
+    return (
+        tag.replace(",", "-")
+        .replace(" ", "_")
+        .replace('"', "")
+        .replace("'", "")
+    )
+
+
+def clean_all_tag_fields(apps, schema_editor):
+    updated_count = {}
+    # Iterate over each model in the app
+    for model_name in model_names:
+        TaggedModel = apps.get_model("dojo", model_name)
+        for field_name in ["tags", "inherited_tags"]:
+            # Make sure we have a field to work with here
+            try:
+                _field = TaggedModel._meta.get_field(field_name)
+            except Exception:
+                continue
+            # Only query objects with tags that are in violation and need to be changed
+            dirty_tag_q = (
+                Q(**{f"{field_name}__name__icontains": ","})
+                | Q(**{f"{field_name}__name__icontains": " "})
+                | Q(**{f"{field_name}__name__icontains": '"'})
+                | Q(**{f"{field_name}__name__icontains": "'"})
+            )
+            count = 0
+            # Iterate over each instance that meets the criteria
+            for instance in (
+                TaggedModel.objects.filter(dirty_tag_q).distinct().iterator()
+            ):
+                # Make sure have a tag field first. If not, move on
+                if (
+                    tag_manager := getattr(instance, field_name, None)
+                ) is None or not hasattr(tag_manager, "all"):
+                    continue
+                # Ensure we actually have some tags to work with here
+                if not (raw_tags := list(tag_manager.all().only("name"))):
+                    continue
+                # Clean the tags, and if we still have something, set it through the manager
+                if cleaned := [clean_tag_value(tag.name) for tag in raw_tags]:
+                    tag_manager.set(cleaned, clear=True)
+                    count += 1
path: dojo/db_migrations/0226_migrate_tags.py
user: login: Maffooch
body: Sure, I can add another log
created_at: 2025-04-09T16:48:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2,6 +2,42 @@
 title: 'Upgrading to DefectDojo Version 2.46.x'
 toc_hide: true
 weight: -20250407
-description: No special instructions.
+description: Tag Formatting Changes
 ---
-There are no special instructions for upgrading to 2.46.x. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.46.0) for the contents of the release.
+
+### Tag Formatting Update
+
+Tags can no longer contain the following characters:
+
+- **Commas (,)**
+- **Quotations** (both single `'` and double `"`)
+- **Spaces**
+
+#### Automatic Migration
+
+To ensure a smooth transition, an automatic migration will be applied to existing tags as follows:
+
+- **Commas** → Replaced with **hyphens (`-`)**
+- **Quotations** (single and double) → **Removed**
+- **Spaces** → Replaced with **underscores (`_`)**
+
+#### Examples
+
+- `example,tag` → `example-tag`
+- `'SingleQuoted'` → `SingleQuoted`
+- `"DoubleQuoted"` → `DoubleQuoted`
+- `space separated tag` → `space_separated_tag`
+
+#### Why This Change?
+
+This update improves consistency, enhances search capabilities, and aligns with best practices for tag formatting.
+
+#### Recommended Actions
+
+We recommend reviewing your current tags, specifically CI/CD processes before upgrading to ensure they align with the new format.
path: docs/content/en/open_source/upgrading/2.46.md
user: login: Maffooch
body: Sure thing
created_at: 2025-04-09T16:48:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,96 @@
+# Generated by Django 5.0.8 on 2024-09-12 18:22
+
+import logging
+from django.db import migrations
+from django.db.models import Q
+
+logger = logging.getLogger(__name__)
+
+model_names = [
+    "Product",
+    "Endpoint",
+    "Engagement",
+    "Test",
+    "Finding",
+    "Finding_Template",
+    "App_Analysis",
+    "Objects_Product",
+]
+
+
+def clean_tag_value(tag: str) -> str:
+    """
+    Clean each tag value by:
+    - Converting all commas to hyphens
+    - Converting all spaces to underscores
+    - Removing all single/double quotes
+    """
+    return (
+        tag.replace(",", "-")
+        .replace(" ", "_")
+        .replace('"', "")
+        .replace("'", "")
+    )
+
+
+def clean_all_tag_fields(apps, schema_editor):
+    updated_count = {}
+    # Iterate over each model in the app
+    for model_name in model_names:
+        TaggedModel = apps.get_model("dojo", model_name)
+        for field_name in ["tags", "inherited_tags"]:
+            # Make sure we have a field to work with here
+            try:
+                _field = TaggedModel._meta.get_field(field_name)
+            except Exception:
+                continue
+            # Only query objects with tags that are in violation and need to be changed
+            dirty_tag_q = (
+                Q(**{f"{field_name}__name__icontains": ","})
+                | Q(**{f"{field_name}__name__icontains": " "})
+                | Q(**{f"{field_name}__name__icontains": '"'})
+                | Q(**{f"{field_name}__name__icontains": "'"})
+            )
+            count = 0
+            # Iterate over each instance that meets the criteria
+            for instance in (
+                TaggedModel.objects.filter(dirty_tag_q).distinct().iterator()
+            ):
+                # Make sure have a tag field first. If not, move on
+                if (
+                    tag_manager := getattr(instance, field_name, None)
+                ) is None or not hasattr(tag_manager, "all"):
+                    continue
+                # Ensure we actually have some tags to work with here
+                if not (raw_tags := list(tag_manager.all().only("name"))):
+                    continue
+                # Clean the tags, and if we still have something, set it through the manager
+                if cleaned := [clean_tag_value(tag.name) for tag in raw_tags]:
+                    tag_manager.set(cleaned, clear=True)
+                    count += 1
path: dojo/db_migrations/0226_migrate_tags.py
user: login: Maffooch
body: Added a summary of the changes. Please see the comments in the migration for a sample of what the changes will look like
created_at: 2025-04-14T20:23:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,96 @@
+# Generated by Django 5.0.8 on 2024-09-12 18:22
+
+import logging
+from django.db import migrations
+from django.db.models import Q
+
+logger = logging.getLogger(__name__)
+
+model_names = [
+    "Product",
+    "Endpoint",
+    "Engagement",
+    "Test",
+    "Finding",
+    "Finding_Template",
+    "App_Analysis",
+    "Objects_Product",
+]
+
+
+def clean_tag_value(tag: str) -> str:
+    """
+    Clean each tag value by:
+    - Converting all commas to hyphens
+    - Converting all spaces to underscores
+    - Removing all single/double quotes
+    """
+    return (
+        tag.replace(",", "-")
+        .replace(" ", "_")
+        .replace('"', "")
+        .replace("'", "")
+    )
+
+
+def clean_all_tag_fields(apps, schema_editor):
+    updated_count = {}
+    # Iterate over each model in the app
+    for model_name in model_names:
+        TaggedModel = apps.get_model("dojo", model_name)
+        for field_name in ["tags", "inherited_tags"]:
+            # Make sure we have a field to work with here
+            try:
+                _field = TaggedModel._meta.get_field(field_name)
+            except Exception:
+                continue
+            # Only query objects with tags that are in violation and need to be changed
+            dirty_tag_q = (
+                Q(**{f"{field_name}__name__icontains": ","})
+                | Q(**{f"{field_name}__name__icontains": " "})
+                | Q(**{f"{field_name}__name__icontains": '"'})
+                | Q(**{f"{field_name}__name__icontains": "'"})
+            )
+            count = 0
+            # Iterate over each instance that meets the criteria
+            for instance in (
+                TaggedModel.objects.filter(dirty_tag_q).distinct().iterator()
+            ):
+                # Make sure have a tag field first. If not, move on
+                if (
+                    tag_manager := getattr(instance, field_name, None)
+                ) is None or not hasattr(tag_manager, "all"):
+                    continue
+                # Ensure we actually have some tags to work with here
+                if not (raw_tags := list(tag_manager.all().only("name"))):
+                    continue
+                # Clean the tags, and if we still have something, set it through the manager
+                if cleaned := [clean_tag_value(tag.name) for tag in raw_tags]:
path: dojo/db_migrations/0226_migrate_tags.py
user: login: Maffooch
body: Djangos m2m mangers will not allow an "empty" objects, so we should be good here
created_at: 2025-04-14T20:24:23Z
author_association: CONTRIBUTOR

PR comments:
user: login: dryrunsecurity[bot]
created_at: 2025-04-08T01:45:29Z
body: ### DryRun Security Summary

DefectDojo version 2.46.x implements comprehensive tag validation and sanitization improvements to enhance security by preventing potential injection vulnerabilities and standardizing tag formatting across the application.

<details>
<summary>Expand for full summary</summary>
<br/>

Summary: DefectDojo version 2.46.x introduces comprehensive tag formatting changes across multiple components, implementing stricter tag validation and cleaning mechanisms to improve input sanitization and prevent potential security risks.

Security Findings:

1. Tag Validation Improvements
   - Systematic input validation for tags across multiple forms and API endpoints
   - Prevents tags containing:
     * Spaces
     * Commas
     * Single quotes
     * Double quotes
   - Helps mitigate potential injection and parsing vulnerabilities

2. Logging Considerations
   - Potential information disclosure risk in migration logs
   - Recommendation to ensure proper log configuration and rotation

3. Input Sanitization
   - Automatic tag cleaning during database migration
   - Replaces problematic characters with safe alternatives
   - Removes quotes and standardizes tag formatting

4. Error Handling
   - Consistent use of validation errors for non-compliant tags
   - Rejects invalid tag requests early in processing pipeline
   - Prevents downstream processing of malformed inputs

5. Operational Security
   - Changes will require users to modify existing tags and CI/CD processes
   - Enforces stricter tag formatting rules across the application
</details>

[View PR in the DryRun Dashboard.](https://app.dryrun.security/pull-requests/?org=DefectDojo)
author_association: NONE
user: login: valentijnscholten
created_at: 2025-04-08T18:01:44Z
body: Have you considered using `bulk_update` to make it faster. Not sure of this is possible with the way tags are implemented. 
Or using raw sql?
Otherwise it might be helpful to output some kind of (info level) logging every 1000 models/tags. We see  some reports on the #defectdojo slack where people think the migration is frozen or they don't realize it's not done yet or restart the initializer while it's still migrating, etc.
author_association: MEMBER
user: login: Maffooch
created_at: 2025-04-09T16:49:24Z
body: > Have you considered using bulk_update to make it faster.

I did try to take this method, but was stuck on the m2m relationship. Will see what I can do about logging
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-04-11T14:04:09Z
body: This pull request has conflicts, please resolve those before we can evaluate the pull request.
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-04-14T20:19:52Z
body: Conflicts have been resolved. A maintainer will review the pull request shortly.
author_association: CONTRIBUTOR
user: login: dryrunsecurity[bot]
created_at: 2025-04-14T20:20:37Z
body: <img src="https://d2ksm3urlgyvb9.cloudfront.net/images/dryrun-comment-banner.svg" alt="DryRun Security">

This pull request introduces enhanced input validation for tags, focusing on preventing injection attacks and potential security risks, while also recommending careful migration procedures and improved error handling to protect system integrity and data privacy.

<details>
<summary>
:thought_balloon: Unconfirmed Findings (5)
</summary>
<br>

| **Vulnerability** | Input Validation Enhancements |
|:---|:---|
| **Description** | New tag validation mechanism implemented across multiple files, preventing tags from containing commas, quotation marks (single and double), and spaces. Focuses on input sanitization and blocking potential special characters. |

| **Vulnerability** | Potential Logging Sensitivity |
|:---|:---|
| **Description** | Migration logs may expose tag transformation details, raising potential privacy and security concerns. Recommendation includes securing and rotating log files to mitigate information disclosure risks. |

| **Vulnerability** | Security Considerations in Tag Processing |
|:---|:---|
| **Description** | Proactive approach to preventing injection attacks through consistent validation across forms and API endpoints. Uses regex-based validation to block special characters in tags, enhancing overall system security. |

| **Vulnerability** | Migration Process Risks |
|:---|:---|
| **Description** | Potential impact on existing systems and integrations during version upgrade. Recommends database backup before upgrade and warns of possible data transformation risks during migration process. |

| **Vulnerability** | Error Handling Improvements |
|:---|:---|
| **Description** | Utilizes RestFrameworkValidationError to catch and reject invalid tags, implementing early validation in the processing pipeline to prevent malformed tag inputs from being processed. |
</details>

---

*All finding details can be found in the **[DryRun Security Dashboard](https://app.dryrun.security/pull-requests/?org=DefectDojo)**.*
author_association: NONE
user: login: Maffooch
created_at: 2025-04-14T20:26:57Z
body: > Have you considered using bulk_update to make it faster. Not sure of this is possible with the way tags are implemented.
Or using raw sql?

Turns out m2m is not supported by bulk update. I did some research on this, and it turns out that using the `.set` method is the best way to this sort of thing. 

> Otherwise it might be helpful to output some kind of (info level) logging every 1000 models/tags.

I went with every 100 in case things went slow for some users. This will at least provide some good feedback if needed to manage expectations
author_association: CONTRIBUTOR
user: login: valentijnscholten
created_at: 2025-04-15T16:37:45Z
body: Not blocking for merge: Could there be more characters that could cause problems? I was wondering if there should be an allowlist regex instead of block the 3 charactes in this PR.
What happens if someone posts a tag with one of these bad boys in it:
![image](https://github.com/user-attachments/assets/1e3489d4-3132-47be-a046-ce8a1eecc158)

author_association: MEMBER
user: login: cneill
created_at: 2025-04-16T16:45:10Z
body: > Not blocking for merge: Could there be more characters that could cause problems? I was wondering if there should be an allowlist regex instead of block the 3 charactes in this PR.

I think this is a good idea if Tagulous doesn't already reject random non-printable characters, but I would guess that it rejects such characters already? If not, maybe worth a follow-up PR
author_association: CONTRIBUTOR
user: login: Maffooch
created_at: 2025-04-16T21:00:18Z
body: Let's save that for another time 😅 
author_association: CONTRIBUTOR