Repository: getsentry/sentry

Issue:
Support "ignored fields" for Jira organization integration like legacy Jira project plugin does
<!--

Note that the Sentry team has finite resources and priorities that are not
always visible on GitHub. If your issue doesn't align with our priorities it's
unlikely it will be worked on. If we're interested in a particular feature however,
we'll follow up and ask you to submit an RFC to talk about it in more detail.

-->

## Summary
Allow an admin to configure the Jira Integration to hide Jira fields in the
issue creation modal, specified by name on the settings page.

## Motivation
The motivation is the same as to why the Jira plugin already supports this
feature: from a bug triage perspective, having all fields of an issue type
present in the modal can be overwhelming noise, esp. since many of those fields
may be intended to be filled out after the issue is in JIRA (e.g. a tech plan
field).

## Additional Context

Existing "legacy integration" feature 🙌 : 
![image](https://user-images.githubusercontent.com/549473/97115019-5c63d100-16b1-11eb-9582-1183665bb79c.png)

Missing from "organization integration" 😭 :
![image](https://user-images.githubusercontent.com/549473/97115045-81584400-16b1-11eb-8c7e-efaeb34df892.png)

Here is an example from my organization's production Sentry that shows the
creation modal being a bad time for developers (note the scrollbar):
![image](https://user-images.githubusercontent.com/549473/97115166-6cc87b80-16b2-11eb-8150-3b418ca01560.png)

The ideal input format is not obvious to me:

- AFAIK, a "comma-separated list" is only found in the legacy plugins,
  so not sure if this was a concerted effort to avoid these.
- Fingerprinting / grouping UIs have a single-line line per rule that allows
  comments, which is fairly nice (esp. since it would be nice to annotate what
  field name what "customfield_10020" refers to).
- A multi-select that actually hits the Jira API to query all fields would be
  interesting, but from a brief investigation `{"type": "select", "multi": True}`
  didn't work so well, and is complicated by the fact that it'd probably
  need to query all fields across all issue types... which could be large /
  unpleasant to navigate unless there was a better way of exploring all the
  fields... which sounds like more work than its worth.

Personally, I've been okay with the legacy integration's approach... it's an
infrequently tweaked list. I've worked around the issue of no comments by
including non-sense values interspliced, e.g.:
```
customfield_100020,<-- this_is_story_points_field,customfield_100021,<-- tech plan field
```
but first-class support for comments would be nice, e.g.:
```
# story points field
customfield_100020

# tech plan field
customfield_100021
```



Discussion:

Resolving patch:
diff --git a/src/sentry/integrations/jira/integration.py b/src/sentry/integrations/jira/integration.py
--- a/src/sentry/integrations/jira/integration.py
+++ b/src/sentry/integrations/jira/integration.py
@@ -3,6 +3,7 @@
 import logging
 import six
 from operator import attrgetter
+import re
 
 from django.conf import settings
 from django.core.urlresolvers import reverse
@@ -24,6 +25,7 @@
 )
 from sentry.integrations.issues import IssueSyncMixin
 from sentry.models import IntegrationExternalProject, Organization, OrganizationIntegration, User
+from sentry.utils.compat import filter
 from sentry.utils.http import absolute_uri
 from sentry.utils.decorators import classproperty
 
@@ -102,6 +104,7 @@ class JiraIntegration(IntegrationInstallation, IssueSyncMixin):
     inbound_status_key = "sync_status_reverse"
     outbound_assignee_key = "sync_forward_assignment"
     inbound_assignee_key = "sync_reverse_assignment"
+    issues_ignored_fields_key = "issues_ignored_fields"
 
     @classproperty
     def use_email_scope(cls):
@@ -163,6 +166,15 @@ def get_organization_config(self):
                     "When a ticket is assigned in Jira, assign its linked Sentry issue to the same user."
                 ),
             },
+            {
+                "name": self.issues_ignored_fields_key,
+                "label": "Ignored Fields",
+                "type": "textarea",
+                "placeholder": _('e.g. "components, security, customfield_10006"'),
+                "help": _(
+                    "Comma-separated list of Jira fields that you don't want to show in issue creation form"
+                ),
+            },
         ]
 
         client = self.get_client()
@@ -220,6 +232,17 @@ def update_organization_config(self, data):
                     unresolved_status=statuses["on_unresolve"],
                 )
 
+        if self.issues_ignored_fields_key in data:
+            ignored_fields_text = data.pop(self.issues_ignored_fields_key)
+            # While we describe the config as a "comma-separated list", users are likely to
+            # accidentally use newlines, so we explicitly handle that case. On page
+            # refresh, they will see how it got interpreted as `get_config_data` will
+            # re-serialize the config as a comma-separated list.
+            ignored_fields_list = filter(
+                None, [field.strip() for field in re.split(r"[,\n\r]+", ignored_fields_text)]
+            )
+            data[self.issues_ignored_fields_key] = ignored_fields_list
+
         config.update(data)
         self.org_integration.update(config=config)
 
@@ -235,6 +258,9 @@ def get_config_data(self):
                 "on_resolve": pm.resolved_status,
             }
         config["sync_status_forward"] = sync_status_forward
+        config[self.issues_ignored_fields_key] = ", ".join(
+            config.get(self.issues_ignored_fields_key, "")
+        )
         return config
 
     def sync_metadata(self):
@@ -276,6 +302,9 @@ def get_persisted_default_config_fields(self):
     def get_persisted_user_default_config_fields(self):
         return ["reporter"]
 
+    def get_persisted_ignored_fields(self):
+        return self.org_integration.config.get(self.issues_ignored_fields_key, [])
+
     def get_group_description(self, group, event, **kwargs):
         output = [
             u"Sentry Issue: [{}|{}]".format(
@@ -575,15 +604,13 @@ def get_create_issue_config(self, group, user, **kwargs):
 
         # title is renamed to summary before sending to Jira
         standard_fields = [f["name"] for f in fields] + ["summary"]
-
-        # TODO(jess): are we going to allow ignored fields?
-        # ignored_fields = (self.get_option('ignored_fields', group.project) or '').split(',')
         ignored_fields = set(
             k
             for k, v in six.iteritems(issue_type_meta["fields"])
             if v["name"] in HIDDEN_ISSUE_FIELDS["names"]
         )
         ignored_fields.update(HIDDEN_ISSUE_FIELDS["keys"])
+        ignored_fields.update(self.get_persisted_ignored_fields())
 
         # apply ordering to fields based on some known built-in Jira fields.
         # otherwise weird ordering occurs.


Test patch:
diff --git a/tests/sentry/integrations/jira/test_integration.py b/tests/sentry/integrations/jira/test_integration.py
--- a/tests/sentry/integrations/jira/test_integration.py
+++ b/tests/sentry/integrations/jira/test_integration.py
@@ -544,6 +544,52 @@ def get_client():
                 },
             ]
 
+    def test_get_create_issue_config_with_ignored_fields(self):
+        org = self.organization
+        self.login_as(self.user)
+        event = self.store_event(
+            data={
+                "event_id": "a" * 32,
+                "message": "message",
+                "timestamp": self.min_ago,
+                "stacktrace": copy.deepcopy(DEFAULT_EVENT_DATA["stacktrace"]),
+            },
+            project_id=self.project.id,
+        )
+        group = event.group
+        installation = self.integration.get_installation(org.id)
+
+        def get_client():
+            return MockJiraApiClient()
+
+        with mock.patch.object(installation, "get_client", get_client):
+            # Initially all fields are present
+            fields = installation.get_create_issue_config(group, self.user)
+            field_names = [field["name"] for field in fields]
+            assert field_names == [
+                "project",
+                "title",
+                "description",
+                "issuetype",
+                "labels",
+                "customfield_10200",
+                "customfield_10300",
+            ]
+
+            installation.org_integration.config = {"issues_ignored_fields": ["customfield_10200"]}
+            # After ignoring "customfield_10200", it no longer shows up
+            installation.org_integration.save()
+            fields = installation.get_create_issue_config(group, self.user)
+            field_names = [field["name"] for field in fields]
+            assert field_names == [
+                "project",
+                "title",
+                "description",
+                "issuetype",
+                "labels",
+                "customfield_10300",
+            ]
+
     def test_get_create_issue_config_with_default_and_param(self):
         org = self.organization
         self.login_as(self.user)
@@ -913,7 +959,7 @@ def test_sync_assignee_outbound_use_email_api(self):
         assert assign_issue_response.status_code == 200
         assert assign_issue_response.request.body == b'{"accountId": "deadbeef123"}'
 
-    def test_update_organization_config(self):
+    def test_update_organization_config_sync_keys(self):
         org = self.organization
         self.login_as(self.user)
 
@@ -1028,6 +1074,43 @@ def test_update_organization_config(self):
             == 0
         )
 
+    def test_update_organization_config_issues_keys(self):
+        org = self.organization
+        self.login_as(self.user)
+
+        integration = Integration.objects.create(provider="jira", name="Example Jira")
+        integration.add_organization(org, self.user)
+
+        installation = integration.get_installation(org.id)
+        org_integration = OrganizationIntegration.objects.get(
+            organization_id=org.id, integration_id=integration.id
+        )
+        assert "issues_ignored_fields" not in org_integration.config
+
+        # Parses user-supplied CSV
+        installation.update_organization_config(
+            {"issues_ignored_fields": "\nhello world ,,\ngoodnight\nmoon , ,"}
+        )
+        org_integration = OrganizationIntegration.objects.get(
+            organization_id=org.id, integration_id=integration.id
+        )
+        assert org_integration.config.get("issues_ignored_fields") == [
+            "hello world",
+            "goodnight",
+            "moon",
+        ]
+
+        # No-ops if updated value is not specified
+        installation.update_organization_config({})
+        org_integration = OrganizationIntegration.objects.get(
+            organization_id=org.id, integration_id=integration.id
+        )
+        assert org_integration.config.get("issues_ignored_fields") == [
+            "hello world",
+            "goodnight",
+            "moon",
+        ]
+
     def test_get_config_data(self):
         org = self.organization
         self.login_as(self.user)
@@ -1063,8 +1146,34 @@ def test_get_config_data(self):
             "sync_reverse_assignment": True,
             "sync_status_reverse": True,
             "sync_status_forward": {"12345": {"on_resolve": "done", "on_unresolve": "in_progress"}},
+            "issues_ignored_fields": "",
         }
 
+    def test_get_config_data_issues_keys(self):
+        org = self.organization
+        self.login_as(self.user)
+
+        integration = Integration.objects.create(provider="jira", name="Example Jira")
+        integration.add_organization(org, self.user)
+
+        installation = integration.get_installation(org.id)
+        org_integration = OrganizationIntegration.objects.get(
+            organization_id=org.id, integration_id=integration.id
+        )
+
+        # If config has not be configured yet, uses empty string fallback
+        assert "issues_ignored_fields" not in org_integration.config
+        assert installation.get_config_data().get("issues_ignored_fields") == ""
+
+        # List is serialized as comma-separated list
+        org_integration.config["issues_ignored_fields"] = ["hello world", "goodnight", "moon"]
+        org_integration.save()
+        installation = integration.get_installation(org.id)
+        assert (
+            installation.get_config_data().get("issues_ignored_fields")
+            == "hello world, goodnight, moon"
+        )
+
     def test_create_comment(self):
         org = self.organization