Repository: zulip/zulip

Issue:
Change the API endpoint of custom profile fields to not have the name/hint/etc.
Change the API endpoint of custom profile fields to not have the name/hint/etc. to be passed in a request as mandatory, instead of supporting passing them as optional.

_Originally posted by @timabbott in https://github.com/zulip/zulip/issues/21588#issuecomment-1253017422_


Discussion:
@zulipbot claim
Hello @otemitay!

Thanks for your interest in Zulip! You have attempted to claim an issue without the label "help wanted". You can only claim and submit pull requests for issues with the [help wanted](https://github.com/zulip/zulip/issues?q=is%3Aopen+is%3Aissue+no%3Aassignee+label%3A%22help+wanted%22) label.

If this is your first time here, we recommend reading our [guide for new contributors](https://zulip.readthedocs.io/en/latest/overview/contributing.html) before getting started.

Hello @zulip/server-api members, this issue was labeled with the "area: api" label, so you may want to check it out!

<!-- areaLabelAddition -->

@zulipbot claim
Welcome to Zulip, @gopivaibhav! We just sent you an invite to collaborate on this repository at https://github.com/zulip/zulip/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:

- Join me on the [Zulip developers' server](https://chat.zulip.org), to get help, chat about this issue, and meet the other developers.
- [Unwatch this repository](https://help.github.com/articles/unwatching-repositories/), so that you don't get 100 emails a day.

As you work on this issue, you'll also want to refer to the [Zulip code contribution guide](https://zulip.readthedocs.io/en/latest/contributing/index.html), as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!

@gopivaibhav You have been unassigned from this issue because you have not made any updates for over 14 days. Please feel free to reclaim the issue if you decide to pick up again. Thanks!

@zulipbot claim
Welcome to Zulip, @dhruv-goyal-10! We just sent you an invite to collaborate on this repository at https://github.com/zulip/zulip/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:

- Join me on the [Zulip developers' server](https://chat.zulip.org), to get help, chat about this issue, and meet the other developers.
- [Unwatch this repository](https://help.github.com/articles/unwatching-repositories/), so that you don't get 100 emails a day.

As you work on this issue, you'll also want to refer to the [Zulip code contribution guide](https://zulip.readthedocs.io/en/latest/contributing/index.html), as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!

@dhruv-goyal-10 You have been unassigned from this issue because you have not made any updates for over 14 days. Please feel free to reclaim the issue if you decide to pick up again. Thanks!

@yogesh-sirsat @timabbott Can I request more context on this issue? Why do we want to make this change and what is the expected end result.

(Note: I wasn't able to find the convo related this on CZO that the other PR mentions, so asking here)
For the `PATCH /realm/profile_fields/{field.id}` endpoint, it is required to pass `name` and `hint` fields in the request data even if we want to change some other field like `display_in_profile_summary`. I have not tested the behavior on not passing these fields - but from reading the code I think the behavior is - 
1) If we do not pass `name` in request data, an error will be raised because we do not allow empty names.
2) If we do not pass `hint` in request data, the `hint` field for the custom profile field becomes empty string.

Instead the API should behave that if any of the fields is not present in the request data, then they should remain same as before like what we do for most of the other endpoints, like `PATCH /realm` endpoint for updating realm settings.
Thanks Sahil ! 

@zulipbot claim
@shubham-padia You have been unassigned from this issue because you have not made any updates for over 14 days. Please feel free to reclaim the issue if you decide to pick up again. Thanks!


Resolving patch:
diff --git a/version.py b/version.py
--- a/version.py
+++ b/version.py
@@ -33,7 +33,7 @@
 # Changes should be accompanied by documentation explaining what the
 # new level means in api_docs/changelog.md, as well as "**Changes**"
 # entries in the endpoint's documentation in `zulip.yaml`.
-API_FEATURE_LEVEL = 251
+API_FEATURE_LEVEL = 252
 
 # Bump the minor PROVISION_VERSION to indicate that folks should provision
 # only when going from an old version of the code to a newer version. Bump
diff --git a/zerver/actions/custom_profile_fields.py b/zerver/actions/custom_profile_fields.py
--- a/zerver/actions/custom_profile_fields.py
+++ b/zerver/actions/custom_profile_fields.py
@@ -101,21 +101,34 @@ def remove_custom_profile_field_value_if_required(
 def try_update_realm_custom_profile_field(
     realm: Realm,
     field: CustomProfileField,
-    name: str,
-    hint: str = "",
+    name: Optional[str] = None,
+    hint: Optional[str] = None,
     field_data: Optional[ProfileFieldData] = None,
-    display_in_profile_summary: bool = False,
-    required: bool = False,
+    display_in_profile_summary: Optional[bool] = None,
+    required: Optional[bool] = None,
 ) -> None:
-    field.name = name
-    field.hint = hint
-    field.display_in_profile_summary = display_in_profile_summary
-    field.required = required
-    if field.field_type in (CustomProfileField.SELECT, CustomProfileField.EXTERNAL_ACCOUNT):
-        if field.field_type == CustomProfileField.SELECT:
-            assert field_data is not None
+    if name is not None:
+        field.name = name
+    if hint is not None:
+        field.hint = hint
+    if required is not None:
+        field.required = required
+    if display_in_profile_summary is not None:
+        field.display_in_profile_summary = display_in_profile_summary
+
+    if field.field_type in (
+        CustomProfileField.SELECT,
+        CustomProfileField.EXTERNAL_ACCOUNT,
+    ):
+        # If field_data is None, field_data is unchanged and there is no need for
+        # comparing field_data values.
+        if field_data is not None and field.field_type == CustomProfileField.SELECT:
             remove_custom_profile_field_value_if_required(field, field_data)
-        field.field_data = orjson.dumps(field_data or {}).decode()
+
+        # If field.field_data is the default empty string, we will set field_data
+        # to an empty dict.
+        if field_data is not None or field.field_data == "":
+            field.field_data = orjson.dumps(field_data or {}).decode()
     field.save()
     notify_realm_custom_profile_fields(realm)
 
diff --git a/zerver/views/custom_profile_fields.py b/zerver/views/custom_profile_fields.py
--- a/zerver/views/custom_profile_fields.py
+++ b/zerver/views/custom_profile_fields.py
@@ -116,21 +116,55 @@ def validate_custom_profile_field(
     validate_display_in_profile_summary_field(field_type, display_in_profile_summary)
 
 
+def validate_custom_profile_field_update(
+    field: CustomProfileField,
+    display_in_profile_summary: Optional[bool] = None,
+    field_data: Optional[ProfileFieldData] = None,
+    name: Optional[str] = None,
+    hint: Optional[str] = None,
+) -> None:
+    if name is None:
+        name = field.name
+    if hint is None:
+        hint = field.hint
+    if field_data is None:
+        if field.field_data == "":
+            # We're passing this just for validation, sinec the function won't
+            # accept a string. This won't change the actual value.
+            field_data = {}
+        else:
+            field_data = orjson.loads(field.field_data)
+    if display_in_profile_summary is None:
+        display_in_profile_summary = field.display_in_profile_summary
+
+    assert field_data is not None
+    validate_custom_profile_field(
+        name,
+        hint,
+        field.field_type,
+        field_data,
+        display_in_profile_summary,
+    )
+
+
 check_profile_field_data: Validator[ProfileFieldData] = check_dict(
     value_validator=check_union([check_dict(value_validator=check_string), check_string])
 )
 
 
 def update_only_display_in_profile_summary(
-    requested_name: str,
-    requested_hint: str,
-    requested_field_data: ProfileFieldData,
     existing_field: CustomProfileField,
+    requested_field_data: Optional[ProfileFieldData] = None,
+    requested_name: Optional[str] = None,
+    requested_hint: Optional[str] = None,
 ) -> bool:
     if (
-        requested_name != existing_field.name
-        or requested_hint != existing_field.hint
-        or requested_field_data != orjson.loads(existing_field.field_data)
+        (requested_name is not None and requested_name != existing_field.name)
+        or (requested_hint is not None and requested_hint != existing_field.hint)
+        or (
+            requested_field_data is not None
+            and requested_field_data != orjson.loads(existing_field.field_data)
+        )
     ):
         return False
     return True
@@ -208,11 +242,13 @@ def update_realm_custom_profile_field(
     request: HttpRequest,
     user_profile: UserProfile,
     field_id: int,
-    name: str = REQ(default="", converter=lambda var_name, x: x.strip()),
-    hint: str = REQ(default=""),
-    field_data: ProfileFieldData = REQ(default={}, json_validator=check_profile_field_data),
-    display_in_profile_summary: bool = REQ(default=False, json_validator=check_bool),
-    required: bool = REQ(default=False, json_validator=check_bool),
+    name: Optional[str] = REQ(default=None, converter=lambda var_name, x: x.strip()),
+    hint: Optional[str] = REQ(default=None),
+    field_data: Optional[ProfileFieldData] = REQ(
+        default=None, json_validator=check_profile_field_data
+    ),
+    required: Optional[bool] = REQ(default=None, json_validator=check_bool),
+    display_in_profile_summary: Optional[bool] = REQ(default=None, json_validator=check_bool),
 ) -> HttpResponse:
     realm = user_profile.realm
     try:
@@ -233,20 +269,21 @@ def update_realm_custom_profile_field(
         # of default external account types, but not any others.
         #
         # TODO: Make the name/hint/field_data parameters optional, and
-        # just require that None was passed for all of them for this case.
+        # explicitly require that the client passes None for all of them for this case.
+        # Right now, for name/hint/field_data we allow the client to send the existing
+        # values for the respective fields. After this TODO is done, we will only allow
+        # the client to pass None values if the field is unchanged.
         and is_default_external_field(field.field_type, orjson.loads(field.field_data))
-        and not update_only_display_in_profile_summary(name, hint, field_data, field)
+        and not update_only_display_in_profile_summary(field, field_data, name, hint)
     ):
         raise JsonableError(_("Default custom field cannot be updated."))
 
-    validate_custom_profile_field(
-        name, hint, field.field_type, field_data, display_in_profile_summary
-    )
+    validate_custom_profile_field_update(field, display_in_profile_summary, field_data, name, hint)
     try:
         try_update_realm_custom_profile_field(
-            realm,
-            field,
-            name,
+            realm=realm,
+            field=field,
+            name=name,
             hint=hint,
             field_data=field_data,
             display_in_profile_summary=display_in_profile_summary,


Test patch:
diff --git a/zerver/tests/test_custom_profile_data.py b/zerver/tests/test_custom_profile_data.py
--- a/zerver/tests/test_custom_profile_data.py
+++ b/zerver/tests/test_custom_profile_data.py
@@ -494,7 +494,6 @@ def test_update(self) -> None:
         result = self.client_patch(
             f"/json/realm/profile_fields/{field.id}",
             info={
-                "name": "New phone number",
                 "hint": "*" * 81,
             },
         )
@@ -504,8 +503,6 @@ def test_update(self) -> None:
         result = self.client_patch(
             f"/json/realm/profile_fields/{field.id}",
             info={
-                "name": "New phone number",
-                "hint": "New contact number",
                 "display_in_profile_summary": "invalid value",
             },
         )
@@ -515,8 +512,6 @@ def test_update(self) -> None:
         result = self.client_patch(
             f"/json/realm/profile_fields/{field.id}",
             info={
-                "name": "New phone number",
-                "hint": "New contact number",
                 "required": "invalid value",
             },
         )
@@ -533,8 +528,7 @@ def test_update(self) -> None:
             },
         )
         self.assert_json_success(result)
-
-        field = CustomProfileField.objects.get(id=field.id, realm=realm)
+        field.refresh_from_db()
         self.assertEqual(CustomProfileField.objects.count(), self.original_count)
         self.assertEqual(field.name, "New phone number")
         self.assertEqual(field.hint, "New contact number")
@@ -542,6 +536,18 @@ def test_update(self) -> None:
         self.assertEqual(field.display_in_profile_summary, True)
         self.assertEqual(field.required, True)
 
+        # Not sending required should not set it to false.
+        result = self.client_patch(
+            f"/json/realm/profile_fields/{field.id}",
+            info={
+                "hint": "New hint",
+            },
+        )
+        self.assert_json_success(result)
+        field.refresh_from_db()
+        self.assertEqual(field.hint, "New hint")
+        self.assertEqual(field.required, True)
+
         result = self.client_patch(
             f"/json/realm/profile_fields/{field.id}",
             info={"name": "Name ", "display_in_profile_summary": "true"},
@@ -554,7 +560,7 @@ def test_update(self) -> None:
         field = CustomProfileField.objects.get(name="Favorite editor", realm=realm)
         result = self.client_patch(
             f"/json/realm/profile_fields/{field.id}",
-            info={"name": "Favorite editor", "field_data": "invalid"},
+            info={"field_data": "invalid"},
         )
         self.assert_json_error(result, 'Argument "field_data" is not valid JSON.')
 
@@ -566,7 +572,7 @@ def test_update(self) -> None:
         ).decode()
         result = self.client_patch(
             f"/json/realm/profile_fields/{field.id}",
-            info={"name": "Favorite editor", "field_data": field_data},
+            info={"field_data": field_data},
         )
         self.assert_json_error(result, "field_data is not a dict")
 
@@ -580,19 +586,28 @@ def test_update(self) -> None:
         result = self.client_patch(
             f"/json/realm/profile_fields/{field.id}",
             info={
-                "name": "Favorite editor",
                 "field_data": field_data,
                 "display_in_profile_summary": "true",
             },
         )
         self.assert_json_success(result)
 
+        # Not sending display_in_profile_summary should not set it to false.
+        result = self.client_patch(
+            f"/json/realm/profile_fields/{field.id}",
+            info={
+                "hint": "Fav editor",
+            },
+        )
+        field.refresh_from_db()
+        self.assertEqual(field.hint, "Fav editor")
+        self.assertEqual(field.display_in_profile_summary, True)
+        self.assert_json_success(result)
+
         field = CustomProfileField.objects.get(name="Birthday", realm=realm)
         result = self.client_patch(
             f"/json/realm/profile_fields/{field.id}",
             info={
-                "name": field.name,
-                "hint": field.hint,
                 "display_in_profile_summary": "true",
             },
         )
@@ -600,6 +615,26 @@ def test_update(self) -> None:
             result, "Only 2 custom profile fields can be displayed in the profile summary."
         )
 
+        # Empty string for hint should set it to an empty string
+        result = self.client_patch(
+            f"/json/realm/profile_fields/{field.id}",
+            info={"hint": ""},
+        )
+        self.assert_json_success(result)
+        field.refresh_from_db()
+        self.assertEqual(field.hint, "")
+
+        field = CustomProfileField.objects.get(name="Favorite editor", realm=realm)
+
+        # Empty field_data should not be allowed
+        result = self.client_patch(
+            f"/json/realm/profile_fields/{field.id}",
+            info={
+                "field_data": {},
+            },
+        )
+        self.assert_json_error(result, "Field must have at least one choice.")
+
     def test_update_is_aware_of_uniqueness(self) -> None:
         self.login("iago")
         realm = get_realm("zulip")
diff --git a/zerver/tests/test_events.py b/zerver/tests/test_events.py
--- a/zerver/tests/test_events.py
+++ b/zerver/tests/test_events.py
@@ -1390,7 +1390,11 @@ def test_custom_profile_fields_events(self) -> None:
 
         events = self.verify_action(
             lambda: try_update_realm_custom_profile_field(
-                realm, field, name, hint=hint, display_in_profile_summary=display_in_profile_summary
+                realm=realm,
+                field=field,
+                name=name,
+                hint=hint,
+                display_in_profile_summary=display_in_profile_summary,
             )
         )
         check_custom_profile_fields("events[0]", events[0])
@@ -1416,7 +1420,9 @@ def test_pronouns_type_support_in_custom_profile_fields_events(self) -> None:
 
         hint = "What pronouns should people use to refer you?"
         events = self.verify_action(
-            lambda: try_update_realm_custom_profile_field(realm, field, name, hint=hint),
+            lambda: try_update_realm_custom_profile_field(
+                realm=realm, field=field, name=name, hint=hint
+            ),
             pronouns_field_type_supported=False,
         )
         check_custom_profile_fields("events[0]", events[0])