Repository: zulip/zulip

Issue:
API documentation doesn't display documentation for drafts parameters usefully
Following #18074, we have API documentation for the new drafts synchronization endpoints.  However, while the `GET` endpoint renders the detailed documentation about individual fields within a `draft` object, the other endpoints do not, e.g.:

![image](https://user-images.githubusercontent.com/2746074/127413043-641fc8ec-0519-4a8a-9040-ab24fec2fa7e.png)



Discussion:
Hello @zulip/server-api members, this issue was labeled with the "area: documentation (api and integrations)" label, so you may want to check it out!

<!-- areaLabelAddition -->
zulipbot claim
@timabbott - I think the issue is in the `render_table` function in `zerver/lib/markdown/api_arguments_table_generator.py`. Unlike the `render_table` function for return values (`zerver/lib/markdown/api_return_values_table_generator.py`), the parameters function doesn't recursively call on itself when it encounters an 'object' - like a draft in this issue. Generally, this isn't an issue because the majority of parameters in the currently documented endpoints are basic data types (boolean, string, int).

@EmmalineLake - have you been looking into this too? I noticed you tried to put a claim on the issue last week. If so, let me know if I can help. If not, then I might try to see what a fix might look like.
I have been looking into it, and it is funny you commented today, because I've been working on it all day.  It is more of a stretch project for me than I had anticipated, but I would love to work on it together, if you are up for working with somewhat of a newbie.  I would definitely benefit from help.  Let me know what you think.
Totally! I'm also a newbie, so no worries there. I'll send you a PM in chat.zulip so that we don't clutter up the github issue.
@timabbott 
This is a clarification question.  I want to make sure  Lauryn and I are on the right track. 

In the `GET` endpoint, the fields (items) within the draft are in the "Responses" section, and rendered on the webpage.  (YAML  line 4436 and https://chat.zulip.org/api/get-drafts)
In the `POST` and `PATCH` endpoints, the fields (item) within the draft are contained in the "Parameters" section, and aren't rendered at all.  (YAML lines 4489 and https://chat.zulip.org/api/create-drafts ; 4964 and https://chat.zulip.org/api/edit-draft).
In the `DELETE` endpoint, the fields within the draft aren't in the YAML or the webapge. (https://chat.zulip.org/api/delete-draft).

The goal is to list the draft schemas in the parameter section of the respective webpages? 
If so , should the formatting match the current parameter formatting or look more like the response formatting from the `GET` page?  

If this is the case, would it make sense to do the following? 
In the `api_arguments_table_generator` file:
1. Add and format a `<div>` above the example in the HTML  in the `render_table` function to contain this information on the website? line 109
2. Add a line to the `table.append` function (wording pending)? line166
3.  Add a recursive call to the `data type` (?) section of the `arguments` for-loop to loop through the draft object? line 128

Thanks in advance!






I don't know this code super well; @orientor or @MSurfer20 is likely better positioned to help.  I think adding a missing recursive call makes sense, but I haven't looked at the code myself.
@orientor or @MSurfer20 
See this [CZO chat](https://chat.zulip.org/#narrow/stream/19-documentation/topic/rendering.20api.20documentation.20from.20YAML) for further questions.  The above question isn't correct.
Thanks!

Resolving patch:
diff --git a/zerver/lib/markdown/api_arguments_table_generator.py b/zerver/lib/markdown/api_arguments_table_generator.py
--- a/zerver/lib/markdown/api_arguments_table_generator.py
+++ b/zerver/lib/markdown/api_arguments_table_generator.py
@@ -17,6 +17,37 @@
 
 REGEXP = re.compile(r"\{generate_api_arguments_table\|\s*(.+?)\s*\|\s*(.+)\s*\}")
 
+API_PARAMETER_TEMPLATE = """
+<div class="api-argument" id="parameter-{argument}">
+    <p class="api-argument-name"><strong>{argument}</strong> <span class="api-field-type">{type}</span> {required} {deprecated} <a href="#parameter-{argument}" class="api-argument-hover-link"><i class="fa fa-chain"></i></a></p>
+    <div class="api-example">
+        <span class="api-argument-example-label">Example</span>: <code>{example}</code>
+    </div>
+    <div class="api-description">{description}{object_details}</div>
+    <hr>
+</div>
+""".strip()
+
+OBJECT_DETAILS_TEMPLATE = """
+<p><strong>{argument}</strong> object details:</p>
+<ul>
+{values}
+</ul>
+""".strip()
+
+OBJECT_LIST_ITEM_TEMPLATE = """
+<li>
+<code>{value}</code>: <span class=api-field-type>{data_type}</span> {description}{object_details}
+</li>
+""".strip()
+
+OBJECT_DESCRIPTION_TEMPLATE = """
+{description}
+<p>{additional_information}</p>
+""".strip()
+
+OBJECT_CODE_TEMPLATE = "<code>{value}</code>".strip()
+
 
 class MarkdownArgumentsTableGenerator(Extension):
     def __init__(self, configs: Mapping[str, Any] = {}) -> None:
@@ -80,7 +111,7 @@ def run(self, lines: List[str]) -> List[str]:
                         arguments = json_obj[doc_name]
 
                 if arguments:
-                    text = self.render_table(arguments)
+                    text = self.render_parameters(arguments)
                 # We want to show this message only if the parameters
                 # description doesn't say anything else.
                 elif is_openapi_format and get_parameters_description(endpoint, method) == "":
@@ -101,22 +132,13 @@ def run(self, lines: List[str]) -> List[str]:
                 done = True
         return lines
 
-    def render_table(self, arguments: Sequence[Mapping[str, Any]]) -> List[str]:
-        # TODO: Fix naming now that this no longer renders a table.
-        table = []
-        argument_template = """
-<div class="api-argument" id="parameter-{argument}">
-    <p class="api-argument-name"><strong>{argument}</strong> <span class="api-field-type">{type}</span> {required} {deprecated} <a href="#parameter-{argument}" class="api-argument-hover-link"><i class="fa fa-chain"></i></a></p>
-    <div class="api-example">
-        <span class="api-argument-example-label">Example</span>: <code>{example}</code>
-    </div>
-    <div class="api-description">{description}</div>
-    <hr>
-</div>"""
+    def render_parameters(self, arguments: Sequence[Mapping[str, Any]]) -> List[str]:
+        parameters = []
 
         md_engine = markdown.Markdown(extensions=[])
         arguments = sorted(arguments, key=lambda argument: "deprecated" in argument)
         for argument in arguments:
+            name = argument.get("argument") or argument.get("name")
             description = argument["description"]
             oneof = ["`" + str(item) + "`" for item in argument.get("schema", {}).get("enum", [])]
             if oneof:
@@ -163,18 +185,100 @@ def render_table(self, arguments: Sequence[Mapping[str, Any]]) -> List[str]:
             else:
                 deprecated_block = ""
 
-            table.append(
-                argument_template.format(
-                    argument=argument.get("argument") or argument.get("name"),
+            object_block = ""
+            # TODO: There are some endpoint parameters with object properties
+            # that are not defined in `zerver/openapi/zulip.yaml`
+            if "object" in data_type:
+                if "schema" in argument:
+                    object_schema = argument["schema"]
+                else:
+                    object_schema = argument["content"]["application/json"]["schema"]
+
+                if "items" in object_schema and "properties" in object_schema["items"]:
+                    object_block = self.render_object_details(object_schema["items"], str(name))
+                elif "properties" in object_schema:
+                    object_block = self.render_object_details(object_schema, str(name))
+
+            parameters.append(
+                API_PARAMETER_TEMPLATE.format(
+                    argument=name,
                     example=escape_html(example),
                     required=required_block,
                     deprecated=deprecated_block,
                     description=md_engine.convert(description),
-                    type=data_type,
+                    type=(data_type),
+                    object_details=object_block,
+                )
+            )
+
+        return parameters
+
+    def render_object_details(self, schema: Mapping[str, Any], name: str) -> str:
+        md_engine = markdown.Markdown(extensions=[])
+        li_elements = []
+
+        object_values = schema.get("properties", {})
+        for value in object_values:
+
+            description = ""
+            if "description" in object_values[value]:
+                description = object_values[value]["description"]
+
+            # check for default, enum, required or example in documentation
+            additions: List[str] = []
+
+            default = object_values.get(value, {}).get("default")
+            if default is not None:
+                formatted_default = OBJECT_CODE_TEMPLATE.format(value=json.dumps(default))
+                additions += f"Defaults to {formatted_default}. "
+
+            enums = object_values.get(value, {}).get("enum")
+            if enums is not None:
+                formatted_enums = [
+                    OBJECT_CODE_TEMPLATE.format(value=json.dumps(enum)) for enum in enums
+                ]
+                additions += "Must be one of: {}. ".format(", ".join(formatted_enums))
+
+            if "required" in schema:
+                if value in schema["required"]:
+                    additions += "Required value. "
+                else:
+                    additions += "Optional value. "
+
+            if "example" in object_values[value]:
+                example = json.dumps(object_values[value]["example"])
+                formatted_example = OBJECT_CODE_TEMPLATE.format(value=escape_html(example))
+                additions += f"Example: {formatted_example}"
+
+            if len(additions) > 0:
+                additional_information = "".join(additions).strip()
+                description_final = OBJECT_DESCRIPTION_TEMPLATE.format(
+                    description=md_engine.convert(description),
+                    additional_information=additional_information,
                 )
+            else:
+                description_final = md_engine.convert(description)
+
+            data_type = generate_data_type(object_values[value])
+
+            details = ""
+            if "object" in data_type and "properties" in object_values[value]:
+                details += self.render_object_details(object_values[value], str(value))
+
+            li = OBJECT_LIST_ITEM_TEMPLATE.format(
+                value=value,
+                data_type=data_type,
+                description=description_final,
+                object_details=details,
             )
 
-        return table
+            li_elements.append(li)
+
+        object_details = OBJECT_DETAILS_TEMPLATE.format(
+            argument=name,
+            values="\n".join(li_elements),
+        )
+        return object_details
 
 
 def makeExtension(*args: Any, **kwargs: str) -> MarkdownArgumentsTableGenerator:


Test patch: