Repository: ansible-collections/community.general

Issue:
Add capabilities to the Redfish SimpleUpdate redfish_command
### Summary

The Redfish Firmware Updater process described [here (See Chapter 6)](https://www.dmtf.org/sites/default/files/standards/documents/DSP2062_1.0.0.pdf) allows for asynchronous polling of the update job by querying the returned "TaskMonitor". Currently, the Redfish Command handled by:

https://github.com/ansible-collections/community.general/blob/8508e3fa6f433c74bedd8ccb1319d295485e6a7f/plugins/module_utils/redfish_utils.py#L1339-L1398

only returns True/False on the job submission, but no other details about the job. This adds extra complexity in searching for the Task after the fact and manually polling.

It would be great if the RF utils module could return Task Monitors where appropriate for long running jobs and It would be even more awesome if the module could abstract the async job process away from the user:

- Accept a "Wait for completion" parameter that would instruct the module to poll the Task and return status.
- Accept the `@Redfish.OperationApplyTime` parameter for more control over the update process timing.

See also [Redfish Task Presentation](https://www.dmtf.org/sites/default/files/Redfish%20School%20-%20Tasks_1.pdf)

### Issue Type

Feature Idea

### Component Name

Redfish Utils

### Additional Information

When using the URI module a SimpleUpdate returns:

```
{
...
  "json": {
    "@odata.context": "/redfish/v1/$metadata#Task.Task",
    "@odata.id": "/redfish/v1/TaskService/Tasks/2",
    "@odata.type": "#Task.v1_3_0.Task",
    "Description": "Update Task",
    "Id": "2",
    "Messages": [
      {}
    ],
    "Name": "Task 2",
    "Payload": {
      "HttpOperation": "POST",
      "JsonBody": "{\"ImageURI\":\"http://updatefile\"}",
      "TargetUri": "/redfish/v1/UpdateService/Actions/UpdateService.SimpleUpdate"
    },
    "StartTime": "2022-02-07T17:12:41Z",
    "TaskMonitor": "/redfish/v1/TaskService/TaskMonitors/2",
    "TaskState": "New"
  },
  "location": "https://1.2.3.4/redfish/v1/TaskService/TaskMonitors/2",
  "msg": "OK (unknown bytes)",
...
}
```

Where the `TaskMonitor` and `TaskState` are returned for further use.

```
The Redfish Command SimpleUpdate only returns:
TASK [Simple update] ************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
changed: [localhost] => {"attempts": 1, "changed": true, "msg": "Action was successful", "session": {}}

TASK [debug] ********************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "bios_upgrade": {
        "attempts": 1,
        "changed": true,
        "failed": false,
        "msg": "Action was successful",
        "session": {}
    }
}
```

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
None

If these files are incorrect, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
Agreed this should be added to the module. One complexity to consider though is the flow might require user intervention in order for the update process to make forward progress (such as performing system resets). Would it be reasonable to have the module issue the reset request automatically as it appears in the job? Having the playbook pause indefinitely for the user to do an operation via other methods doesn't seem like a good approach. Maybe we can make the assumption that if a user issues this type of request that they're prepared for the system to be reset?
@felixfontein do you have any guidance from an "Ansible best practices" perspective? Is it safe to assume that if a user runs a playbook, they want the desired outcome regardless of the side effects? If not, is it okay for a playbook to sit and wait for a user to perform some other operation (like invoke a system reset)?
@mraineri hmm, that's tricky. It depends a lot on what the side-effects are, resp. how obvious they are :) If the command is 'reset system' it's not surprising that the system is restarted. If the command is 'upgrade firmware' some might only expect that the upgrade will happen on the next restart, and not that the system is restarted right now. But also if the operation requires user interaction and the user isn't aware that it does, that wouldn't be very helpful.

I would definitely make the wait optional, and potentially outsource it to another module, like have the module return some task monitor ID, and have another module that accepts one or more task monitor IDs and waits for all of them to complete. That would keep the original module simpler and allow to add more complexity to the wait module without impacting the original module. (You might want to have wait timeouts, or if multiple task monitor IDs are specified just wait until N of them finished and not all, or ...)
@felixfontein I've been thinking about how to approach this a bit more based on your feedback. Here is what I'm thinking:

1) Change SimpleUpdate to add new parameters that are allowed by the Redfish standard (at least from this issue, we might only be missing OperationApplyTime).
2) Add to the output of SimpleUpdate the task handle returned by the service.
3) Add a new command (proposing "CheckUpdateProgress") that takes a task handle for querying the service; the output will be the progress of the update, if any resets are needed, other information, and the task handle (in case the update progress is redirected elsewhere).
4) Add a new command (proposing "PerformInterventions", but I think the name needs some work) that takes a task handle, and it will walk the status reported by the service to perform any requested operations (such as reset operations to devices or systems).

Essentially using these commands from an end-to-end perspective would look something like this:

```
update_status, handle = SimpleUpdate()
while( handle ) {
    update_status, handle, intervention_required = CheckUpdateProgress( handle )
    if intervention_required {
        PerformInterventions( handle )
    }
}
print_update_results( update_status )
```

I'm considering this approach simply because all of the above operations will not block the caller waiting for some event, so it gives the users the flexibility to decide how frequently to poll, how to handle pauses in the update progress, and other types of situations where you might need to wait for something to happen.
@mraineri I think that sounds great! Having these 'building blocks' enables users to do everything they want, even though it might require some extra work on the playbook / role level (multiple tasks for one operation).
Thanks for the feedback; I'll work on this when I'm back to work.

Resolving patch:
diff --git a/plugins/module_utils/redfish_utils.py b/plugins/module_utils/redfish_utils.py
--- a/plugins/module_utils/redfish_utils.py
+++ b/plugins/module_utils/redfish_utils.py
@@ -143,7 +143,7 @@ def get_request(self, uri):
         except Exception as e:
             return {'ret': False,
                     'msg': "Failed GET request to '%s': '%s'" % (uri, to_text(e))}
-        return {'ret': True, 'data': data, 'headers': headers}
+        return {'ret': True, 'data': data, 'headers': headers, 'resp': resp}
 
     def post_request(self, uri, pyld):
         req_headers = dict(POST_HEADERS)
@@ -155,6 +155,11 @@ def post_request(self, uri, pyld):
                             force_basic_auth=basic_auth, validate_certs=False,
                             follow_redirects='all',
                             use_proxy=True, timeout=self.timeout)
+            try:
+                data = json.loads(to_native(resp.read()))
+            except Exception as e:
+                # No response data; this is okay in many cases
+                data = None
             headers = dict((k.lower(), v) for (k, v) in resp.info().items())
         except HTTPError as e:
             msg = self._get_extended_message(e)
@@ -169,7 +174,7 @@ def post_request(self, uri, pyld):
         except Exception as e:
             return {'ret': False,
                     'msg': "Failed POST request to '%s': '%s'" % (uri, to_text(e))}
-        return {'ret': True, 'headers': headers, 'resp': resp}
+        return {'ret': True, 'data': data, 'headers': headers, 'resp': resp}
 
     def patch_request(self, uri, pyld, check_pyld=False):
         req_headers = dict(PATCH_HEADERS)
@@ -1384,11 +1389,82 @@ def get_software_inventory(self):
         else:
             return self._software_inventory(self.software_uri)
 
+    def _operation_results(self, response, data, handle=None):
+        """
+        Builds the results for an operation from task, job, or action response.
+
+        :param response: HTTP response object
+        :param data: HTTP response data
+        :param handle: The task or job handle that was last used
+        :return: dict containing operation results
+        """
+
+        operation_results = {'status': None, 'messages': [], 'handle': None, 'ret': True,
+                             'resets_requested': []}
+
+        if response.status == 204:
+            # No content; successful, but nothing to return
+            # Use the Redfish "Completed" enum from TaskState for the operation status
+            operation_results['status'] = 'Completed'
+        else:
+            # Parse the response body for details
+
+            # Determine the next handle, if any
+            operation_results['handle'] = handle
+            if response.status == 202:
+                # Task generated; get the task monitor URI
+                operation_results['handle'] = response.getheader('Location', handle)
+
+            # Pull out the status and messages based on the body format
+            if data is not None:
+                response_type = data.get('@odata.type', '')
+                if response_type.startswith('#Task.') or response_type.startswith('#Job.'):
+                    # Task and Job have similar enough structures to treat the same
+                    operation_results['status'] = data.get('TaskState', data.get('JobState'))
+                    operation_results['messages'] = data.get('Messages', [])
+                else:
+                    # Error response body, which is a bit of a misnomer since it's used in successful action responses
+                    operation_results['status'] = 'Completed'
+                    if response.status >= 400:
+                        operation_results['status'] = 'Exception'
+                    operation_results['messages'] = data.get('error', {}).get('@Message.ExtendedInfo', [])
+            else:
+                # No response body (or malformed); build based on status code
+                operation_results['status'] = 'Completed'
+                if response.status == 202:
+                    operation_results['status'] = 'New'
+                elif response.status >= 400:
+                    operation_results['status'] = 'Exception'
+
+            # Clear out the handle if the operation is complete
+            if operation_results['status'] in ['Completed', 'Cancelled', 'Exception', 'Killed']:
+                operation_results['handle'] = None
+
+            # Scan the messages to see if next steps are needed
+            for message in operation_results['messages']:
+                message_id = message['MessageId']
+
+                if message_id.startswith('Update.1.') and message_id.endswith('.OperationTransitionedToJob'):
+                    # Operation rerouted to a job; update the status and handle
+                    operation_results['status'] = 'New'
+                    operation_results['handle'] = message['MessageArgs'][0]
+                    operation_results['resets_requested'] = []
+                    # No need to process other messages in this case
+                    break
+
+                if message_id.startswith('Base.1.') and message_id.endswith('.ResetRequired'):
+                    # A reset to some device is needed to continue the update
+                    reset = {'uri': message['MessageArgs'][0], 'type': message['MessageArgs'][1]}
+                    operation_results['resets_requested'].append(reset)
+
+        return operation_results
+
     def simple_update(self, update_opts):
         image_uri = update_opts.get('update_image_uri')
         protocol = update_opts.get('update_protocol')
         targets = update_opts.get('update_targets')
         creds = update_opts.get('update_creds')
+        apply_time = update_opts.get('update_apply_time')
 
         if not image_uri:
             return {'ret': False, 'msg':
@@ -1439,11 +1515,65 @@ def simple_update(self, update_opts):
                 payload["Username"] = creds.get('username')
             if creds.get('password'):
                 payload["Password"] = creds.get('password')
+        if apply_time:
+            payload["@Redfish.OperationApplyTime"] = apply_time
         response = self.post_request(self.root_uri + update_uri, payload)
         if response['ret'] is False:
             return response
         return {'ret': True, 'changed': True,
-                'msg': "SimpleUpdate requested"}
+                'msg': "SimpleUpdate requested",
+                'update_status': self._operation_results(response['resp'], response['data'])}
+
+    def get_update_status(self, update_handle):
+        """
+        Gets the status of an update operation.
+
+        :param handle: The task or job handle tracking the update
+        :return: dict containing the response of the update status
+        """
+
+        if not update_handle:
+            return {'ret': False, 'msg': 'Must provide a handle tracking the update.'}
+
+        # Get the task or job tracking the update
+        response = self.get_request(self.root_uri + update_handle)
+        if response['ret'] is False:
+            return response
+
+        # Inspect the response to build the update status
+        return self._operation_results(response['resp'], response['data'], update_handle)
+
+    def perform_requested_update_operations(self, update_handle):
+        """
+        Performs requested operations to allow the update to continue.
+
+        :param handle: The task or job handle tracking the update
+        :return: dict containing the result of the operations
+        """
+
+        # Get the current update status
+        update_status = self.get_update_status(update_handle)
+        if update_status['ret'] is False:
+            return update_status
+
+        changed = False
+
+        # Perform any requested updates
+        for reset in update_status['resets_requested']:
+            resp = self.post_request(self.root_uri + reset['uri'], {'ResetType': reset['type']})
+            if resp['ret'] is False:
+                # Override the 'changed' indicator since other resets may have
+                # been successful
+                resp['changed'] = changed
+                return resp
+            changed = True
+
+        msg = 'No operations required for the update'
+        if changed:
+            # Will need to consider finetuning this message if the scope of the
+            # requested operations grow over time
+            msg = 'One or more components reset to continue the update'
+        return {'ret': True, 'changed': changed, 'msg': msg}
 
     def get_bios_attributes(self, systems_uri):
         result = {}
diff --git a/plugins/modules/redfish_command.py b/plugins/modules/redfish_command.py
--- a/plugins/modules/redfish_command.py
+++ b/plugins/modules/redfish_command.py
@@ -161,6 +161,24 @@
         description:
           - Password for retrieving the update image.
         type: str
+  update_apply_time:
+    required: false
+    description:
+      - Time when to apply the update.
+    type: str
+    choices:
+      - Immediate
+      - OnReset
+      - AtMaintenanceWindowStart
+      - InMaintenanceWindowOnReset
+      - OnStartUpdateRequest
+    version_added: '6.1.0'
+  update_handle:
+    required: false
+    description:
+      - Handle to check the status of an update in progress.
+    type: str
+    version_added: '6.1.0'
   virtual_media:
     required: false
     description:
@@ -508,6 +526,15 @@
         username: operator
         password: supersecretpwd
 
+  - name: Perform requested operations to continue the update
+    community.general.redfish_command:
+      category: Update
+      command: PerformRequestedOperations
+      baseuri: "{{ baseuri }}"
+      username: "{{ username }}"
+      password: "{{ password }}"
+      update_handle: /redfish/v1/TaskService/TaskMonitors/735
+
   - name: Insert Virtual Media
     community.general.redfish_command:
       category: Systems
@@ -610,6 +637,20 @@
     returned: always
     type: str
     sample: "Action was successful"
+return_values:
+    description: Dictionary containing command-specific response data from the action.
+    returned: on success
+    type: dict
+    version_added: 6.1.0
+    sample: {
+        "update_status": {
+            "handle": "/redfish/v1/TaskService/TaskMonitors/735",
+            "messages": [],
+            "resets_requested": [],
+            "ret": true,
+            "status": "New"
+        }
+    }
 '''
 
 from ansible.module_utils.basic import AnsibleModule
@@ -630,12 +671,13 @@
     "Manager": ["GracefulRestart", "ClearLogs", "VirtualMediaInsert",
                 "VirtualMediaEject", "PowerOn", "PowerForceOff", "PowerForceRestart",
                 "PowerGracefulRestart", "PowerGracefulShutdown", "PowerReboot"],
-    "Update": ["SimpleUpdate"]
+    "Update": ["SimpleUpdate", "PerformRequestedOperations"],
 }
 
 
 def main():
     result = {}
+    return_values = {}
     module = AnsibleModule(
         argument_spec=dict(
             category=dict(required=True),
@@ -667,6 +709,9 @@ def main():
                     password=dict(no_log=True)
                 )
             ),
+            update_apply_time=dict(choices=['Immediate', 'OnReset', 'AtMaintenanceWindowStart',
+                                            'InMaintenanceWindowOnReset', 'OnStartUpdateRequest']),
+            update_handle=dict(),
             virtual_media=dict(
                 type='dict',
                 options=dict(
@@ -721,7 +766,9 @@ def main():
         'update_image_uri': module.params['update_image_uri'],
         'update_protocol': module.params['update_protocol'],
         'update_targets': module.params['update_targets'],
-        'update_creds': module.params['update_creds']
+        'update_creds': module.params['update_creds'],
+        'update_apply_time': module.params['update_apply_time'],
+        'update_handle': module.params['update_handle'],
     }
 
     # Boot override options
@@ -859,6 +906,10 @@ def main():
         for command in command_list:
             if command == "SimpleUpdate":
                 result = rf_utils.simple_update(update_opts)
+                if 'update_status' in result:
+                    return_values['update_status'] = result['update_status']
+            elif command == "PerformRequestedOperations":
+                result = rf_utils.perform_requested_update_operations(update_opts['update_handle'])
 
     # Return data back or fail with proper message
     if result['ret'] is True:
@@ -866,7 +917,8 @@ def main():
         changed = result.get('changed', True)
         session = result.get('session', dict())
         module.exit_json(changed=changed, session=session,
-                         msg='Action was successful')
+                         msg='Action was successful',
+                         return_values=return_values)
     else:
         module.fail_json(msg=to_native(result['msg']))
 
diff --git a/plugins/modules/redfish_info.py b/plugins/modules/redfish_info.py
--- a/plugins/modules/redfish_info.py
+++ b/plugins/modules/redfish_info.py
@@ -58,6 +58,12 @@
       - Timeout in seconds for HTTP requests to OOB controller.
     default: 10
     type: int
+  update_handle:
+    required: false
+    description:
+      - Handle to check the status of an update in progress.
+    type: str
+    version_added: '6.1.0'
 
 author: "Jose Delarosa (@jose-delarosa)"
 '''
@@ -247,6 +253,15 @@
       username: "{{ username }}"
       password: "{{ password }}"
 
+  - name: Get the status of an update operation
+    community.general.redfish_info:
+      category: Update
+      command: GetUpdateStatus
+      baseuri: "{{ baseuri }}"
+      username: "{{ username }}"
+      password: "{{ password }}"
+      update_handle: /redfish/v1/TaskService/TaskMonitors/735
+
   - name: Get Manager Services
     community.general.redfish_info:
       category: Manager
@@ -324,7 +339,8 @@
                 "GetChassisThermals", "GetChassisInventory", "GetHealthReport"],
     "Accounts": ["ListUsers"],
     "Sessions": ["GetSessions"],
-    "Update": ["GetFirmwareInventory", "GetFirmwareUpdateCapabilities", "GetSoftwareInventory"],
+    "Update": ["GetFirmwareInventory", "GetFirmwareUpdateCapabilities", "GetSoftwareInventory",
+               "GetUpdateStatus"],
     "Manager": ["GetManagerNicInventory", "GetVirtualMedia", "GetLogs", "GetNetworkProtocols",
                 "GetHealthReport", "GetHostInterfaces", "GetManagerInventory"],
 }
@@ -350,7 +366,8 @@ def main():
             username=dict(),
             password=dict(no_log=True),
             auth_token=dict(no_log=True),
-            timeout=dict(type='int', default=10)
+            timeout=dict(type='int', default=10),
+            update_handle=dict(),
         ),
         required_together=[
             ('username', 'password'),
@@ -372,6 +389,9 @@ def main():
     # timeout
     timeout = module.params['timeout']
 
+    # update handle
+    update_handle = module.params['update_handle']
+
     # Build root URI
     root_uri = "https://" + module.params['baseuri']
     rf_utils = RedfishUtils(creds, root_uri, timeout, module)
@@ -482,6 +502,8 @@ def main():
                     result["software"] = rf_utils.get_software_inventory()
                 elif command == "GetFirmwareUpdateCapabilities":
                     result["firmware_update_capabilities"] = rf_utils.get_firmware_update_capabilities()
+                elif command == "GetUpdateStatus":
+                    result["update_status"] = rf_utils.get_update_status(update_handle)
 
         elif category == "Sessions":
             # execute only if we find SessionService resources


Test patch: