Repository: ansible-collections/community.general

Issue:
KeyError: 'MessageId' in community.general.redfish_command (SimpleUpdate)
### Summary

I get the following error with a simple SimpleUpdate playbook. Please note that the update is performed on the HPE Server. The playbook just yields an error and does not provide me with a reply, which I need in order to continue and monitor the task. Any help is much appreciated. Thanks for reading.

The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1699008195.459113-4025-185255856440091/AnsiballZ_redfish_command.py", line 107, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1699008195.459113-4025-185255856440091/AnsiballZ_redfish_command.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1699008195.459113-4025-185255856440091/AnsiballZ_redfish_command.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.community.general.plugins.modules.redfish_command', init_globals=dict(_module_fqn='ansible_collections.community.general.plugins.modules.redfish_command', _modlib_path=modlib_path),
  File "/usr/lib/python3.9/runpy.py", line 210, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_community.general.redfish_command_payload_9ix7y_t7/ansible_community.general.redfish_command_payload.zip/ansible_collections/community/general/plugins/modules/redfish_command.py", line 1052, in <module>
  File "/tmp/ansible_community.general.redfish_command_payload_9ix7y_t7/ansible_community.general.redfish_command_payload.zip/ansible_collections/community/general/plugins/modules/redfish_command.py", line 1029, in main
  File "/tmp/ansible_community.general.redfish_command_payload_9ix7y_t7/ansible_community.general.redfish_command_payload.zip/ansible_collections/community/general/plugins/module_utils/redfish_utils.py", line 1741, in simple_update
  File "/tmp/ansible_community.general.redfish_command_payload_9ix7y_t7/ansible_community.general.redfish_command_payload.zip/ansible_collections/community/general/plugins/module_utils/redfish_utils.py", line 1661, in _operation_results
KeyError: 'MessageId'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1699008195.459113-4025-185255856440091/AnsiballZ_redfish_command.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1699008195.459113-4025-185255856440091/AnsiballZ_redfish_command.py\", line 99, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1699008195.459113-4025-185255856440091/AnsiballZ_redfish_command.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.general.plugins.modules.redfish_command', init_globals=dict(_module_fqn='ansible_collections.community.general.plugins.modules.redfish_command', _modlib_path=modlib_path),\n  File \"/usr/lib/python3.9/runpy.py\", line 210, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_community.general.redfish_command_payload_9ix7y_t7/ansible_community.general.redfish_command_payload.zip/ansible_collections/community/general/plugins/modules/redfish_command.py\", line 1052, in <module>\n  File \"/tmp/ansible_community.general.redfish_command_payload_9ix7y_t7/ansible_community.general.redfish_command_payload.zip/ansible_collections/community/general/plugins/modules/redfish_command.py\", line 1029, in main\n  File \"/tmp/ansible_community.general.redfish_command_payload_9ix7y_t7/ansible_community.general.redfish_command_payload.zip/ansible_collections/community/general/plugins/module_utils/redfish_utils.py\", line 1741, in simple_update\n  File \"/tmp/ansible_community.general.redfish_command_payload_9ix7y_t7/ansible_community.general.redfish_command_payload.zip/ansible_collections/community/general/plugins/module_utils/redfish_utils.py\", line 1661, in _operation_results\nKeyError: 'MessageId'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

### Issue Type

Bug Report

### Component Name

- name: Upload and install new FW package
  community.general.redfish_command:
    category: Update
    command: SimpleUpdate
    baseuri: "{{ item.value.ILOIP }}"
    username: "admin"
    password: "{{ item.value.ILOPW }}"
    update_image_uri: "{{ location_to_ip[item.value.Locationletter] ~ '/path/to/flash.signed.flash' }}"
    update_protocol: "HTTP"
      #ignore_errors: yes
      #register: bios_update_result
      #delegate_to: localhost

### Ansible Version

```console (paste below)
$ ansible --version

```
ansible [core 2.14.3]
  config file = /root/gits/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /root/.local/lib/python3.9/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

### Community.general Version

```console (paste below)
$ ansible-galaxy collection list community.general

```
# /usr/lib/python3/dist-packages/ansible_collections
Collection        Version
----------------- -------
community.general 1.3.6

# /root/.local/lib/python3.9/site-packages/ansible_collections
Collection        Version
----------------- -------
community.general 6.4.0

# /root/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
community.general 7.5.0


### Configuration

```console (paste below)
$ ansible-config dump --only-changed

```
CONFIG_FILE() = /root/gits/ansible/ansible.cfg
DEFAULT_FORKS(/root/gits/ansible/ansible.cfg) = 20
DEFAULT_HOST_LIST(/root/gits/ansible/ansible.cfg) = ['REDACTED']
HOST_KEY_CHECKING(/root/gits/ansible/ansible.cfg) = False

### OS / Environment

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)

```
Playbook and error can be found in the previous fields. 

I have also extracted the task into a standalone playbook that experiences the same error:

---
- hosts: all
  name: Test
  gather_facts: False


  tasks:
    - name: Upload and install new FW package and move it to the FW-Inventory of the BMC
      community.general.redfish_command:
        category: Update
        command: SimpleUpdate
        baseuri: IPADDRESS
        username: "admin"
        password: "SUPERSAFEPW"
        update_image_uri: "IP/PATH/TO/FLASH.signed.flash"
        update_protocol: "HTTP"


### Expected Results

When performing the POST manually I get the following reply:

{
    "@odata.context": "/redfish/v1/$metadata#Task.Task",
    "@odata.etag": "W/\"D793BCE6\"",
    "@odata.id": "/redfish/v1/TaskService/Tasks/12",
    "@odata.type": "#Task.v1_3_0.Task",
    "Id": "12",
    "Description": "iLO Task",
    "Messages": [
        {}
    ],
    "Name": "Task 12",
    "Payload": {
        "HttpOperation": "POST",
        "JsonBody": "{\"ImageURI\":\"REDACTED/path/to/flash.signed.flash\",\"TransferProtocol\":\"HTTP\"}",
        "TargetUri": "/redfish/v1/UpdateService/Actions/UpdateService.SimpleUpdate"
    },
    "StartTime": "2023-11-03T10:59:53Z",
    "TaskMonitor": "/redfish/v1/TaskService/TaskMonitors/12",
    "TaskState": "New"
}




### Actual Results

```console (paste below)

```

See error message in summary

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
* [`plugins/modules/redfish_command.py`](https://github.com/ansible-collections/community.general/blob/main/plugins/modules/redfish_command.py)

If these files are incorrect, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @TSKushal @bhavya06 @jyundt @mraineri @rajeevkallur @renxulei @tomasg2012 @xmadsen
[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Unfortunately iLO is breaking the requirements for the Task resource. In the "Messages" array, there is one member, and it contains an empty object. Per the schema definition for the message object, "MessageId" is mandatory. If iLO isn't trying to show a message, then it's supposed to use an empty array for "Messages".

So, it either needs to be...

```
{
    ...
    "Messages": []
    ...
}
```

or

```
{
    ...
    "Messages": [
        { "MessageId": "Some Message Identifier" }
    ]
    ...
}
```

We should be able to add an HPE-specific workaround here, but this really needs to be fed back to HPE to fix in their firmware.
@Ralece do you have any support contacts for HPE where you can direct this issue?
@mraineri Thank you for your reply! I think the error might have been on my end. Your reply indicating that this is related to the hpe fw, led me to reinstall/update the ILO FW and I do not get the error any more. I will do some more research from my end and share it if needed. 

So the issue is solved as it is generally possible with the new ILO FW. Older ILO Versions might however still receive the error message I have posted without catching it. I mean its only an inconvenience at this point since the task is still being performed. I kind of still need some return values like the TaskMonitorID in order to monitor the tasks, though I might get there via workaround. 

I would therefore be fine with closing thread this as solved. If you still require me to contact HPE, please let me know!

Thanks!
That's great that they've already fixed it! No need to reach out to HPE anymore!

I think it still might be worth adding workarounds to this; folks using Ansible would want to update their firmware to newer versions, so a workaround here would be generally useful.

Resolving patch:
diff --git a/plugins/module_utils/redfish_utils.py b/plugins/module_utils/redfish_utils.py
--- a/plugins/module_utils/redfish_utils.py
+++ b/plugins/module_utils/redfish_utils.py
@@ -1658,7 +1658,10 @@ def _operation_results(self, response, data, handle=None):
 
             # Scan the messages to see if next steps are needed
             for message in operation_results['messages']:
-                message_id = message['MessageId']
+                message_id = message.get('MessageId')
+                if message_id is None:
+                    # While this is invalid, treat the lack of a MessageId as "no message"
+                    continue
 
                 if message_id.startswith('Update.1.') and message_id.endswith('.OperationTransitionedToJob'):
                     # Operation rerouted to a job; update the status and handle


Test patch: