Repository: ansible-collections/community.aws

Issue:
msk_cluster - Cannot create a cluster w/ authentication sasl_scram
### Summary

When I do 

```- name: Create MSK cluster
  community.aws.msk_cluster:
    name: "{{ item.name }}"
....................................................................
    authentication:
      sasl_scram: true
....................................................................
```

i get an error:

```
"msg": "Failed to create kafka cluster: Parameter validation failed:\nInvalid type for parameter ClientAuthentication.Sasl.Scram, value: True, type: <class 'bool'>, valid types: <class 'dict'>"
```

Full traceback:
```
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_community.aws.msk_cluster_payload_roovgjj3/ansible_community.aws.msk_cluster_payload.zip/ansible_collections/community/aws/plugins/modules/msk_cluster.py", line 484, in create_or_update_cluster
  File "/tmp/ansible_community.aws.msk_cluster_payload_roovgjj3/ansible_community.aws.msk_cluster_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/modules.py", line 354, in deciding_wrapper
    return retrying_wrapper(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ansible_community.aws.msk_cluster_payload_roovgjj3/ansible_community.aws.msk_cluster_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 119, in _retry_wrapper
    return _retry_func(
           ^^^^^^^^^^^^
  File "/tmp/ansible_community.aws.msk_cluster_payload_roovgjj3/ansible_community.aws.msk_cluster_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 69, in _retry_func
    return func()
           ^^^^^^
  File "/home/.../.local/lib/python3.11/site-packages/botocore/client.py", line 530, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../.local/lib/python3.11/site-packages/botocore/client.py", line 919, in _make_api_call
    request_dict = self._convert_to_request_dict(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../.local/lib/python3.11/site-packages/botocore/client.py", line 990, in _convert_to_request_dict
    request_dict = self._serializer.serialize_to_request(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../.local/lib/python3.11/site-packages/botocore/validate.py", line 381, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter ClientAuthentication.Sasl.Scram, value: True, type: <class 'bool'>, valid types: <class 'dict'>
failed: [localhost] (item={'name': '...-msk-dev-2', 'configuration': '...-msk-dev-conf', 'version': '2.8.1', 'nodes': 3, 'ebs_volume_gb': 256, 'enhanced_monitoring': 'PER_TOPIC_PER_BROKER', 'instance_type': 'kafka.t3.small', 'open_monitoring': {'jmx_exporter': False, 'node_exporter': True}, 'subnets': ['subnet-...', 'subnet-...', 'subnet-...'], 'security_groups': ['sg-...'], 'tags': {'Payer': '...'}}) => {
    "ansible_loop_var": "item",
    "boto3_version": "1.26.72",
    "botocore_version": "1.29.72",
    "changed": false,
    "invocation": {
        "module_args": {
            "access_key": null,
            "authentication": {
                "sasl_scram": true,
                "tls_ca_arn": null
            },
```

I can create the cluster w/o `authentication` but it creates an `Unauthenticated` cluster.



### Issue Type

Bug Report

### Component Name

msk_cluster

### Ansible Version

ansible [core 2.14.3]
  config file = .../infrastructure/ansible.cfg
  configured module search path = ['/home/.../.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.11/site-packages/ansible
  ansible collection location = /home/.../.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.11.2 (main, Feb  8 2023, 00:00:00) [GCC 12.2.1 20221121 (Red Hat 12.2.1-4)] (/usr/bin/python3)
  jinja version = 3.0.3
  libyaml = True
```


### Collection Versions

```
$ ansible-galaxy collection list
[DEPRECATION WARNING]: DEFAULT_GATHER_SUBSET option, the module_defaults keyword is a more generic version and can apply to all calls to the M(ansible.builtin.gather_facts) or 
M(ansible.builtin.setup) actions, use module_defaults instead. This feature will be removed from ansible-core in version 2.18. Deprecation warnings can be disabled by setting 
deprecation_warnings=False in ansible.cfg.

# /home/.../.ansible/collections/ansible_collections
Collection        Version
----------------- -------
amazon.aws        5.4.0  
ansible.posix     1.5.1  
community.aws     5.4.0  
community.general 6.5.0  
community.mongodb 1.5.1  
community.mysql   3.6.0  

# /usr/lib/python3.11/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    5.2.0  
ansible.netcommon             4.1.0  
ansible.posix                 1.5.1  
ansible.utils                 2.9.0  
ansible.windows               1.13.0 
arista.eos                    6.0.0  
awx.awx                       21.12.0
azure.azcollection            1.14.0 
check_point.mgmt              4.0.0  
chocolatey.chocolatey         1.4.0  
cisco.aci                     2.4.0  
cisco.asa                     4.0.0  
cisco.dnac                    6.6.3  
cisco.intersight              1.0.23 
cisco.ios                     4.3.1  
cisco.iosxr                   4.1.0  
cisco.ise                     2.5.12 
cisco.meraki                  2.15.1 
cisco.mso                     2.2.1  
cisco.nso                     1.0.3  
cisco.nxos                    4.1.0  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.4  
community.aws                 5.2.0  
community.azure               2.0.0  
community.ciscosmb            1.0.5  
community.crypto              2.11.0 
community.digitalocean        1.23.0 
community.dns                 2.5.1  
community.docker              3.4.2  
community.fortios             1.0.0  
community.general             6.4.0  
community.google              1.0.0  
community.grafana             1.5.4  
community.hashi_vault         4.1.0  
community.hrobot              1.7.0  
community.libvirt             1.2.0  
community.mongodb             1.5.1  
community.mysql               3.6.0  
community.network             5.0.0  
community.okd                 2.3.0  
community.postgresql          2.3.2  
community.proxysql            1.5.1  
community.rabbitmq            1.2.3  
community.routeros            2.7.0  
community.sap                 1.0.0  
community.sap_libs            1.4.0  
community.skydive             1.0.0  
community.sops                1.6.1  
community.vmware              3.4.0  
community.windows             1.12.0 
community.zabbix              1.9.2  
containers.podman             1.10.1 
cyberark.conjur               1.2.0  
cyberark.pas                  1.0.17 
dellemc.enterprise_sonic      2.0.0  
dellemc.openmanage            6.3.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
dellemc.powerflex             1.5.0  
dellemc.unity                 1.5.0  
f5networks.f5_modules         1.22.1 
fortinet.fortimanager         2.1.7  
fortinet.fortios              2.2.2  
frr.frr                       2.0.0  
gluster.gluster               1.0.2  
google.cloud                  1.1.2  
grafana.grafana               1.1.1  
hetzner.hcloud                1.10.0 
hpe.nimble                    1.1.4  
ibm.qradar                    2.1.0  
ibm.spectrum_virtualize       1.11.0 
infinidat.infinibox           1.3.12 
infoblox.nios_modules         1.4.1  
inspur.ispim                  1.3.0  
inspur.sm                     2.3.0  
junipernetworks.junos         4.1.0  
kubernetes.core               2.4.0  
lowlydba.sqlserver            1.3.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.22.0
netapp.elementsw              21.7.0 
netapp.ontap                  22.3.0 
netapp.storagegrid            21.11.1
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.4.0  
netbox.netbox                 3.11.0 
ngine_io.cloudstack           2.3.0  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.3  
openstack.cloud               1.10.0 
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   2.4.1  
purestorage.flasharray        1.17.0 
purestorage.flashblade        1.10.0 
purestorage.fusion            1.3.0  
sensu.sensu_go                1.13.2 
splunk.es                     2.1.0  
t_systems_mms.icinga_director 1.32.0 
theforeman.foreman            3.9.0  
vmware.vmware_rest            2.2.0  
vultr.cloud                   1.7.0  
vyos.vyos                     4.0.0  
wti.remote                    1.0.4  

```


### AWS SDK versions

```
$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: mitch@garnaat.com
License: MIT
Location: /usr/local/lib/python3.11/site-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.26.72
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /home/.../.local/lib/python3.11/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.29.72
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /home/.../.local/lib/python3.11/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: awscli, boto3, s3transfer

```


### Configuration

```$ ansible-config dump --only-changed
[DEPRECATION WARNING]: DEFAULT_GATHER_SUBSET option, the module_defaults keyword is a more generic version and can apply to all calls to the M(ansible.builtin.gather_facts) or 
M(ansible.builtin.setup) actions, use module_defaults instead. This feature will be removed from ansible-core in version 2.18. Deprecation warnings can be disabled by setting 
deprecation_warnings=False in ansible.cfg.
CACHE_PLUGIN(/home/.../infrastructure/ansible.cfg) = jsonfile
CACHE_PLUGIN_CONNECTION(/home/.../infrastructure/ansible.cfg) = ~/.ansible/cache
CACHE_PLUGIN_TIMEOUT(/home/.../infrastructure/ansible.cfg) = 3600
CALLBACKS_ENABLED(/home/.../infrastructure/ansible.cfg) = ['timer', 'profile_tasks', 'profile_roles']
CONFIG_FILE() = /home/.../infrastructure/ansible.cfg
DEFAULT_ASK_PASS(/home/.../infrastructure/ansible.cfg) = False
DEFAULT_EXECUTABLE(/home/.../infrastructure/ansible.cfg) = /bin/bash
DEFAULT_FORCE_HANDLERS(/home/.../infrastructure/ansible.cfg) = True
DEFAULT_FORKS(/home/.../infrastructure/ansible.cfg) = 15
DEFAULT_GATHERING(/home/.../infrastructure/ansible.cfg) = smart
DEFAULT_GATHER_SUBSET(/home/.../infrastructure/ansible.cfg) = ['all']
DEFAULT_HOST_LIST(/home/.../infrastructure/ansible.cfg) = ['/home/.../infrastructure/envs']
DEFAULT_LOG_PATH(/home/.../infrastructure/ansible.cfg) = /home/.../.ansible/ansible.log
DEFAULT_MANAGED_STR(/home/.../infrastructure/ansible.cfg) = Ansible managed! DON'T CHANGE THIS FILE BY HAND! You were warned!
DEFAULT_ROLES_PATH(/home/.../infrastructure/ansible.cfg) = ['/home/.../infrastructure/roles']
DEFAULT_TIMEOUT(/home/.../infrastructure/ansible.cfg) = 30
DEPRECATION_WARNINGS(/home/.../infrastructure/ansible.cfg) = True
HOST_KEY_CHECKING(/home/.../infrastructure/ansible.cfg) = False
INVENTORY_ENABLED(/home/.../infrastructure/ansible.cfg) = ['yaml', 'aws_ec2', 'ini']
RETRY_FILES_ENABLED(/home/.../infrastructure/ansible.cfg) = False
SHOW_CUSTOM_STATS(/home/.../infrastructure/ansible.cfg) = True


```


### OS / Environment

Fedora release 37 (Thirty Seven)


### Steps to Reproduce

```
- name: Create MSK cluster
  community.aws.msk_cluster:
    name: "{{ item.name }}"
....................................................................
    authentication:
      sasl_scram: true
```


### Expected Results

Create a MSK cluster w/ SASL/SCRAM authentication.

### Actual Results

```
"msg": "Failed to create kafka cluster: Parameter validation failed:\nInvalid type for parameter ClientAuthentication.Sasl.Scram, value: True, type: <class 'bool'>, valid types: <class 'dict'>"
```

As I'm working on this I can jump in at any time to further debug this.

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
@gabriel-preda-adswizz thanks for your bug report.

https://github.com/ansible-collections/community.aws/blob/main/plugins/modules/msk_cluster.py#L383-L392  
the generated object is wrong.

ref https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kafka/client/create_cluster.html

```diff
diff --git a/plugins/modules/msk_cluster.py b/plugins/modules/msk_cluster.py
index 65c9edea..1b045c05 100644
--- a/plugins/modules/msk_cluster.py
+++ b/plugins/modules/msk_cluster.py
@@ -384,11 +384,13 @@ def prepare_create_options(module):
         c_params["ClientAuthentication"] = {}
         if module.params["authentication"].get("sasl_scram"):
             c_params["ClientAuthentication"]["Sasl"] = {
-                "Scram": module.params["authentication"]["sasl_scram"]
+                "Scram": {
+                    "Enable": module.params["authentication"]["sasl_scram"]
             }
         if module.params["authentication"].get("tls_ca_arn"):
             c_params["ClientAuthentication"]["Tls"] = {
-                "CertificateAuthorityArnList": module.params["authentication"]["tls_ca_arn"]
+                "CertificateAuthorityArnList": module.params["authentication"]["tls_ca_arn"],
+                'Enabled': True
             }
 
     c_params.update(prepare_enhanced_monitoring_options(module))
```

this might work.  
@gabriel-preda-adswizz do you have some time for work on a fix to contribute or can you test if this patch fixes the issue for you?
Thanx @markuman.

The patch is working for me.

However aside from the above I found this in output:
```
   "invocation": {
        "module_args": {
            "access_key": null,
            "authentication": {
                "sasl_scram": true,
                "tls_ca_arn": null
            }
```

I didn't set anything about `tls_ca_arn`, I don't understand why I have that line in there.
I only did:
```
authentication:
  sasl_scram: true
```

Thanx for the fast turnout (I can work w/ the patched version for some time).
> I didn't set anything about `tls_ca_arn`, I don't understand why I have that line in there.

That's because the module treated `tls_ca_arn` as `default(false)` if not provided.

See: https://github.com/ansible-collections/community.aws/blob/main/plugins/modules/msk_cluster.py#L715

So this is also a documentation bug, because the default value is missing there.
* https://github.com/ansible-collections/community.aws/blob/main/plugins/modules/msk_cluster.py#L118-L121
* https://docs.ansible.com/ansible/devel//collections/community/aws/msk_cluster_module.html#parameter-authentication/tls_ca_arn
Ah not. It's wrong. There is no default. Basically the empty key must/can be popped out.
So your patch is ok for me.

What next? It's your contribution :1st_place_medal: 
@gabriel-preda-adswizz If you have time for it, feel free to make a pull request with
* the patch/fix
* a changelog fragment
* and expand the integration test that covers the bug
Hi @markuman,

In preparing the pull request I've extended the fix for IAM authentication and also the posibility to disable unauthenticated clients that were not previously covered.

Now I'm puzzled about the tests.
There are a myriad of combinations :) and I'm still thinking in how to reorganize or only add those.
> Now I'm puzzled about the tests.
> There are a myriad of combinations :) and I'm still thinking in how to reorganize or only add those.

Mostly it helps to create a new task file that covers just that case/bug.  
Example: https://github.com/ansible-collections/community.mysql/pull/503/files  
New task file for the scenario `revoke_only_grant.yml` is included in the `main.yml` of the integration test.


Resolving patch:
diff --git a/plugins/modules/msk_cluster.py b/plugins/modules/msk_cluster.py
--- a/plugins/modules/msk_cluster.py
+++ b/plugins/modules/msk_cluster.py
@@ -125,7 +125,15 @@
             sasl_scram:
                 description: SASL/SCRAM authentication is enabled or not.
                 type: bool
-                default: False
+            sasl_iam:
+                version_added: 5.5.0
+                description: IAM authentication is enabled or not.
+                type: bool
+            unauthenticated:
+                version_added: 5.5.0
+                description: Option to explicitly turn on or off authentication
+                type: bool
+                default: True
     enhanced_monitoring:
         description: Specifies the level of monitoring for the MSK cluster.
         choices:
@@ -385,13 +393,21 @@ def prepare_create_options(module):
 
     if module.params["authentication"]:
         c_params["ClientAuthentication"] = {}
-        if module.params["authentication"].get("sasl_scram"):
-            c_params["ClientAuthentication"]["Sasl"] = {
-                "Scram": module.params["authentication"]["sasl_scram"]
-            }
+        if module.params["authentication"].get("sasl_scram") or module.params["authentication"].get("sasl_iam"):
+            sasl = {}
+            if module.params["authentication"].get("sasl_scram"):
+                sasl["Scram"] = {"Enabled": True}
+            if module.params["authentication"].get("sasl_iam"):
+                sasl["Iam"] = {"Enabled": True}
+            c_params["ClientAuthentication"]["Sasl"] = sasl
         if module.params["authentication"].get("tls_ca_arn"):
             c_params["ClientAuthentication"]["Tls"] = {
-                "CertificateAuthorityArnList": module.params["authentication"]["tls_ca_arn"]
+                "CertificateAuthorityArnList": module.params["authentication"]["tls_ca_arn"],
+                "Enabled": True,
+            }
+        if module.params["authentication"].get("unauthenticated"):
+            c_params["ClientAuthentication"] = {
+                "Unauthenticated": {"Enabled": True},
             }
 
     c_params.update(prepare_enhanced_monitoring_options(module))
@@ -716,7 +732,9 @@ def main():
             type="dict",
             options=dict(
                 tls_ca_arn=dict(type="list", elements="str", required=False),
-                sasl_scram=dict(type="bool", default=False),
+                sasl_scram=dict(type="bool", required=False),
+                sasl_iam=dict(type="bool", required=False),
+                unauthenticated=dict(type="bool", default=True, required=False),
             ),
         ),
         enhanced_monitoring=dict(


Test patch:
diff --git a/tests/integration/targets/msk_cluster-auth/aliases b/tests/integration/targets/msk_cluster-auth/aliases
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/msk_cluster-auth/aliases
@@ -0,0 +1,4 @@
+cloud/aws
+time=46m
+
+msk_cluster
diff --git a/tests/integration/targets/msk_cluster-auth/defaults/main.yml b/tests/integration/targets/msk_cluster-auth/defaults/main.yml
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/msk_cluster-auth/defaults/main.yml
@@ -0,0 +1,19 @@
+---
+vpc_name: "{{ resource_prefix }}-mskc-a"
+vpc_cidr: '10.{{ 256 | random(seed=resource_prefix) }}.0.0/16'
+vpc_subnets:
+  - '10.{{ 256 | random(seed=resource_prefix) }}.100.0/24'
+  - '10.{{ 256 | random(seed=resource_prefix) }}.101.0/24'
+vpc_subnet_name_prefix: "{{ resource_prefix }}"
+
+msk_config_name: "{{ resource_prefix }}-msk-cluster-auth"
+msk_cluster_name: "{{ tiny_prefix }}-msk-cluster-auth"
+msk_version: 2.8.1
+msk_broker_nodes: 2
+
+tags_create:
+  key1: "value1"
+  key2: "value2"
+tags_update:
+  key2: "value2"
+  key3: "value3"
diff --git a/tests/integration/targets/msk_cluster-auth/meta/main.yml b/tests/integration/targets/msk_cluster-auth/meta/main.yml
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/msk_cluster-auth/meta/main.yml
@@ -0,0 +1 @@
+dependencies: []
diff --git a/tests/integration/targets/msk_cluster-auth/tasks/main.yml b/tests/integration/targets/msk_cluster-auth/tasks/main.yml
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/msk_cluster-auth/tasks/main.yml
@@ -0,0 +1,91 @@
+---
+- name: aws_msk_cluster integration tests
+  module_defaults:
+    group/aws:
+      aws_access_key: "{{ aws_access_key }}"
+      aws_secret_key: "{{ aws_secret_key }}"
+      security_token: "{{ security_token | default(omit) }}"
+      region: "{{ aws_region }}"
+  collections:
+    - amazon.aws
+  block:
+    - name: collect availability zone info
+      aws_az_info:
+      register: az_info
+
+    - name: assert there are at least two zones
+      assert:
+        that: az_info.availability_zones | length >= 2
+
+    - name: create vpc
+      ec2_vpc_net:
+        state: present
+        cidr_block: '{{ vpc_cidr }}'
+        name: '{{ vpc_name }}'
+      register: vpc
+
+    - name: create subnets
+      ec2_vpc_subnet:
+        state: present
+        cidr: '{{ item }}'
+        az: '{{ az_info.availability_zones[index].zone_name }}'
+        vpc_id: '{{ vpc.vpc.id }}'
+        tags:
+          Name: '{{ vpc_subnet_name_prefix }}-subnet-{{ index }}'
+      loop: "{{ vpc_subnets }}"
+      loop_control:
+        index_var: index
+      register: subnets
+
+    - set_fact:
+        subnet_ids: '{{ subnets | community.general.json_query("results[].subnet.id") | list }}'
+
+     # ============================================================
+    - name: create msk configuration
+      aws_msk_config:
+        name: "{{ msk_config_name }}"
+        state: "present"
+        kafka_versions:
+          - "{{ msk_version }}"
+      register: msk_config
+
+    - name: create test with sasl_iam
+      include_tasks: test_create_auth.yml
+
+  always:
+
+    - name: delete msk cluster
+      aws_msk_cluster:
+        name: "{{ msk_cluster_name }}"
+        state: absent
+        wait: true
+      ignore_errors: yes
+
+    - name: remove msk configuration
+      aws_msk_config:
+        name: "{{ msk_config_name }}"
+        state: absent
+      ignore_errors: yes
+
+    - name: remove subnets
+      ec2_vpc_subnet:
+        state: absent
+        cidr: '{{ item }}'
+        vpc_id: '{{ vpc.vpc.id }}'
+      loop: "{{ vpc_subnets }}"
+      ignore_errors: yes
+      register: removed_subnets
+      until: removed_subnets is succeeded
+      retries: 5
+      delay: 5
+
+    - name: remove the vpc
+      ec2_vpc_net:
+        state: absent
+        cidr_block: '{{ vpc_cidr }}'
+        name: '{{ vpc_name }}'
+      ignore_errors: yes
+      register: removed_vpc
+      until: removed_vpc is success
+      retries: 5
+      delay: 5
diff --git a/tests/integration/targets/msk_cluster-auth/tasks/test_create_auth.yml b/tests/integration/targets/msk_cluster-auth/tasks/test_create_auth.yml
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/msk_cluster-auth/tasks/test_create_auth.yml
@@ -0,0 +1,101 @@
+---
+- name: create a msk cluster with authentication flipped from default (check mode)
+  aws_msk_cluster:
+    name: "{{ msk_cluster_name }}"
+    state: "present"
+    version: "{{ msk_version }}"
+    nodes: "{{ msk_broker_nodes }}"
+    ebs_volume_size: 10
+    authentication:
+      sasl_iam: true
+      sasl_scram: true
+      unauthenticated: false
+    subnets: "{{ subnet_ids }}"
+    wait: true
+    tags: "{{ tags_create }}"
+    configuration_arn: "{{ msk_config.arn }}"
+    configuration_revision: "{{ msk_config.revision }}"
+  check_mode: yes
+  register: msk_cluster
+
+- name: assert that the msk cluster be created
+  assert:
+    that:
+      - msk_cluster is changed
+
+- name: create a msk cluster with authentication flipped from default
+  aws_msk_cluster:
+    name: "{{ msk_cluster_name }}"
+    state: "present"
+    version: "{{ msk_version }}"
+    nodes: "{{ msk_broker_nodes }}"
+    ebs_volume_size: 10
+    authentication:
+      sasl_iam: true
+      sasl_scram: true
+      unauthenticated: false
+    subnets: "{{ subnet_ids }}"
+    wait: true
+    tags: "{{ tags_create }}"
+    configuration_arn: "{{ msk_config.arn }}"
+    configuration_revision: "{{ msk_config.revision }}"
+  register: msk_cluster
+
+- name: assert that the msk cluster is created
+  assert:
+    that:
+      - msk_cluster is changed
+
+- name: validate return values
+  assert:
+    that:
+      - "'cluster_info' in msk_cluster"
+      - "'bootstrap_broker_string' in msk_cluster"
+      - "'key1' in msk_cluster.cluster_info.tags"
+      - "msk_cluster.cluster_info.tags.key1 == 'value1'"
+      - "msk_cluster.cluster_info.cluster_name == msk_cluster_name"
+      - "msk_cluster.cluster_info.number_of_broker_nodes == msk_broker_nodes"
+      - "msk_cluster.cluster_info.broker_node_group_info.instance_type == 'kafka.t3.small'"
+      - "msk_cluster.cluster_info.broker_node_group_info.storage_info.ebs_storage_info.volume_size == 10"
+      - "msk_cluster.cluster_info.client_authentication.sasl.iam.enabled == true"
+      - "msk_cluster.cluster_info.client_authentication.sasl.scram.enabled == true"
+      # Not always returned by API
+      # - "msk_cluster.cluster_info.client_authentication.unauthenticated.enabled == false"
+      - "msk_cluster.cluster_info.open_monitoring.prometheus.jmx_exporter.enabled_in_broker == false"
+      - "msk_cluster.cluster_info.cluster_arn.startswith('arn:aws:kafka:{{ aws_region }}:')"
+
+- name: create a msk cluster with authentication flipped from default (idempotency)
+  aws_msk_cluster:
+    name: "{{ msk_cluster_name }}"
+    state: "present"
+    version: "{{ msk_version }}"
+    nodes: "{{ msk_broker_nodes }}"
+    ebs_volume_size: 10
+    authentication:
+      sasl_iam: true
+      sasl_scram: true
+      unauthenticated: false
+    subnets: "{{ subnet_ids }}"
+    wait: true
+    tags: "{{ tags_create }}"
+    configuration_arn: "{{ msk_config.arn }}"
+    configuration_revision: "{{ msk_config.revision }}"
+  register: msk_cluster
+
+- name: assert that the msk cluster wasn't changed
+  assert:
+    that:
+      - msk_cluster is not changed
+
+### Keep delete simple as we're not checking delete here
+- name: delete msk cluster
+  aws_msk_cluster:
+    name: "{{ msk_cluster_name }}"
+    state: "absent"
+    wait: true
+  register: msk_cluster
+
+- name: assert that the msk cluster is changed
+  assert:
+    that:
+      - msk_cluster is changed
diff --git a/tests/integration/targets/msk_cluster/defaults/main.yml b/tests/integration/targets/msk_cluster/defaults/main.yml
--- a/tests/integration/targets/msk_cluster/defaults/main.yml
+++ b/tests/integration/targets/msk_cluster/defaults/main.yml
@@ -8,7 +8,7 @@ vpc_subnet_name_prefix: "{{ resource_prefix }}"
 
 msk_config_name: "{{ resource_prefix }}-msk-cluster"
 msk_cluster_name: "{{ tiny_prefix }}-msk-cluster"
-msk_version: 2.6.0
+msk_version: 2.8.1
 msk_broker_nodes: 2
 
 tags_create: