Repository: ansible-collections/amazon.aws

Issue:
collection amazon.aws.lambda_event missing parameter
### Summary

"msg": "Error creating stream source event mapping: An error occurred (InvalidParameterValueException) when calling the CreateEventSourceMapping operation: **Maximum batch window in seconds must be greater than 0 if maximum batch size is greater than 10**",
  "exception": "  File \"/tmp/ansible_amazon.aws.lambda_event_payload_u9cfyt84/ansible_amazon.aws.lambda_event_payload.zip/ansible_collections/amazon/aws/plugins/modules/lambda_event.py\", line 371, in lambda_event_stream\n  File \"/usr/local/lib/python3.9/site-packages/botocore/client.py\", line 514, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n  File \"/usr/local/lib/python3.9/site-packages/botocore/client.py\", line 938, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n",

### Issue Type

Bug Report

### Component Name

amazon.aws.lambda_event

### Ansible Version

```console (paste below)
$ ansible --version

"ansible [core 2.14.6]",
    "  config file = /etc/ansible/ansible.cfg",
    "  configured module search path = ['/home/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']",
    "  ansible python module location = /usr/lib/python3.9/site-packages/ansible",
    "  ansible collection location = /runner/requirements_collections:/home/runner/.ansible/collections:/usr/share/ansible/collections",
    "  executable location = /usr/bin/ansible",
    "  python version = 3.9.16 (main, May 31 2023, 12:21:58) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)] (/usr/bin/python3.9)",
    "  jinja version = 3.1.2",
    "  libyaml = True"

```


### Collection Versions

```console (paste below)
$ ansible-galaxy collection list
# /usr/share/ansible/collections/ansible_collections
Collection              Version
----------------------- -------
amazon.aws              5.1.0  
ansible.controller      4.3.0  
ansible.netcommon       4.1.0  
ansible.network         1.2.0  
ansible.posix           1.3.0  
ansible.security        1.0.0  
ansible.snmp            1.0.1  
ansible.utils           2.7.0  
ansible.windows         1.12.0 
ansible.yang            1.0.0  
arista.eos              6.0.0  
cisco.asa               4.0.0  
cisco.ios               4.4.0  
cisco.iosxr             5.0.0  
cisco.nxos              4.1.0  
cloud.common            2.1.2  
frr.frr                 2.0.0  
ibm.qradar              2.1.0  
junipernetworks.junos   4.0.0  
kubernetes.core         2.3.2  
openvswitch.openvswitch 2.1.0  
redhat.insights         1.0.7  
redhat.openshift        2.2.0  
redhat.rhv              2.1.0  
redhat.satellite        3.3.0  
servicenow.itsm         1.3.3  
splunk.es               2.1.0  
trendmicro.deepsec      2.0.0  
vmware.vmware_rest      2.2.0  
vyos.vyos               4.0.0  

# /runner/requirements_collections/ansible_collections
Collection           Version
-------------------- -------
amazon.aws           7.3.0  
community.aws        7.1.0  
community.general    8.4.0  
community.mysql      3.9.0  
community.postgresql 3.4.0  

```


### AWS SDK versions

```console (paste below)
$ pip show boto boto3 botocore

Name: boto3
Version: 1.24.95
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.27.95
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: jmespath, urllib3, python-dateutil
Required-by: s3transfer, boto3
```


### Configuration

```console (paste below)
$ ansible-config dump --only-changed
[0;33mANSIBLE_FORCE_COLOR(env: ANSIBLE_FORCE_COLOR) = True[0m
[0;33mCOLLECTIONS_PATHS(env: ANSIBLE_COLLECTIONS_PATHS) = ['/runner/requirements_collections', '/home/runner/.ansible/collections', '/usr/share/ansible/collections'][0m
[0;33mCONFIG_FILE() = /etc/ansible/ansible.cfg[0m
[0;33mDEFAULT_CALLBACK_PLUGIN_PATH(env: ANSIBLE_CALLBACK_PLUGINS) = ['/runner/artifacts/1345365/callback'][0m
[0;33mDEFAULT_ROLES_PATH(env: ANSIBLE_ROLES_PATH) = ['/runner/requirements_roles', '/home/runner/.ansible/roles', '/usr/share/ansible/roles', '/etc/ansible/roles'][0m
[0;33mDEFAULT_STDOUT_CALLBACK(env: ANSIBLE_STDOUT_CALLBACK) = awx_display[0m
[0;33mHOST_KEY_CHECKING(env: ANSIBLE_HOST_KEY_CHECKING) = False[0m
[0;33mINVENTORY_UNPARSED_IS_FAILED(env: ANSIBLE_INVENTORY_UNPARSED_FAILED) = True[0m
[0;33mRETRY_FILES_ENABLED(env: ANSIBLE_RETRY_FILES_ENABLED) = False[0m

```


### OS / Environment

_No response_

### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)
  - name: Task 11 - Setup lambda event source
    amazon.aws.lambda_event:
      aws_access_key: "{{ aws.connect.aws_access_key_id }}"
      aws_secret_key: "{{ aws.connect.aws_secret_access_key }}"
      region: "{{ aws.arn.region }}"
      state: present
      event_source: sqs
      function_name: "{{ eim_environment }}-{{ service.lambda.function.name }}"
      source_params:
        source_arn: "{{ queue.queue_arn }}"
        enabled: true
        batch_size: 700
    register: event

```


### Expected Results

I expect to have access to a variable, MaximumBatchingWindowInSeconds, to support this statement:

> Related setting: For streams and Amazon SQS event sources, when you set BatchSize to a value greater than 10, you must set MaximumBatchingWindowInSeconds to at least 1.
> 
> Type: Integer
> 
> Valid Range: Minimum value of 0. Maximum value of 300.

[](https://docs.aws.amazon.com/lambda/latest/api/API_CreateEventSourceMapping.html#lambda-CreateEventSourceMapping-request-MaximumBatchingWindowInSeconds)


### Actual Results

```console (paste below)
Error creating stream source event mapping: An error occurred (InvalidParameterValueException) when calling the CreateEventSourceMapping operation: Maximum batch window in seconds must be greater than 0 if maximum batch size is greater than 10
 "exception": "  File \"/tmp/ansible_amazon.aws.lambda_event_payload_u9cfyt84/ansible_amazon.aws.lambda_event_payload.zip/ansible_collections/amazon/aws/plugins/modules/lambda_event.py\", line 371, in lambda_event_stream\n  File \"/usr/local/lib/python3.9/site-packages/botocore/client.py\", line 514, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n  File \"/usr/local/lib/python3.9/site-packages/botocore/client.py\", line 938, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n"
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:

Resolving patch:
diff --git a/plugins/modules/lambda_event.py b/plugins/modules/lambda_event.py
--- a/plugins/modules/lambda_event.py
+++ b/plugins/modules/lambda_event.py
@@ -54,22 +54,28 @@
     type: str
   source_params:
     description:
-      -  Sub-parameters required for event source.
+      - Sub-parameters required for event source.
     suboptions:
       source_arn:
         description:
-          -  The Amazon Resource Name (ARN) of the SQS queue, Kinesis stream or DynamoDB stream that is the event source.
+          - The Amazon Resource Name (ARN) of the SQS queue, Kinesis stream or DynamoDB stream that is the event source.
         type: str
         required: true
       enabled:
         description:
-          -  Indicates whether AWS Lambda should begin polling or readin from the event source.
+          - Indicates whether AWS Lambda should begin polling or readin from the event source.
         default: true
         type: bool
       batch_size:
         description:
-          -  The largest number of records that AWS Lambda will retrieve from your event source at the time of invoking your function.
-        default: 100
+          - The largest number of records that AWS Lambda will retrieve from your event source at the time of invoking your function.
+          - Amazon Kinesis - Default V(100). Max V(10000).
+          - Amazon DynamoDB Streams - Default V(100). Max V(10000).
+          - Amazon Simple Queue Service - Default V(10). For standard queues the max is V(10000). For FIFO queues the max is V(10).
+          - Amazon Managed Streaming for Apache Kafka - Default V(100). Max V(10000).
+          - Self-managed Apache Kafka - Default C(100). Max V(10000).
+          - Amazon MQ (ActiveMQ and RabbitMQ) - Default V(100). Max V(10000).
+          - DocumentDB - Default V(100). Max V(10000).
         type: int
       starting_position:
         description:
@@ -84,6 +90,15 @@
         elements: str
         choices: [ReportBatchItemFailures]
         version_added: 5.5.0
+      maximum_batching_window_in_seconds:
+        description:
+          - The maximum amount of time, in seconds, that Lambda spends gathering records before invoking the function.
+          - You can configure O(source_params.maximum_batching_window_in_seconds) to any value from V(0) seconds to V(300) seconds in increments of seconds.
+          - For streams and Amazon SQS event sources, when O(source_params.batch_size) is set to a value greater than V(10),
+            O(source_params.maximum_batching_window_in_seconds) defaults to V(1).
+          - O(source_params.maximum_batching_window_in_seconds) is not supported by FIFO queues.
+        type: int
+        version_added: 8.0.0
     required: true
     type: dict
 extends_documentation_fragment:
@@ -135,6 +150,7 @@
     type: list
 """
 
+import copy
 import re
 
 try:
@@ -228,6 +244,37 @@ def get_qualifier(module):
 # ---------------------------------------------------------------------------------------------------
 
 
+def set_default_values(module, source_params):
+    _source_params_cpy = copy.deepcopy(source_params)
+
+    if module.params["event_source"].lower() == "sqs":
+        # Default 10. For standard queues the max is 10,000. For FIFO queues the max is 10.
+        _source_params_cpy.setdefault("batch_size", 10)
+
+        if source_params["source_arn"].endswith(".fifo"):
+            if _source_params_cpy["batch_size"] > 10:
+                module.fail_json(msg="For FIFO queues the maximum batch_size is 10.")
+            if _source_params_cpy.get("maximum_batching_window_in_seconds"):
+                module.fail_json(
+                    msg="maximum_batching_window_in_seconds is not supported by Amazon SQS FIFO event sources."
+                )
+        else:
+            if _source_params_cpy["batch_size"] >= 10000:
+                module.fail_json(msg="For standard queue batch_size must be between lower than 10000.")
+
+    elif module.params["event_source"].lower() == "stream":
+        # Default 100.
+        _source_params_cpy.setdefault("batch_size", 100)
+
+        if not (100 <= _source_params_cpy["batch_size"] <= 10000):
+            module.fail_json(msg="batch_size for streams must be between 100 and 10000")
+
+    if _source_params_cpy["batch_size"] > 10 and not _source_params_cpy.get("maximum_batching_window_in_seconds"):
+        _source_params_cpy["maximum_batching_window_in_seconds"] = 1
+
+    return _source_params_cpy
+
+
 def lambda_event_stream(module, client):
     """
     Adds, updates or deletes lambda stream (DynamoDb, Kinesis) event notifications.
@@ -252,13 +299,8 @@ def lambda_event_stream(module, client):
     else:
         module.fail_json(msg="Source parameter 'source_arn' is required for stream event notification.")
 
-    # check if optional sub-parameters are valid, if present
-    batch_size = source_params.get("batch_size")
-    if batch_size:
-        try:
-            source_params["batch_size"] = int(batch_size)
-        except ValueError:
-            module.fail_json(msg=f"Source parameter 'batch_size' must be an integer, found: {batch_size}")
+    if state == "present":
+        source_params = set_default_values(module, source_params)
 
     # optional boolean value needs special treatment as not present does not imply False
     source_param_enabled = module.boolean(source_params.get("enabled", "True"))
@@ -285,6 +327,10 @@ def lambda_event_stream(module, client):
             api_params.update(Enabled=source_param_enabled)
             if source_params.get("batch_size"):
                 api_params.update(BatchSize=source_params.get("batch_size"))
+            if source_params.get("maximum_batching_window_in_seconds"):
+                api_params.update(
+                    MaximumBatchingWindowInSeconds=source_params.get("maximum_batching_window_in_seconds")
+                )
             if source_params.get("function_response_types"):
                 api_params.update(FunctionResponseTypes=source_params.get("function_response_types"))
 
@@ -347,7 +393,18 @@ def main():
         state=dict(required=False, default="present", choices=["present", "absent"]),
         lambda_function_arn=dict(required=True, aliases=["function_name", "function_arn"]),
         event_source=dict(required=False, default="stream", choices=source_choices),
-        source_params=dict(type="dict", required=True),
+        source_params=dict(
+            type="dict",
+            required=True,
+            options=dict(
+                source_arn=dict(type="str", required=True),
+                enabled=dict(type="bool", default=True),
+                batch_size=dict(type="int"),
+                starting_position=dict(type="str", choices=["TRIM_HORIZON", "LATEST"]),
+                function_response_types=dict(type="list", elements="str", choices=["ReportBatchItemFailures"]),
+                maximum_batching_window_in_seconds=dict(type="int"),
+            ),
+        ),
         alias=dict(required=False, default=None),
         version=dict(type="int", required=False, default=0),
     )


Test patch:
diff --git a/tests/integration/targets/lambda_event/tasks/main.yml b/tests/integration/targets/lambda_event/tasks/main.yml
--- a/tests/integration/targets/lambda_event/tasks/main.yml
+++ b/tests/integration/targets/lambda_event/tasks/main.yml
@@ -6,11 +6,10 @@
       secret_key: "{{ aws_secret_key }}"
       session_token: "{{ security_token | default(omit) }}"
       region: "{{ aws_region }}"
-  collections:
-    - community.general
   block:
     - name: Create test resources setup
       ansible.builtin.import_tasks: setup.yml
+
     - name: Create DynamoDB stream event mapping (trigger) - check_mode
       amazon.aws.lambda_event:
         state: present
diff --git a/tests/unit/plugins/modules/test_lambda_event.py b/tests/unit/plugins/modules/test_lambda_event.py
--- a/tests/unit/plugins/modules/test_lambda_event.py
+++ b/tests/unit/plugins/modules/test_lambda_event.py
@@ -4,6 +4,7 @@
 # This file is part of Ansible
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
+from contextlib import nullcontext as does_not_raise
 from copy import deepcopy
 from unittest.mock import MagicMock
 from unittest.mock import patch
@@ -12,6 +13,7 @@
 
 from ansible_collections.amazon.aws.plugins.modules.lambda_event import get_qualifier
 from ansible_collections.amazon.aws.plugins.modules.lambda_event import lambda_event_stream
+from ansible_collections.amazon.aws.plugins.modules.lambda_event import set_default_values
 from ansible_collections.amazon.aws.plugins.modules.lambda_event import validate_params
 
 mock_get_qualifier = "ansible_collections.amazon.aws.plugins.modules.lambda_event.get_qualifier"
@@ -205,6 +207,7 @@ def test_lambda_event_stream_create_event_missing_starting_position(ansible_aws_
         "event_source": "stream",
         "source_params": {
             "source_arn": "arn:aws:sqs:us-east-2:123456789012:ansible-test-sqs",
+            "maximum_batching_window_in_seconds": 1,
             "batch_size": 200,
         },
         "alias": None,
@@ -235,6 +238,7 @@ def test_lambda_event_stream_create_event_missing_starting_position(ansible_aws_
                 "event_source": "stream",
                 "source_params": {
                     "source_arn": "arn:aws:sqs:us-east-2:123456789012:ansible-test-sqs",
+                    "maximum_batching_window_in_seconds": 1,
                     "batch_size": 250,
                     "starting_position": "END",
                     "function_response_types": ["ReportBatchItemFailures"],
@@ -247,6 +251,7 @@ def test_lambda_event_stream_create_event_missing_starting_position(ansible_aws_
                 "EventSourceArn": "arn:aws:sqs:us-east-2:123456789012:ansible-test-sqs",
                 "StartingPosition": "END",
                 "Enabled": True,
+                "MaximumBatchingWindowInSeconds": 1,
                 "BatchSize": 250,
                 "FunctionResponseTypes": ["ReportBatchItemFailures"],
             },
@@ -258,6 +263,7 @@ def test_lambda_event_stream_create_event_missing_starting_position(ansible_aws_
                 "event_source": "stream",
                 "source_params": {
                     "source_arn": "arn:aws:sqs:us-east-2:123456789012:ansible-test-sqs",
+                    "maximum_batching_window_in_seconds": 1,
                     "batch_size": 250,
                     "starting_position": "END",
                     "function_response_types": ["ReportBatchItemFailures"],
@@ -271,6 +277,7 @@ def test_lambda_event_stream_create_event_missing_starting_position(ansible_aws_
                 "EventSourceArn": "arn:aws:sqs:us-east-2:123456789012:ansible-test-sqs",
                 "StartingPosition": "END",
                 "Enabled": False,
+                "MaximumBatchingWindowInSeconds": 1,
                 "BatchSize": 250,
                 "FunctionResponseTypes": ["ReportBatchItemFailures"],
             },
@@ -282,6 +289,7 @@ def test_lambda_event_stream_create_event_missing_starting_position(ansible_aws_
                 "event_source": "sqs",
                 "source_params": {
                     "source_arn": "arn:aws:sqs:us-east-2:123456789012:ansible-test-sqs",
+                    "maximum_batching_window_in_seconds": 1,
                     "batch_size": 101,
                 },
                 "alias": None,
@@ -291,6 +299,7 @@ def test_lambda_event_stream_create_event_missing_starting_position(ansible_aws_
                 "FunctionName": "sqs_consumer",
                 "EventSourceArn": "arn:aws:sqs:us-east-2:123456789012:ansible-test-sqs",
                 "Enabled": True,
+                "MaximumBatchingWindowInSeconds": 1,
                 "BatchSize": 101,
             },
         ),
@@ -375,3 +384,161 @@ def test_lambda_event_stream_update_event(
         api_params.update({"FunctionName": function_name, "UUID": existing_event_source[0]["UUID"]})
         assert dict(changed=True, events=event_source_updated) == result
         client.update_event_source_mapping.assert_called_once_with(**api_params)
+
+
+@pytest.mark.parametrize(
+    "params, expected, exception, message, source_type",
+    [
+        (
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052.fifo",
+                "enabled": True,
+                "batch_size": 100,
+                "starting_position": None,
+                "function_response_types": None,
+                "maximum_batching_window_in_seconds": None,
+            },
+            None,
+            pytest.raises(SystemExit),
+            "For FIFO queues the maximum batch_size is 10.",
+            "sqs",
+        ),
+        (
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052.fifo",
+                "enabled": True,
+                "batch_size": 10,
+                "starting_position": None,
+                "function_response_types": None,
+                "maximum_batching_window_in_seconds": 1,
+            },
+            None,
+            pytest.raises(SystemExit),
+            "maximum_batching_window_in_seconds is not supported by Amazon SQS FIFO event sources.",
+            "sqs",
+        ),
+        (
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052.fifo",
+                "enabled": True,
+                "batch_size": 10,
+                "starting_position": None,
+                "function_response_types": None,
+                "maximum_batching_window_in_seconds": None,
+            },
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052.fifo",
+                "enabled": True,
+                "batch_size": 10,
+                "starting_position": None,
+                "function_response_types": None,
+                "maximum_batching_window_in_seconds": None,
+            },
+            does_not_raise(),
+            None,
+            "sqs",
+        ),
+        (
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052",
+                "enabled": True,
+                "batch_size": 11000,
+                "starting_position": None,
+                "function_response_types": None,
+                "maximum_batching_window_in_seconds": None,
+            },
+            None,
+            pytest.raises(SystemExit),
+            "For standard queue batch_size must be lower than 10000.",
+            "sqs",
+        ),
+        (
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052",
+                "enabled": True,
+                "batch_size": 100,
+                "starting_position": None,
+                "function_response_types": None,
+                "maximum_batching_window_in_seconds": None,
+            },
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052",
+                "enabled": True,
+                "batch_size": 100,
+                "starting_position": None,
+                "function_response_types": None,
+                "maximum_batching_window_in_seconds": 1,
+            },
+            does_not_raise(),
+            None,
+            "sqs",
+        ),
+        (
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052",
+                "enabled": True,
+                "starting_position": None,
+                "function_response_types": None,
+                "maximum_batching_window_in_seconds": None,
+            },
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052",
+                "enabled": True,
+                "batch_size": 100,
+                "starting_position": None,
+                "function_response_types": None,
+                "maximum_batching_window_in_seconds": 1,
+            },
+            does_not_raise(),
+            None,
+            "stream",
+        ),
+        (
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052",
+                "enabled": True,
+                "starting_position": None,
+                "function_response_types": None,
+            },
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052",
+                "enabled": True,
+                "batch_size": 10,
+                "starting_position": None,
+                "function_response_types": None,
+            },
+            does_not_raise(),
+            None,
+            "sqs",
+        ),
+        (
+            {
+                "source_arn": "arn:aws:sqs:us-east-1:123456789012:ansible-test-28277052",
+                "enabled": True,
+                "batch_size": 10,
+                "starting_position": None,
+                "function_response_types": None,
+                "maximum_batching_window_in_seconds": None,
+            },
+            None,
+            pytest.raises(SystemExit),
+            "batch_size for streams must be between 100 and 10000",
+            "stream",
+        ),
+    ],
+)
+def test__set_default_values(params, expected, exception, message, source_type):
+    result = None
+    module = MagicMock()
+    module.check_mode = False
+    module.params = {
+        "event_source": source_type,
+        "source_params": params,
+    }
+    module.fail_json = MagicMock()
+    module.fail_json.side_effect = SystemExit(message)
+    with exception as e:
+        result = set_default_values(module, params)
+    assert message is None or message in str(e)
+    if expected is not None:
+        assert result == expected