Repository: getmoto/moto
Pull request #7984 (2024-08-15 19:34:57)

Admin: Support Py 3.13
The SES changes are due to a logic bug that only manifests in Python3.13.

When no CC/BCC fields are set, the `fieldvalues` would be set to something like: `['to@example.com, foo@example.com', '', '']`
Sending that to the builtin `emails.utils.getaddresses`-method in Python 3.12 results in:
`[('', 'to@example.com'), ('', 'foo@example.com'), ('', '')]`

But in Python 3.13, this results in an empty list.

This PR ensures that there are no empty strings in `fieldvalues`, which in turns means that the `getaddresses`-method keeps working.

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/getmoto/moto/pull/7984?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 94.37%. Comparing base [(`459be00`)](https://app.codecov.io/gh/getmoto/moto/commit/459be0049ae1bd2a7c26fb521475cfc44ededd81?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) to head [(`4317323`)](https://app.codecov.io/gh/getmoto/moto/commit/4317323d9c4858f9dcaf3ede1525ce905d6cdb0d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto).

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##           master    #7984   +/-   ##
=======================================
  Coverage   94.37%   94.37%           
=======================================
  Files        1125     1125           
  Lines       96356    96356           
=======================================
  Hits        90935    90935           
  Misses       5421     5421           
```

| [Flag](https://app.codecov.io/gh/getmoto/moto/pull/7984/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) | Coverage Δ | |
|---|---|---|
| [servertests](https://app.codecov.io/gh/getmoto/moto/pull/7984/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) | `28.93% <ø> (ø)` | |
| [unittests](https://app.codecov.io/gh/getmoto/moto/pull/7984/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) | `94.34% <ø> (+<0.01%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/getmoto/moto/pull/7984?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto).


Patch:
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index f38344ca2..2b8c40350 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -8,7 +8,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [ 3.8, 3.9, "3.10", "3.11", "3.12" ]
+        python-version: [ 3.8, 3.9, "3.10", "3.11", "3.12", "3.13.0-rc.1" ]
 
     steps:
       - uses: actions/checkout@v4
@@ -30,6 +30,11 @@ jobs:
         if: ${{ steps.pip-cache.outputs.cache-hit != 'true' }}
         run: |
           python -m pip install --upgrade pip
+      - name: Install XML dependencies
+        if: ${{ matrix.python-version == '3.13.0-rc.1' }}
+        run: |
+          echo "The libxml dependency needs these system packages to compile in Python 3.13"
+          sudo apt install -y libxml2-dev libxslt-dev
       - name: Install project dependencies
         if: ${{ steps.pip-cache.outputs.cache-hit != 'true' }}
         run: |
@@ -42,7 +47,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [3.8, 3.9, "3.10", "3.11", "3.12"]
+        python-version: [3.8, 3.9, "3.10", "3.11", "3.12", "3.13.0-rc.1"]
     steps:
     - uses: actions/checkout@v4
     - name: Set up Python ${{ matrix.python-version }}
diff --git a/moto/autoscaling/models.py b/moto/autoscaling/models.py
index c1265c795..7ff98b502 100644
--- a/moto/autoscaling/models.py
+++ b/moto/autoscaling/models.py
@@ -1171,6 +1171,8 @@ class AutoScalingBackend(BaseBackend):
                 "Valid requests must contain either LaunchTemplate, LaunchConfigurationName "
                 "or MixedInstancesPolicy parameter."
             )
+        if name not in self.autoscaling_groups:
+            raise ValidationError("AutoScalingGroup name not found - null")
 
         group = self.autoscaling_groups[name]
         group.update(
diff --git a/moto/emr/models.py b/moto/emr/models.py
index fc14a121a..5922d17a2 100644
--- a/moto/emr/models.py
+++ b/moto/emr/models.py
@@ -10,10 +10,9 @@ from moto.emr.exceptions import (
     ResourceNotFoundException,
     ValidationException,
 )
-from moto.utilities.utils import get_partition
+from moto.utilities.utils import CamelToUnderscoresWalker, get_partition
 
 from .utils import (
-    CamelToUnderscoresWalker,
     EmrSecurityGroupManager,
     random_cluster_id,
     random_instance_group_id,
diff --git a/moto/emr/utils.py b/moto/emr/utils.py
index 18f070c85..8b3a08226 100644
--- a/moto/emr/utils.py
+++ b/moto/emr/utils.py
@@ -3,10 +3,7 @@ import re
 import string
 from typing import Any, Dict, Iterator, List, Tuple
 
-from moto.core.utils import (
-    camelcase_to_underscores,
-    iso_8601_datetime_with_milliseconds,
-)
+from moto.core.utils import iso_8601_datetime_with_milliseconds
 from moto.moto_api._internal import mock_random as random
 
 
@@ -129,37 +126,6 @@ class Unflattener:
             return len(container) >= i
 
 
-class CamelToUnderscoresWalker:
-    """A class to convert the keys in dict/list hierarchical data structures from CamelCase to snake_case (underscores)"""
-
-    @staticmethod
-    def parse(x: Any) -> Any:  # type: ignore[misc]
-        if isinstance(x, dict):
-            return CamelToUnderscoresWalker.parse_dict(x)
-        elif isinstance(x, list):
-            return CamelToUnderscoresWalker.parse_list(x)
-        else:
-            return CamelToUnderscoresWalker.parse_scalar(x)
-
-    @staticmethod
-    def parse_dict(x: Dict[str, Any]) -> Dict[str, Any]:  # type: ignore[misc]
-        temp = {}
-        for key in x.keys():
-            temp[camelcase_to_underscores(key)] = CamelToUnderscoresWalker.parse(x[key])
-        return temp
-
-    @staticmethod
-    def parse_list(x: Any) -> Any:  # type: ignore[misc]
-        temp = []
-        for i in x:
-            temp.append(CamelToUnderscoresWalker.parse(i))
-        return temp
-
-    @staticmethod
-    def parse_scalar(x: Any) -> Any:  # type: ignore[misc]
-        return x
-
-
 class ReleaseLabel:
     version_re = re.compile(r"^emr-(\d+)\.(\d+)\.(\d+)$")
 
diff --git a/moto/events/models.py b/moto/events/models.py
index 1efe0e359..9b43ba579 100644
--- a/moto/events/models.py
+++ b/moto/events/models.py
@@ -7,7 +7,7 @@ from collections import OrderedDict
 from enum import Enum, unique
 from json import JSONDecodeError
 from operator import eq, ge, gt, le, lt
-from typing import Any, Dict, List, Optional
+from typing import TYPE_CHECKING, Any, Dict, List, Optional
 
 import requests
 
@@ -29,18 +29,32 @@ from moto.events.exceptions import (
     ValidationException,
 )
 from moto.moto_api._internal import mock_random as random
-from moto.secretsmanager import secretsmanager_backends
 from moto.utilities.arns import parse_arn
 from moto.utilities.paginator import paginate
 from moto.utilities.tagging_service import TaggingService
-from moto.utilities.utils import ARN_PARTITION_REGEX, get_partition
+from moto.utilities.utils import (
+    ARN_PARTITION_REGEX,
+    CamelToUnderscoresWalker,
+    get_partition,
+)
 
 from .utils import _BASE_EVENT_MESSAGE, PAGINATION_MODEL, EventMessageType
 
+if TYPE_CHECKING:
+    from moto.secretsmanager.models import SecretsManagerBackend
+
 # Sentinel to signal the absence of a field for `Exists` pattern matching
 UNDEFINED = object()
 
 
+def get_secrets_manager_backend(
+    account_id: str, region: str
+) -> "SecretsManagerBackend":
+    from moto.secretsmanager import secretsmanager_backends
+
+    return secretsmanager_backends[account_id][region]
+
+
 class Rule(CloudFormationModel):
     def __init__(
         self,
@@ -768,15 +782,23 @@ class Connection(BaseModel):
         self.state = "AUTHORIZED"
 
         connection_id = f"{self.name}/{self.uuid}"
-        secretsmanager_backend = secretsmanager_backends[account_id][region_name]
+        secretsmanager_backend = get_secrets_manager_backend(account_id, region_name)
+        secret_value = {}
+        for key, value in self.auth_parameters.items():
+            if key == "InvocationHttpParameters":
+                secret_value.update({"InvocationHttpParameters": value})
+            else:
+                secret_value.update(value)
+        secret_value = CamelToUnderscoresWalker.parse(secret_value)
+
         secret = secretsmanager_backend.create_secret(
-            name=f"{connection_id}/auth",
-            secret_string=json.dumps(self.auth_parameters),
-            replica_regions=list(),
+            name=f"events!connection/{connection_id}/auth",
+            secret_string=json.dumps(secret_value),
+            replica_regions=[],
             force_overwrite=False,
             secret_binary=None,
             description=f"Auth parameters for Eventbridge connection {connection_id}",
-            tags=list(),
+            tags=[{"Key": "aws:secretsmanager:owningService", "Value": "events"}],
             kms_key_id=None,
             client_request_token=None,
         )
@@ -817,7 +839,6 @@ class Connection(BaseModel):
             - The original response also has:
                 - LastAuthorizedTime (number)
                 - LastModifiedTime (number)
-                - SecretArn (string)
                 - StateReason (string)
             - At the time of implementing this, there was no place where to set/get
             those attributes. That is why they are not in the response.
@@ -1842,6 +1863,16 @@ class EventsBackend(BaseBackend):
         if not connection:
             raise ResourceNotFoundException(f"Connection '{name}' does not exist.")
 
+        # Delete Secret
+        secretsmanager_backend = get_secrets_manager_backend(
+            self.account_id, self.region_name
+        )
+        secretsmanager_backend.delete_secret(
+            secret_id=connection.secret_arn,
+            recovery_window_in_days=None,
+            force_delete_without_recovery=True,
+        )
+
         return connection.describe_short()
 
     def create_api_destination(
diff --git a/moto/secretsmanager/models.py b/moto/secretsmanager/models.py
index 6f38516aa..5a3cf7f05 100644
--- a/moto/secretsmanager/models.py
+++ b/moto/secretsmanager/models.py
@@ -914,7 +914,7 @@ class SecretsManagerBackend(BaseBackend):
     def delete_secret(
         self,
         secret_id: str,
-        recovery_window_in_days: int,
+        recovery_window_in_days: Optional[int],
         force_delete_without_recovery: bool,
     ) -> Tuple[str, str, float]:
         if recovery_window_in_days is not None and (
diff --git a/moto/ses/models.py b/moto/ses/models.py
index e8d26a945..d7bb447cd 100644
--- a/moto/ses/models.py
+++ b/moto/ses/models.py
@@ -358,7 +358,9 @@ class SESBackend(BaseBackend):
                     f"Did not have authority to send from email {source}"
                 )
 
-        fieldvalues = [message.get(header, "") for header in ["TO", "CC", "BCC"]]
+        fieldvalues = [
+            message[header] for header in ["TO", "CC", "BCC"] if header in message
+        ]
         destinations += [
             formataddr((realname, email_address))
             for realname, email_address in getaddresses(fieldvalues)
diff --git a/moto/utilities/utils.py b/moto/utilities/utils.py
index 77fae680e..8fd3b1641 100644
--- a/moto/utilities/utils.py
+++ b/moto/utilities/utils.py
@@ -123,3 +123,36 @@ class LowercaseDict(MutableMapping[str, Any]):
 
     def _keytransform(self, key: str) -> str:
         return key.lower()
+
+
+class CamelToUnderscoresWalker:
+    """A class to convert the keys in dict/list hierarchical data structures from CamelCase to snake_case (underscores)"""
+
+    @staticmethod
+    def parse(x: Any) -> Any:  # type: ignore[misc]
+        if isinstance(x, dict):
+            return CamelToUnderscoresWalker.parse_dict(x)
+        elif isinstance(x, list):
+            return CamelToUnderscoresWalker.parse_list(x)
+        else:
+            return CamelToUnderscoresWalker.parse_scalar(x)
+
+    @staticmethod
+    def parse_dict(x: Dict[str, Any]) -> Dict[str, Any]:  # type: ignore[misc]
+        from moto.core.utils import camelcase_to_underscores
+
+        temp = {}
+        for key in x.keys():
+            temp[camelcase_to_underscores(key)] = CamelToUnderscoresWalker.parse(x[key])
+        return temp
+
+    @staticmethod
+    def parse_list(x: Any) -> Any:  # type: ignore[misc]
+        temp = []
+        for i in x:
+            temp.append(CamelToUnderscoresWalker.parse(i))
+        return temp
+
+    @staticmethod
+    def parse_scalar(x: Any) -> Any:  # type: ignore[misc]
+        return x
diff --git a/setup.cfg b/setup.cfg
index 28b41da35..03d5fdb5c 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -15,6 +15,7 @@ classifiers =
     Programming Language :: Python :: 3.10
     Programming Language :: Python :: 3.11
     Programming Language :: Python :: 3.12
+    Programming Language :: Python :: 3.13
     License :: OSI Approved :: Apache Software License
     Topic :: Software Development :: Testing
 keywords = aws ec2 s3 boto3 mock


Test patch:
diff --git a/.github/workflows/tests_decoratormode.yml b/.github/workflows/tests_decoratormode.yml
index a34885137..8cdc43aba 100644
--- a/.github/workflows/tests_decoratormode.yml
+++ b/.github/workflows/tests_decoratormode.yml
@@ -7,7 +7,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [3.8, 3.9, "3.10", "3.11", "3.12"]
+        python-version: [3.8, 3.9, "3.10", "3.11", "3.12", "3.13.0-rc.1"]
 
     steps:
     - uses: actions/checkout@v4
diff --git a/tests/test_autoscaling/test_autoscaling.py b/tests/test_autoscaling/test_autoscaling.py
index a052b8640..614b75e79 100644
--- a/tests/test_autoscaling/test_autoscaling.py
+++ b/tests/test_autoscaling/test_autoscaling.py
@@ -7,7 +7,7 @@ from botocore.exceptions import ClientError
 
 from moto import mock_aws
 from moto.core import DEFAULT_ACCOUNT_ID as ACCOUNT_ID
-from tests import EXAMPLE_AMI_ID
+from tests import EXAMPLE_AMI_ID, aws_verified
 
 from .utils import setup_instance_with_networking, setup_networking
 
@@ -651,6 +651,22 @@ def test_update_autoscaling_group_max_size_desired_capacity_change():
     assert len(group["Instances"]) == 5
 
 
+@aws_verified
+@pytest.mark.aws_verified
+def test_update_unknown_group():
+    client = boto3.client("autoscaling", region_name="us-east-1")
+    with pytest.raises(ClientError) as exc:
+        client.update_auto_scaling_group(
+            AutoScalingGroupName="fake-asg",
+            MinSize=2,
+            MaxSize=2,
+            DesiredCapacity=2,
+        )
+    err = exc.value.response["Error"]
+    assert err["Code"] == "ValidationError"
+    assert err["Message"] == "AutoScalingGroup name not found - null"
+
+
 @mock_aws
 def test_autoscaling_describe_policies():
     mocked_networking = setup_networking()
diff --git a/tests/test_events/test_events.py b/tests/test_events/test_events.py
index 995cac188..b2fe8dd78 100644
--- a/tests/test_events/test_events.py
+++ b/tests/test_events/test_events.py
@@ -3,6 +3,8 @@ import random
 import unittest
 import warnings
 from datetime import datetime, timezone
+from time import sleep
+from uuid import uuid4
 
 import boto3
 import pytest
@@ -11,6 +13,7 @@ from botocore.exceptions import ClientError
 from moto import mock_aws, settings
 from moto.core import DEFAULT_ACCOUNT_ID as ACCOUNT_ID
 from moto.core.utils import iso_8601_datetime_without_milliseconds
+from tests import aws_verified
 
 RULES = [
     {"Name": "test1", "ScheduleExpression": "rate(5 minutes)"},
@@ -2312,6 +2315,95 @@ def test_create_and_update_connection():
     assert "CreationTime" in description
 
 
+@aws_verified
+@pytest.mark.aws_verified
+@pytest.mark.parametrize(
+    "auth_type,auth_parameters",
+    [
+        (
+            "API_KEY",
+            {"ApiKeyAuthParameters": {"ApiKeyName": "test", "ApiKeyValue": "test"}},
+        ),
+        ("BASIC", {"BasicAuthParameters": {"Username": "un", "Password": "pw"}}),
+    ],
+    ids=["auth_params", "basic_auth_params"],
+)
+@pytest.mark.parametrize(
+    "with_headers", [True, False], ids=["with_headers", "without_headers"]
+)
+def test_kms_key_is_created(auth_type, auth_parameters, with_headers):
+    client = boto3.client("events", "us-east-1")
+    secrets = boto3.client("secretsmanager", "us-east-1")
+    sts = boto3.client("sts", "us-east-1")
+
+    name = f"event_{str(uuid4())[0:6]}"
+    account_id = sts.get_caller_identity()["Account"]
+    connection_deleted = False
+    if with_headers:
+        auth_parameters["InvocationHttpParameters"] = {
+            "HeaderParameters": [
+                {"Key": "k1", "Value": "v1", "IsValueSecret": True},
+                {"Key": "k2", "Value": "v2", "IsValueSecret": False},
+            ]
+        }
+    else:
+        auth_parameters.pop("InvocationHttpParameters", None)
+
+    client.create_connection(
+        Name=name,
+        AuthorizationType=auth_type,
+        AuthParameters=auth_parameters,
+    )
+    try:
+        description = client.describe_connection(Name=name)
+        secret_arn = description["SecretArn"]
+        assert secret_arn.startswith(
+            f"arn:aws:secretsmanager:us-east-1:{account_id}:secret:events!connection/{name}/"
+        )
+
+        secret = secrets.describe_secret(SecretId=secret_arn)
+        assert secret["Name"].startswith(f"events!connection/{name}/")
+        assert secret["Tags"] == [
+            {"Key": "aws:secretsmanager:owningService", "Value": "events"}
+        ]
+
+        if auth_type == "BASIC":
+            expected_secret = {"username": "un", "password": "pw"}
+        else:
+            expected_secret = {"api_key_name": "test", "api_key_value": "test"}
+        if with_headers:
+            expected_secret["invocation_http_parameters"] = {
+                "header_parameters": [
+                    {"key": "k1", "value": "v1", "is_value_secret": True},
+                    {"key": "k2", "value": "v2", "is_value_secret": False},
+                ]
+            }
+
+        secret_value = secrets.get_secret_value(SecretId=secret_arn)
+        assert json.loads(secret_value["SecretString"]) == expected_secret
+
+        client.delete_connection(Name=name)
+        connection_deleted = True
+
+        secret_deleted = False
+        attempts = 0
+        while not secret_deleted and attempts < 5:
+            try:
+                attempts += 1
+                secrets.describe_secret(SecretId=secret_arn)
+                sleep(1)
+            except ClientError as e:
+                secret_deleted = (
+                    e.response["Error"]["Code"] == "ResourceNotFoundException"
+                )
+
+        if not secret_deleted:
+            assert False, f"Should have automatically deleted secret {secret_arn}"
+    finally:
+        if not connection_deleted:
+            client.delete_connection(Name=name)
+
+
 @mock_aws
 def test_update_unknown_connection():
     client = boto3.client("events", "eu-north-1")