Repository: getmoto/moto
Pull request #6817 (2023-09-15 16:21:39)

ELBv2: added validation for create_target_group() and set_ip_address_type()
# Fixed
- `create_target_group`: parameter validation added for `TargetType == 'lambda'`
- `set_ip_address_type `:  parameter validation added for ip address type

# Test
- Adjusted test case and added new scenarios


Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7510c4272..27d821f5d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,6 +1,49 @@
 Moto Changelog
 ==============
 
+4.2.3
+-----
+Docker Digest for 4.2.3: <autopopulateddigest>
+
+    New Services:
+        * RoboMaker:
+            * create_robot_application()
+            * delete_robot_application()
+            * describe_robot_application()
+            * list_robot_applications()
+
+    New Methods:
+        * ElasticBeanstalk:
+            * delete_application()
+
+        * Events:
+            * create_partner_event_source()
+            * delete_partner_event_source()
+            * describe_event_source()
+            * describe_partner_event_source()
+            * put_partner_events()
+
+    Miscellaneous:
+        * Core: The mocked ACCESS_KEY has been changed from `foobar_key` to `FOOBARKEY`, to align with AWS guidelines
+        * Core: set_initial_no_auth_action_count() now supports SSM actions
+        * Core: Fixed a memory leak when comparing requests (introduced in 4.1.13)
+        * Athena: get_query_execution() now returns a StatementType dependent on the provided query
+        * DynamoDB: query() now throws an exception when the KeyConditionExpression contains a literal value
+        * EBS: put_snapshot_block() now supports random bytes
+        * EC2: describe_transit_gateways() now supports filtering by tags
+        * ELBv2: describe_target_groups() now throws an exception for invalid input parameters
+        * ELBv2: describe_target_groups() now sorts the result before returning it
+        * ELBv2: create_target_group() now has improved validation and default values
+        * ELBv2: create_rule() now creates links between the TargetGroups and LoadBalancers
+        * Events: put_events() now support HTTP targets
+        * IAM policy validation now takes the provided Resource-argument into account when validating access to STS-resources
+        * IAM: get_role() now returns the LastUsed-parameter, provided the role was assumed and used
+        * KMS: sign/verify now uses the original message when signing, not the base64-version
+        * Lambda: invoke() now loads any Layers provided in the create_function()
+        * S3: put_bucket_logging() now supports bucket policies (as well as ACP's)
+        * S3: Range requests are now more permissive (following AWS' behaviour)
+        * SFN: list_executions() now returns the StopDate-attribute
+
 4.2.2
 ------
 Docker Digest for 4.2.2: <autopopulateddigest>
diff --git a/IMPLEMENTATION_COVERAGE.md b/IMPLEMENTATION_COVERAGE.md
index c53bfa33e..3fbb89966 100644
--- a/IMPLEMENTATION_COVERAGE.md
+++ b/IMPLEMENTATION_COVERAGE.md
@@ -2763,7 +2763,7 @@
 
 ## elasticbeanstalk
 <details>
-<summary>12% implemented</summary>
+<summary>14% implemented</summary>
 
 - [ ] abort_environment_update
 - [ ] apply_environment_managed_action
@@ -2776,7 +2776,7 @@
 - [X] create_environment
 - [ ] create_platform_version
 - [ ] create_storage_location
-- [ ] delete_application
+- [X] delete_application
 - [ ] delete_application_version
 - [ ] delete_configuration_template
 - [ ] delete_environment_configuration
diff --git a/docs/docs/services/elasticbeanstalk.rst b/docs/docs/services/elasticbeanstalk.rst
index 45efbb313..93e54cdaf 100644
--- a/docs/docs/services/elasticbeanstalk.rst
+++ b/docs/docs/services/elasticbeanstalk.rst
@@ -36,7 +36,7 @@ elasticbeanstalk
 - [X] create_environment
 - [ ] create_platform_version
 - [ ] create_storage_location
-- [x] delete_application
+- [X] delete_application
 - [ ] delete_application_version
 - [ ] delete_configuration_template
 - [ ] delete_environment_configuration
diff --git a/docs/docs/services/kms.rst b/docs/docs/services/kms.rst
index 6017b6003..b6ce534a5 100644
--- a/docs/docs/services/kms.rst
+++ b/docs/docs/services/kms.rst
@@ -78,9 +78,8 @@ kms
 - [X] revoke_grant
 - [X] schedule_key_deletion
 - [X] sign
-  Sign message using generated private key.
-
-        - signing_algorithm is ignored and hardcoded to RSASSA_PSS_SHA_256
+  
+        Sign message using generated private key.
 
         - grant_tokens are not implemented
         
@@ -92,9 +91,8 @@ kms
 - [X] update_key_description
 - [ ] update_primary_region
 - [X] verify
-  Verify message using public key from generated private key.
-
-        - signing_algorithm is ignored and hardcoded to RSASSA_PSS_SHA_256
+  
+        Verify message using public key from generated private key.
 
         - grant_tokens are not implemented
         
diff --git a/docs/docs/services/robomaker.rst b/docs/docs/services/robomaker.rst
index 70df30899..33c099ab3 100644
--- a/docs/docs/services/robomaker.rst
+++ b/docs/docs/services/robomaker.rst
@@ -12,8 +12,6 @@
 robomaker
 =========
 
-.. autoclass:: moto.robomaker.models.RoboMakerBackend
-
 |start-h3| Example usage |end-h3|
 
 .. sourcecode:: python
diff --git a/moto/dynamodb/models/dynamo_type.py b/moto/dynamodb/models/dynamo_type.py
index 33bad7d48..d6554a25c 100644
--- a/moto/dynamodb/models/dynamo_type.py
+++ b/moto/dynamodb/models/dynamo_type.py
@@ -1,3 +1,4 @@
+import base64
 import copy
 import decimal
 
@@ -195,6 +196,10 @@ class DynamoType(object):
 
     def to_json(self) -> Dict[str, Any]:
         # Returns a regular JSON object where the value can still be/contain a DynamoType
+        if self.is_binary():
+            # Binary data cannot be represented in JSON
+            # AWS returns a base64-encoded value - the SDK's then convert that back
+            return {self.type: base64.b64encode(self.value).decode("utf-8")}
         return {self.type: self.value}
 
     def to_regular_json(self) -> Dict[str, Any]:
@@ -212,6 +217,8 @@ class DynamoType(object):
                 val.to_regular_json() if isinstance(val, DynamoType) else val
                 for val in value
             ]
+        if self.is_binary():
+            value = base64.b64decode(value)
         return {self.type: value}
 
     def compare(self, range_comparison: str, range_objs: List[Any]) -> bool:
@@ -236,6 +243,9 @@ class DynamoType(object):
     def is_map(self) -> bool:
         return self.type == DDBType.MAP
 
+    def is_binary(self) -> bool:
+        return self.type == DDBType.BINARY
+
     def same_type(self, other: "DynamoType") -> bool:
         return self.type == other.type
 
diff --git a/moto/dynamodb/models/utilities.py b/moto/dynamodb/models/utilities.py
index 227f34bc0..6d4636e78 100644
--- a/moto/dynamodb/models/utilities.py
+++ b/moto/dynamodb/models/utilities.py
@@ -14,7 +14,7 @@ def dynamo_json_dump(dynamo_object: Any) -> str:
 
 
 def bytesize(val: str) -> int:
-    return len(val.encode("utf-8"))
+    return len(val if isinstance(val, bytes) else val.encode("utf-8"))
 
 
 def find_nested_key(
diff --git a/moto/elbv2/models.py b/moto/elbv2/models.py
index 2ae4f3f51..69c1f13ac 100644
--- a/moto/elbv2/models.py
+++ b/moto/elbv2/models.py
@@ -1104,6 +1104,14 @@ Member must satisfy regular expression pattern: {expression}"
                     param = "VPC ID" if param == "vpc_id" else param.lower()
                     raise ValidationError(f"A {param} must be specified")
 
+        if target_type == "lambda":
+            for param in ["protocol", "port", "vpc_id"]:
+                if kwargs.get(param) is not None:
+                    param = "VPC ID" if param == "vpc_id" else param.capitalize()
+                    raise ValidationError(
+                        f"{param} cannot be specified for target groups with target type 'lambda'"
+                    )
+
         if kwargs.get("vpc_id"):
             from moto.ec2.exceptions import InvalidVPCIdError
 
@@ -1568,10 +1576,10 @@ Member must satisfy regular expression pattern: {expression}"
         return modified_rules
 
     def set_ip_address_type(self, arn: str, ip_type: str) -> None:
-        if ip_type not in ("internal", "dualstack"):
+        if ip_type not in ("ipv4", "dualstack"):
             raise RESTError(
-                "InvalidParameterValue",
-                "IpAddressType must be either internal | dualstack",
+                "ValidationError",
+                f"1 validation error detected: Value '{ip_type}' at 'ipAddressType' failed to satisfy constraint: Member must satisfy enum value set: [ipv4, dualstack]",
             )
 
         balancer = self.load_balancers.get(arn)


Test patch:
diff --git a/tests/test_dynamodb/test_dynamodb.py b/tests/test_dynamodb/test_dynamodb.py
index 07c405071..667c0dfa4 100644
--- a/tests/test_dynamodb/test_dynamodb.py
+++ b/tests/test_dynamodb/test_dynamodb.py
@@ -4,6 +4,7 @@ from decimal import Decimal
 
 import boto3
 from boto3.dynamodb.conditions import Attr, Key
+from boto3.dynamodb.types import Binary
 import re
 from moto import mock_dynamodb, settings
 from moto.core import DEFAULT_ACCOUNT_ID as ACCOUNT_ID
@@ -5727,6 +5728,30 @@ def test_projection_expression_execution_order():
     )
 
 
+@mock_dynamodb
+def test_projection_expression_with_binary_attr():
+    dynamo_resource = boto3.resource("dynamodb", region_name="us-east-1")
+    dynamo_resource.create_table(
+        TableName="test",
+        AttributeDefinitions=[
+            {"AttributeName": "pk", "AttributeType": "S"},
+            {"AttributeName": "sk", "AttributeType": "S"},
+        ],
+        KeySchema=[
+            {"AttributeName": "pk", "KeyType": "HASH"},
+            {"AttributeName": "sk", "KeyType": "RANGE"},
+        ],
+        BillingMode="PAY_PER_REQUEST",
+    )
+    table = dynamo_resource.Table("test")
+    table.put_item(Item={"pk": "pk", "sk": "sk", "key": b"value\xbf"})
+    assert table.get_item(
+        Key={"pk": "pk", "sk": "sk"},
+        ExpressionAttributeNames={"#key": "key"},
+        ProjectionExpression="#key",
+    )["Item"] == {"key": Binary(b"value\xbf")}
+
+
 @mock_dynamodb
 def test_invalid_projection_expressions():
     table_name = "test-projection-expressions-table"
diff --git a/tests/test_elbv2/test_elbv2.py b/tests/test_elbv2/test_elbv2.py
index 19db045e6..67c398187 100644
--- a/tests/test_elbv2/test_elbv2.py
+++ b/tests/test_elbv2/test_elbv2.py
@@ -1159,6 +1159,11 @@ def test_set_ip_address_type():
 
     client.set_ip_address_type(LoadBalancerArn=arn, IpAddressType="dualstack")
 
+    with pytest.raises(ClientError) as ex:
+        client.set_ip_address_type(LoadBalancerArn=arn, IpAddressType="internal")
+    err = ex.value.response["Error"]
+    assert err["Code"] == "ValidationError"
+
 
 @mock_elbv2
 @mock_ec2
diff --git a/tests/test_elbv2/test_elbv2_target_groups.py b/tests/test_elbv2/test_elbv2_target_groups.py
index 1370cb8fe..347c7f211 100644
--- a/tests/test_elbv2/test_elbv2_target_groups.py
+++ b/tests/test_elbv2/test_elbv2_target_groups.py
@@ -375,6 +375,7 @@ def test_target_group_attributes():
 
     # check if Names filter works
     response = conn.describe_target_groups(Names=[])
+    assert len(response["TargetGroups"]) == 1
     response = conn.describe_target_groups(Names=["a-target"])
     assert len(response["TargetGroups"]) == 1
     target_group_arn = target_group["TargetGroupArn"]
@@ -466,7 +467,6 @@ def test_describe_target_groups():
     response, vpc, _, _, _, conn = create_load_balancer()
 
     lb_arn = response["LoadBalancers"][0]["LoadBalancerArn"]
-    assert "LoadBalancerArn" in response["LoadBalancers"][0]
 
     groups = conn.describe_target_groups()["TargetGroups"]
     assert len(groups) == 0
@@ -564,6 +564,20 @@ def test_describe_target_groups():
     assert groups[1]["TargetGroupName"] == "d-target"
 
 
+@mock_elbv2
+@mock_ec2
+def test_describe_target_groups_with_empty_load_balancer():
+    response, _, _, _, _, conn = create_load_balancer()
+
+    lb_arn = response["LoadBalancers"][0]["LoadBalancerArn"]
+
+    with pytest.raises(ClientError) as exc:
+        conn.describe_target_groups(LoadBalancerArn=lb_arn)
+    err = exc.value.response["Error"]
+    assert err["Code"] == "TargetGroupNotFound"
+    assert err["Message"] == "One or more target groups not found"
+
+
 @mock_elbv2
 @mock_ec2
 def test_modify_target_group():
@@ -881,3 +895,36 @@ def test_create_target_group_validation_error():
     err = ex.value.response["Error"]
     assert err["Code"] == "ValidationError"
     assert err["Message"] == "Health check interval must be greater than the timeout."
+
+    with pytest.raises(ClientError) as ex:
+        elbv2.create_target_group(Name="a-target", TargetType="lambda", Port=8080)
+    err = ex.value.response["Error"]
+    assert err["Code"] == "ValidationError"
+    assert (
+        err["Message"]
+        == "Port cannot be specified for target groups with target type 'lambda'"
+    )
+
+    with pytest.raises(ClientError) as ex:
+        elbv2.create_target_group(
+            Name="a-target", TargetType="lambda", VpcId="non-existing"
+        )
+    err = ex.value.response["Error"]
+    assert err["Code"] == "ValidationError"
+    assert (
+        err["Message"]
+        == "VPC ID cannot be specified for target groups with target type 'lambda'"
+    )
+
+    with pytest.raises(ClientError) as ex:
+        elbv2.create_target_group(
+            Name="a-target",
+            TargetType="lambda",
+            Protocol="HTTP",
+        )
+    err = ex.value.response["Error"]
+    assert err["Code"] == "ValidationError"
+    assert (
+        err["Message"]
+        == "Protocol cannot be specified for target groups with target type 'lambda'"
+    )
diff --git a/tests/test_s3/test_s3_bucket_policy.py b/tests/test_s3/test_s3_bucket_policy.py
index 8a910c61c..7a100fcf4 100644
--- a/tests/test_s3/test_s3_bucket_policy.py
+++ b/tests/test_s3/test_s3_bucket_policy.py
@@ -34,54 +34,37 @@ class TestBucketPolicy:
     def teardown_class(cls):
         cls.server.stop()
 
-    xfail_reason = "S3 logic for resource-based policy is not yet correctly implemented, see https://github.com/getmoto/moto/pull/6799#issuecomment-1712799688"
-
     @pytest.mark.parametrize(
-        "kwargs,status",
+        "kwargs,boto3_status,unauthorized_status",
         [
-            ({}, 200),
-            ({"resource": "arn:aws:s3:::mybucket/test_txt"}, 200),
-            pytest.param(
-                {"resource": "arn:aws:s3:::notmybucket/*"},
-                403,
-                marks=pytest.mark.xfail(reason=xfail_reason),
-            ),
-            pytest.param(
-                {"resource": "arn:aws:s3:::mybucket/other*"},
-                403,
-                marks=pytest.mark.xfail(reason=xfail_reason),
-            ),
-            ({"resource": ["arn:aws:s3:::mybucket", "arn:aws:s3:::mybucket/*"]}, 200),
-            pytest.param(
-                {
-                    "resource": [
-                        "arn:aws:s3:::notmybucket",
-                        "arn:aws:s3:::notmybucket/*",
-                    ]
-                },
-                403,
-                marks=pytest.mark.xfail(reason=xfail_reason),
-            ),
-            pytest.param(
-                {"resource": ["arn:aws:s3:::mybucket", "arn:aws:s3:::notmybucket/*"]},
-                403,
-                marks=pytest.mark.xfail(reason=xfail_reason),
-            ),
-            pytest.param(
-                {"effect": "Deny"}, 403, marks=pytest.mark.xfail(reason=xfail_reason)
-            ),
+            # The default policy is to allow access to 'mybucket/*'
+            ({}, 200, 200),
+            # We'll also allow access to the specific key
+            ({"resource": "arn:aws:s3:::mybucket/test_txt"}, 200, 200),
+            # We're allowing authorized access to an unrelated bucket
+            # Accessing our key is allowed for authenticated users, as there is no explicit deny
+            # It should block unauthenticated (public) users, as there is no explicit allow
+            ({"resource": "arn:aws:s3:::notmybucket/*"}, 200, 403),
+            # Verify public access when the policy contains multiple resources
+            ({"resource": ["arn:aws:s3:::other", "arn:aws:s3:::mybucket/*"]}, 200, 200),
+            # Deny all access, for any resource
+            ({"effect": "Deny"}, 403, 403),
+            # We don't explicitly deny authenticated access
+            # We'll deny an unrelated resource, but that should not affect anyone
+            # It should block unauthorized users, as there is no explicit allow
+            ({"resource": "arn:aws:s3:::notmybucket/*", "effect": "Deny"}, 200, 403),
         ],
     )
-    def test_block_or_allow_get_object(self, kwargs, status):
+    def test_block_or_allow_get_object(self, kwargs, boto3_status, unauthorized_status):
         self._put_policy(**kwargs)
 
-        if status == 200:
+        if boto3_status == 200:
             self.client.get_object(Bucket="mybucket", Key="test_txt")
         else:
             with pytest.raises(ClientError):
                 self.client.get_object(Bucket="mybucket", Key="test_txt")
 
-        assert requests.get(self.key_name).status_code == status
+        assert requests.get(self.key_name).status_code == unauthorized_status
 
     def test_block_put_object(self):
         # Block Put-access