Repository: localstack/localstack

Issue:
bug: SAM Deploy with Image Based Lambdas that have a CompanionStack throws error: No updates are to be performed.
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

When using image based lambdas that have companion stacks subsequent applies throws the following error:
```
Error: An error occurred (ValidationError) when calling the UpdateStack operation: Unable to update stack "localstack-companion-stack-e4f29f11-CompanionStack": No updates are to be performed.

For more information please visit: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html
```

This first apply works when it's just a create, but any subsequent apply fails and it does not respect the `no-fail-on-empty-changeset` flag. This issue appears to only happen with CompanionStack which is created when you are using a parameter in the `ImageUri` parameter of `AWS::Serverless::Function`. 

### Expected Behavior

Subsequent SAM deploys should not check companionstack for changes and shouldn't throw an error if there are no updates as that is expected.

### How are you starting LocalStack?

With a docker-compose file

### Steps To Reproduce

#### How are you starting localstack (e.g., `bin/localstack` command, arguments, or `docker-compose.yml`)

    docker-compose up

#### template.yaml
```
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  localstack-companionstack

  Sample SAM Template for localstack-companionstack

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Parameters:

  LegacyStackName:
    Type: String

  Tag:
    Type: String

Globals:

  Function:
    Timeout: 30
    MemorySize: 320
    Environment:
      Variables:
        AWS_ACCOUNT_ID: !Ref AWS::AccountId

Resources:
  HelloWorldApi:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: localstack-companionstack
      PackageType: Image
      ImageConfig:
        Command:
        - handler.handler
      ImageUri: !Sub 
        ${AWS::AccountId}.dkr.ecr.us-east-1.amazonaws.com/${LegacyStackName}:${Tag}  # End of Resource
```

#### samconfig.toml
```
# More information about the configuration file can be found here:
# https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html
version = 0.1

[default]
[default.global.parameters]
stack_name = "localstack-companion-stack"
s3_bucket = "aws-sam-cli-managed-test"
s3_prefix = "companionstack-test"
region = "us-east-1"


[default.build.parameters]
cached = true
parallel = true

[default.validate.parameters]
lint = true

[default.deploy.parameters]
capabilities = "CAPABILITY_IAM"
confirm_changeset = true
parameter_overrides = [
    "LegacyStackName=companionstack-test",
    "Tag=latest",
]
[default.sync.parameters]
watch = true

[default.local_start_api.parameters]
warm_containers = "EAGER"

[default.local_start_lambda.parameters]
warm_containers = "EAGER"
```

#### Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)
```
awslocal ecr create-repository --repository-name companionstack-test
awslocal s3api create-bucket --bucket aws-sam-cli-managed-test
docker build -t "000000000000.dkr.ecr.us-east-1.localhost.localstack.cloud:4510/companionstack-test" .
docker push 000000000000.dkr.ecr.us-east-1.localhost.localstack.cloud:4510/companionstack-test:latest
samlocal build --config-file samconfig.toml --template-file template.yaml --debug
samlocal deploy --config-file samconfig.toml  --no-fail-on-empty-changeset --no-confirm-changeset --resolve-image-repos
```
Then re-run sam build and deploy:
```
samlocal build --config-file samconfig.toml --template-file template.yaml --debug
samlocal deploy --config-file samconfig.toml  --no-fail-on-empty-changeset --no-confirm-changeset --resolve-image-repos
```

### Environment

```markdown
- OS: OSX 14 
- LocalStack: Localstack pro latest
- SAM 1.110.0
```


### Anything else?

#### Example Dockerfile
```
# Simple Dockerfile illustrating Lambda container images in LocalStack

FROM public.ecr.aws/lambda/python:3.11

COPY ./handler.py ./

CMD [ "handler.handler" ]
```

#### Example handler.py
```
def handler(event, context):
    print('Hello from LocalStack Lambda container image!')
```


Discussion:
Hey @Chad-Bailer-Regal! Thanks for the reproduction instructions, really helpful!

Currently, support for UPDATE operations on resources is limited. Prefer stack re-creation over stack update at this time. (see [docs](https://docs.localstack.cloud/user-guide/aws/cloudformation/#features)) 

You can use `localstack restart` to restart localstack within the container (it's pretty fast) and then re-deploy. You can also use `localstack state export/import` to automate pre-seed of localstack with your ecr registry.

Resolving patch:
diff --git a/localstack/services/cloudformation/provider.py b/localstack/services/cloudformation/provider.py
--- a/localstack/services/cloudformation/provider.py
+++ b/localstack/services/cloudformation/provider.py
@@ -1,3 +1,4 @@
+import copy
 import json
 import logging
 import re
@@ -351,6 +352,7 @@ def update_stack(
             stack_name=stack_name,
         )
 
+        raw_new_template = copy.deepcopy(template)
         try:
             template = template_preparer.transform_template(
                 context.account_id,
@@ -362,6 +364,9 @@ def update_stack(
                 resolved_stack_conditions,
                 resolved_parameters,
             )
+            processed_template = copy.deepcopy(
+                template
+            )  # copying it here since it's being mutated somewhere downstream
         except FailedTransformationException as e:
             stack.add_stack_event(
                 stack.stack_name,
@@ -384,6 +389,9 @@ def update_stack(
             deployer.update_stack(new_stack)
         except NoStackUpdates as e:
             stack.set_stack_status("UPDATE_COMPLETE")
+            if raw_new_template != processed_template:
+                # processed templates seem to never return an exception here
+                return UpdateStackOutput(StackId=stack.stack_id)
             raise ValidationError(str(e))
         except Exception as e:
             stack.set_stack_status("UPDATE_FAILED")


Test patch:
diff --git a/tests/aws/services/cloudformation/api/test_stacks.py b/tests/aws/services/cloudformation/api/test_stacks.py
--- a/tests/aws/services/cloudformation/api/test_stacks.py
+++ b/tests/aws/services/cloudformation/api/test_stacks.py
@@ -221,10 +221,12 @@ def test_list_stack_resources_for_removed_resource(self, deploy_cfn_template, aw
         statuses = {res["ResourceStatus"] for res in resources}
         assert statuses == {"UPDATE_COMPLETE"}
 
-    @markers.aws.needs_fixing
-    def test_update_stack_with_same_template_withoutchange(self, deploy_cfn_template, aws_client):
+    @markers.aws.validated
+    def test_update_stack_with_same_template_withoutchange(
+        self, deploy_cfn_template, aws_client, snapshot
+    ):
         template = load_file(
-            os.path.join(os.path.dirname(__file__), "../../../templates/fifo_queue.json")
+            os.path.join(os.path.dirname(__file__), "../../../templates/simple_no_change.yaml")
         )
         stack = deploy_cfn_template(template=template)
 
@@ -236,9 +238,29 @@ def test_update_stack_with_same_template_withoutchange(self, deploy_cfn_template
                 StackName=stack.stack_name
             )
 
-        error_message = str(ctx.value)
-        assert "UpdateStack" in error_message
-        assert "No updates are to be performed." in error_message
+        snapshot.match("no_change_exception", ctx.value.response)
+
+    @markers.aws.validated
+    def test_update_stack_with_same_template_withoutchange_transformation(
+        self, deploy_cfn_template, aws_client
+    ):
+        template = load_file(
+            os.path.join(
+                os.path.dirname(__file__),
+                "../../../templates/simple_no_change_with_transformation.yaml",
+            )
+        )
+        stack = deploy_cfn_template(template=template)
+
+        # transformations will always work even if there's no change in the template!
+        aws_client.cloudformation.update_stack(
+            StackName=stack.stack_name,
+            TemplateBody=template,
+            Capabilities=["CAPABILITY_AUTO_EXPAND"],
+        )
+        aws_client.cloudformation.get_waiter("stack_update_complete").wait(
+            StackName=stack.stack_name
+        )
 
     @markers.aws.validated
     def test_update_stack_actual_update(self, deploy_cfn_template, aws_client):
diff --git a/tests/aws/services/cloudformation/api/test_stacks.snapshot.json b/tests/aws/services/cloudformation/api/test_stacks.snapshot.json
--- a/tests/aws/services/cloudformation/api/test_stacks.snapshot.json
+++ b/tests/aws/services/cloudformation/api/test_stacks.snapshot.json
@@ -1165,5 +1165,21 @@
         }
       }
     }
+  },
+  "tests/aws/services/cloudformation/api/test_stacks.py::TestStacksApi::test_update_stack_with_same_template_withoutchange": {
+    "recorded-date": "07-05-2024, 08:34:18",
+    "recorded-content": {
+      "no_change_exception": {
+        "Error": {
+          "Code": "ValidationError",
+          "Message": "No updates are to be performed.",
+          "Type": "Sender"
+        },
+        "ResponseMetadata": {
+          "HTTPHeaders": {},
+          "HTTPStatusCode": 400
+        }
+      }
+    }
   }
 }
diff --git a/tests/aws/services/cloudformation/api/test_stacks.validation.json b/tests/aws/services/cloudformation/api/test_stacks.validation.json
--- a/tests/aws/services/cloudformation/api/test_stacks.validation.json
+++ b/tests/aws/services/cloudformation/api/test_stacks.validation.json
@@ -20,6 +20,12 @@
   "tests/aws/services/cloudformation/api/test_stacks.py::TestStacksApi::test_stack_update_resources": {
     "last_validated_date": "2022-08-29T22:13:26+00:00"
   },
+  "tests/aws/services/cloudformation/api/test_stacks.py::TestStacksApi::test_update_stack_with_same_template_withoutchange": {
+    "last_validated_date": "2024-05-07T08:35:29+00:00"
+  },
+  "tests/aws/services/cloudformation/api/test_stacks.py::TestStacksApi::test_update_stack_with_same_template_withoutchange_transformation": {
+    "last_validated_date": "2024-05-07T09:26:39+00:00"
+  },
   "tests/aws/services/cloudformation/api/test_stacks.py::test_blocked_stack_deletion": {
     "last_validated_date": "2023-09-06T09:01:18+00:00"
   },
diff --git a/tests/aws/templates/simple_no_change.yaml b/tests/aws/templates/simple_no_change.yaml
new file mode 100644
--- /dev/null
+++ b/tests/aws/templates/simple_no_change.yaml
@@ -0,0 +1,3 @@
+Resources:
+  MyTopic:
+    Type: AWS::SNS::Topic
diff --git a/tests/aws/templates/simple_no_change_with_transformation.yaml b/tests/aws/templates/simple_no_change_with_transformation.yaml
new file mode 100644
--- /dev/null
+++ b/tests/aws/templates/simple_no_change_with_transformation.yaml
@@ -0,0 +1,4 @@
+Transform: AWS::Serverless-2016-10-31
+Resources:
+  MyTopic:
+    Type: AWS::SNS::Topic