Repository: localstack/localstack

Issue:
bug: update CDK exists stack resources on next run
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

I have my code here, its working perfect first time i run cdklocal deploy, if i run it again and remove eg. s3 bucket or my codeartifact domain its will sending a lots of error in my docker container.

```python
from constructs import Construct
from aws_cdk import (
    App, Stack,
    aws_codeartifact as codeartifact,
    aws_s3 as s3,
)


class LocalTestStack(Stack):
    def __init__(self, scope: Construct, id: str, **kwargs):
        super().__init__(scope, id, **kwargs)

        cfn_domain = codeartifact.CfnDomain(
            self,
            "TestCodeArtifactDomain",
            domain_name="DemoDomain"
        )

        s3_bucket_test = s3.Bucket(self, 'TestBucket')
        s3_bucket_test.grant_public_access()

        s3_bucket_test2 = s3.Bucket(self, 'HestBucket')
        s3_bucket_test2.grant_public_access()


app = App()
LocalTestStack(app, "LocalTestStack")
app.synth()
```

my log look like this, look like some localhost proxy post request there are failed.

```
2022-05-04T12:56:44.202:WARNING:localstack.utils.server.http2_server: Error in proxy handler for request POST http://localhost:4566/: 'ParameterValue' Traceback (most recent call last):

  File "/opt/code/localstack/localstack/utils/server/http2_server.py", line 185, in index

    result = await run_sync(handler, request, data)

  File "/opt/code/localstack/localstack/utils/asyncio.py", line 85, in run_sync

    return await loop.run_in_executor(thread_pool, copy_context().run, func_wrapped)

  File "/opt/code/localstack/localstack/utils/threads.py", line 39, in run

    result = self.func(self.params, **kwargs)

  File "/opt/code/localstack/localstack/utils/asyncio.py", line 30, in _run

    return fn(*args, **kwargs)

  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 973, in handler

    return modify_and_forward(

  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 514, in wrapper

    value = func(*args, **kwargs)

  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 594, in modify_and_forward

    listener_result = listener.forward_request(

  File "/opt/code/localstack/.venv/lib/python3.10/site-packages/localstack_ext/services/edge.py.enc", line 132, in forward_request

  File "/opt/code/localstack/localstack/services/edge.py", line 170, in forward_request

    result = do_forward_request(api, method, path, data, headers, port=port)

  File "/opt/code/localstack/localstack/utils/patch.py", line 38, in proxy

    return new(target, *args, **kwargs)

  File "/opt/code/localstack/.venv/lib/python3.10/site-packages/localstack_ext/services/edge.py.enc", line 128, in do_forward_request

  File "/opt/code/localstack/localstack/services/edge.py", line 225, in do_forward_request

    result = do_forward_request_inmem(api, method, path, data, headers, port=port)

  File "/opt/code/localstack/localstack/services/edge.py", line 249, in do_forward_request_inmem

    response = modify_and_forward(

  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 514, in wrapper

    value = func(*args, **kwargs)

  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 594, in modify_and_forward

    listener_result = listener.forward_request(

  File "/opt/code/localstack/localstack/http/adapters.py", line 51, in forward_request

    response = self.request(request)

  File "/opt/code/localstack/localstack/aws/proxy.py", line 42, in request

    response = self.skeleton.invoke(context)

  File "/opt/code/localstack/localstack/aws/skeleton.py", line 153, in invoke

    return self.dispatch_request(context, instance)

  File "/opt/code/localstack/localstack/aws/skeleton.py", line 165, in dispatch_request

    result = handler(context, instance) or {}

  File "/opt/code/localstack/localstack/aws/skeleton.py", line 117, in __call__

    return self.fn(*args, **kwargs)

  File "/opt/code/localstack/localstack/aws/api/core.py", line 95, in operation_marker

    return fn(*args, **kwargs)

  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 951, in execute_change_set

    deployer.apply_change_set(change_set)

  File "/opt/code/localstack/localstack/utils/cloudformation/template_deployer.py", line 1219, in apply_change_set

    self.apply_changes(

  File "/opt/code/localstack/localstack/utils/cloudformation/template_deployer.py", line 1490, in apply_changes

    self.init_resource_status(old_resources, action="UPDATE")

  File "/opt/code/localstack/localstack/utils/cloudformation/template_deployer.py", line 1338, in init_resource_status

    stack.set_resource_status(resource_id, "%s_IN_PROGRESS" % action)

  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 183, in set_resource_status

    self._set_resource_status_details(resource_id, physical_res_id=physical_res_id)

  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 192, in _set_resource_status_details

    resource = self.resources.get(resource_id)

  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 255, in resources

    add_params(defaults=False)

  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 250, in add_params

    else param["ParameterValue"]

KeyError: 'ParameterValue'
```

### Expected Behavior

Expect to keep my stack up to date, its very important when you need to test a new configuration from time to time, and not a "fresh" run.

### How are you starting LocalStack?

With the `localstack` script

### Steps To Reproduce

Steps are wirte in the start.

### Environment

```markdown
- OS: macOS 12.3.1
- LocalStack: 0.14.2.7
- cdklocal: 2.22.0 (build 1db4b16)
- Docker Desktop: 4.7.1
- Docker Engine: 20.10.14
```


### Anything else?

_No response_


Discussion:
Welcome to LocalStack! Thanks for reporting your first issue and our team will be working towards fixing the issue for you or reach out for more background information. We recommend joining our [Slack Community](https://localstack.cloud/contact/) for real-time help and drop a message to LocalStack Pro Support if you are a Pro user! If you are willing to contribute towards fixing this issue, please have a look at our [contributing guidelines](https://github.com/localstack/.github/blob/main/CONTRIBUTING.md) and our [developer guide](https://docs.localstack.cloud/developer-guide/).
I am also seeing this. After updating localstack to `latest` I can no longer re-deploy the CDK bootstrap, which is run currently for all test runs.

`> cdklocal bootstrap aws://000000000000/us-east-1`

```
localstack/localstack   latest       564f2f3bc1e5   46 minutes ago   1.53GB
```

```
2022-05-06T18:43:35.693:INFO:localstack.utils.threads: Thread run method <function AdaptiveThreadPool.submit.<locals>._run at 0x7f62a3a96320>(None) failed: 'ParameterValue' Traceback (most recent call last):
  File "/opt/code/localstack/localstack/utils/threads.py", line 39, in run
    result = self.func(self.params, **kwargs)
  File "/opt/code/localstack/localstack/utils/asyncio.py", line 30, in _run
    return fn(*args, **kwargs)
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 973, in handler
    return modify_and_forward(
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 514, in wrapper
    value = func(*args, **kwargs)
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 594, in modify_and_forward
    listener_result = listener.forward_request(
  File "/opt/code/localstack/localstack/services/edge.py", line 170, in forward_request
    result = do_forward_request(api, method, path, data, headers, port=port)
  File "/opt/code/localstack/localstack/services/edge.py", line 225, in do_forward_request
    result = do_forward_request_inmem(api, method, path, data, headers, port=port)
  File "/opt/code/localstack/localstack/services/edge.py", line 249, in do_forward_request_inmem
    response = modify_and_forward(
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 514, in wrapper
    value = func(*args, **kwargs)
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 594, in modify_and_forward
    listener_result = listener.forward_request(
  File "/opt/code/localstack/localstack/http/adapters.py", line 51, in forward_request
    response = self.request(request)
  File "/opt/code/localstack/localstack/aws/proxy.py", line 42, in request
    return self.skeleton.invoke(context)
  File "/opt/code/localstack/localstack/aws/skeleton.py", line 153, in invoke
    return self.dispatch_request(context, instance)
  File "/opt/code/localstack/localstack/aws/skeleton.py", line 165, in dispatch_request
    result = handler(context, instance) or {}
  File "/opt/code/localstack/localstack/aws/skeleton.py", line 117, in __call__
    return self.fn(*args, **kwargs)
  File "/opt/code/localstack/localstack/aws/api/core.py", line 95, in operation_marker
    return fn(*args, **kwargs)
  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 951, in execute_change_set
    deployer.apply_change_set(change_set)
  File "/opt/code/localstack/localstack/utils/cloudformation/template_deployer.py", line 1219, in apply_change_set
    self.apply_changes(
  File "/opt/code/localstack/localstack/utils/cloudformation/template_deployer.py", line 1490, in apply_changes
    self.init_resource_status(old_resources, action="UPDATE")
  File "/opt/code/localstack/localstack/utils/cloudformation/template_deployer.py", line 1338, in init_resource_status
    stack.set_resource_status(resource_id, "%s_IN_PROGRESS" % action)
  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 183, in set_resource_status
    self._set_resource_status_details(resource_id, physical_res_id=physical_res_id)
  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 192, in _set_resource_status_details
    resource = self.resources.get(resource_id)
  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 255, in resources
    add_params(defaults=False)
  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 250, in add_params
    else param["ParameterValue"]
KeyError: 'ParameterValue'

2022-05-06T18:43:35.693:DEBUG:localstack.utils.threads: FINISHED: <Future at 0x7f62c0de82b0 state=finished raised KeyError>
2022-05-06T18:43:35.694:WARNING:localstack.utils.server.http2_server: Error in proxy handler for request POST http://localhost:4566/: 'ParameterValue' Traceback (most recent call last):
  File "/opt/code/localstack/localstack/utils/server/http2_server.py", line 185, in index
    result = await run_sync(handler, request, data)
  File "/opt/code/localstack/localstack/utils/asyncio.py", line 85, in run_sync
    return await loop.run_in_executor(thread_pool, copy_context().run, func_wrapped)
  File "/opt/code/localstack/localstack/utils/threads.py", line 39, in run
    result = self.func(self.params, **kwargs)
  File "/opt/code/localstack/localstack/utils/asyncio.py", line 30, in _run
    return fn(*args, **kwargs)
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 973, in handler
    return modify_and_forward(
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 514, in wrapper
    value = func(*args, **kwargs)
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 594, in modify_and_forward
    listener_result = listener.forward_request(
  File "/opt/code/localstack/localstack/services/edge.py", line 170, in forward_request
    result = do_forward_request(api, method, path, data, headers, port=port)
  File "/opt/code/localstack/localstack/services/edge.py", line 225, in do_forward_request
    result = do_forward_request_inmem(api, method, path, data, headers, port=port)
  File "/opt/code/localstack/localstack/services/edge.py", line 249, in do_forward_request_inmem
    response = modify_and_forward(
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 514, in wrapper
    value = func(*args, **kwargs)
  File "/opt/code/localstack/localstack/services/generic_proxy.py", line 594, in modify_and_forward
    listener_result = listener.forward_request(
  File "/opt/code/localstack/localstack/http/adapters.py", line 51, in forward_request
    response = self.request(request)
  File "/opt/code/localstack/localstack/aws/proxy.py", line 42, in request
    return self.skeleton.invoke(context)
  File "/opt/code/localstack/localstack/aws/skeleton.py", line 153, in invoke
    return self.dispatch_request(context, instance)
  File "/opt/code/localstack/localstack/aws/skeleton.py", line 165, in dispatch_request
    result = handler(context, instance) or {}
  File "/opt/code/localstack/localstack/aws/skeleton.py", line 117, in __call__
    return self.fn(*args, **kwargs)
  File "/opt/code/localstack/localstack/aws/api/core.py", line 95, in operation_marker
    return fn(*args, **kwargs)
  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 951, in execute_change_set
    deployer.apply_change_set(change_set)
  File "/opt/code/localstack/localstack/utils/cloudformation/template_deployer.py", line 1219, in apply_change_set
    self.apply_changes(
  File "/opt/code/localstack/localstack/utils/cloudformation/template_deployer.py", line 1490, in apply_changes
    self.init_resource_status(old_resources, action="UPDATE")
  File "/opt/code/localstack/localstack/utils/cloudformation/template_deployer.py", line 1338, in init_resource_status
    stack.set_resource_status(resource_id, "%s_IN_PROGRESS" % action)
  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 183, in set_resource_status
    self._set_resource_status_details(resource_id, physical_res_id=physical_res_id)
  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 192, in _set_resource_status_details
    resource = self.resources.get(resource_id)
  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 255, in resources
    add_params(defaults=False)
  File "/opt/code/localstack/localstack/services/cloudformation/provider.py", line 250, in add_params
    else param["ParameterValue"]
KeyError: 'ParameterValue'
```
Stopping `localstack` and restarting seems to be the only way to get around this.
Should also note that CDK hangs

```
Bootstrapping CDK...
 ⏳  Bootstrapping environment aws://000000000000/us-east-1...
Trusted accounts for deployment: (none)
Trusted accounts for lookup: (none)
Using default execution policy of 'arn:aws:iam::aws:policy/AdministratorAccess'. Pass '--cloudformation-execution-policies' to customize.
CDKToolkit: creating CloudFormation changeset...
```
This should be patched in the most recent build now. Please pull `localstack/localstack:latest` again and let us know if you're still experiencing these issues. :+1: 
> This should be patched in the most recent build now. Please pull `localstack/localstack:latest` again and let us know if you're still experiencing these issues. 👍

Thanks a lot for fast react, I will check on work on Monday :)
> This should be patched in the most recent build now. Please pull `localstack/localstack:latest` again and let us know if you're still experiencing these issues. 👍

Its working almost, if i do it with S3 bucket, if i comment codeartifact out after first deploy i hit a new error

```
LocalTestStack: deploying...
[0%] start: Publishing 8ec40cbf3d86e269ce9af726dcff3555d0e8cd740d2f86d828fb8e7ad811b22d:current_account-current_region
[100%] success: Published 8ec40cbf3d86e269ce9af726dcff3555d0e8cd740d2f86d828fb8e7ad811b22d:current_account-current_region
LocalTestStack: creating CloudFormation changeset...

 ❌  LocalTestStack failed: Error: The stack named LocalTestStack failed to deploy: CREATE_FAILED (Deployment failed)
    at prepareAndExecuteChangeSet (/Users/paris.kejser/.nvm/versions/node/v16.15.0/lib/node_modules/aws-cdk/lib/api/deploy-stack.ts:385:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at CdkToolkit.deploy (/Users/paris.kejser/.nvm/versions/node/v16.15.0/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:209:24)
    at initCommandLine (/Users/paris.kejser/.nvm/versions/node/v16.15.0/lib/node_modules/aws-cdk/lib/cli.ts:341:12)

The stack named LocalTestStack failed to deploy: CREATE_FAILED (Deployment failed)
```

Its only happen on codeartifact line.
Hi @ParisNakitaKejser-EB!

AWS CodeArtifact is currently not supported by LocalStack, which results in the error message you are facing.
This can also be found in the logs during deployment via `cdklocal deploy`.

For example, this is the output during deployment of your Stack:
```
localstack_main | 2022-05-19T10:26:15.212:WARNING:localstack.utils.cloudformation.template_deployer: Unexpected resource type CodeArtifact::Domain when resolving references of resource TestCodeArtifactDomain: {"Type": "AWS::CodeArtifact::Domain", "Metadata": {"aws:cdk:path": "UpdateCdkStack/TestCodeArtifactDomain"}, "LogicalResourceId": "TestCodeArtifactDomain", "Properties": {"DomainName": "DemoDomain"}}. To find out if CodeArtifact::Domain is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation
localstack_main | 2022-05-19T10:26:15.212:WARNING:localstack.utils.cloudformation.template_deployer: Unable to deploy resource type "CodeArtifact::Domain": {"Type": "AWS::CodeArtifact::Domain", "Metadata": {"aws:cdk:path": "UpdateCdkStack/TestCodeArtifactDomain"}, "LogicalResourceId": "TestCodeArtifactDomain", "Properties": {"DomainName": "DemoDomain"}}
```

The log output during the deployment attempting to delete the CodeArtifact:
```
localstack_main | 2022-05-19T10:30:32.697:WARNING:localstack.utils.cloudformation.template_deployer: Action delete for resource type CodeArtifact::Domain not available. To find out if CodeArtifact::Domain is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation
localstack_main | 2022-05-19T10:30:32.734:WARNING:localstack.utils.cloudformation.template_deployer: Unexpected resource type CodeArtifact::Domain when resolving references of resource TestCodeArtifactDomain: {"Type": "AWS::CodeArtifact::Domain", "Metadata": {"aws:cdk:path": "UpdateCdkStack/TestCodeArtifactDomain"}, "LogicalResourceId": "TestCodeArtifactDomain", "Properties": {"DomainName": "DemoDomain"}, "PhysicalResourceId": null}. To find out if CodeArtifact::Domain is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation
```


> Hi @ParisNakitaKejser-EB!
> 
> AWS CodeArtifact is currently not supported by LocalStack, which results in the error message you are facing. This can also be found in the logs during deployment via `cdklocal deploy`.
> 
> For example, this is the output during deployment of your Stack:
> 
> ```
> localstack_main | 2022-05-19T10:26:15.212:WARNING:localstack.utils.cloudformation.template_deployer: Unexpected resource type CodeArtifact::Domain when resolving references of resource TestCodeArtifactDomain: {"Type": "AWS::CodeArtifact::Domain", "Metadata": {"aws:cdk:path": "UpdateCdkStack/TestCodeArtifactDomain"}, "LogicalResourceId": "TestCodeArtifactDomain", "Properties": {"DomainName": "DemoDomain"}}. To find out if CodeArtifact::Domain is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation
> localstack_main | 2022-05-19T10:26:15.212:WARNING:localstack.utils.cloudformation.template_deployer: Unable to deploy resource type "CodeArtifact::Domain": {"Type": "AWS::CodeArtifact::Domain", "Metadata": {"aws:cdk:path": "UpdateCdkStack/TestCodeArtifactDomain"}, "LogicalResourceId": "TestCodeArtifactDomain", "Properties": {"DomainName": "DemoDomain"}}
> ```
> 
> The log output during the deployment attempting to delete the CodeArtifact:
> 
> ```
> localstack_main | 2022-05-19T10:30:32.697:WARNING:localstack.utils.cloudformation.template_deployer: Action delete for resource type CodeArtifact::Domain not available. To find out if CodeArtifact::Domain is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation
> localstack_main | 2022-05-19T10:30:32.734:WARNING:localstack.utils.cloudformation.template_deployer: Unexpected resource type CodeArtifact::Domain when resolving references of resource TestCodeArtifactDomain: {"Type": "AWS::CodeArtifact::Domain", "Metadata": {"aws:cdk:path": "UpdateCdkStack/TestCodeArtifactDomain"}, "LogicalResourceId": "TestCodeArtifactDomain", "Properties": {"DomainName": "DemoDomain"}, "PhysicalResourceId": null}. To find out if CodeArtifact::Domain is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation
> ```

Its sad its sound break a redeploy, its fine its not supported but its stille a bug you can deploy first time and then its break secound time, becures then its mean you can run a test run in localstack as you want before sending to AWS.

Resolving patch:
diff --git a/localstack/utils/cloudformation/template_deployer.py b/localstack/utils/cloudformation/template_deployer.py
--- a/localstack/utils/cloudformation/template_deployer.py
+++ b/localstack/utils/cloudformation/template_deployer.py
@@ -1615,6 +1615,13 @@ def do_apply_changes_in_loop(self, changes, stack, stack_name: str):
                         if not self.all_resource_dependencies_satisfied(resource):
                             j += 1
                             continue
+                    elif action == "Remove":
+                        should_remove = self.prepare_should_deploy_change(
+                            resource_id, change, stack, new_resources
+                        )
+                        if not should_remove:
+                            del changes[j]
+                            continue
                     self.apply_change(change, stack=stack)
                     changes_done.append(change)
                     del changes[j]
@@ -1668,6 +1675,13 @@ def prepare_should_deploy_change(self, resource_id, change, stack, new_resources
                     resource.get("Type"),
                 )
                 return False
+        elif action == "Remove":
+            should_remove = self.is_deployable_resource(resource)
+            if not should_remove:
+                LOG.debug(
+                    f"Action 'remove' not yet implemented for CF resource type {resource.get('Type')}"
+                )
+            return should_remove
         return True
 
     def apply_change(self, change, stack):


Test patch:
diff --git a/tests/integration/cloudformation/test_cloudformation_changesets.py b/tests/integration/cloudformation/test_cloudformation_changesets.py
--- a/tests/integration/cloudformation/test_cloudformation_changesets.py
+++ b/tests/integration/cloudformation/test_cloudformation_changesets.py
@@ -8,6 +8,7 @@
 from localstack.testing.aws.util import is_aws_cloud
 from localstack.utils.common import short_uid
 from localstack.utils.generic.wait_utils import wait_until
+from localstack.utils.sync import poll_condition
 
 
 # TODO: refactor file and remove this compatibility fn
@@ -382,3 +383,52 @@ def test_delete_change_set_nonexisting(cfn_client):
         cfn_client.delete_change_set(ChangeSetName="DoesNotExist")
 
     assert ex.value.response["Error"]["Code"] == "ResourceNotFoundException"
+
+
+@pytest.mark.aws_validated
+def test_create_and_then_remove_non_supported_resource_change_set(deploy_cfn_template):
+    # first deploy cfn with a CodeArtifact resource that is not actually supported
+    stack = deploy_cfn_template(
+        template=load_template_raw("code_artifact_template.yaml"),
+        parameters={"CADomainName": f"domainname-{short_uid()}"},
+    )
+
+    # removal of CodeArtifact should not throw exception
+    deploy_cfn_template(
+        is_update=True,
+        template=load_template_raw("code_artifact_remove_template.yaml"),
+        stack_name=stack.stack_name,
+    )
+
+
+@pytest.mark.aws_validated
+def test_create_and_then_remove_supported_resource_change_set(deploy_cfn_template, s3_client):
+    first_bucket_name = f"test-bucket-1-{short_uid()}"
+    second_bucket_name = f"test-bucket-2-{short_uid()}"
+
+    stack = deploy_cfn_template(
+        template=load_template_raw("for_removal_setup.yaml"),
+        template_mapping={
+            "first_bucket_name": first_bucket_name,
+            "second_bucket_name": second_bucket_name,
+        },
+    )
+
+    available_buckets = s3_client.list_buckets()
+    bucket_names = [bucket["Name"] for bucket in available_buckets["Buckets"]]
+    assert first_bucket_name in bucket_names
+    assert second_bucket_name in bucket_names
+
+    deploy_cfn_template(
+        is_update=True,
+        template=load_template_raw("for_removal_remove.yaml"),
+        template_mapping={"first_bucket_name": first_bucket_name},
+        stack_name=stack.stack_name,
+    )
+
+    def assert_bucket_gone():
+        available_buckets = s3_client.list_buckets()
+        bucket_names = [bucket["Name"] for bucket in available_buckets["Buckets"]]
+        return first_bucket_name in bucket_names and second_bucket_name not in bucket_names
+
+    poll_condition(condition=assert_bucket_gone, timeout=20, interval=5)
diff --git a/tests/integration/templates/code_artifact_remove_template.yaml b/tests/integration/templates/code_artifact_remove_template.yaml
new file mode 100644
--- /dev/null
+++ b/tests/integration/templates/code_artifact_remove_template.yaml
@@ -0,0 +1,134 @@
+Resources:
+  TestBucket560B80BC:
+    Type: AWS::S3::Bucket
+    UpdateReplacePolicy: Delete
+    DeletionPolicy: Delete
+    Metadata:
+      aws:cdk:path: UpdateCdkStack/TestBucket/Resource
+  TestBucketPolicyBA12ED38:
+    Type: AWS::S3::BucketPolicy
+    Properties:
+      Bucket:
+        Ref: TestBucket560B80BC
+      PolicyDocument:
+        Statement:
+          - Action: s3:GetObject
+            Effect: Allow
+            Principal:
+              AWS: "*"
+            Resource:
+              Fn::Join:
+                - ""
+                - - Fn::GetAtt:
+                      - TestBucket560B80BC
+                      - Arn
+                  - /*
+        Version: "2012-10-17"
+    Metadata:
+      aws:cdk:path: UpdateCdkStack/TestBucket/Policy/Resource
+  HestBucketABE4AE1C:
+    Type: AWS::S3::Bucket
+    UpdateReplacePolicy: Delete
+    DeletionPolicy: Delete
+    Metadata:
+      aws:cdk:path: UpdateCdkStack/HestBucket/Resource
+  HestBucketPolicy276ECA48:
+    Type: AWS::S3::BucketPolicy
+    Properties:
+      Bucket:
+        Ref: HestBucketABE4AE1C
+      PolicyDocument:
+        Statement:
+          - Action: s3:GetObject
+            Effect: Allow
+            Principal:
+              AWS: "*"
+            Resource:
+              Fn::Join:
+                - ""
+                - - Fn::GetAtt:
+                      - HestBucketABE4AE1C
+                      - Arn
+                  - /*
+        Version: "2012-10-17"
+    Metadata:
+      aws:cdk:path: UpdateCdkStack/HestBucket/Policy/Resource
+  CDKMetadata:
+    Type: AWS::CDK::Metadata
+    Properties:
+      Analytics: v2:deflate64:H4sIAAAAAAAA/0XIQQqDMBCF4bO4T6ZaKXRdewDRA5R0jHSqzkAyUiR49yoWunr/+85QXCDP3Cda7AY70hNSqw4Hs9EjoXTeBaXeoULV810mR2xiCek24+DVbPirY2oZCZc/H39dd2l8lDmg37sS7khJeDX1oi/hUwlXKPLsHYlsmFlp8tAc+wUWGrSMpgAAAA==
+    Metadata:
+      aws:cdk:path: UpdateCdkStack/CDKMetadata/Default
+    Condition: CDKMetadataAvailable
+Conditions:
+  CDKMetadataAvailable:
+    Fn::Or:
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - af-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-northeast-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-northeast-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-southeast-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-southeast-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ca-central-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - cn-north-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - cn-northwest-1
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-central-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-north-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-3
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - me-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - sa-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-east-2
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-west-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-west-2
diff --git a/tests/integration/templates/code_artifact_template.yaml b/tests/integration/templates/code_artifact_template.yaml
new file mode 100644
--- /dev/null
+++ b/tests/integration/templates/code_artifact_template.yaml
@@ -0,0 +1,145 @@
+Parameters:
+  CADomainName:
+    Description: Name of the CodeArtifact domain
+    Type: String
+    Default: testingDomainName
+Resources:
+  TestCodeArtifactDomain:
+    Type: AWS::CodeArtifact::Domain
+    Properties:
+      DomainName: !Ref CADomainName
+    Metadata:
+      aws:cdk:path: UpdateCdkStack/TestCodeArtifactDomain
+  TestBucket560B80BC:
+    Type: AWS::S3::Bucket
+    UpdateReplacePolicy: Delete
+    DeletionPolicy: Delete
+    Metadata:
+      aws:cdk:path: UpdateCdkStack/TestBucket/Resource
+  TestBucketPolicyBA12ED38:
+    Type: AWS::S3::BucketPolicy
+    Properties:
+      Bucket:
+        Ref: TestBucket560B80BC
+      PolicyDocument:
+        Statement:
+          - Action: s3:GetObject
+            Effect: Allow
+            Principal:
+              AWS: "*"
+            Resource:
+              Fn::Join:
+                - ""
+                - - Fn::GetAtt:
+                      - TestBucket560B80BC
+                      - Arn
+                  - /*
+        Version: "2012-10-17"
+    Metadata:
+      aws:cdk:path: UpdateCdkStack/TestBucket/Policy/Resource
+  HestBucketABE4AE1C:
+    Type: AWS::S3::Bucket
+    UpdateReplacePolicy: Delete
+    DeletionPolicy: Delete
+    Metadata:
+      aws:cdk:path: UpdateCdkStack/HestBucket/Resource
+  HestBucketPolicy276ECA48:
+    Type: AWS::S3::BucketPolicy
+    Properties:
+      Bucket:
+        Ref: HestBucketABE4AE1C
+      PolicyDocument:
+        Statement:
+          - Action: s3:GetObject
+            Effect: Allow
+            Principal:
+              AWS: "*"
+            Resource:
+              Fn::Join:
+                - ""
+                - - Fn::GetAtt:
+                      - HestBucketABE4AE1C
+                      - Arn
+                  - /*
+        Version: "2012-10-17"
+    Metadata:
+      aws:cdk:path: UpdateCdkStack/HestBucket/Policy/Resource
+  CDKMetadata:
+    Type: AWS::CDK::Metadata
+    Properties:
+      Analytics: v2:deflate64:H4sIAAAAAAAA/0XIQQqDMBCF4bO4T6ZaKXRdewDRA5R0jHSqzkAyUiR49yoWunr/+85QXCDP3Cda7AY70hNSqw4Hs9EjoXTeBaXeoULV810mR2xiCek24+DVbPirY2oZCZc/H39dd2l8lDmg37sS7khJeDX1oi/hUwlXKPLsHYlsmFlp8tAc+wUWGrSMpgAAAA==
+    Metadata:
+      aws:cdk:path: UpdateCdkStack/CDKMetadata/Default
+    Condition: CDKMetadataAvailable
+Conditions:
+  CDKMetadataAvailable:
+    Fn::Or:
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - af-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-northeast-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-northeast-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-southeast-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-southeast-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ca-central-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - cn-north-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - cn-northwest-1
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-central-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-north-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-3
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - me-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - sa-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-east-2
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-west-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-west-2
diff --git a/tests/integration/templates/for_removal_remove.yaml b/tests/integration/templates/for_removal_remove.yaml
new file mode 100644
--- /dev/null
+++ b/tests/integration/templates/for_removal_remove.yaml
@@ -0,0 +1,109 @@
+Resources:
+  TestBucket560B80BC:
+    Type: AWS::S3::Bucket
+    UpdateReplacePolicy: Delete
+    Properties:
+      BucketName: {{ first_bucket_name }}
+    DeletionPolicy: Delete
+    Metadata:
+      aws:cdk:path: RemoveCdkStack/TestBucket/Resource
+  TestBucketPolicyBA12ED38:
+    Type: AWS::S3::BucketPolicy
+    Properties:
+      Bucket:
+        Ref: TestBucket560B80BC
+      PolicyDocument:
+        Statement:
+          - Action: s3:GetObject
+            Effect: Allow
+            Principal:
+              AWS: "*"
+            Resource:
+              Fn::Join:
+                - ""
+                - - Fn::GetAtt:
+                      - TestBucket560B80BC
+                      - Arn
+                  - /*
+        Version: "2012-10-17"
+    Metadata:
+      aws:cdk:path: RemoveCdkStack/TestBucket/Policy/Resource
+  CDKMetadata:
+    Type: AWS::CDK::Metadata
+    Properties:
+      Analytics: v2:deflate64:H4sIAAAAAAAA/0XIQQqDMBCF4bO4T6ZaKXRdewDRA5R0jHSqzkAyUiR49yoWunr/+85QXCDP3Cda7AY70hNSqw4Hs9EjoXTeBaXeoULV810mR2xiCek24+DVbPirY2oZCZc/H39dd2l8lDmg37sS7khJeDX1oi/hUwlXKPLsHYlsmFlp8tAc+wUWGrSMpgAAAA==
+    Metadata:
+      aws:cdk:path: RemoveCdkStack/CDKMetadata/Default
+    Condition: CDKMetadataAvailable
+Conditions:
+  CDKMetadataAvailable:
+    Fn::Or:
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - af-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-northeast-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-northeast-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-southeast-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-southeast-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ca-central-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - cn-north-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - cn-northwest-1
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-central-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-north-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-3
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - me-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - sa-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-east-2
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-west-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-west-2
diff --git a/tests/integration/templates/for_removal_setup.yaml b/tests/integration/templates/for_removal_setup.yaml
new file mode 100644
--- /dev/null
+++ b/tests/integration/templates/for_removal_setup.yaml
@@ -0,0 +1,138 @@
+Resources:
+  TestBucket560B80BC:
+    Type: AWS::S3::Bucket
+    UpdateReplacePolicy: Delete
+    Properties:
+      BucketName: {{ first_bucket_name }}
+    DeletionPolicy: Delete
+    Metadata:
+      aws:cdk:path: RemoveCdkStack/TestBucket/Resource
+  TestBucketPolicyBA12ED38:
+    Type: AWS::S3::BucketPolicy
+    Properties:
+      Bucket:
+        Ref: TestBucket560B80BC
+      PolicyDocument:
+        Statement:
+          - Action: s3:GetObject
+            Effect: Allow
+            Principal:
+              AWS: "*"
+            Resource:
+              Fn::Join:
+                - ""
+                - - Fn::GetAtt:
+                      - TestBucket560B80BC
+                      - Arn
+                  - /*
+        Version: "2012-10-17"
+    Metadata:
+      aws:cdk:path: RemoveCdkStack/TestBucket/Policy/Resource
+  HestBucketABE4AE1C:
+    Type: AWS::S3::Bucket
+    Properties:
+      BucketName: {{ second_bucket_name }}
+    UpdateReplacePolicy: Delete
+    DeletionPolicy: Delete
+    Metadata:
+      aws:cdk:path: RemoveCdkStack/HestBucket/Resource
+  HestBucketPolicy276ECA48:
+    Type: AWS::S3::BucketPolicy
+    Properties:
+      Bucket:
+        Ref: HestBucketABE4AE1C
+      PolicyDocument:
+        Statement:
+          - Action: s3:GetObject
+            Effect: Allow
+            Principal:
+              AWS: "*"
+            Resource:
+              Fn::Join:
+                - ""
+                - - Fn::GetAtt:
+                      - HestBucketABE4AE1C
+                      - Arn
+                  - /*
+        Version: "2012-10-17"
+    Metadata:
+      aws:cdk:path: RemoveCdkStack/HestBucket/Policy/Resource
+  CDKMetadata:
+    Type: AWS::CDK::Metadata
+    Properties:
+      Analytics: v2:deflate64:H4sIAAAAAAAA/0XIQQqDMBCF4bO4T6ZaKXRdewDRA5R0jHSqzkAyUiR49yoWunr/+85QXCDP3Cda7AY70hNSqw4Hs9EjoXTeBaXeoULV810mR2xiCek24+DVbPirY2oZCZc/H39dd2l8lDmg37sS7khJeDX1oi/hUwlXKPLsHYlsmFlp8tAc+wUWGrSMpgAAAA==
+    Metadata:
+      aws:cdk:path: RemoveCdkStack/CDKMetadata/Default
+    Condition: CDKMetadataAvailable
+Conditions:
+  CDKMetadataAvailable:
+    Fn::Or:
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - af-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-northeast-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-northeast-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-southeast-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ap-southeast-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - ca-central-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - cn-north-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - cn-northwest-1
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-central-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-north-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-2
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - eu-west-3
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - me-south-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - sa-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-east-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-east-2
+      - Fn::Or:
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-west-1
+          - Fn::Equals:
+              - Ref: AWS::Region
+              - us-west-2