Migrate resources across CDK stacks
Tags: aws, cdk, migrate, resources
Published: 2022-12-29 08:02:42

When you start building infrastructure as code. You will run into the situation that, you want to split a stack into separate ones. Or, you deployed a resource in the wrong stack. This is all part of the development lifecycle. You try things, some succeed, some will fail. But deployed production resources are hard to rebuild. They either have a complex data migration path or you can't afford the downtime.

If you accept that some resources or located in unexpected stacks. Your complexity of your infrastructure will increase. And complexity will lead to incidents in the future.

In this blog post I will explain how you can move a deployed resource from one CDK stack to another.

Introduction
------------

First of all, it is important to realize that CDK generates CloudFormation. CloudFormation offers functionality to bring existing resources under it’s control. This means you can import an already deployed resource into a CloudFormation stack.
Now how does that relate to CDK?

Lets assume you created a CDK stack that created a DynamoDB table. This table contains information or is in use. In these scenarios you cannot delete the resource and recreate it. That would lead to data loss or downtime. So in this scenario we could migrate the resource from one stack to another.

Prerequisites
-------------

You need to have the following command line tools in place.

* [AWS CLI](https://aws.amazon.com/cli/)
* [jq](https://stedolan.github.io/jq/)

Make sure you have valid credentials in the correct account.

Step 1 – Remove the resource from the existing stack
----------------------------------------------------

Download the existing CloudFormation template:

```bash
aws cloudformation get-template \
    --output json \
    --stack-name <my-cloudformation-stack-name> | jq -r '.TemplateBody' > original-template.yml
cp original-template.yml modified-template.yml
```

In the template `modified-template.yml` you need to locate the resource you want to move to your “new” stack. Once you found it you need to add the following `DeletionPolicy: Retain` to the resource.

```yaml
Resources:
  MyTable794EDED1:
    Type: AWS::DynamoDB::Table
    Properties:
      KeySchema:
        - AttributeName: id
          KeyType: HASH
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
      ProvisionedThroughput:
        ReadCapacityUnits: 5
        WriteCapacityUnits: 5
      TableName: Games
    UpdateReplacePolicy: Retain
    DeletionPolicy: Retain
```

This will preserve the resource when you delete it from the stack. Now we will update the CloudFormation stack:

```bash
aws cloudformation update-stack \
    --stack-name <my-cloudformation-stack-name> \
    --template-body file://modified-template.yml
cp modified-template.yml final-template.yml
```

The next step depends a bit on your situation. In some cases you can remove the complete stack because, there are no other resources in the stack. Or because they are not needed anymore. But in other cases you need to move out a resource. (A DynamoDB Table in this example)

When you don’t need the other resources anymore you can delete the complete stack. If you want to preserve the rest of the resources. You will need to remove the resource from the template that you want to move. So we remove `MyTable794EDED1` from the `final-template.yml` file.

By re-running the `update-stack` command again. We will instruct CloudFormation to remove the resource. But, because we have the `Retain` policy set. The resource will only be removed from the stack.

```bash
aws cloudformation update-stack \
    --stack-name <my-cloudformation-stack-name> \
    --template-body file://final-template.yml
```

Step 2 – Import the resource
----------------------------

When you move resources around you either move them to a new or an existing stack. When you deal with an existing stack you need the existing template first.
If you are updating an existing stack, continue with “Download the existing stack”. Otherwise you can skip that section.

### Download the existing stack

Download the existing CloudFormation template:

```bash
aws cloudformation get-template \
    --output json \
    --stack-name <my-other-cloudformation-stack-name> | jq -r '.TemplateBody' > template.yml
```

### Import the existing resource

When you want to move the resource to a new stack. You need to create a new `template.yml` file. If you want to move the resource in an existing stack you already have the existing template ready.

To import the resource we need the resource definition. You can copy that from the `modified-template.yml` file from step 1. I could recommend this because the CloudFormation definition should match the deployed resource. The second thing is it needs to have the `DeletionPolicy: Retain` definition. So our example will look like this:

```yaml
Resources:
  MyTable794EDED1:
    Type: AWS::DynamoDB::Table
    Properties:
      KeySchema:
        - AttributeName: id
          KeyType: HASH
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
      ProvisionedThroughput:
        ReadCapacityUnits: 5
        WriteCapacityUnits: 5
      TableName: Games
    UpdateReplacePolicy: Retain
    DeletionPolicy: Retain
```

Next we need to create a file, lets call it `import.json`. In this file we will define the following for each resource what we import:

* **AWS Resource type**, the type of the resource itself.
* **LogicalResourceId**, the identifier in the template. In this example that would be `Games`.
* **ResourceIdentifier**, the identifier of the existing resource.

As you can see this file contains the relation between the stack and what is already deployed. The file looks as followed:

```json
[
    {
        "ResourceType":"AWS::DynamoDB::Table",
        "LogicalResourceId":"MyTable794EDED1",
        "ResourceIdentifier": {
            "TableName":"Games"
        }
    }
]
```

Next, you will need deploy the stack. You can do this with the following commands:

```bash
aws cloudformation create-change-set \
    --stack-name <my-other-cloudformation-stack-name> \
    --change-set-name ImportChangeSet \
    --change-set-type IMPORT \
    --template-body file://template.yml \
    --resources-to-import file://import.json
aws cloudformation wait change-set-create-complete --change-set-name ImportChangeSet --stack-name <my-other-cloudformation-stack-name>
aws cloudformation execute-change-set --change-set-name ImportChangeSet --stack-name <my-other-cloudformation-stack-name>
```

Step 3 – Redeploy using CDK
---------------------------

At this point the DynamoDB table is part of the new stack. But when you redeploy the CDK stack it will not render the DynamoDB resource definition. So for this we will need to add the definition to the CDK stack.
For this example I used the following CDK snippet. But when you move from one CDK stack to another you can just move the relevant code.

```python
dynamodb.Table(self, "MyTable",
    table_name="GamesTable",
    partition_key=dynamodb.Attribute(name="id", type=dynamodb.AttributeType.STRING),
    billing_mode=dynamodb.BillingMode.PROVISIONED
)
```

Now in theory this should render the same template you used during the import. But if there are changes CloudFormation will thread them as updates. It tries to change the resource to its new state.
After you deployed using CDK its business as usual.

Conclusion
----------

By reducing complexity in your infrastructure. You reduce the chance on failures and incidents. This reduction can be achieved by moving resources to more logical stacks. The process itself does contain manual steps. But when your resources contain data or cannot afford downtime. This might be your only solution.