Migrating from ECS to EKS
Tags: amazon, awsecs, awseks
Published: 2023-09-22 11:40:05


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mb1h05e49vo4x3w7p4ga.png)

Not! Given the ease of configuration, & deployments choosing ECS wasn’t the wrong choice at all. Additionally, from a cost perspective, since ECS doesn’t charge you for orchestrating, it turns out to be much cheaper than EKS. EKS takes a $72 fee per cluster. Until you actually breach the VPC and infra limits that ECS has to offer, going the EKS route was the way to go. Given that you’re reading this, I guess you have, and it’s time for a makeover!


![Image description](https://miro.medium.com/v2/resize:fit:640/0*7aKUXhjyb2H_zgMr.gif)

If you’re thinking about moving out of ECS, and to EKS, think of these points like Pokemon — gotta catch ’em all. Okay, not all but at least 3!

1. Breaking out of the AWS vendor lock-in ECS comes with a tightly coupled vendor

lock-in to AWS services even for things like logging, & observability. These prices increase significantly after you’ve hit some scale and the costs quickly spiral out of control.
‍[this article](https://www.wednesday.is/writing-articles/aws-ecs-breaking-observability-vendor-lock-in-signoz) by Kshitij Gang takes you through the ABCs of breaking out of the AWS vendor lock-in with ECS.

2. De-risking yourself by having multi-cloud support
‍ Let’s face it your application shouldn’t ever be down. But what can you do if AWS itself is down? Distributing workloads across multi-cloud infrastructure providers makes your product resilient to vendor downtimes, as well as de-risks you from the perils of relying deeply on a single infrastructure vendor.

3. Configuring granular access control for and over resources
‍ Kubernetes allows more granular control over resources within the cluster and the individual components that interact with it. It provides a more advanced security model, including role-based access control and encryption.

4. Leveraging state-of-the-art community plugins, extensions, and tools
‍ Kubernetes community is able to build creative solutions to complex problems which we are able to leverage as opposed to ECS with is closed source, and we are only dependent on the AWS ECS Roadmap for changes

5. Support for increased scale
‍ ECS has hard limits in terms of hardware, and networking components and there’s no way out of it. Once you’re breaching these limits there is no straightforward way to solve this. Additionally, if you’re using a fargate cluster time to allocate compute also increases as you increase the size of the instance that you’re allocating. That’s another consideration that you need to make. With EKS you can do a combination of node groups with fargate, so you have some provisioned hardware with the ability to scale on demand


![Image description](https://uploads-ssl.webflow.com/5f3a212688d83d3bea8f0438/645e257ba2377005a1c8da1f_giphy.gif)

## How to migrate from ECS to EKS

Current system architecture with ECS -

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/77hlxmppkhv0lecpbtco.png)

System architecture once we migrate to EKS -


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lbdwmoascp5ahptdve42.png)

For the migration, we will use the existing vpc, subnet, and docker image from the ECS.

We will be using the [ecs-eks-migration](https://github.com/bhaveshpawar07/migrate-ecs-eks) repo, go ahead and clone it to get started with the migration.

To start off we have some prerequisites

- [helm](https://helm.sh/docs/intro/install/)
- [yq](https://github.com/mikefarah/yq#install)
- [docker desktop](https://www.docker.com/products/docker-desktop/)
- [AWS cli](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
- [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl)
- [eksctl](https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html)

Apart from this we also need to have subnets id beforehand, subnets to be tagged, nat gateway, and Image URI

To get the image URI,

1. Go to ECS and then click on your service
2. Click on the “Configurations and task” tab and then click on the task that's running
3. Click on the running task and then you will see containers in the task
4. Click on the container and you will see the Image URI, copy that we will need it.

![Image description](https://uploads-ssl.webflow.com/5f3a212688d83d3bea8f0438/645df2c9d013ab43563ab626_imageuri.gif)

Steps to get image uri from ecs.

Now that you are ready, let's get started.

![Image description](https://uploads-ssl.webflow.com/5f3a212688d83d3bea8f0438/645df4b7e1ab4f62abd7baa6_captain-america-cap.gif)

Get the subnet ids from the ecs
 a. Go to your ECS
 b. Click on your service, then go to the networking tab you will find vpc and subnets id’s
      2. Now the first thing we do is tag the public and private subnets, [why do we tag them](https://eksctl.io/usage/vpc-configuration/#use-existing-vpc-other-custom-configuration)?
           a. Add kubernetes.io/cluster/<name> and set it to either shared or owned here <name> is your eks cluster name that we will be creating, in our case it will be applicationName-envName. Do this for all your private and public subnets.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/72b2jgollagj22xm10a9.png)

b. Add kubernetes.io/role/internal-elb tag set to 1 for private subnets

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2y2a5kr8hcr4lfpd7l2s.png)

      c. Add kubernetes.io/role/elb tag set to 1 for public subnets

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wco0pjewzcjkg2lnw96p.png)

3. Once tagging is done we will now create a  NAT gateway and associate it with the route table

    a.  Open NAT Gateway from the VPC service

    b.  Click on Create NAT gateway.

    c.   Enter the name for the NAT gateway, select a **PUBLIC** public subnet, and allocate elastic IP.

    d.  Create NAT gateway

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g0y1ets83iwcvtn6dot0.png)

 e.  Now, we will add routing to our NAT gateway. Go to your private subnet and click on the route table tab and then open the route table.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ktfz3jh52ysuxrfp6jwg.png)

  f.  Click on edit routes and add a route for 0.0.0.0/0 to the nat gateway


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/562ruszui87nzcogpeke.png)

   g.  Click save changes, and then on the route table page click on the subnet associations tab and edit subnet associations

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8f60zziwq55ywfz82d6b.png)

  h.  Now select your private subnets and click save


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e0w7eu1mh0kpfx0zxzql.png)

4. All the additional work is done now we just need to run a few scripts and boom 💥 we will be on EKS.

5. We will be using [ecs-eks-migration](https://github.com/bhaveshpawar07/migrate-ecs-eks) as the base, but don’t worry the contents of this repo will work for your workload as well. You just need to make sure that the values that you pass are correct.

`git clone https://github.com/bhaveshpawar07/migrate-ecs-eks.git`

6. Before starting we will be requiring the following parameters for the command to work.
   **a.  region**
    Your AWS account region where you want to deploy.

     **b.  applicationName**
    You can give any name here, this will be used as your helm app and eks cluster name.

     **c.  env**
     Again this will be part of your cluster name, as said earlier and also will be used for your helm application.

     **d.  awsAccountId**
     Your AWS account ID.

     **e.  pvtSubnets**
     These will be the private subnets where your current ecs is deployed
    **Example‍**

    ‍pvtSubnets=privatesubnet1,privatesubnet2

    **f.  pubSubnets**
     These will be the public subnets where your current ecs is deployed.

    **Example**

      **pubSubnets**=publicsubnet1,publicsubnet2‍

> _💡 PS - Using your private and public subnets, the EKS cluster will be deployed in your existing VPC._

     **g.  port**

    The Port on which your application is running will be used in the k8s app as the target port.

     **h.  imageLocation**
Finally the ARN of your existing ECS Image that you want to use for the deployment‍

> 💡 PS - You can find the image from your ECS→Task→Click on your tasks and inside configuration look out for containers and there you will find “Image URI”‍

7. That's it once you have all the parameters ready replace them in the below command and execute the command.

`make create-cluster region={yourRegion}\
 applicationName={yourApplicationName}\
 env={yourEnv}\
 awsAccountId={yourAwsAccountId}\
 pvtSubnet={yourPrivateSubnets}\
 pubSubnet={yourPublicSubnets}\
 port={yourPort}\
 imageLocation={yourImageArn}`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2h65i110nw9xthbht3zj.png)

Here starts the magic, get your 🍿 and just watch things being set up, this command can take from 15 to 30 mins.

## Let's understand the magic behind
The **_make create-cluster_** script does a lot of things starting from creating a cluster to setting up a load balancer, let's have a detailed look at the script.

##Create cluster _make create-cluster_
First things first, the script will create an EKS Cluster for you in the existing vpc using the private and public subnets given in the command.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/te04v3vqwr3954oboi9f.png)

Once the cluster is created, it will then update the local kubectl context, so that you can use the kubectl command locally and access the cluster.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/93grqu1yb6ydgcwfplog.png)

## Create helm application _make helm-app
_Once the cluster is ready, we will be using Helm to create and install applications on the cluster.

The make helm-app command does a lot of things, first, it will create a chart directory for the application

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ds9ptcnl4jyahd1lzn0a.png)

Then the script will take the default values file and change it using yq to match our application needs. It will set the correct port for the service to listen, it will set the image path to the existing ECS image, also it will set the host and other ingress details.

You can go through the file here and read about each command being executed.

## Install helm application _make helm-upgrade_

Once we have our helm application ready with all the correct values, its time to now install the application.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8oi6lh7ky5s8ok1mkacb.png)

The upgrade command with the install flag will install the application.

Interestingly if you want multiple env for your application, then you can simply duplicate the existing **_values.$env.yaml_** file and change the env name in the new file and then pass the file to the upgrade command with one more **--values** flag and it will work.

And ofcourse you will have to change the name of the duplicated file to something like **_values.{newenv}.yaml_**

Lets consider you need to create a qa env from the dev -

1. Copy the file and change its name
**_cp values.dev.yaml values.qa.yaml_**

2. Change the relevant values, for example

`env:
  configmap:
    data:
      ENVIRONMENT_NAME: qa`

3. Now run this command to create an application for your qa env. Note this will use the same cluster as your existing application. In order to have separation of       concerns you can either use a virtual cluster, or namespaces

`helm upgrade --install\
 $applicationName-$newenv\
  ./helm/$applicationName\
  --values=./helm/$applicationName/values.$env.yaml\
  --values=./helm/$applicationName/values.$newenv.yaml\
  --debug --wait --force`

Once you run this helm application will consider the **_$env.yaml_** as the base file and then **_$newenv.yaml_** will be used as an overriding file and the changes will be made. Also you will have to change your application name in order to install it.

## Creating an IAM account _make create-iamaccount_

Now before we create the load balancer and get the DNS which is the most exciting part ( only if it succeeds 😅) We will first create an IAM service account which will allow our EKS cluster to securely access AWS resources.

To start with we will first associate OIDC with our cluster

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n5j2g4bs6xafnldjumud.png)

Then we will create an IAM policy that will be attached to our service account, which will allow a lot of things , along with the creation of load balancer and target group.

We will using this [policy](https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.4/docs/install/iam_policy.json), you can see what is allows your service to do.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l755d74p7b8u7ef1xn7r.png)

Once the policy is created , finally we will create the service account and attach the policy to it.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rnhk5u099oa0pedagmct.png)

## Creating an application load balancer _make create-alb_
Now the final part, we will update our helm repo and add in the [https://aws.github.io/eks-charts](https://aws.github.io/eks-charts), from which we will install the application load balancer


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4oqdt6idscbj8kxzlx3l.png)

Then we get our vpc id, to create the load balancer in the same vpc

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f7wf6b12dijptumkq1rx.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k5y7wzxpomr9ohmpe9wu.png)

Now we will install the load balancer controller , which will create a load balancer and we will also attach the service account that we had created for the it to have access to AWS services.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tme65arpcca413s7tz9n.png)

Ok ok , I know even if after doing so much we haven't reached the end, but here is the last command which will create the CRD’s for the AWS load balancer controller.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ros56dcpfmn975xf9n8x.png)

🥱 Phew, that was quite a lot to digest! If you’ve followed along, pat yourself on the back first following along, and second cause, while you were reading this your EKS cluster, was created and is now already up ( hopefully🤞🏻) and now in just a few minutes you will be having your load balancer up and running🕺🏻

Once this command completes all you have to do is check for the newly created load balancer, grab the DNS name and you are done.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eyivcf2bvn6ddxg82e2a.png)

Alternatively, you can hit kubectl get ingress in your terminal and you can get the load balancer address.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/plzty91xmh4a73z7zu6y.png)

Before you access the load balancer make sure to check in the AWS console if the load balancer is Running or in a Provisioning state.

Once it is in a Running state, you can then access it.

In my case accessing the URL hits my server and logs a message saying “Api is serving”

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h9m8u6epkde8w48629o9.png)

Now, once your EKS cluster is up and running you can do whatever you want with your ECS, even this


![Image description](https://uploads-ssl.webflow.com/5f3a212688d83d3bea8f0438/645e257ec56a319597181f10_pir%25C3%25B3mano-meme.gif)

**💀❗Just make sure you do not delete the image, the ECR, and the vpc as we are using them**

## Where to go from here?

Ok, it’s kinda crazy but we actually figured out when to migrate our workloads from ECS to EKS, and exactly how it should be done. That’s insane homie. Go now, brag to your friends, colleagues, and your boss. Ah, before you go, take this CD pipeline to automate your set-up, won’t you?

⚠️ Before you add the below jobs to your existing CD pipeline, make sure to copy the helm folder from the project/location where you ran the migration commands (i.e the [ecs-eks-migration](https://github.com/bhaveshpawar07/migrate-ecs-eks) clone location) to your actual application folder.

Also, make sure to have these secrets in your GitHub repo:

**AWS_ACCESS_KEY_ID**

      Your AWS access key

**AWS_SECRET_ACCESS_KEY**
      Yo. r AWS secret access key

**AWS_REGION**

      Your AWS region

**PROJECT_NAME**

    Your application name has to be the same as the one you gave will creating the helm app, if it's not you need to make some changes to the cd file accordingly.

    Dont, add the below

**env.ENVIRONMENT_NAME**

    We will set this variable based on the branch we are deploying to.

`- name: Set branch name
  id: vars
  run: echo ::set-output name=stage::${GITHUB_REF#refs/*/}
- name: Set env.ENVIRONMENT_NAME #1 
  run: |
    if [[ ${{ steps.vars.outputs.stage }} == 'main' ]]; then
        echo "ENVIRONMENT_NAME=prod" >> "$GITHUB_ENV"
    else
        echo "ENVIRONMENT_NAME=${{ steps.vars.outputs.stage }}" >> "$GITHUB_ENV"
    fi
- name: Configure AWS credentials
  uses: aws-actions/configure-aws-credentials@v1
  with:
    aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
    aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
    aws-region: ${{ secrets.AWS_REGION }}
- name: Login to Amazon ECR
  id: login-ecr
  uses: aws-actions/amazon-ecr-login@v1
- name: Build, tag, and push docker image to Amazon ECR #2
  env:
    REGISTRY: ${{ steps.login-ecr.outputs.registry }}
    REPOSITORY: ${{ secrets.PROJECT_NAME }}-${{ env.ENVIRONMENT_NAME }}
    IMAGE_TAG: ${{ github.sha }}
  run: |
    docker build -t $REGISTRY/$REPOSITORY:$IMAGE_TAG .
    docker push $REGISTRY/$REPOSITORY:$IMAGE_TAG
- name: Deploy to EKS #3
	uses: peymanmortazavi/eks-helm-deploy@v2.2 
  with:
    aws-access-key-id: ${{ secrets.AWS_ACCESS__KEY_ID }}
    aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
    aws-region: ${{ secrets.AWS_REGION }}
    cluster-name: ${{ secrets.PROJECT_NAME }}-${{ env.ENVIRONMENT_NAME }}
    config-files: ./helm/${{ secrets.PROJECT_NAME }}/values.${{ env.ENVIRONMENT_NAME }}.yaml
    values: image.tag=${{ github.sha }},image.repository=${{ steps.login-ecr.outputs.registry }}/${{ secrets.PROJECT_NAME }}-${{ env.ENVIRONMENT_NAME }}
    name: ${{ secrets.PROJECT_NAME }}-${{ env.ENVIRONMENT_NAME }}
    chart-path: ./helm/${{ secrets.PROJECT_NAME }}`

Let's understand what are the jobs-

**1. Set env.ENVIRONMENT_NAME**

2. In this job we are setting the ENVIRONMENT_NAME variable based on the current branch name, you can change the env name or branch name according to your requirements.

3. **Build, tag, and push the docker image to Amazon ECR** We will build our image and push it to the ECR, with proper tags. Before pushing we are logging into Amazon ECR in the previous step(Login to Amazon ECR)

4. **Deploy to EKS** In this final setup, we will use eks-helm-deploy action to log in to eks and deploy our chart, read more about the action [here](https://github.com/peymanmortazavi/eks-helm-deploy).

      And if you’re looking for help with moving your workloads from ECS → EKS or you are stuck with other ECS issues just hit us up at [work@wednesday.is](work@wednesday.is) and sit tight while we wave our magic wand! Ok, I’ll let you in on a little secret, Aseer KT is working on an amazing terraform provider that will allow you to deploy your workloads across multiple cloud providers. Keep an eye out on our [Articles Space](https://www.wednesday.is/writing-articles) and we’ll holler once it’s out.

_This was originally published at [https://www.wednesday.is/writing-articles/migrating-ecs-eks-step-by-step-guide](https://www.wednesday.is). Come say Hi :) and let us know if we can help you design and build digital products._