AWS: Fargate — capabilities, comparison with Lambda/EC2 and usage with AWS EKS
Tags: serverless, aws, kubernetes, devops
Published: 2023-05-13 05:51:44

---
title: AWS: Fargate — capabilities, comparison with Lambda/EC2 and usage with AWS EKS
published: true
date: 2023-05-07 03:05:58 UTC
tags: serverless,aws,kubernetes,devops
canonical_url: https://rtfm.co.ua/en/aws-fargate-capabilities-comparison-with-lambda-ec2-and-usage-with-aws-eks/
---

### AWS: Fargate — capabilities, comparison with Lambda/EC2, and usage with AWS EKS

![](https://cdn-images-1.medium.com/max/1024/1*RotXy7XKKz_-U59wSQ57sA.jpeg)

AWS Fargate is another serverless solution from Amazon that takes over infrastructure management, saving the user the need to spend time configuring EC2 instances, operating systems, container management systems, etc.

In general, when I was getting acquainted with Fargate, I found a great video from AWS re:Invent 2022, which very well explained (and showed) the [Shared Responsibility model in AWS](https://aws.amazon.com/compliance/shared-responsibility-model/) — for which parts of the system Amazon is responsible, and for which the user is responsible, so I highly recommend to review — [AWS re:Invent 2022 — A close look at AWS Fargate and AWS App Runner](https://www.youtube.com/watch?v=MZBbhqt6bQs&t=1469s&ab_channel=AWSEvents).

If you look at the diagram from this video, Fargate’s role is well represented there:

![](https://cdn-images-1.medium.com/max/1024/0*qRMinvYgGVGDvz0J.png)

That is, AWS takes care of everything related to the servers and the operating system and its components, while we only have to create and run a container.

At the same time, AWS Fargate can be used together with AWS Elastic Container Service or AWS Elastic Kubernetes Service, and today we will see how to make it works with AWS EKS.

### AWS Fargate vs AWS Lambda

The very first question I had when I started reading about Fargate was why if there is already [AWS Lambda](https://rtfm.co.ua/en/aws-lambda-functions-and-overview-and-integration-with-aws-api-gateway/)? What is the difference? By the way, the video also talks about [AWS App Runner](https://aws.amazon.com/apprunner/) — another serverless service from Amazon, but not about it now (and by the way we also have [Knative](https://rtfm.co.ua/en/knative-serverless-for-kubernetes-an-overview-and-launch-in-minikube/), although it is not about AWS).

#### Functionality

So, conceptually — AWS Fargate is CaaS, i.e. _Container as a Service_, while AWS Lambda is FaaS, i.e. _Function as a Service_: to work with Fargate, you need to have a built docker image (or any other corresponding [Open Container Initiative](https://www.opencontainers.org/) specification), while to work with AWS Lambda you only need the code — Lambda itself will “pack” it into a container and run it.

Also, with Fargate you have to configure the autoscaling of containers, while Lambda does it automatically. In addition, containers in Fargate will not scale to zero when there is no work — for this, you need to stop [Fargate tasks](https://docs.aws.amazon.com/AmazonECS/latest/userguide/what-is-fargate.html) yourself (or simply scale pods to zero in Kubernetes), while Lambda functions will be stopped as soon as events which triggered the launch of these functions will stop coming to them.

#### Cost and payment for the service

At the same time, both services have the same payment model — _pay-as-you-go_, that is, you pay only for the time when your container or function is executed, although there are differences: in Fargate, payment is charged exactly for the CPU/RAM consumed per second, whereas with Labmda you pay for each function call and for the time it runs. See [AWS Fargate Pricing](https://aws.amazon.com/fargate/pricing/).

#### Use Cases

Fargate is well-suited for running long-running tasks where you have more options to customize your work environment and you’ll have fewer restrictions on CPU/RAM, disk storage, and don’t have such significant limits on the size of data you can send/receive.

Lambda, on the other hand, allows you to run code faster (because you don’t need to build a container image), auto-scaling out of the box, monitoring, and is good for short-term tasks.

### AWS Fargate vs AWS EKS EC2 Node Groups

During the launching of the first pods in EKS using Fargate, another question arose — well, what about EC2?

In this comparison, Fargate has the following advantages:

- faster scaling
- can be a more cost-effective solution than EC2
- no need for security patches (although Managed Node Groups apparently also install patches themselves)

Disadvantages of Fargate:

- less control over the infrastructure
- sometimes an EC2 solution can be more cost-effective
- restrictions on CPU/Memory and instance types (for example, it is not possible to use GPU)

With EC2, you have more control over the infrastructure and instance types (GPU, network, etc.), but it requires more engineering work (startup, server maintenance, monitoring), and you pay for the servers regardless of whether they run on them work or not.

In general, you can simply have both Node Groups and Fargate instances for different pods in one EKS cluster.

### Amazon EKS and AWS Fargate

So, in EKS, our Pods have to start on some servers. Usually, NodeGrops ([Managed](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) and/or [Self-managed](https://docs.aws.amazon.com/eks/latest/userguide/worker.html) ) are used for this, which are regular AWS EC2 instances, but instead of virtual machines we can use AWS Fargate, see [Amazon EKS nodes](https://docs.aws.amazon.com/eks/latest/userguide/eks-compute.html).

Which Pods will be launched is configured in [Fargate profiles](https://docs.aws.amazon.com/eks/latest/userguide/fargate-profile.html), which are part of your EKS cluster. EKS itself integrates with Fargate through controllers that are built into the EKS service and work on its [control plane](https://rtfm.co.ua/en/kubernetes-part-1-architecture-and-main-components-overview/#Kubernetes_core_services_aka_Kubernetes_Control_Plane). In addition, Fargate has a separate scheduler for running pods — the fargate-scheduler (as opposed to the default-scheduler, which is responsible for running pods on EC2 instances).

See [AWS Fargate](https://docs.aws.amazon.com/eks/latest/userguide/fargate.html).

#### Fargate in EKS considerations

When planning to use AWS Fargate with your EKS, keep in mind that:

- Network Load Balancers and Application Load Balancers when using Fargate must be with [IP targets](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html#target-group-ip-address-type)
- DaemonSets are not supported
- Privileged containers are not supported
- Pods in Fargate cannot have HostPorteitherHostNetwork
- Pods in Fargate can only be launched in private subnets
- you can use the [Vertical Pod Autoscaler](https://rtfm.co.ua/en/kubernetes-vertical-pod-scaling-with-vertical-pod-autoscaler/) to set resources.requests for Pods on startup, and then scale them using the [Horizontal Pod Autoscaler](https://rtfm.co.ua/en/kubernetes-horizontalpodautoscaler-an-overview-with-examples/)
- [Amazon EC2 instance metadata service](https://rtfm.co.ua/en/aws-security-instance-metadata-service-v1-vs-imds-v2-kubernetes-pod-and-docker-containers/) (IMDS) is not supported in Fargate
- [Amazon VPC CNI plugin for Amazon EKS](https://github.com/aws/amazon-vpc-cni-plugins) is used on Fargate nodes, alternative [CNI plugins](https://docs.aws.amazon.com/eks/latest/userguide/alternate-cni-plugins.html) are not supported
- you can use the [Amazon EBS CSI controller](https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi.html) with Fargate, but you cannot connect additional EBS
- when using [Kubernetes jobs](https://kubernetes.io/docs/concepts/workloads/controllers/job/) with Fargate, it is important to delete these jobs after finishing their work even if Failed, otherwise, they will continue to use Fargate nodes and you will continue to pay
- maximum 16 vCPU and 120 GB RAM (see [AWS Fargate increases compute and memory resource configurations by 4x](https://aws.amazon.com/about-aws/whats-new/2022/09/aws-fargate-increases-compute-memory-resource-configurations-4x/) )
- AWS ECS seems to already have the possibility to use [Fargate Spot instances](https://aws.amazon.com/blogs/aws/aws-fargate-spot-now-generally-available/), but EKS does not have this yet (but there is a [GitHub issue](https://github.com/aws/containers-roadmap/issues/622) with such a request)

See all in [AWS Fargate considerations](https://docs.aws.amazon.com/eks/latest/userguide/fargate.html#fargate-considerations).

#### Creating an EKS cluster

Here we will do everything by “clickops”, and another time will deploy EKS using Terraform or AWS CDK.

#### EKS cluster IAM role

First, we need an AIM role through which the future cluster will communicate with Amazon services, see [Amazon EKS cluster IAM role](https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html#create-service-role).

Go to AIM, click _Create role_, leave “AWS service” in _Trusted entity type_, and select _EKS — Cluster_ from the list in _Use case_:

![](https://cdn-images-1.medium.com/max/820/0*yxEEYfW55Qc1jJ5H.png)

On the _Add permissions page_, leave it as default, and click Next:

Next, set the name of the role and press _Create Role_:

![](https://cdn-images-1.medium.com/max/887/0*FinBl0-9kktx3Jtu.png)

#### Створення VPC та Subnets

I haven’t created networking in AWS Console for a few years… Especially by hand.

Okay, what do we need:

- VPC
- divide it into several subnets in different AvailabilityZones
- part — public, with Internet Gateway — for any AWS LoadBalancer
- some are private, Kubernetes Pods will live here, and we will create a NAT Gateway for them
- and there was also something with SecurityGroups

Let’s go.

And — oh… Here we are) The VPC creation interface has been remastered really cool…

Go to the VPC, create a new network, select _VPC and more_ — I’m just curious how it is, and it looks really convenient — now, we can create everything at once, instead of spending half an hour clicking on the VPC dashboard, and then thinking about where we messed up in the route tables.

So, create a VPC with subnets in each Availability Zone, create public subnets with an Internet Gateway, and private subnets with a NAT Gateway in each Availability Zone (more expensive, but more reliable, if this is done for Production). Also, let’s add a [VPC S3 endpoints](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html) — it is not needed now, but in general, it is a very useful thing in terms of security and cost-effectiveness:

![](https://cdn-images-1.medium.com/max/1024/0*eZy_RaUd-Z_wKW9T.png)

Below check that _DNS hostnames_ and _DNS resolution_ are enabled — this is a requirement for Fargate to work (described in the same [AWS Fargate considerations](https://docs.aws.amazon.com/eks/latest/userguide/fargate.html#fargate-considerations) if you missed it):

![](https://cdn-images-1.medium.com/max/495/0*ZWlHNu6X45FEpywL.png)

Click _Create_.

Well, they did it really cool!

![](https://cdn-images-1.medium.com/max/814/0*0GzqYgor2HVG5JJE.png)

Wait for the creation of the resources:

![](https://cdn-images-1.medium.com/max/1024/0*tyHVFPIycbYlv6kI.png)

#### Creating an EKS cluster

Go to the Elastic Kubernetes Service, click _Add cluster > Create_:

![](https://cdn-images-1.medium.com/max/258/0*gq2aso2B9bs0c0-H.png)

Specify the name of the cluster, select the IAM role create above, and click Next (the _Secrets encryption_ is something new, need to test it somehow):

![](https://cdn-images-1.medium.com/max/1024/0*uYA6JKb3FZ53mpYe.png)

Next, select our VPS — the subnets will be pulled up automatically, select a SecurityGroup below, for now, you can default from our VPC:

![](https://cdn-images-1.medium.com/max/1024/0*Vb_E9Z09-VO-k9LY.png)

Next, configure access to our cluster.

#### AWS Fargate Pod hangs in Pending status

Looking ahead a little — about a possible problem. After setting up almost everything, the test Pod hung in the Pedning state with the following messages:

```
$ kubectl describe pod nginx
Name: nginx
Namespace: default
…
Labels: eks.amazonaws.com/fargate-profile=fargate-test-eu-central-1-profile
…
Status: Pending
…
Warning LoggingDisabled 11m fargate-scheduler Disabled logging because aws-logging configmap was not found. configmap “aws-logging” not found
Warning FailedScheduling 8m48s fargate-scheduler Pod provisioning timed out (will retry) for pod: default/nginx
```

I have checked everything, and googled it — Fargate profiles, SecurityGroup EKS, NAT Gateways in subnets — everything was correct everywhere. WTF?

It turned out that when I wrote a little below in this post:

> Actually, of course, it is desirable to disable public access to the API, and go through a private endpoint, for example, through a VPN. But for now, let’s fill in both varviants — public and private, just will set a limit on IP.

I ran further and did not switch access to the cluster from the default value _of Public_ to _Public and private_, which led to the problem.

And this is stated in the first lines of the documentation, and I even quoted it when describing the process:

> Without the private endpoint enabled, the CIDR blocks that you specify for public access must include the outbound sources from your VPC.

But I just missed switching the option and spent half an hour trying to understand whose problem it was.

Okay, let’s move on — this time it’s right.

So, of course, it is desirable to disable public access to the Kubernetes API and go through a private endpoint, for example, through a VPN. But for now, let’s include both options — public and private, just will set a limit on my home IP.

Find the IP:

```
$ curl ifconfig.me
217. ***.***.253
```

Add it to the allowed:

![](https://cdn-images-1.medium.com/max/854/0*itXfH1kodOztbzH2.png)

Next, configure logging. Here, too, it is desirable to include if not all, then at least the logs of the API server, Audit or Authenticator, and Scheduler:

![](https://cdn-images-1.medium.com/max/1024/0*fkBiAm6lTNY3vz0i.png)

In the next step, choose Addons, and can enable [GuardDuty](https://aws.amazon.com/guardduty/):

![](https://cdn-images-1.medium.com/max/1024/0*TvcQyvg9YVKCzVLs.png)

On the next page, I don’t see what can be changed — leave it, and click _Next_:

![](https://cdn-images-1.medium.com/max/1024/0*PX_Fj1iNbCRdWmBX.png)

And finally, the last thing is a review, and create a cluster:

![](https://cdn-images-1.medium.com/max/922/0*SKGphMfCaSCPku1A.png)

It used to take 15–20 minutes, but there was news that Amazon has sped up the process.

#### kubectl configuration

To check that everything is fine with the cluster and for further testing, add a next context to your `~/.kube/config`.

Run AWS CLI with the update-kubeconfig command:

```
$ aws --profile setevoy eks update-kubeconfig — name fargate-test-eu-central-1-cluster — alias setevoy-fargate-test-eu-central-1-cluster
Added new context setevoy-fargate-test-eu-central-1-cluster to /home/setevoy/.kube/config
```

Check:

```
$ kubectl get pod — all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-cbbbbb9cb-2hhx2 0/1 Pending 0 18h
kube-system coredns-cbbbbb9cb-4xf2w 0/1 Pending 0 18h
```

So far, we have only two pods with CoreDNS, currently in the Pending state because neither Worker Nodes nor Fargate profiles are available.

Let’s go to Fargate.

### Running AWS Fargate

Now that we have an EKS cluster, it’s time to connect Fargate.

See the documentation on the [Getting started with AWS Fargate using Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/fargate-getting-started.html), and the first thing we pay attention to is Security Groups on Worker Nodes:

> If you restrict access to the public endpoint of your cluster using CIDR blocks, we recommend that you also enable private endpoint access. This way, Fargate pods can communicate with the cluster. Without the private endpoint enabled, the CIDR blocks that you specify for public access must include the outbound sources from your VPC.

In our case, there are no Node Groups, so we can continue.

#### EKS Pod execution IAM Role

First, we need to add a role to the AIM that will allow Kubernetes Pods on Fargate to communicate with Amazon, see [Amazon EKS pod execution IAM role](https://docs.aws.amazon.com/eks/latest/userguide/pod-execution-role.html).

Go to _AIM > Roles > Create role_.

This time we choose _EKS — Fragate pod_ :

![](https://cdn-images-1.medium.com/max/926/0*LI5OnWFvDxhF4pc5.png)

Click _Next_, on the _Add permissions page_ leave it as it is (you can check the permissions here), go further, enter a name, and click _Create role_:

![](https://cdn-images-1.medium.com/max/1024/0*CF1uhOiSYQ7z_4Bl.png)

After creation, find the Role, go to the _Trust relationships_ tab, and edit the role:

![](https://cdn-images-1.medium.com/max/1024/0*eScFfgeoItV2XmAy.jpg)

Describe a new access policy for this role:

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Condition": {
         "ArnLike": {
            "aws:SourceArn": "arn:aws:eks:eu-central-1:26***286:fargateprofile/fargate-test-eu-central-1-cluster/*"
         }
      },
      "Principal": {
        "Service": "eks-fargate-pods.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
```

Click on the _Update policy_, and proceed to the creation of a Fargate profile.

#### Creating Fargate profile

The Fargate profile describes exactly which Pods from the Kubernetes cluster will run in Fargate, see [AWS Fargate profile](https://docs.aws.amazon.com/eks/latest/userguide/fargate-profile.html).

To create a profile, go to the EKS, choose your cluster, and on the _Compute_ tab at the bottom find the _Add Fargate profile_:

![](https://cdn-images-1.medium.com/max/1024/0*kEsu5PDfHxTxetsO.png)

Set a name, the AIM role will be automatically substituted as well as the private subnets of our VPC:

![](https://cdn-images-1.medium.com/max/1024/0*E5cmjINRyjzFOA43.png)

Pod selectors describe which Pods from which namespaces will be launched with this Fargate profile.

Now for the test let there be all of them, but in general, it is possible to create different profiles for different types of Pods with a selection by namespaces and/or labels that are added to the pods:

![](https://cdn-images-1.medium.com/max/1024/0*BbqiMcIbTAfua08o.png)

Next, check that everything is correct and create a profile:

![](https://cdn-images-1.medium.com/max/814/0*C2xs92Pe9kS0I7jK.png)

Creation took 5 minutes.

#### Running Kubernetes Pods at Fargate

So, for now, we only have two Pods with the CoreDNS service, which are in Pending status.

In order for CoreDNS Pods to start in Fargate, edit their Deployment and remove the annotation `eks.amazonaws.com/compute-type: ec2`:

![](https://cdn-images-1.medium.com/max/409/0*yg6H8jF-QKqevhjW.png)

And check in a minute or two:

```
$ kubectl -n kube-system get pod
NAME READY STATUS RESTARTS AGE
coredns-75694977b5-m7smc 0/1 Pending 0 5m48s
coredns-75694977b5-tgwdl 1/1 Running 0 50s
```

The first one is running, good.

And after launching the first Pods, on the Compute tab of our cluster, we should see the Fargate nodes in the EKS cluster:

![](https://cdn-images-1.medium.com/max/1024/0*YrCE2fit2KoLAMiu.png)

While the second Pod with CoreDNS starts, let’s add a regular test Pod with Nginx to see how it works again:

```
$ kubectl run nginx --image=nginx
pod/nginx created
```

And in a couple of minutes check the Pods:

```
$ kubectl get pod --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default nginx 1/1 Running 0 2m27s
kube-system coredns-75694977b5-r8d6b 1/1 Running 0 46s
kube-system coredns-75694977b5-tgwdl 1/1 Running 0 5m52s
```

And Fargate nodes:

![](https://cdn-images-1.medium.com/max/1024/0*u5OMcnniuvXsCurO.png)

Regarding the event “ **_Disabled logging because aws-logging configmap was not found_** ” — you can configure logging separately, see [Fargate logging](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html), but now this does not affect Pods startup.

Another interesting thing to know is that the IAM role for our pods is added to `aws-auth` the cluster's ConfigMap:

```
$ kubectl -n kube-system get cm aws-auth -o yaml
apiVersion: v1
data:
mapRoles: |
- groups:
- system:bootstrappers
- system:nodes
- system:node-proxier
rolearn: arn:aws:iam::264***286:role/AmazonEKSFargatePodExecutionRole-fargate-test-eu-central-1
username: system:node:{{SessionName}}
```

That seems to be all for now, and you can try to use it instead of regular EC2.

### Useful links

- [AWS re:Invent 2022 — A close look at AWS Fargate and AWS App Runner](https://www.youtube.com/watch?v=MZBbhqt6bQs&t=1469s&ab_channel=AWSEvents)
- [Introduction to AWS Fargate](https://www.geeksforgeeks.org/introduction-to-aws-fargate/)
- [How can I troubleshoot Amazon EKS pods on AWS Fargate that are stuck in a Pending state?](https://repost.aws/knowledge-center/eks-resolve-pending-fargate-pods)
- [Theoretical cost optimization by Amazon ECS launch type: Fargate vs EC2](https://aws.amazon.com/blogs/containers/theoretical-cost-optimization-by-amazon-ecs-launch-type-fargate-vs-ec2/)
- [EKS Fargate: Features and Best Practices](https://www.densify.com/eks-best-practices/eks-fargate/)

_Originally published at_ [_RTFM: Linux, DevOps, and system administration_](https://rtfm.co.ua/en/aws-fargate-capabilities-comparison-with-lambda-ec2-and-usage-with-aws-eks/)_._

* * *