Package: aws-cdk.aws-eks
Amazon EKS Construct Library---AWS CDK v1 has reached End-of-Support on 2023-06-01.
This package is no longer being updated, and users should migrate to AWS CDK v2.For more information on how to migrate, see theMigrating to AWS CDK v2guide.This construct library allows you to defineAmazon Elastic Container Service for Kubernetes (EKS)clusters.
In addition, the library also supports defining Kubernetes resource manifests within EKS clusters.Table Of ContentsQuick StartAPI ReferenceArchitectural OverviewProvisioning clustersManaged node groupsFargate ProfilesSelf-managed nodesEndpoint AccessALB ControllerVPC SupportKubectl SupportARM64 SupportMasters RoleEncryptionPermissions and SecurityApplying Kubernetes ResourcesKubernetes ManifestsHelm ChartsCDK8s ChartsPatching Kubernetes ResourcesQuerying Kubernetes ResourcesUsing existing clustersKnown Issues and LimitationsQuick StartThis example defines an Amazon EKS cluster with the following configuration:Dedicated VPC with default configuration (Implicitly created usingec2.Vpc)A Kubernetes pod with a container based on thepaulbouwer/hello-kubernetesimage.# provisiong a clustercluster=eks.Cluster(self,"hello-eks",version=eks.KubernetesVersion.V1_21)# apply a kubernetes manifest to the clustercluster.add_manifest("mypod",{"api_version":"v1","kind":"Pod","metadata":{"name":"mypod"},"spec":{"containers":[{"name":"hello","image":"paulbouwer/hello-kubernetes:1.5","ports":[{"container_port":8080}]}]}})In order to interact with your cluster throughkubectl, you can use theaws eks update-kubeconfigAWS CLI commandto configure your local kubeconfig. The EKS module will define a CloudFormation output in your stack which contains the command to run. For example:Outputs:
ClusterConfigCommand43AAE40F = aws eks update-kubeconfig --name cluster-xxxxx --role-arn arn:aws:iam::112233445566:role/yyyyyExecute theaws eks update-kubeconfig ...command in your terminal to create or update a local kubeconfig context:$awseksupdate-kubeconfig--namecluster-xxxxx--role-arnarn:aws:iam::112233445566:role/yyyyyAdded new context arn:aws:eks:rrrrr:112233445566:cluster/cluster-xxxxx to /home/boom/.kube/configAnd now you can simply usekubectl:$kubectlgetall-nkube-systemNAME                           READY   STATUS    RESTARTS   AGEpod/aws-node-fpmwv             1/1     Running   0          21mpod/aws-node-m9htf             1/1     Running   0          21mpod/coredns-5cb4fb54c7-q222j   1/1     Running   0          23mpod/coredns-5cb4fb54c7-v9nxx   1/1     Running   0          23m...Architectural OverviewThe following is a qualitative diagram of the various possible components involved in the cluster deployment.+-----------------------------------------------+               +-----------------+
 |                 EKS Cluster                   |    kubectl    |                 |
 |-----------------------------------------------|<-------------+| Kubectl Handler |
 |                                               |               |                 |
 |                                               |               +-----------------+
 | +--------------------+    +-----------------+ |
 | |                    |    |                 | |
 | | Managed Node Group |    | Fargate Profile | |               +-----------------+
 | |                    |    |                 | |               |                 |
 | +--------------------+    +-----------------+ |               | Cluster Handler |
 |                                               |               |                 |
 +-----------------------------------------------+               +-----------------+
    ^                                   ^                          +
    |                                   |                          |
    | connect self managed capacity     |                          | aws-sdk
    |                                   | create/update/delete     |
    +                                   |                          v
 +--------------------+                 +              +-------------------+
 |                    |                 --------------+| eks.amazonaws.com |
 | Auto Scaling Group |                                +-------------------+
 |                    |
 +--------------------+In a nutshell:EKS Cluster- The cluster endpoint created by EKS.Managed Node Group- EC2 worker nodes managed by EKS.Fargate Profile- Fargate worker nodes managed by EKS.Auto Scaling Group- EC2 worker nodes managed by the user.KubectlHandler- Lambda function for invokingkubectlcommands on the cluster - created by CDK.ClusterHandler- Lambda function for interacting with EKS API to manage the cluster lifecycle - created by CDK.A more detailed breakdown of each is provided further down this README.Provisioning clustersCreating a new cluster is done using theClusterorFargateClusterconstructs. The only required property is the kubernetesversion.eks.Cluster(self,"HelloEKS",version=eks.KubernetesVersion.V1_21)You can also useFargateClusterto provision a cluster that uses only fargate workers.eks.FargateCluster(self,"HelloEKS",version=eks.KubernetesVersion.V1_21)NOTE: Only 1 cluster per stack is supported.If you have a use-case for multiple clusters per stack, or would like to understand more about this limitation, seehttps://github.com/aws/aws-cdk/issues/10073.Below you'll find a few important cluster configuration options. First of which is Capacity.
Capacity is the amount and the type of worker nodes that are available to the cluster for deploying resources. Amazon EKS offers 3 ways of configuring capacity, which you can combine as you like:Managed node groupsAmazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters.
With Amazon EKS managed node groups, you don’t need to separately provision or register the Amazon EC2 instances that provide compute capacity to run your Kubernetes applications. You can create, update, or terminate nodes for your cluster with a single operation. Nodes run using the latest Amazon EKS optimized AMIs in your AWS account while node updates and terminations gracefully drain nodes to ensure that your applications stay available.For more details visitAmazon EKS Managed Node Groups.Managed Node Groups are the recommended way to allocate cluster capacity.By default, this library will allocate a managed node group with 2m5.largeinstances (this instance type suits most common use-cases, and is good value for money).At cluster instantiation time, you can customize the number of instances and their type:eks.Cluster(self,"HelloEKS",version=eks.KubernetesVersion.V1_21,default_capacity=5,default_capacity_instance=ec2.InstanceType.of(ec2.InstanceClass.M5,ec2.InstanceSize.SMALL))To access the node group that was created on your behalf, you can usecluster.defaultNodegroup.Additional customizations are available post instantiation. To apply them, set the default capacity to 0, and use thecluster.addNodegroupCapacitymethod:cluster=eks.Cluster(self,"HelloEKS",version=eks.KubernetesVersion.V1_21,default_capacity=0)cluster.add_nodegroup_capacity("custom-node-group",instance_types=[ec2.InstanceType("m5.large")],min_size=4,disk_size=100,ami_type=eks.NodegroupAmiType.AL2_X86_64_GPU)To set node taints, you can settaintsoption.# cluster: eks.Clustercluster.add_nodegroup_capacity("custom-node-group",instance_types=[ec2.InstanceType("m5.large")],taints=[eks.TaintSpec(effect=eks.TaintEffect.NO_SCHEDULE,key="foo",value="bar")])Spot Instances SupportUsecapacityTypeto create managed node groups comprised of spot instances. To maximize the availability of your applications while using
Spot Instances, we recommend that you configure a Spot managed node group to use multiple instance types with theinstanceTypesproperty.For more details visitManaged node group capacity types.# cluster: eks.Clustercluster.add_nodegroup_capacity("extra-ng-spot",instance_types=[ec2.InstanceType("c5.large"),ec2.InstanceType("c5a.large"),ec2.InstanceType("c5d.large")],min_size=3,capacity_type=eks.CapacityType.SPOT)Launch Template SupportYou can specify a launch template that the node group will use. For example, this can be useful if you want to use
a custom AMI or add custom user data.When supplying a custom user data script, it must be encoded in the MIME multi-part archive format, since Amazon EKS merges with its own user data. Visit theLaunch Template Docsfor mode details.# cluster: eks.Clusteruser_data="""MIME-Version: 1.0Content-Type: multipart/mixed; boundary="==MYBOUNDARY=="--==MYBOUNDARY==Content-Type: text/x-shellscript; charset="us-ascii"#!/bin/bashecho "Running custom user data script"--==MYBOUNDARY==--\\"""lt=ec2.CfnLaunchTemplate(self,"LaunchTemplate",launch_template_data=ec2.CfnLaunchTemplate.LaunchTemplateDataProperty(instance_type="t3.small",user_data=Fn.base64(user_data)))cluster.add_nodegroup_capacity("extra-ng",launch_template_spec=eks.LaunchTemplateSpec(id=lt.ref,version=lt.attr_latest_version_number))Note that when using a custom AMI, Amazon EKS doesn't merge any user data. Which means you do not need the multi-part encoding. and are responsible for supplying the required bootstrap commands for nodes to join the cluster.
In the following example,/ect/eks/bootstrap.shfrom the AMI will be used to bootstrap the node.# cluster: eks.Clusteruser_data=ec2.UserData.for_linux()user_data.add_commands("set -o xtrace",f"/etc/eks/bootstrap.sh{cluster.clusterName}")lt=ec2.CfnLaunchTemplate(self,"LaunchTemplate",launch_template_data=ec2.CfnLaunchTemplate.LaunchTemplateDataProperty(image_id="some-ami-id",# custom AMIinstance_type="t3.small",user_data=Fn.base64(user_data.render())))cluster.add_nodegroup_capacity("extra-ng",launch_template_spec=eks.LaunchTemplateSpec(id=lt.ref,version=lt.attr_latest_version_number))You may specify oneinstanceTypein the launch template or multipleinstanceTypesin the node group,but not both.For more details visitLaunch Template Support.Graviton 2 instance types are supported includingc6g,m6g,r6gandt4g.Fargate profilesAWS Fargate is a technology that provides on-demand, right-sized compute
capacity for containers. With AWS Fargate, you no longer have to provision,
configure, or scale groups of virtual machines to run containers. This removes
the need to choose server types, decide when to scale your node groups, or
optimize cluster packing.You can control which pods start on Fargate and how they run with Fargate
Profiles, which are defined as part of your Amazon EKS cluster.SeeFargate Considerationsin the AWS EKS User Guide.You can add Fargate Profiles to any EKS cluster defined in your CDK app
through theaddFargateProfile()method. The following example adds a profile
that will match all pods from the "default" namespace:# cluster: eks.Clustercluster.add_fargate_profile("MyProfile",selectors=[eks.Selector(namespace="default")])You can also directly use theFargateProfileconstruct to create profiles under different scopes:# cluster: eks.Clustereks.FargateProfile(self,"MyProfile",cluster=cluster,selectors=[eks.Selector(namespace="default")])To create an EKS cluster thatonlyuses Fargate capacity, you can useFargateCluster.
The following code defines an Amazon EKS cluster with a default Fargate Profile that matches all pods from the "kube-system" and "default" namespaces. It is also configured torun CoreDNS on Fargate.cluster=eks.FargateCluster(self,"MyCluster",version=eks.KubernetesVersion.V1_21)FargateClusterwill create a defaultFargateProfilewhich can be accessed via the cluster'sdefaultProfileproperty. The created profile can also be customized by passing options as withaddFargateProfile.NOTE: Classic Load Balancers and Network Load Balancers are not supported on
pods running on Fargate. For ingress, we recommend that you use theALB Ingress
Controlleron Amazon EKS (minimum version v1.1.4).Self-managed nodesAnother way of allocating capacity to an EKS cluster is by using self-managed nodes.
EC2 instances that are part of the auto-scaling group will serve as worker nodes for the cluster.
This type of capacity is also commonly referred to asEC2 Capacity* orEC2 Nodes.For a detailed overview please visitSelf Managed Nodes.Creating an auto-scaling group and connecting it to the cluster is done using thecluster.addAutoScalingGroupCapacitymethod:# cluster: eks.Clustercluster.add_auto_scaling_group_capacity("frontend-nodes",instance_type=ec2.InstanceType("t2.medium"),min_capacity=3,vpc_subnets=ec2.SubnetSelection(subnet_type=ec2.SubnetType.PUBLIC))To connect an already initialized auto-scaling group, use thecluster.connectAutoScalingGroupCapacity()method:# cluster: eks.Cluster# asg: autoscaling.AutoScalingGroupcluster.connect_auto_scaling_group_capacity(asg)To connect a self-managed node group to an imported cluster, use thecluster.connectAutoScalingGroupCapacity()method:# cluster: eks.Cluster# asg: autoscaling.AutoScalingGroupimported_cluster=eks.Cluster.from_cluster_attributes(self,"ImportedCluster",cluster_name=cluster.cluster_name,cluster_security_group_id=cluster.cluster_security_group_id)imported_cluster.connect_auto_scaling_group_capacity(asg)In both cases, thecluster security groupwill be automatically attached to
the auto-scaling group, allowing for traffic to flow freely between managed and self-managed nodes.Note:The defaultupdateTypefor auto-scaling groups does not replace existing nodes. Since security groups are determined at launch time, self-managed nodes that were provisioned with version1.78.0or lower, will not be updated.
To apply the new configuration on all your self-managed nodes, you'll need to replace the nodes using theUpdateType.REPLACING_UPDATEpolicy for theupdateTypeproperty.You can customize the/etc/eks/boostrap.shscript, which is responsible
for bootstrapping the node to the EKS cluster. For example, you can usekubeletExtraArgsto add custom node labels or taints.# cluster: eks.Clustercluster.add_auto_scaling_group_capacity("spot",instance_type=ec2.InstanceType("t3.large"),min_capacity=2,bootstrap_options=eks.BootstrapOptions(kubelet_extra_args="--node-labels foo=bar,goo=far",aws_api_retry_attempts=5))To disable bootstrapping altogether (i.e. to fully customize user-data), setbootstrapEnabledtofalse.
You can also configure the cluster to use an auto-scaling group as the default capacity:cluster=eks.Cluster(self,"HelloEKS",version=eks.KubernetesVersion.V1_21,default_capacity_type=eks.DefaultCapacityType.EC2)This will allocate an auto-scaling group with 2m5.largeinstances (this instance type suits most common use-cases, and is good value for money).
To access theAutoScalingGroupthat was created on your behalf, you can usecluster.defaultCapacity.
You can also independently create anAutoScalingGroupand connect it to the cluster using thecluster.connectAutoScalingGroupCapacitymethod:# cluster: eks.Cluster# asg: autoscaling.AutoScalingGroupcluster.connect_auto_scaling_group_capacity(asg)This will add the necessary user-data to access the apiserver and configure all connections, roles, and tags needed for the instances in the auto-scaling group to properly join the cluster.Spot InstancesWhen using self-managed nodes, you can configure the capacity to use spot instances, greatly reducing capacity cost.
To enable spot capacity, use thespotPriceproperty:# cluster: eks.Clustercluster.add_auto_scaling_group_capacity("spot",spot_price="0.1094",instance_type=ec2.InstanceType("t3.large"),max_capacity=10)Spot instance nodes will be labeled withlifecycle=Ec2Spotand tainted withPreferNoSchedule.TheAWS Node Termination HandlerDaemonSetwill be
installed fromAmazon EKS Helm chart repositoryon these nodes.
The termination handler ensures that the Kubernetes control plane responds appropriately to events that
can cause your EC2 instance to become unavailable, such asEC2 maintenance eventsandEC2 Spot interruptionsand helps gracefully stop all pods running on spot nodes that are about to be
terminated.Handler Version:1.7.0Chart Version:0.9.5To disable the installation of the termination handler, set thespotInterruptHandlerproperty tofalse. This applies both toaddAutoScalingGroupCapacityandconnectAutoScalingGroupCapacity.BottlerocketBottlerocketis a Linux-based open-source operating system that is purpose-built by Amazon Web Services for running containers on virtual machines or bare metal hosts.Bottlerocketis supported when using managed nodegroups or self-managed auto-scaling groups.To create a Bottlerocket managed nodegroup:# cluster: eks.Clustercluster.add_nodegroup_capacity("BottlerocketNG",ami_type=eks.NodegroupAmiType.BOTTLEROCKET_X86_64)The following example will create an auto-scaling group of 2t3.smallLinux instances running with theBottlerocketAMI.# cluster: eks.Clustercluster.add_auto_scaling_group_capacity("BottlerocketNodes",instance_type=ec2.InstanceType("t3.small"),min_capacity=2,machine_image_type=eks.MachineImageType.BOTTLEROCKET)The specific Bottlerocket AMI variant will be auto selected according to the k8s version for thex86_64architecture.
For example, if the Amazon EKS cluster version is1.17, the Bottlerocket AMI variant will be auto selected asaws-k8s-1.17behind the scene.SeeVariantsfor more details.Please note Bottlerocket does not allow to customize bootstrap options andbootstrapOptionsproperties is not supported when you create theBottlerocketcapacity.For more details about Bottlerocket, seeBottlerocket FAQsandBottlerocket Open Source Blog.Endpoint AccessWhen you create a new cluster, Amazon EKS creates an endpoint for the managed Kubernetes API server that you use to communicate with your cluster (using Kubernetes management tools such askubectl)By default, this API server endpoint is public to the internet, and access to the API server is secured using a combination of
AWS Identity and Access Management (IAM) and native KubernetesRole Based Access Control(RBAC).You can configure thecluster endpoint accessby using theendpointAccessproperty:cluster=eks.Cluster(self,"hello-eks",version=eks.KubernetesVersion.V1_21,endpoint_access=eks.EndpointAccess.PRIVATE)The default value iseks.EndpointAccess.PUBLIC_AND_PRIVATE. Which means the cluster endpoint is accessible from outside of your VPC, but worker node traffic andkubectlcommands issued by this library stay within your VPC.Alb ControllerSome Kubernetes resources are commonly implemented on AWS with the help of theALB Controller.From the docs:AWS Load Balancer Controller is a controller to help manage Elastic Load Balancers for a Kubernetes cluster.It satisfies Kubernetes Ingress resources by provisioning Application Load Balancers.It satisfies Kubernetes Service resources by provisioning Network Load Balancers.To deploy the controller on your EKS cluster, configure thealbControllerproperty:eks.Cluster(self,"HelloEKS",version=eks.KubernetesVersion.V1_21,alb_controller=eks.AlbControllerOptions(version=eks.AlbControllerVersion.V2_4_1))Querying the controller pods should look something like this:❯ kubectl get pods -n kube-systemNAME                                            READY   STATUS    RESTARTS   AGEaws-load-balancer-controller-76bd6c7586-d929p   1/1     Running   0          109maws-load-balancer-controller-76bd6c7586-fqxph   1/1     Running   0          109m......Every Kubernetes manifest that utilizes the ALB Controller is effectively dependant on the controller.
If the controller is deleted before the manifest, it might result in dangling ELB/ALB resources.
Currently, the EKS construct library does not detect such dependencies, and they should be done explicitly.For example:# cluster: eks.Clustermanifest=cluster.add_manifest("manifest",{})ifcluster.alb_controller:manifest.node.add_dependency(cluster.alb_controller)VPC SupportYou can specify the VPC of the cluster using thevpcandvpcSubnetsproperties:# vpc: ec2.Vpceks.Cluster(self,"HelloEKS",version=eks.KubernetesVersion.V1_21,vpc=vpc,vpc_subnets=[ec2.SubnetSelection(subnet_type=ec2.SubnetType.PRIVATE_WITH_NAT)])Note: Isolated VPCs (i.e with no internet access) are not currently supported. Seehttps://github.com/aws/aws-cdk/issues/12171If you do not specify a VPC, one will be created on your behalf, which you can then access viacluster.vpc. The cluster VPC will be associated to any EKS managed capacity (i.e Managed Node Groups and Fargate Profiles).Please note that thevpcSubnetsproperty defines the subnets where EKS will place thecontrol planeENIs. To choose
the subnets where EKS will place the worker nodes, please refer to theProvisioning clusterssection above.If you allocate self managed capacity, you can specify which subnets should the auto-scaling group use:# vpc: ec2.Vpc# cluster: eks.Clustercluster.add_auto_scaling_group_capacity("nodes",vpc_subnets=ec2.SubnetSelection(subnets=vpc.private_subnets),instance_type=ec2.InstanceType("t2.medium"))There are two additional components you might want to provision within the VPC.Kubectl HandlerTheKubectlHandleris a Lambda function responsible to issuingkubectlandhelmcommands against the cluster when you add resource manifests to the cluster.The handler association to the VPC is derived from theendpointAccessconfiguration. The rule of thumb is:If the cluster VPC can be associated, it will be.Breaking this down, it means that if the endpoint exposes private access (viaEndpointAccess.PRIVATEorEndpointAccess.PUBLIC_AND_PRIVATE), and the VPC containsprivatesubnets, the Lambda function will be provisioned inside the VPC and use the private subnets to interact with the cluster. This is the common use-case.If the endpoint does not expose private access (viaEndpointAccess.PUBLIC)orthe VPC does not contain private subnets, the function will not be provisioned within the VPC.If your use-case requires control over the IAM role that the KubeCtl Handler assumes, a custom role can be passed through the ClusterProps (askubectlLambdaRole) of the EKS Cluster construct.Cluster HandlerTheClusterHandleris a set of Lambda functions (onEventHandler,isCompleteHandler) responsible for interacting with the EKS API in order to control the cluster lifecycle. To provision these functions inside the VPC, set theplaceClusterHandlerInVpcproperty totrue. This will place the functions inside the private subnets of the VPC based on the selection strategy specified in thevpcSubnetsproperty.You can configure the environment of the Cluster Handler functions by specifying it at cluster instantiation. For example, this can be useful in order to configure an http proxy:# proxy_instance_security_group: ec2.SecurityGroupcluster=eks.Cluster(self,"hello-eks",version=eks.KubernetesVersion.V1_21,cluster_handler_environment={"https_proxy":"http://proxy.myproxy.com"},## If the proxy is not open publicly, you can pass a security group to the# Cluster Handler Lambdas so that it can reach the proxy.#cluster_handler_security_group=proxy_instance_security_group)Kubectl SupportThe resources are created in the cluster by runningkubectl applyfrom a python lambda function.By default, CDK will create a new python lambda function to apply your k8s manifests. If you want to use an existing kubectl provider function, for example with tight trusted entities on your IAM Roles - you can import the existing provider and then use the imported provider when importing the cluster:handler_role=iam.Role.from_role_arn(self,"HandlerRole","arn:aws:iam::123456789012:role/lambda-role")kubectl_provider=eks.KubectlProvider.from_kubectl_provider_attributes(self,"KubectlProvider",function_arn="arn:aws:lambda:us-east-2:123456789012:function:my-function:1",kubectl_role_arn="arn:aws:iam::123456789012:role/kubectl-role",handler_role=handler_role)cluster=eks.Cluster.from_cluster_attributes(self,"Cluster",cluster_name="cluster",kubectl_provider=kubectl_provider)EnvironmentYou can configure the environment of this function by specifying it at cluster instantiation. For example, this can be useful in order to configure an http proxy:cluster=eks.Cluster(self,"hello-eks",version=eks.KubernetesVersion.V1_21,kubectl_environment={"http_proxy":"http://proxy.myproxy.com"})RuntimeThe kubectl handler useskubectl,helmand theawsCLI in order to
interact with the cluster. These are bundled into AWS Lambda layers included in
the@aws-cdk/lambda-layer-awscliand@aws-cdk/lambda-layer-kubectlmodules.You can specify a customlambda.LayerVersionif you wish to use a different
version of these tools. The handler expects the layer to include the following
three executables:helm/helm
kubectl/kubectl
awscli/awsSee more information in theDockerfilefor @aws-cdk/lambda-layer-awscli
and theDockerfilefor @aws-cdk/lambda-layer-kubectl.layer=lambda_.LayerVersion(self,"KubectlLayer",code=lambda_.Code.from_asset("layer.zip"))Now specify when the cluster is defined:# layer: lambda.LayerVersion# vpc: ec2.Vpccluster1=eks.Cluster(self,"MyCluster",kubectl_layer=layer,vpc=vpc,cluster_name="cluster-name",version=eks.KubernetesVersion.V1_21)# orcluster2=eks.Cluster.from_cluster_attributes(self,"MyCluster",kubectl_layer=layer,vpc=vpc,cluster_name="cluster-name")MemoryBy default, the kubectl provider is configured with 1024MiB of memory. You can use thekubectlMemoryoption to specify the memory size for the AWS Lambda function:# or# vpc: ec2.Vpceks.Cluster(self,"MyCluster",kubectl_memory=Size.gibibytes(4),version=eks.KubernetesVersion.V1_21)eks.Cluster.from_cluster_attributes(self,"MyCluster",kubectl_memory=Size.gibibytes(4),vpc=vpc,cluster_name="cluster-name")ARM64 SupportInstance types withARM64architecture are supported in both managed nodegroup and self-managed capacity. Simply specify an ARM64instanceType(such asm6g.medium), and the latest
Amazon Linux 2 AMI for ARM64 will be automatically selected.# cluster: eks.Cluster# add a managed ARM64 nodegroupcluster.add_nodegroup_capacity("extra-ng-arm",instance_types=[ec2.InstanceType("m6g.medium")],min_size=2)# add a self-managed ARM64 nodegroupcluster.add_auto_scaling_group_capacity("self-ng-arm",instance_type=ec2.InstanceType("m6g.medium"),min_capacity=2)Masters RoleWhen you create a cluster, you can specify amastersRole. TheClusterconstruct will associate this role with thesystem:mastersRBACgroup, giving it super-user access to the cluster.# role: iam.Roleeks.Cluster(self,"HelloEKS",version=eks.KubernetesVersion.V1_21,masters_role=role)If you do not specify it, a default role will be created on your behalf, that can be assumed by anyone in the account withsts:AssumeRolepermissions for this role.This is the role you see as part of the stack outputs mentioned in theQuick Start.$awseksupdate-kubeconfig--namecluster-xxxxx--role-arnarn:aws:iam::112233445566:role/yyyyyAdded new context arn:aws:eks:rrrrr:112233445566:cluster/cluster-xxxxx to /home/boom/.kube/configEncryptionWhen you create an Amazon EKS cluster, envelope encryption of Kubernetes secrets using the AWS Key Management Service (AWS KMS) can be enabled.
The documentation oncreating a clustercan provide more details about the customer master key (CMK) that can be used for the encryption.You can use thesecretsEncryptionKeyto configure which key the cluster will use to encrypt Kubernetes secrets. By default, an AWS Managed key will be used.This setting can only be specified when the cluster is created and cannot be updated.secrets_key=kms.Key(self,"SecretsKey")cluster=eks.Cluster(self,"MyCluster",secrets_encryption_key=secrets_key,version=eks.KubernetesVersion.V1_21)You can also use a similar configuration for running a cluster built using the FargateCluster construct.secrets_key=kms.Key(self,"SecretsKey")cluster=eks.FargateCluster(self,"MyFargateCluster",secrets_encryption_key=secrets_key,version=eks.KubernetesVersion.V1_21)The Amazon Resource Name (ARN) for that CMK can be retrieved.# cluster: eks.Clustercluster_encryption_config_key_arn=cluster.cluster_encryption_config_key_arnPermissions and SecurityAmazon EKS provides several mechanism of securing the cluster and granting permissions to specific IAM users and roles.AWS IAM MappingAs described in theAmazon EKS User Guide, you can map AWS IAM users and roles toKubernetes Role-based access control (RBAC).The Amazon EKS construct manages theaws-authConfigMapKubernetes resource on your behalf and exposes an API through thecluster.awsAuthfor mapping
users, roles and accounts.Furthermore, when auto-scaling group capacity is added to the cluster, the IAM instance role of the auto-scaling group will be automatically mapped to RBAC so nodes can connect to the cluster. No manual mapping is required.For example, let's say you want to grant an IAM user administrative privileges on your cluster:# cluster: eks.Clusteradmin_user=iam.User(self,"Admin")cluster.aws_auth.add_user_mapping(admin_user,groups=["system:masters"])A convenience method for mapping a role to thesystem:mastersgroup is also available:# cluster: eks.Cluster# role: iam.Rolecluster.aws_auth.add_masters_role(role)Cluster Security GroupWhen you create an Amazon EKS cluster, acluster security groupis automatically created as well. This security group is designed to allow all traffic from the control plane and managed node groups to flow freely
between each other.The ID for that security group can be retrieved after creating the cluster.# cluster: eks.Clustercluster_security_group_id=cluster.cluster_security_group_idNode SSH AccessIf you want to be able to SSH into your worker nodes, you must already have an SSH key in the region you're connecting to and pass it when
you add capacity to the cluster. You must also be able to connect to the hosts (meaning they must have a public IP and you
should be allowed to connect to them on port 22):SeeSSH into nodesfor a code example.If you want to SSH into nodes in a private subnet, you should set up a bastion host in a public subnet. That setup is recommended, but is
unfortunately beyond the scope of this documentation.Service AccountsWith services account you can provide Kubernetes Pods access to AWS resources.# cluster: eks.Cluster# add service accountservice_account=cluster.add_service_account("MyServiceAccount")bucket=s3.Bucket(self,"Bucket")bucket.grant_read_write(service_account)mypod=cluster.add_manifest("mypod",{"api_version":"v1","kind":"Pod","metadata":{"name":"mypod"},"spec":{"service_account_name":service_account.service_account_name,"containers":[{"name":"hello","image":"paulbouwer/hello-kubernetes:1.5","ports":[{"container_port":8080}]}]}})# create the resource after the service account.mypod.node.add_dependency(service_account)# print the IAM role arn for this service accountCfnOutput(self,"ServiceAccountIamRole",value=service_account.role.role_arn)Note that usingserviceAccount.serviceAccountNameabovedoes nottranslate into a resource dependency.
This is why an explicit dependency is needed. Seehttps://github.com/aws/aws-cdk/issues/9910for more details.It is possible to pass annotations and labels to the service account.# cluster: eks.Cluster# add service account with annotations and labelsservice_account=cluster.add_service_account("MyServiceAccount",annotations={"eks.amazonaws.com/sts-regional-endpoints":"false"},labels={"some-label":"with-some-value"})You can also add service accounts to existing clusters.
To do so, pass theopenIdConnectProviderproperty when you import the cluster into the application.# or create a new one using an existing issuer url# issuer_url: str# you can import an existing providerprovider=eks.OpenIdConnectProvider.from_open_id_connect_provider_arn(self,"Provider","arn:aws:iam::123456:oidc-provider/oidc.eks.eu-west-1.amazonaws.com/id/AB123456ABC")provider2=eks.OpenIdConnectProvider(self,"Provider",url=issuer_url)cluster=eks.Cluster.from_cluster_attributes(self,"MyCluster",cluster_name="Cluster",open_id_connect_provider=provider,kubectl_role_arn="arn:aws:iam::123456:role/service-role/k8sservicerole")service_account=cluster.add_service_account("MyServiceAccount")bucket=s3.Bucket(self,"Bucket")bucket.grant_read_write(service_account)Note that adding service accounts requires runningkubectlcommands against the cluster.
This means you must also pass thekubectlRoleArnwhen importing the cluster.
SeeUsing existing Clusters.Applying Kubernetes ResourcesThe library supports several popular resource deployment mechanisms, among which are:Kubernetes ManifestsTheKubernetesManifestconstruct orcluster.addManifestmethod can be used
to apply Kubernetes resource manifests to this cluster.When usingcluster.addManifest, the manifest construct is defined within the cluster's stack scope. If the manifest contains
attributes from a different stack which depend on the cluster stack, a circular dependency will be created and you will get a synth time error.
To avoid this, directly usenew KubernetesManifestto create the manifest in the scope of the other stack.The following examples will deploy thepaulbouwer/hello-kubernetesservice on the cluster:# cluster: eks.Clusterapp_label={"app":"hello-kubernetes"}deployment={"api_version":"apps/v1","kind":"Deployment","metadata":{"name":"hello-kubernetes"},"spec":{"replicas":3,"selector":{"match_labels":app_label},"template":{"metadata":{"labels":app_label},"spec":{"containers":[{"name":"hello-kubernetes","image":"paulbouwer/hello-kubernetes:1.5","ports":[{"container_port":8080}]}]}}}}service={"api_version":"v1","kind":"Service","metadata":{"name":"hello-kubernetes"},"spec":{"type":"LoadBalancer","ports":[{"port":80,"target_port":8080}],"selector":app_label}}# option 1: use a constructeks.KubernetesManifest(self,"hello-kub",cluster=cluster,manifest=[deployment,service])# or, option2: use `addManifest`cluster.add_manifest("hello-kub",service,deployment)ALB Controller IntegrationTheKubernetesManifestconstruct can detect ingress resources inside your manifest and automatically add the necessary annotations
so they are picked up by the ALB Controller.SeeAlb ControllerTo that end, it offers the following properties:ingressAlb- Signal that the ingress detection should be done.ingressAlbScheme- Which ALB scheme should be applied. Defaults tointernal.Adding resources from a URLThe following example will deploy the resource manifest hosting on remote server:// This example is only available in TypeScript

import * as yaml from 'js-yaml';
import * as request from 'sync-request';

declare const cluster: eks.Cluster;
const manifestUrl = 'https://url/of/manifest.yaml';
const manifest = yaml.safeLoadAll(request('GET', manifestUrl).getBody());
cluster.addManifest('my-resource', manifest);DependenciesThere are cases where Kubernetes resources must be deployed in a specific order.
For example, you cannot define a resource in a Kubernetes namespace before the
namespace was created.You can represent dependencies betweenKubernetesManifests usingresource.node.addDependency():# cluster: eks.Clusternamespace=cluster.add_manifest("my-namespace",{"api_version":"v1","kind":"Namespace","metadata":{"name":"my-app"}})service=cluster.add_manifest("my-service",{"metadata":{"name":"myservice","namespace":"my-app"},"spec":{}})service.node.add_dependency(namespace)NOTE:when aKubernetesManifestincludes multiple resources (either directly
or throughcluster.addManifest()) (e.g.cluster.addManifest('foo', r1, r2, r3,...)), these resources will be applied as a single manifest viakubectland will be applied sequentially (the standard behavior inkubectl).Since Kubernetes manifests are implemented as CloudFormation resources in the
CDK. This means that if the manifest is deleted from your code (or the stack is
deleted), the nextcdk deploywill issue akubectl deletecommand and the
Kubernetes resources in that manifest will be deleted.Resource PruningWhen a resource is deleted from a Kubernetes manifest, the EKS module will
automatically delete these resources by injecting aprune labelto all
manifest resources. This label is then passed tokubectl apply --prune.Pruning is enabled by default but can be disabled through thepruneoption
when a cluster is defined:eks.Cluster(self,"MyCluster",version=eks.KubernetesVersion.V1_21,prune=False)Manifests ValidationThekubectlCLI supports applying a manifest by skipping the validation.
This can be accomplished by setting theskipValidationflag totruein theKubernetesManifestprops.# cluster: eks.Clustereks.KubernetesManifest(self,"HelloAppWithoutValidation",cluster=cluster,manifest=[{"foo":"bar"}],skip_validation=True)Helm ChartsTheHelmChartconstruct orcluster.addHelmChartmethod can be used
to add Kubernetes resources to this cluster using Helm.When usingcluster.addHelmChart, the manifest construct is defined within the cluster's stack scope. If the manifest contains
attributes from a different stack which depend on the cluster stack, a circular dependency will be created and you will get a synth time error.
To avoid this, directly usenew HelmChartto create the chart in the scope of the other stack.The following example will install theNGINX Ingress Controllerto your cluster using Helm.# cluster: eks.Cluster# option 1: use a constructeks.HelmChart(self,"NginxIngress",cluster=cluster,chart="nginx-ingress",repository="https://helm.nginx.com/stable",namespace="kube-system")# or, option2: use `addHelmChart`cluster.add_helm_chart("NginxIngress",chart="nginx-ingress",repository="https://helm.nginx.com/stable",namespace="kube-system")Helm charts will be installed and updated usinghelm upgrade --install, where a few parameters
are being passed down (such asrepo,values,version,namespace,wait,timeout, etc).
This means that if the chart is added to CDK with the same release name, it will try to update
the chart in the cluster.Additionally, thechartAssetproperty can be anaws-s3-assets.Asset. This allows the use of local, private helm charts.importaws_cdk.aws_s3_assetsass3_assets# cluster: eks.Clusterchart_asset=s3_assets.Asset(self,"ChartAsset",path="/path/to/asset")cluster.add_helm_chart("test-chart",chart_asset=chart_asset)OCI ChartsOCI charts are also supported.
Also replace the${VARS}with appropriate values.# cluster: eks.Cluster# option 1: use a constructeks.HelmChart(self,"MyOCIChart",cluster=cluster,chart="some-chart",repository="oci://${ACCOUNT_ID}.dkr.ecr.${ACCOUNT_REGION}.amazonaws.com/${REPO_NAME}",namespace="oci",version="0.0.1")Helm charts are implemented as CloudFormation resources in CDK.
This means that if the chart is deleted from your code (or the stack is
deleted), the nextcdk deploywill issue ahelm uninstallcommand and the
Helm chart will be deleted.When there is noreleasedefined, a unique ID will be allocated for the release based
on the construct path.By default, all Helm charts will be installed concurrently. In some cases, this
could cause race conditions where two Helm charts attempt to deploy the same
resource or if Helm charts depend on each other. You can usechart.node.addDependency()in order to declare a dependency order between
charts:# cluster: eks.Clusterchart1=cluster.add_helm_chart("MyChart",chart="foo")chart2=cluster.add_helm_chart("MyChart",chart="bar")chart2.node.add_dependency(chart1)CDK8s ChartsCDK8sis an open-source library that enables Kubernetes manifest authoring using familiar programming languages. It is founded on the same technologies as the AWS CDK, such asconstructsandjsii.To learn more about cdk8s, visit theGetting Startedtutorials.The EKS module natively integrates with cdk8s and allows you to apply cdk8s charts on AWS EKS clusters via thecluster.addCdk8sChartmethod.In addition tocdk8s, you can also usecdk8s+, which provides higher level abstraction for the core kubernetes api objects.
You can think of it like theL2constructs for Kubernetes. Any othercdk8sbased libraries are also supported, for examplecdk8s-debore.To get started, add the following dependencies to yourpackage.jsonfile:"dependencies":{"cdk8s":"^1.0.0","cdk8s-plus-21":"^1.0.0-beta.38","constructs":"^3.3.69"}Note that here we are usingcdk8s-plus-21as we are targeting Kubernetes version 1.21.0. If you operate a different kubernetes version, you should
use the correspondingcdk8s-plus-XXlibrary.
SeeSelect the appropriate cdk8s+ libraryfor more details.Similarly to how you would create a stack by extending@aws-cdk/core.Stack, we recommend you create a chart of your own that extendscdk8s.Chart,
and add your kubernetes resources to it. You can useaws-cdkconstruct attributes and properties inside yourcdk8sconstruct freely.In this example we create a chart that accepts ans3.Bucketand passes its name to a kubernetes pod as an environment variable.Notice that the chart must accept aconstructs.Constructtype as its scope, not an@aws-cdk/core.Constructas you would normally use.
For this reason, to avoid possible confusion, we will create the chart in a separate file:+ my-chart.tsimportaws_cdk.aws_s3ass3importconstructsasconstructsimportcdk8sascdk8simportcdk8s_plus_21askplusclassMyChart(cdk8s.Chart):def__init__(self,scope,id,*,bucket):super().__init__(scope,id)kplus.Pod(self,"Pod",containers=[kplus.Container(image="my-image",env_variables={"BUCKET_NAME":kplus.EnvValue.from_value(bucket.bucket_name)})])Then, in your AWS CDK app:# cluster: eks.Cluster# some bucket..bucket=s3.Bucket(self,"Bucket")# create a cdk8s chart and use `cdk8s.App` as the scope.my_chart=MyChart(cdk8s.App(),"MyChart",bucket=bucket)# add the cdk8s chart to the clustercluster.add_cdk8s_chart("my-chart",my_chart)Custom CDK8s ConstructsYou can also compose a few stockcdk8s+constructs into your own custom construct. However, since mixing scopes betweenaws-cdkandcdk8sis currently not supported, theConstructclass
you'll need to use is the one from theconstructsmodule, and not from@aws-cdk/corelike you normally would.
This is why we usednew cdk8s.App()as the scope of the chart above.importconstructsasconstructsimportcdk8sascdk8simportcdk8s_plus_21askplusapp=cdk8s.App()chart=cdk8s.Chart(app,"my-chart")classLoadBalancedWebService(constructs.Construct):def__init__(self,scope,id,props):super().__init__(scope,id)deployment=kplus.Deployment(chart,"Deployment",replicas=props.replicas,containers=[kplus.Container(image=props.image)])deployment.expose_via_service(port=props.port,service_type=kplus.ServiceType.LOAD_BALANCER)Manually importing k8s specs and CRD'sIf you find yourself unable to usecdk8s+, or just like to directly use thek8snative objects or CRD's, you can do so by manually importing them using thecdk8s-cli.SeeImporting kubernetes objectsfor detailed instructions.Patching Kubernetes ResourcesTheKubernetesPatchconstruct can be used to update existing kubernetes
resources. The following example can be used to patch thehello-kubernetesdeployment from the example above with 5 replicas.# cluster: eks.Clustereks.KubernetesPatch(self,"hello-kub-deployment-label",cluster=cluster,resource_name="deployment/hello-kubernetes",apply_patch={"spec":{"replicas":5}},restore_patch={"spec":{"replicas":3}})Querying Kubernetes ResourcesTheKubernetesObjectValueconstruct can be used to query for information about kubernetes objects,
and use that as part of your CDK application.For example, you can fetch the address of aLoadBalancertype service:# cluster: eks.Cluster# query the load balancer addressmy_service_address=eks.KubernetesObjectValue(self,"LoadBalancerAttribute",cluster=cluster,object_type="service",object_name="my-service",json_path=".status.loadBalancer.ingress[0].hostname")# pass the address to a lambda functionproxy_function=lambda_.Function(self,"ProxyFunction",handler="index.handler",code=lambda_.Code.from_inline("my-code"),runtime=lambda_.Runtime.NODEJS_14_X,environment={"my_service_address":my_service_address.value})Specifically, since the above use-case is quite common, there is an easier way to access that information:# cluster: eks.Clusterload_balancer_address=cluster.get_service_load_balancer_address("my-service")Using existing clustersThe Amazon EKS library allows defining Kubernetes resources such asKubernetes
manifestsandHelm chartson clusters
that are not defined as part of your CDK app.First, you'll need to "import" a cluster to your CDK app. To do that, use theeks.Cluster.fromClusterAttributes()static method:cluster=eks.Cluster.from_cluster_attributes(self,"MyCluster",cluster_name="my-cluster-name",kubectl_role_arn="arn:aws:iam::1111111:role/iam-role-that-has-masters-access")Then, you can useaddManifestoraddHelmChartto define resources inside
your Kubernetes cluster. For example:# cluster: eks.Clustercluster.add_manifest("Test",{"api_version":"v1","kind":"ConfigMap","metadata":{"name":"myconfigmap"},"data":{"Key":"value","Another":"123454"}})At the minimum, when importing clusters forkubectlmanagement, you will need
to specify:clusterName- the name of the cluster.kubectlRoleArn- the ARN of an IAM role mapped to thesystem:mastersRBAC
role. If the cluster you are importing was created using the AWS CDK, the
CloudFormation stack has an output that includes an IAM role that can be used.
Otherwise, you can create an IAM role and map it tosystem:mastersmanually.
The trust policy of this role should include the thearn:aws::iam::${accountId}:rootprincipal in order to allow the execution
role of the kubectl resource to assume it.If the cluster is configured with private-only or private and restricted public
Kubernetesendpoint access, you must also specify:kubectlSecurityGroupId- the ID of an EC2 security group that is allowed
connections to the cluster's control security group. For example, the EKS managedcluster security group.kubectlPrivateSubnetIds- a list of private VPC subnets IDs that will be used
to access the Kubernetes endpoint.LoggingEKS supports cluster logging for 5 different types of events:API requests to the cluster.Cluster access via the Kubernetes API.Authentication requests into the cluster.State of cluster controllers.Scheduling decisions.You can enable logging for each one separately using theclusterLoggingproperty. For example:cluster=eks.Cluster(self,"Cluster",# ...version=eks.KubernetesVersion.V1_21,cluster_logging=[eks.ClusterLoggingTypes.API,eks.ClusterLoggingTypes.AUTHENTICATOR,eks.ClusterLoggingTypes.SCHEDULER])Known Issues and LimitationsOne cluster per stackService Account dependenciesSupport isolated VPCs