Package: ecsdep
IntroduceECS deploy usingdocker composeandterraform.You need to manage justymlfile fordocker composeand:ecsdepclustercreate
ecsdepserviceupThat's all.PrequisitionGitlab Repository Read CredentialCreate Gitlab access token inProject Settings > Access Tokens,
then checkread/write registrygrants.With this token, create secret in AWShttps://console.aws.amazon.com/secretsmanager/{"username":"<gitlab username>","password":"<access token>"}And savesecret arn.Issuing Domain CertificationIt need AWS domain certification for linking load balancerS3 Bucket For Terraform State BackendCreate bucket liketerraform.my-company.comSSH Key PairsSSH key file, for generating key file,ssh-keygen-trsa-b2048-C"email@example.com"-f./mykeypairIt makes 2 files.mykeypairis private key andmykeypair.pubis public key.
Please keep private key file carefully for accessing your EC2 instances.AWS Access Key For ECS DeployCreate policy.{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["acm:*","application-autoscaling:*","autoscaling:*","cloudformation:*","cognito-identity:*","ec2:*","ecs:*","elasticloadbalancing:*","iam:*","kms:DescribeKey","kms:ListAliases","kms:ListKeys","logs:*","route53:*","s3:*","secretsmanager:*","servicediscovery:*","tag:GetResources"],"Resource":["*"]},{"Effect":"Allow","Action":["ecr:GetAuthorizationToken","ecr:BatchCheckLayerAvailability","ecr:GetDownloadUrlForLayer","ecr:BatchGetImage","logs:CreateLogStream","logs:PutLogEvents"],"Resource":"*"},{"Action":"iam:PassRole","Effect":"Allow","Resource":["*"],"Condition":{"StringLike":{"iam:PassedToService":"ecs-tasks.amazonaws.com"}}},{"Action":"iam:PassRole","Effect":"Allow","Resource":["arn:aws:iam::*:role/ecsInstanceRole*"],"Condition":{"StringLike":{"iam:PassedToService":["ec2.amazonaws.com","ec2.amazonaws.com.cn"]}}},{"Action":"iam:PassRole","Effect":"Allow","Resource":["arn:aws:iam::*:role/ecsAutoscaleRole*"],"Condition":{"StringLike":{"iam:PassedToService":["application-autoscaling.amazonaws.com","application-autoscaling.amazonaws.com.cn"]}}}]}Create user and save access key.Starting Deploy DockerLocal DeployDocker containsterrform,awscliandecsdep.dockerrun-d--privileged\--nameecsdep\-v/path/to/myproject:/app\hansroh/dep:dind
dockerexec-itecsdepbashWithin docker,pip3install-UecsdepGitlab CI/CD DeployAdd these lines into.gitlab-ci.yml:image: hansroh/dep:latest
services:
  - name: docker:dind
    alias: dind-service
before_script:
  - pip3 install -U ecsdepSetup Deploy EnvironmentDocker Logindockerlogin-u<username>-p<personalaccesstoken>registry.gitlab.comConfoguring AWS Access KeyFirst of all, configure AWS access key,awsconfiguresetaws_access_key_id<value>
awsconfiguresetaws_secret_access_key<value>
awsconfiguresetregionap-northeast-2Creating ECS ClusterCreate file withdocker-compose.ymlordocker.ecs.yml.Terraform Settingx-terraform:provider:awsregion:ap-northeast-2template-version:1.1state-backend:region:"ap-northeast-2"bucket:"states-data"key-prefix:"terraform/ecs-cluster"Make sure you create s3 bucket.Cluster Settingsx-ecs-cluster:name:my-clusterpublic-key-file:"mykeypair.pub"instance-type:t3.mediumami:amzn2-ami-ecs-hvm-*-x86_64-*autoscaling:desired:2min:2max:10cpu:60memory:60target-capacity:0loadbalancer:cert-name:mydoamin.comvpc:cidr_block:10.0.0.0/16It creates resources like:Load BalancerVPC with subnets: 10.0.10.0/24, 10.0.20.0/24 and 10.0.30.0/24Cluster Instance Auto Scaling Launch ConfigurationCluster Auto Scaling GroupClusterCluster-Level Auto Scalingautoscaling.cpumean that your cpu reservations of your containers
reaches 60% of all cluster instances's CPU units, scale out ECS instances.
If 0, nothing will be happen.autoscaling.memoryis same.autoscaling.target-capacitytry to keep new instances by percent ofdesired.value is 100: auto scaling group doesn't need to scale in or scale outvalue under 100: at least one instance that's not running a non-daemon taskInstance TypeIf you specifiedx-ecs-gpus,instance-typemust have GPU.VPCIf you consider VPC peering, choose carefullycidr_block.
If you would like to create default VPC, just removevpckey.For VPC peering,x-ecs-cluster:
  vpc:
    peering_vpc_ids:
      - default # mean default VPC
      - vpc-e2ecb79ef6da46Note that this will be work only if your own VPCs.Container AWS Resouce Access Policiesx-ecs-cluster:
  task-iam-policies:
    - arn:aws:iam::aws:policy/AmazonS3FullAccessCreating/Destroying ECS ClusterFinally you can create cluster,ecsdepfind default yaml named asdocker-compose.ymlorcompose.ecs.yml.ecsdepclustercreate
ecsdepclusterdestroySpecifying file is also possible.ecsdep-f/path/to/docker-compose.ymlclustercreate
ecsdep-f/path/to/docker-compose.ymlclusterdestroyCreating S3 Bucket and Cognito Poolx-ecs-cluster:
  s3-cors-hosts:
    - mayservice.comConfiguring ContainsersThis example launch 1 container namedskitai-app.version:'3.3'services:skitai-app:image:registry.gitlab.com/skitai/ecsdepcontainer_name:skitai-appbuild:context:.dockerfile:./Dockerfileports:-5000:5000Make sure service name (skitai-app) is same ascontainer_name.And test image build and container.docker-composebuild
docker-composeup-d
docker-composedownAdding ECS Related SettingsSpecify Deploy ContainersAddservices.skitai-app.deploykey. Otherwise this container will not be included in ECS service.services:skitai-app:image:registry.gitlab.com/skitai/ecsdepdeploy:Docker Container Registry Pull Credentialservices:skitai-app:image:registry.gitlab.com/skitai/ecsdepx-ecs-pull-credentials:arn:aws:secretsmanager:ap-northeast-2:000000000:secret:gitlab/registry/mysecret-PrENMFSeePrequisitionsection.Logging (Optaional)For integrating Cloud Watch log group.services:skitai-app:logging:x-ecs-driver:awslogsContainer Health Checking (Optaional)healthcheck:test:-"CMD-SHELL"-"wget-O/dev/null-qhttp://localhost:5000/ping||exit1"Container Level Reosurce RequirementsFor minimal memory reservation - called soft memory limit,services:skitai-app:deploy:resources:reservations:memory:"256M"For hard memory limit,deploy:resources:reservations:memory:"256M"limits:memory:"320M"Make sure hard limits must be greater or equal than reservation value.For miniaml CPU units reservation,deploy:resources:reservations:cpus:"1024"1024 means 1 vCPU. ECS will deploys container which fulfills these reservation requirements.
Value ofcpusmust be string.Maybe you are using GPU,deploy:resources:reservations:devices:-driver:nvidiacount:1capabilities:[gpu]Butecsdepignores above statement. Just addx-ecs-gpus.deploy:resources:reservations:x-ecs-gpus:1devices:...Ok, test docker build and container.docker-composebuildConfiguring ECS Task Definitionx-ecs-service:name:ecsdepstages:default:env-service-stage:"qa"hosts:["qa.myservice.com"]listener-priority:100production:env-service-stage:"production"hosts:["myservice.com"]listener-priority:101autoscaling:min:3max:7loadbalancing:pathes:-/*protocol:httphealthcheck:path:"/ping"matcher:"200,301,302,404"deploy:compatibilities:-ec2resources:limits:memory:256Mcpus:"1024"autoscaling:desired:1min:1max:4cpu:100memory:80strategy:minimum_healthy_percent:50maximum_percent:150Stagingstagescan make deploy stages likeproduction,qaorstaging.
It find your current environment variable namedSERVICE_STAGE.exportSERVICE_STAGE=qaCurrent deploy stages is selected by currentSERVICE_STAGEvalue matched withenv-service-stage.IfSERVICE_STAGEisqa, your container routedqa.myservice.comby load balancer.Deployment Strategystrategy.maximum_percent: The upper limit (as a percentage of the service's desiredCount) of the number of running tasks that can be running in a service during a deploymentstrategy.minimum_healthy_percent: The lower limit (as a percentage of the service's desiredCount) of the number of running tasks that must remain running and healthy in a service during a deploymentAuto ScalingDefault container-level auto scaling settings are placed inx-ecs-service.autoscaling.
But each stages can overwrite this values byx-ecs-service.[workspace name].autoscaling.Load Balancing (Optional)Your container routed toloadbalancing.pathesby your load balancer.Using FargateIf you wnatfargatedeploy, setx-ecs-service.deploy.compatibilitiesto- fargateResource LimitingIt is required when 'fargate' laubching, but it is optional for 'ec2' launching.x-ecs-service.deploy.autoscaling.cpuandx-ecs-service.deploy.autoscaling.memoryare both percentages of reserved cpu units or memory MBytes.Ifx-ecs-service.deploy.resources.limits.cpusis defined, service cannot use units
over this value.Ifx-ecs-service.deploy.resources.limits.memoryis defined and your container over
this value, container will be terminated.Deploying Serviceyou need env varCI_COMMIT_SHA. It is used as image tag with first 8chars from git commit hash string. it will be provided on gitlab runner, but at local testing,latesetis just OK.exportCI_COMMIT_SHA=latestexportSERVICE_STAGE=qa

ecsdep-fdep/compose.ecs.ymlserviceupWhenever commandingecsdep service up, your containers will be deployed to ECS by rolling update way.As a results, AWS resources will be created.Task DefinitionUpdate Service and RunNote: Sometimes, service-level auto-scaling settings are not applied at initial deploy.
I don't know why but please deploy twice in this case.Shutdown/Remove ServiceecsdepservicedownDeploying Other Services Into ClusterIt is recommended cluster settings are kept in your main app only.In other service's yml, keepx-ecs-cluster.nameonly and remove otherx-ecs-clustersettings.You just care aboutx-ecs-serviceandservicesdefinition.services:skitai-app-2:...x-terrform:...x-ecs-cluster:name:my-clusterDeploying Service With Multiple Containersservices:skitai-app:deploy:ports:-5000healthcheck:test:-"CMD-SHELL"-"wget-O/dev/null-qhttp://localhost:5000||exit1"skitai-nginx:depends_on:-skitai-appx-ecs-wait-conditions:-HEALTHYports:-"80:80"Make sure only single loadbalancable container can have host port
mapping like "80:80", other wise just docker internal port like "5000".Deploying Non Web ServiceGet rid of ports and load balancing settings like:services.your-app.portsx-ecs-service.loadbalancingx-ecs-service.stages.default.hostsx-ecs-service.stages.default.listener-priorityUsing Secretsversion:'3.3'services:skitai-app:environment:DB_PASSWORD=$DB_PASSWORDsecrets:DB_PASSWORD:name:"arn:aws:secretsmanager:ap-northeast-1:0000000000:secret:gitlab/registry/hansroh-PrENMF:DBPASSWORD::"external:trueAt ECS deploy time, environmentDB_PASSWORDwill be overwritten withsecrets.DB_PASSWORD.namevalue by AWS ECS Service.