Building a Simple CI/CD Pipeline with GitLab, Terraform, and Amazon Web Services
Tags: terraform, gitlab, aws, devops
Published: 2021-05-12 13:17:29

---
title: Building a Simple CI/CD Pipeline with GitLab, Terraform, and Amazon Web Services
published: true
description: How to build a deployment pipeline for websites using GitLab and Terraform.
tags: terraform,gitlab,aws,devops
---

# How to build a CI/CD pipeline using GitLab for your business's website.


### Prerequisites
1. A [GitLab](gitlab.com) account
2. An [AWS](aws.amazon.com) account
3. [Terraform](terraform.io) is installed
4. A [KeyBase](keybase.io) account
5. A domain managed in Route53
6. An ACM certificate for your domain.

### Set up the infrastructure
We'll be using Terraform to build out the infrastructure. For the website, all we'll need is an S3 bucket and a CloudFront deployment.

Create a file named main.tf and paste this into. You can change the bucket name to whatever you want, just make sure you set this correctly later on in another file (you'll see).

```terraform
variable "bucket_name" {
  default = "website.example.com"
}

variable "cnames" {
  type    = list(string)
  default = ["example.com", "www.example.com"]
}

variable "certificate_arn" {
  default = "arn:::acm::example23132423"
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "bucket" {
  bucket = "${var.bucket_name}"
  acl    = "private"
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Sid": "AddPerm",
          "Effect": "Allow",
          "Principal": "*",
          "Action": "s3:GetObject",
          "Resource": "arn:aws:s3:::${var.bucket_name}/*"
      }
  ]
}
EOF

  website {
    index_document = "index.html"
    error_document = "index.html"
  }

  tags = {
    billing = "${var.billing_tag}"
  }
}

locals {
  s3_origin_id = "S3-${var.bucket_name}"
}

resource "aws_cloudfront_distribution" "s3_distribution" {
  origin {
    domain_name = "${aws_s3_bucket.bucket.bucket_regional_domain_name}"
    origin_id = "${local.s3_origin_id}"
  }

  wait_for_deployment = false

  enabled = true
  is_ipv6_enabled = true
  default_root_object = "index.html"

  aliases = "${var.cnames}"

  default_cache_behavior {
    allowed_methods = ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"]
    cached_methods = ["GET", "HEAD"]
    target_origin_id = "${local.s3_origin_id}"

    forwarded_values {
      query_string = false

      cookies {
        forward = "none"
      }
    }

    viewer_protocol_policy = "redirect-to-https"
    min_ttl = 0
    default_ttl = 3600
    max_ttl = 86400
  }


  price_class = "PriceClass_100"

  restrictions {
    geo_restriction {
      restriction_type = "none"
    }
  }

  custom_error_response {
    error_code = 403
    error_caching_min_ttl = 0
    response_code = 200
    response_page_path = "/index.html"
  }

  viewer_certificate {
    acm_certificate_arn = "${var.certificate_arn}"
    ssl_support_method = "sni-only"
    minimum_protocol_version = "TLSv1.2_2018"
  }
}

```

Now create the bucket
```
terraform init
terraform apply -auto-approve
```

Awesome! All of our infrastructure is set up in AWS and now we need to set up our GitLab runner!

### Configure our runner
In GitLab, create a repository for your project, or use an existing repository. We just need to do a few things before our app is ready to deploy.

First, let's set up our runner. We're going to use a shared runner from GitLab. They're free to use up for up to 2,000 minutes of deployments per month -- and they're enabled by default.

![shared runners](https://thepracticaldev.s3.amazonaws.com/i/8elgjh8tvs3el88oi5g5.png)

We need to give it an AWS account to use to deploy to S3. Create another terraform config with this content:

```terraform
variable "keybase_user" {
  description = "A keybase username to encrypt the secret key output."
  default     = "dannextlinklabs"
}

provider "aws" {
  region = "us-east-1"
}


resource "aws_iam_access_key" "gitlab_ci" {
  user    = "${aws_iam_user.gitlab_ci.name}"
  pgp_key = "keybase:${var.keybase_user}"
}

resource "aws_iam_user_policy" "gitlab_ci" {
  name = "gitlab-ci-policy"
  user = "${aws_iam_user.gitlab_ci.name}"

  policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::website.example.com/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "cloudfront:*",
            "Resource": "*"
        }
    ]
}
EOF
}

resource "aws_iam_user" "gitlab_ci" {
  name = "gitlab-ci"
}

output "access_key" {
  value = "${aws_iam_access_key.gitlab_ci.id}"
}

output "secret_access_key" {
  value = "${aws_iam_access_key.gitlab_ci.encrypted_secret}"
}

```

Make sure you set the keybase user to your own keybase user. Okay whatever, run the config.

```
terraform init
terraform apply -auto-approve
```

The terraform config returns an access key and a secret key for this user. We need to decrypt the secret key with the command (this is why you needed to use your own keybase user).
```
terraform output encrypted_secret | base64 --decode | keybase pgp decrypt
```
Now that we have the access key and the secret key for our GitLab user, we need to supply those variables to our runner by adding them to the variables section in the CI/CD settings.

![variables](https://thepracticaldev.s3.amazonaws.com/i/i7pzyvy1erk0a2fgv59k.png)

We set three variables:
```
AWS_ACCESS_KEY_ID - to the key we just got
AWS_SECRET_ACCESS_KEY - to the key we just got
AWS_DEFAULT_REGION - us-east-1
```

### Run a deployment
GitLab CI/CD is based around a file called `.gitlab-ci.yml`. Our file needs to look like this:

```yaml
stages:
  - deploy-s3
  - deploy-cf
  
variables:
  AWS_BUCKET: website.example.com

deploy_s3:
  image: python:3.6
  stage: deploy-s3
  tags:
    - docker
    - gce
  before_script:
    - pip install awscli -q
  script:
    - aws s3 sync . s3://$AWS_BUCKET/ --delete --acl public-read
  only:
    - master

deploy_cf:
  image: python:3.6
  stage: deploy-cf
  tags:
    - docker
    - gce
  before_script:
    - pip install awscli -q
  script:
    - export distId=$(aws cloudfront list-distributions --output=text --query 'DistributionList.Items[*].[Id, DefaultCacheBehavior.TargetOriginId'] | grep "S3-$AWS_BUCKET" | cut -f1) 
    - while read -r dist; do aws cloudfront create-invalidation --distribution-id $dist --paths "/*"; done <<< "$distId"
  only:
    - master

```

This gitlab-ci file sets up two stages: deploy-s3, and deploy-cf. The first stage uploads our application to our S3 bucket, and the second invalidates the CloudFront zone for that bucket to present the new changes to our website! 

This simple configuration is all you need to have a complete CI/CD pipeline for your business' website. 

This post first appeared on our [blog](https://nextlinklabs.com/insights/kubernetes-ci-cd-gitlab-with-helm) where we write about [devops and devops consulting services](https://nextlinklabs.com/services/devops-consulting-services).