Simplifying Keycloak Configuration with Terraform and Terragrunt
Tags: terraform, keycloak, terragrunt, security
Published: 2024-05-04 23:51:48

[Keycloak](https://www.keycloak.org), an open-source identity and access management solution, provides robust authentication and authorization services for modern applications. However, configuring Keycloak instances manually can be tedious and error-prone. In this blog post, we'll explore how to simplify Keycloak configuration using Terraform and Terragrunt, enabling infrastructure as code (IaC) practices for managing Keycloak realms, clients, users, and more.

---

## Why Terraform and Terragrunt?
[Terraform](https://www.terraform.io) is an open-source tool that lets you manage your infrastructure as code. With Terraform, you can define your infrastructure in simple configuration files, and Terraform will automatically create, manage, and update your infrastructure according to those configurations. 

[Terragrunt](https://terragrunt.gruntwork.io) is a thin wrapper for Terraform that provides extra tools for keeping your Terraform configurations DRY, managing remote state, and working with multiple Terraform modules. It helps you maintain clean, DRY, and repeatable Terraform code by providing extra functionality and best practices.

**By leveraging both, we can have easy**:
- **Infrastructure as Code (IaC)**: By defining Keycloak configuration as code, we can version control, review changes, and ensure consistency across environments.
- **Modularization**: Modularize our Keycloak configuration, making it easier to manage complex setups.
- **State Management**: Manage the state of our infrastructure, preventing configuration drift and ensuring that our infrastructure remains in the desired state.

---

## Let us go!

### Install Terraform and Terragrunt
Make sure you have Terraform and Terragrunt installed on your machine. You can find installation instructions on the official Terraform and Terragrunt documentation
- Terraform: https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli 
- Terragrunt: https://terragrunt.gruntwork.io/docs/getting-started/install/ 

### Project Structure 
Creating a one-size-fits-all structure for a Terraform project can be challenging because it largely depends on the specific requirements of each project. Below is the structure I've found most suitable for organizing Keycloak components, concepts, and setup.

<!-- prettier-ignore-start -->
<!-- Don't forget the two tabs! -->
    .
    ├── README.md                 <- The project overview
    ├── .tool-versions            <- Used tools versions (managed by asdf. see https://asdf-vm.com) 
    ├── README.md                 <- The project overview
    ├── modules                   <- Terraform modules
    |   └── common
    │       ├── provider.tf
    │       └── variables.tf
    │       └── output.tf
    │       └── main.tf
    │       └── README.md
    │       └── docs/
    |   └── clients
    │       ├── provider.tf
    │       └── variables.tf
    │       └── output.tf
    │       └── main.tf
    │       └── README.md
    │       └── docs/
    |   └── my-realm
    │       ├── provider.tf
    │       └── variables.tf
    │       └── output.tf
    │       └── main.tf
    │       └── README.md
    │       └── docs/
    |   └── other
    │       ├── provider.tf
    │       └── variables.tf
    │       └── output.tf
    │       └── main.tf
    │       └── README.md
    │       └── docs/
    └── how-to                    <- Documentation
    └── stage                     <- Terraform for environment stage
        ├── .terraform.lock.hcl   <- Terraform lock file
        └── terragrunt.hcl        <- Terragrunt file
        └── env.yaml              <- environment related variables
        └── main.tf               <- environment modules
    └── prod                      <- Terraform for environment prod
        ├── .terraform.lock.hcl   <- Terraform lock file
        └── terragrunt.hcl        <- Terragrunt file
        └── env.yaml              <- environment related variables
        └── main.tf               <- environment modules
    
    └── local                     <- Terraform for environment local
        ├── .terraform.lock.hcl   <- Terraform lock file
        └── terragrunt.hcl        <- Terragrunt file
        └── env.yaml              <- environment related variables
        └── main.tf               <- environment modules
<!-- prettier-ignore-end -->

In this project structure, I've included a modules directory containing a set of modules shared across all environments. Each module includes a `main.tf` file encapsulating the module's resources, along with `input.tf`, `output.tf`, and `variable.tf` files for easy configuration across different environments.

### Common Module 
Let's assume that in the `common` module, we configure realm events by using the `jboss-logging` event listener with some non-default configurations. Below is an example of how the `main.tf` file may look:
```tf
resource "keycloak_realm_events" "realm_events" {
  realm_id                     = var.realm_id
  events_enabled               = true
  events_expiration            = 1800
  admin_events_enabled         = true
  admin_events_details_enabled = true
  ]

  events_listeners = [
    "jboss-logging"
  ]
}
```
To include the `realm_id` variable, it must be defined in the `variables.tf` file as shown below:
```tf
variable "realm_id" {
  description = "Realm ID"
  type        = string
}
```
And we must configure the used providers in our module. In this case, the `provider.tf` will look like as:

```tf
terraform {
  required_providers {
    keycloak = {
      source = "mrparkers/keycloak"
    }
  }
}
```

### Master Realm Module 
In the `realm-master` module, the `main.tf` file should reference the common module. Here is an example of how the `main.tf` file may look:
```tf
data "keycloak_realm" "master" {
  realm = "master"
}

module "realm-master" {
  source = "../../modules/common"
  realm_id = data.keycloak_realm.master.id
}
```
Similarly, the `provider.tf` file should be configured as follows:
```tf
terraform {
  required_providers {
    keycloak = {
      source = "mrparkers/keycloak"
    }
  }
}
```

### Local environment 
In the `main.tf` file, we need to define the master realm in order to reference the `realm-master` module in our project.

```tf
# Define master realm
module "realm-master" {
  source = "../modules/realm-master"
}
```

Within each environment (e.g., `prod`, `stage` and `local`), there's an `env.yaml` file containing all the environment-specific variables.

For example, the `env.yaml` file for the local environment may look like this:
```yaml
---
environment: local
url: http://localhost:8080/keycloak

```
And of course, don't forget to include the `terragrunt-local.hcl` file, which should be defined in the parent module.
```hcl
include "root" {
  path = find_in_parent_folders("terragrunt-local.hcl")
}
```

### Terragrunt configuration
As mentioned earlier, Terragrunt is highly beneficial for keeping your configuration Don't Repeat Yourself (DRY). For example, the `terragrunt.hcl` file may look like this:

```hcl
# Generates the backend for all modules.
remote_state {
  backend = "s3"
  config  = {
    encrypt        = true
    key            = "keycloak/${path_relative_to_include()}/terraform.tfstate"
    region         = "<AWS REGION>"
    bucket         = "terraform-states"
    dynamodb_table = "terraform-lock"
  }
}

# Read the local "env.yaml" in every environment.
locals {
  vars        = yamldecode(file("${path_relative_to_include()}/env.yaml"))
  environment = local.vars.environment
  url         = local.vars.url
}

# Generate the "provider.tf" file for every module.
generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite"
  contents  = <<EOF
terraform {
  required_providers {
    keycloak = {
      source  = "mrparkers/keycloak"
      version = "4.4.0"
    }

    http = {
      source  = "hashicorp/http"
      version = "3.2.1"
    }
  }
}

data "http" "config" {
  url = "<INTERNAL CONFIGURATION URL>"
}

provider "keycloak" {
  client_id     = jsondecode(data.http.config.response_body).terraform-client-id
  client_secret = jsondecode(data.http.config.response_body).terraform-client-secret
  url           = "${local.url}"
}

EOF
}

# Generate the "backend.tf" file for every module.
generate "backend" {
  path      = "backend.tf"
  if_exists = "overwrite"
  contents  = <<EOF
terraform {
  backend "s3" {}
}
EOF
}
```
If you're familiar with Terragrunt, you'll find the `terragrunt.hcl` file above quite familiar, except for the http `config` part, which I'll describe later in section.

In `remote_state`, we use AWS S3 to store the state of our environment configurations. Additionally, we generate the `backend` and `provider` for every environment.

The `locals` block is mainly used to read the `env.yaml` file and assign the variables to local variables.

Lastly, the http `config` is responsible for making an HTTP call to wherever you securely store your environment configurations (at least the Terraform client ID and secret) to pass them to the `keycloak` provider.

The `terragrunt-local.hcl` file is similar to the `terragrunt.hcl` file, except for the remote state configuration, which isn't necessary in this case. The `local` file is primarily for testing your configuration on a local Keycloak cluster setup.  

---

### The Docker Compose 
To run our Terraform configurations, we require a Keycloak cluster setup. In below, we use Docker Compose to start a Keycloak cluster, enabling us to run our local Terraform configurations seamlessly against it.

```yaml
version: '3'

volumes:
  postgres_data:
      driver: local

services:
  postgres:
      image: postgres
      volumes:
        - postgres_data:/var/lib/postgresql/data
      environment:
        POSTGRES_DB: keycloak
        POSTGRES_USER: keycloak
        POSTGRES_PASSWORD: password
      ports:
        - "5432:5432"
  keycloak:
      image: quay.io/keycloak/keycloak:23.0.6
      environment:
        KC_DB_USERNAME: keycloak
        KC_DB_PASSWORD: password
        KC_DB_URL_HOST: postgres
        KC_DB: postgres
        KC_DB_SCHEMA: public
        KC_HTTP_RELATIVE_PATH: /keycloak
        KC_HOSTNAME_ADMIN: 127.0.0.1
        KC_HOSTNAME: localhost
        KEYCLOAK_ADMIN: admin
        KEYCLOAK_ADMIN_PASSWORD: admin
      command:
        - start-dev
      ports:
        - "8080:8080"
        - "8787:8787"
      depends_on:
        - postgres
  config_keycloak:
    image: ubuntu
    volumes:
      - ./keycloak-docker-config.sh:/opt/keycloak-docker-config.sh
    command: ./opt/keycloak-docker-config.sh
    depends_on:
      - keycloak
```
The `keycloak-docker-config.sh` script is primarily used to configure a Terraform client with `admin` privileges, which Terraform will use during its operations.
```sh
#!/bin/bash

apt update -y && apt -y install jq curl

until $(curl --output /dev/null --silent --head --fail http://keycloak:8080/keycloak); do
    printf '.'
    sleep 5
done

# Get access token
TOKEN=$( \
  curl -X POST \
    -H "Content-Type: application/x-www-form-urlencoded" \
    -d "client_id=admin-cli" \
    -d "username=admin" \
    -d "password=admin" \
    -d "grant_type=password" \
    "http://keycloak:8080/keycloak/realms/master/protocol/openid-connect/token" | jq -r '.access_token')

# Create Terraform client (terraform/terraform)
curl -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{"clientId": "terraform", "name": "terraform", "enabled": true, "publicClient": false, "secret": "terraform", "serviceAccountsEnabled": true}' \
  "http://keycloak:8080/keycloak/admin/realms/master/clients"

# Get the Terraform service account user ID
USER_ID=$( \
  curl -X GET \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer ${TOKEN}" \
    "http://keycloak:8080/keycloak/admin/realms/master/users?username=service-account-terraform" | jq -r '.[0].id')

# Get the admin role ID
ROLE_ID=$( \
  curl -X GET \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer ${TOKEN}" \
    "http://keycloak:8080/keycloak/admin/realms/master/roles" | jq -r '.[] | select(.name == "admin") | .id')

# Add the admin role to the Terraform service account user
curl -kv -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '[{"id":"'"${ROLE_ID}"'", "name":"admin"}]' \
  "http://keycloak:8080/keycloak/admin/realms/master/users/$USER_ID/role-mappings/realm"
```
To run it, open the `terminal` and run the below docker compose command
```shell
docker-compose up --build
```

After the docker-compose containers are up and running, navigate to `http://localhost:8080/keycloak` and log in using `admin/admin` as credentials. Ensure that the Terraform client is configured within the master realm.

![Terraform client](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m18il661sbzjk61hmufm.png)

Now it's time to run your Terraform local configurations. Open your terminal and execute the following commands:
```shell
# Navigate to the local environment
$ cd local 
# Ensure that Terraform-related files, including the auto-generated backend.tf and provider.tf, are removed
$ rm -r backend.tf provider.tf terraform.tfstate terraform.tfstate.backup .terraform.lock.hcl .terraform 
# Initialize Terraform to create all necessary files
$ terragrunt init --terragrunt-config terragrunt-local.hcl 
# Apply the Terraform configurations
$ terragrunt apply --terragrunt-config terragrunt-local.hcl 
```
Now, open your web browser and go to `http://localhost:8080/keycloak`. Log in using your admin credentials and make sure your configurations are properly set up there!

## Conclusion
Organizing and managing Keycloak configurations with Terraform can greatly streamline your development process. By following the structure and steps outlined in this guide, you can efficiently set up and maintain your Keycloak environments, ensuring consistency and scalability across your projects. If you have any questions or suggestions, feel free to leave them in the comments below. 

I hope you find it useful!