☸️ How to Deploy a Secured OVHCloud Managed Kubernetes Cluster Using Terraform in 2023
Tags: ovh, terraform, kubernetes, cloud
Published: 2023-05-05 07:58:51

---
title: ☸️ How to Deploy a Secured OVHCloud Managed Kubernetes Cluster Using Terraform in 2023
tags:
  - ovh
  - terraform
  - kubernetes
  - cloud
license: public-domain
cover_image: 'https://raw.githubusercontent.com/bcouetil/articles/main/images/terraform/terraform-ovh-1.jpg'
published: true
id: 1385287
date: '2023-05-05T07:58:51Z'
---

<!-- (masterpiece, best quality), <lora:add_detail:.5>, sharp, intricate details, (((purple excavator))), smoke, large white clouds, ((blue night sky)), stars, moon, ((dark))

Déployer son cluster Kubernetes de production sur OVHcloud en full Terraform, c'est possible 😊

Cluster sécurisé dans un réseau privé, avec une BDD dans ce réseau, un accès internet sortant, et entrant par ingress controller.

La documentation n'est pas toujours simple à trouver et à réunir, aussi je vous ai décrit la partie Terraform dans un article 🇬🇧: https://dev.to/zenika/how-to-deploy-a-secured-ovh-managed-kubernetes-cluster-using-terraform-in-2023-17o7

Grand merci à Olivier Beautier pour son dépôt exemple dans lequel j'ai puisé beaucoup d'information: https://github.com/ovh/public-cloud-examples/tree/main/examples/03

Nous avons mis ça en place ça chez Zenika pour une startup française, et ça fonctionne parfaitement !

L'excuse de la difficulté d'automatisation, pour ne pas faire Cloud souverain 🇫🇷, n'est plus valable en ce qui me concerne. Je pourrai y aller plus sereinement pour nos prochains clients !

#kubernetes #ovh #terraform #devops

-->

{%- # TOC start (generated with https://github.com/derlin/bitdowntoc) -%}

- [Initial thoughts](#initial-thoughts)
- [Prerequisites](#prerequisites)
- [Variables](#variables)
  - [Safe variables](#safe-variables)
  - [Secret variables](#secret-variables)
- [Providers and locals](#providers-and-locals)
- [The enclosing VPC network](#the-enclosing-vpc-network)
- [The actual Kubernetes cluster](#the-actual-kubernetes-cluster)
  - [Kubernetes specific outputs](#kubernetes-specific-outputs)
- [Docker registries](#docker-registries)
  - [Registry specific outputs](#registry-specific-outputs)
  - [Private docker registry creation](#private-docker-registry-creation)
  - [Kubernetes cluster to registry integration](#kubernetes-cluster-to-registry-integration)
- [Optional: PostGreSQL managed database](#optional-postgresql-managed-database)
  - [PostGreSQL specific outputs](#postgresql-specific-outputs)
- [Optional: Cloud storage compatible with S3](#optional-cloud-storage-compatible-with-s3)
  - [S3 specific outputs](#s3-specific-outputs)
- [Wrapping up](#wrapping-up)
- [Further reading](#further-reading)

{%- # TOC end -%}

# Initial thoughts

[Terraform](https://developer.hashicorp.com/terraform/intro) is an infrastructure as code tool that lets you build, change, and version cloud and on-prem resources safely and efficiently.

Surprisingly, complete Terraform examples deploying an OVHCloud Kubernetes cluster with private nodes can be hard to find. This blog post is there to fill that whole. Most pieces of code have been found in a [Github repo gathering OVHCloud Terraform examples](), thanks to [Olivier BEAUTIER](https://github.com/raclepoulpe)

We detail here how to deploy, using Terraform, an OVHCloud secured Kubernetes cluster with the following characteristics:

- A VPC network with private subnets using a gateway to internet
- A Kubernetes cluster with nodes in the private subnets
- Docker registries
- A secured managed PostGreSQL redondant database with restricted access

Resulting architecture is very close to this:

![Secured kubernetes cluster](https://raw.githubusercontent.com/bcouetil/articles/main/images/kubernetes/ovh-private-architecture.png)

# Prerequisites

For the below file blocks to work, you need to know the basics of Terraform.

This example uses Terraform workspaces to segregate environments, which is not a conventional practice. But this won't prevent you from massively take advantage of it.

Before anything else, you need:

- an existing S3 bucket to store Terraform state, please follow [official documentation](https://docs.ovh.com/us/en/public-cloud/use_high_perf_object_storage_terraform_backend_state/) to create one
- an [OpenStack user](https://docs.ovh.com/gb/en/public-cloud/creation-and-deletion-of-openstack-user/) in OVHCloud console
- to follow [this guide](https://breadnet.co.uk/terraform-ovh-openstack/) to get OpenStack variables
- OVHCloud Application key, secret and consumer key that are personal and can be generated [online](https://www.ovh.com/auth/api/createToken?GET=/*&POST=/*&PUT=/*&DELETE=/*)

# Variables

Here are variables used by most resources described below. All are simple values.

## Safe variables

Let's start with safe variables, that you can commit to your repo.

```terraform
variable "region" {
  description = "OVH region from https://www.ovhcloud.com/en/public-cloud/regions-availability/"
  default     = "GRA9"
}

variable "global_region" {
  description = "OVH global location for PostGreSQL and bucket"
  default     = "GRA"
}

variable "project_id" {
  description = "OVH tenant or project ID"
  default = {
    staging = "0ebdc4435exxx"
    prod    = "7eb505ec0fyyy"
  }
}

variable "openstack_tenant" {
  description = "OpenStack provider tenant or project name"
  default = {
    staging = "4795193xxx"
    prod    = "3081355yyy"
  }
}

variable "openstack_user" {
  description = "OpenStack provider user name"
  default = {
    staging = "user-Xnernxxx"
    prod    = "user-wn8YNyyy"
  }
}

variable "vlan_id" {
  description = "VLAN ID for staging and production not to overlap. By default it is max(VLAN IDs on the project) + 1, so they would surely overlap"
  default = {
    staging = "1000"
    prod    = "2000"
  }
}
```

## Secret variables

Let's add secret variables, not suitable for commit to your repo.

```terraform

variable "openstack_password" {
  description = "OpenStack provider password"
  default = {
    staging = "sHwxxx"
    prod    = "pRsyyy"
  }
}

# from https://www.ovh.com/auth/api/createToken?GET=/*&POST=/*&PUT=/*&DELETE=/*
variable "ovh_application_key" {
  description = "OVH provider application key"
  default = {
    staging = "1abbxxx"
    prod    = "b565yyy"
  }
}

# idem
variable "ovh_application_secret" {
  description = "OVH provider application secret"
  default = {
    staging = "7d5f76xxx"
    prod    = "19b419yyy"
  }
}

# idem
variable "ovh_consumer_key" {
  description = "OVH provider consumer key"
  default = {
    staging = "cc080axxx"
    prod    = "99f526yyy"
  }
}
```

![(((purple excavator))), smoke, large white clouds, ((blue night sky)), stars, moon, ((dark))](https://raw.githubusercontent.com/bcouetil/articles/main/images/terraform/terraform-ovh-2.jpg)

# Providers and locals

Here are Terraform providers.

A main difference from main cloud providers is the clear mention of OpenStack, for which you will have to create a provider too, in addition to OVHCloud Terraform provider.

```terraform

terraform {

  backend "s3" {
    bucket                      = "my-app-production-admin"
    key                         = "terraform" # (Required) Path to the state file inside the S3 Bucket. When using a non-default workspace, the state path will be /workspace_key_prefix/workspace_name/key
    region                      = "gra"
    endpoint                    = "https://s3.gra.io.cloud.ovh.net/"
    skip_credentials_validation = true
    skip_region_validation      = true

    ### variables not allowed
    # access_key = from 'terraform init' params
    # secret_key = from 'terraform init' params
  }

  required_providers {
    ovh = {
      source  = "ovh/ovh"
      version = "~> 0.27.0"
    }
    openstack = {
      source  = "terraform-provider-openstack/openstack"
      version = "~> 1.49.0"
    }
  }
  required_version = "~> 1.3.6"
}

provider "ovh" {
  endpoint           = "ovh-eu"                                        # or OVH_ENDPOINT
  application_key    = var.ovh_application_key[terraform.workspace]    # or OVH_APPLICATION_KEY
  application_secret = var.ovh_application_secret[terraform.workspace] # or OVH_APPLICATION_SECRET
  consumer_key       = var.ovh_consumer_key[terraform.workspace]       # or OVH_CONSUMER_KEY
}

# inspired by https://breadnet.co.uk/terraform-ovh-openstack/
# and https://registry.terraform.io/providers/terraform-provider-openstack/openstack/latest/docs
provider "openstack" {
  auth_url            = "https://auth.cloud.ovh.net/v3/"            # Authentication URL
  domain_name         = "default"                                   # Domain name - Always at 'default' for OVHcloud
  region              = var.region                                  # or OS_REGION_NAME
  user_domain_name    = "Default"                                   # or OS_USER_DOMAIN_NAME
  project_domain_name = "Default"                                   # or OS_PROJECT_DOMAIN_NAME
  tenant_id           = var.project_id[terraform.workspace]         # or OS_TENANT_ID / OS_PROJECT_ID
  tenant_name         = var.openstack_tenant[terraform.workspace]   # or OS_TENANT_NAME / OS_PROJECT_NAME
  user_name           = var.openstack_user[terraform.workspace]     # or OS_USERNAME
  password            = var.openstack_password[terraform.workspace] # or OS_PASSWORD
}

}
```

# The enclosing VPC network

Here is a sample Terraform block for the VPC network where the Kubernetes cluster will be created.

```terraform

data "openstack_networking_network_v2" "ext_net" {
  name   = "Ext-Net"
  region = var.region
}

resource "openstack_networking_network_v2" "private_network" {
  name           = "${terraform.workspace}-private-network"
  region         = var.region
  admin_state_up = "true"
}

resource "openstack_networking_subnet_v2" "subnet" {
  network_id      = openstack_networking_network_v2.private_network.id
  region          = var.region
  name            = "${terraform.workspace}-subnet"
  cidr            = "192.168.12.0/24"
  enable_dhcp     = true
  no_gateway      = false
  dns_nameservers = ["1.1.1.1", "1.0.0.1"]

  value_specs = {
    "provider:network_type"    = "vrack"
    "provider:segmentation_id" = var.vlan_id[terraform.workspace]
  }

  allocation_pool {
    start = "192.168.12.100"
    end   = "192.168.12.254"
  }
}

resource "openstack_networking_router_v2" "router" {
  region              = var.region
  name                = "${terraform.workspace}-router"
  admin_state_up      = true
  external_network_id = data.openstack_networking_network_v2.ext_net.id
}

resource "openstack_networking_router_interface_v2" "router_interface" {
  router_id = openstack_networking_router_v2.router.id
  region    = var.region
  subnet_id = openstack_networking_subnet_v2.subnet.id
}
```

# The actual Kubernetes cluster

Now serving the main dish: the Kubernetes cluster.

Node autoscaling is not activated in this example, but nothing prevents you from doing so.

```terraform

resource "ovh_cloud_project_kube" "cluster" {
  service_name = var.project_id[terraform.workspace]
  name         = "${terraform.workspace}-cluster"
  region       = var.region

  private_network_id = openstack_networking_network_v2.private_network.id

  private_network_configuration {
    default_vrack_gateway              = "192.168.12.1"
    private_network_routing_as_default = true
  }

  customization {
    apiserver {
      admissionplugins {
        enabled  = ["NodeRestriction"]
        disabled = ["AlwaysPullImages"] # the long-awaited option <3, see https://github.com/ovh/public-cloud-roadmap/issues/70#issuecomment-1235364408
      }
    }
  }
}

resource "ovh_cloud_project_kube_nodepool" "node_pool" {
  service_name = var.project_id[terraform.workspace]
  name         = "${terraform.workspace}-pool"
  kube_id      = ovh_cloud_project_kube.cluster.id
  flavor_name  = "b2-15"
  ## TODO: configure using https://docs.ovh.com/us/en/kubernetes/configuring-cluster-autoscaler/, not available in terraform
  # autoscale = true
  desired_nodes = 3
  max_nodes     = 3
  min_nodes     = 3

  timeouts {
    create = "1h" # default 20m ; OVH can be real slow on this one, and will consider a duplicate on next run
  }
}

resource "local_sensitive_file" "kubeconfig" {
  content         = ovh_cloud_project_kube.cluster.kubeconfig
  filename        = "${terraform.workspace}.kubeconfig"
  file_permission = "0644"

  depends_on = [ovh_cloud_project_kube.cluster, ovh_cloud_project_kube_nodepool.node_pool]
}
```

## Kubernetes specific outputs

kube-config file is given as an output, don't forget to move it, or define KUBECONFIG viable, to be able to use the cluster locally.

Other cluster users will be able to get kube-config [in console](https://docs.ovh.com/gb/en/kubernetes/configuring-kubectl/).

```terraform
output "kubeconfig_file" {
  value     = ovh_cloud_project_kube.cluster.kubeconfig
  sensitive = true
}
```

# Docker registries

Most of us deploy Kubernetes cluster for custom applications, so here are OVHCloud docker registries blocks.

```terraform

data "ovh_cloud_project_capabilities_containerregistry_filter" "regcap" {
  service_name = var.project_id[terraform.workspace]
  plan_name    = "SMALL"
  region       = var.global_region
}

# no need for sub-registries ; the modules will be the image name
resource "ovh_cloud_project_containerregistry" "my-app_registry" {
  name         = "${terraform.workspace}-registry"
  service_name = data.ovh_cloud_project_capabilities_containerregistry_filter.regcap.service_name
  plan_id      = data.ovh_cloud_project_capabilities_containerregistry_filter.regcap.id
  region       = data.ovh_cloud_project_capabilities_containerregistry_filter.regcap.region
}

resource "ovh_cloud_project_containerregistry_user" "team-user" {
  service_name = ovh_cloud_project_containerregistry.my-app_registry.service_name
  registry_id  = ovh_cloud_project_containerregistry.my-app_registry.id
  email        = "noreply-team@docker-registry.ovh"
  login        = "team-user"
}

resource "ovh_cloud_project_containerregistry_user" "tech-user" {
  service_name = ovh_cloud_project_containerregistry.my-app_registry.service_name
  registry_id  = ovh_cloud_project_containerregistry.my-app_registry.id
  email        = "noreply-tech@docker-registry.ovh"
  login        = "tech-user"
}
```

## Registry specific outputs

Some outputs for docker registries, used in next paragraph.

```terraform

output "docker_registry_url" {
  description = "Docker Registry URL"
  value       = ovh_cloud_project_containerregistry.my-app_registry.url
}

output "docker_registry_credentials_url" {
  description = "Generate Docker registry credentials here"
  value       = "https://www.ovh.com/manager/#/public-cloud/pci/projects/${var.project_id[terraform.workspace]}/private-registry"
}

output "docker_registry_harbor_url" {
  description = "Please create the project 'private' here, connecting using credentials generated on docker_registry_credentials_url"
  value       = "${ovh_cloud_project_containerregistry.my-app_registry.url}/harbor/projects"
}

output "docker_registry_tech_user_password" {
  description = "OVH Docker Registry password for user 'tech-user'"
  value       = ovh_cloud_project_containerregistry_user.tech-user.password
  # won't be printed on 'terraform apply'. You have to run:
  # terraform -chdir=devops/infra/staging-prod output --raw docker_registry_tech_user_password && echo ""
  sensitive = true
}

output "docker_registry_k8s_secret_creation_command" {
  description = "Full command to create the secret"
  value       = "kubectl -n my-app create secret docker-registry ovh-docker-reg-cred --docker-server=${ovh_cloud_project_containerregistry.my-app_registry.url} --docker-username=${ovh_cloud_project_containerregistry_user.tech-user.login} --docker-password=${ovh_cloud_project_containerregistry_user.tech-user.password} --docker-email=${ovh_cloud_project_containerregistry_user.tech-user.email}"
  # won't be printed on 'terraform apply'. You have to run:
  # terraform -chdir=devops/infra/staging-prod output --raw docker_registry_k8s_secret_creation_command && echo ""
  sensitive = true
}

output "docker_registry_team_user_password" {
  description = "OVH Docker Registry password for user 'team-user'"
  value       = ovh_cloud_project_containerregistry_user.team-user.password
  # won't be printed on 'terraform apply'. You have to run:
  # terraform -chdir=devops/infra/staging-prod output --raw docker_registry_team_user_password && echo ""
  sensitive = true
}

output "docker_registry_team_user_login_command" {
  description = "Docker login one-liner"
  value       = "echo ${ovh_cloud_project_containerregistry_user.team-user.password} | docker login --username=registry-user ${ovh_cloud_project_containerregistry.my-app_registry.url} --password-stdin"
  # won't be printed on 'terraform apply'. You have to run:
  # terraform -chdir=devops/infra/staging-prod output --raw docker_registry_team_user_login_command && echo ""
  sensitive = true
}

```

## Private docker registry creation

_NOTE: for now, this is not possible using Terraform, so this action stays manual_

- Connect to `docker_registry_credentials_url` given in Terraform outputs above and generate identification details

- With generated credentials, connect to `docker_registry_harbor_url` given in Terraform outputs

- Create a 'private' project here as private

You will then be able to push images with a full name like `xxxx.gra7.container-registry.ovh.net/private/my-app:my-tag`

## Kubernetes cluster to registry integration

Integration with the Kubernetes cluster is not automatic. You have to get credentials and add them in the cluster as a secret.

- Use terraform outputs:

```shell
terraform output --raw docker_registry_k8s_secret_creation_command && echo ""
```

- Apply the generated command to create the secret

- Tell your service account to use it:

```shell
kubectl -n MY_NAMESPACE patch serviceaccount default -p '{"imagePullSecrets": [{"name": "ovh-docker-reg-cred"}]}'
```

![(((purple excavator))), smoke, large white clouds, ((blue night sky)), stars, moon, ((dark))](https://raw.githubusercontent.com/bcouetil/articles/main/images/terraform/terraform-ovh-3.jpg)

# Optional: PostGreSQL managed database

Best practice for Kubernetes is to stay stateless, so here is an optional secured managed PostGreSQL, which access is only from the Kubernetes cluster. In this example, databases `backend`and `auth-server` are created on the same managed instance.

```terraform

# Inspired by: https://github.com/ovh/public-cloud-examples/tree/main/databases/pgsql
# This example is a bit different and older: https://github.com/ovh/public-cloud-databases-examples/tree/main/databases/postgresql/terraform/hello-world
resource "ovh_cloud_project_database" "pg_database" {
  #   depends_on  = [openstack_networking_network_v2.private_network]
  service_name = var.project_id[terraform.workspace]
  description  = "${terraform.workspace} PostGreSQL Cluster"
  engine       = "postgresql" # one of [postgresql cassandra mysql kafka kafkaConnect]
  version      = "14"
  plan         = "business" # 2 nodes, read replicas planned: https://docs.ovh.com/gb/en/publiccloud/databases/postgresql/capabilities/#plans
  flavor       = "db1-7"    # https://docs.ovh.com/gb/en/publiccloud/databases/postgresql/capabilities/#hardware-resources_1
  nodes {
    region     = var.global_region
    network_id = openstack_networking_network_v2.private_network.id
    subnet_id  = openstack_networking_subnet_v2.subnet.id
  }
  nodes {
    region     = var.global_region
    network_id = openstack_networking_network_v2.private_network.id
    subnet_id  = openstack_networking_subnet_v2.subnet.id
  }
}


resource "ovh_cloud_project_database_database" "auth-server" {
  service_name = ovh_cloud_project_database.pg_database.service_name
  engine       = ovh_cloud_project_database.pg_database.engine
  cluster_id   = ovh_cloud_project_database.pg_database.id
  name         = "auth-server"
}

resource "ovh_cloud_project_database_database" "backend" {
  service_name = ovh_cloud_project_database.pg_database.service_name
  engine       = ovh_cloud_project_database.pg_database.engine
  cluster_id   = ovh_cloud_project_database.pg_database.id
  name         = "backend"
}

resource "ovh_cloud_project_database_ip_restriction" "ip_restriction" {
  engine       = "postgresql"
  cluster_id   = ovh_cloud_project_database.pg_database.id
  service_name = ovh_cloud_project_database.pg_database.service_name
  ip           = "192.168.12.0/24"
}

resource "ovh_cloud_project_database_postgresql_user" "backend" {
  service_name = ovh_cloud_project_database.pg_database.service_name
  cluster_id   = ovh_cloud_project_database.pg_database.id
  name         = "backend" # 'postgres' is a reserved user, detailed message taken from API https://eu.api.ovh.com/console/#/cloud/project/%7BserviceName%7D/database/postgresql/%7BclusterId%7D/user~POST
  roles        = ["replication"]
  # Arbitrary string to change to trigger a password update.
  # Use 'terraform refresh' after 'terraform apply' to update the output with the new password.
  password_reset = "password-reset-on-18-01-2022"
}

resource "ovh_cloud_project_database_postgresql_user" "auth" {
  service_name = ovh_cloud_project_database.pg_database.service_name
  cluster_id   = ovh_cloud_project_database.pg_database.id
  name         = "auth" # 'postgres' is a reserved user, detailed message taken from API https://eu.api.ovh.com/console/#/cloud/project/%7BserviceName%7D/database/postgresql/%7BclusterId%7D/user~POST
  roles        = ["replication"]
  # Arbitrary string to change to trigger a password update.
  # Use 'terraform refresh' after 'terraform apply' to update the output with the new password.
  password_reset = "password-reset-on-18-01-2022"
}

```

## PostGreSQL specific outputs

```Terraform

output "postgresql_database_cluster_id" {
  description = "PostGreSQL database ID"
  value       = ovh_cloud_project_database.pg_database.id
  # database_id = 7f0f38d0-df5e-48ec-a6ed-9d9255837cfc
  # cluster_id = a8a9aec9-96c4-4eff-a1ac-bb02b162b343
}

output "postgresql_database_endpoint" {
  description = "PostGreSQL database endpoint in the form of host:port"
  value       = "${ovh_cloud_project_database.pg_database.endpoints[0].domain}:${ovh_cloud_project_database.pg_database.endpoints[0].port}"
}

output "postgresql_backend_user_password" {
  description = "PostGreSQL database user password"
  value       = ovh_cloud_project_database_postgresql_user.backend.password
  # won't be printed on 'terraform apply'. You have to run:
  # terraform -chdir=devops/infra/staging-prod output --raw postgresql_backend_user_password && echo ""
  sensitive = true
}

output "postgresql_auth_user_password" {
  description = "PostGreSQL database user password"
  value       = ovh_cloud_project_database_postgresql_user.auth.password
  # won't be printed on 'terraform apply'. You have to run:
  # terraform -chdir=devops/infra/staging-prod output --raw postgresql_auth_user_password && echo ""
  sensitive = true
}
```

# Optional: Cloud storage compatible with S3

Another optional yet important part of application hosting, is a S3 compatible storage with access from one user known by the Kubernetes cluster.

There is a [need for the AWS provider](https://github.com/ovh/terraform-provider-ovh/issues/329) for this, else a Swift cloud storage is created. The fully automated Terraform process is:

- creation of an admin S3 user
- use of this user to configure the AWS provider
- creation of the S3 bucket and associated bucket user

First add this in the provider part:

```terraform
terraform {

  [...]

  required_providers {
    [...]
    aws = { # for bucket with S3 API
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }

}

provider "aws" {
  region     = lower(var.global_region)
  access_key = ovh_cloud_project_user_s3_credential.s3_admin_cred.access_key_id
  secret_key = ovh_cloud_project_user_s3_credential.s3_admin_cred.secret_access_key

  #OVH implementation has no STS service
  skip_credentials_validation = true
  skip_requesting_account_id  = true
  # the gra region is unknown to AWS hence skipping is needed.
  skip_region_validation = true
  endpoints {
    s3 = "https://s3.${lower(var.global_region)}.io.cloud.ovh.net"
  }
}
```

Then you can create a S3 compatible cloud storage:

```terraform

# inspired by https://github.com/yomovh/tf-at-ovhcloud pointed by https://github.com/ovh/terraform-provider-ovh/issues/329

resource "ovh_cloud_project_user" "s3_admin" {
  service_name = var.project_id[terraform.workspace]
  description  = "Used to create S3 buckets with Terraform"
  role_names = [
    "objectstore_operator"
  ]
}
resource "ovh_cloud_project_user_s3_credential" "s3_admin_cred" {
  service_name = ovh_cloud_project_user.s3_admin.service_name
  user_id      = ovh_cloud_project_user.s3_admin.id
}

resource "aws_s3_bucket" "backend" {
  # name should be unique because shared by all users on the system
  bucket = "my-app-clients-data-${terraform.workspace}"
}

resource "ovh_cloud_project_user" "backend" {
  service_name = var.project_id[terraform.workspace]
  # username is the user id, and is not customizable
  description = "my-app backend app user"
  role_names = [
    "objectstore_operator"
  ]
}

resource "ovh_cloud_project_user_s3_credential" "backend" {
  service_name = ovh_cloud_project_user.backend.service_name
  user_id      = ovh_cloud_project_user.backend.id
}

resource "ovh_cloud_project_user_s3_policy" "policy" {
  service_name = ovh_cloud_project_user.backend.service_name
  user_id      = ovh_cloud_project_user.backend.id
  policy = jsonencode({
    "Statement": [{
      "Sid": "RWContainer",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket", "s3:ListMultipartUploadParts", "s3:ListBucketMultipartUploads", "s3:AbortMultipartUpload", "s3:GetBucketLocation"],
      "Resource": ["arn:aws:s3:::${aws_s3_bucket.backend.bucket}", "arn:aws:s3:::${aws_s3_bucket.backend.bucket}/*"]
    }]
  })
}
```

## S3 specific outputs

Unlike AWS, you will have to store credentials in cluster, next to your application needing the access, or in a secret manager (such as HashiCorp Vault).

```terraform

output "s3_access_key_id" {
  description = "my-app backend app S3 ACCESS_KEY_ID"
  value       = ovh_cloud_project_user_s3_credential.backend.access_key_id
}

output "s3_secret_access_key" {
  description = "my-app backend app S3 SECRET_ACCESS_KEY"
  value       = ovh_cloud_project_user_s3_credential.backend.secret_access_key
  # won't be printed on 'terraform apply'. You have to run:
  # terraform -chdir=devops/infra/staging-prod output --raw s3_secret_access_key && echo ""
  sensitive = true
}

output "s3_endpoint" {
  description = "my-app backend bucket S3 endpoint"
  value       = "https://s3.${lower(var.global_region)}.io.cloud.ovh.net"
}
```

# Wrapping up

Getting all these pieces of Terraform code together, you should be able to deploy a cluster in one command under 30 minutes.

If you think some code should be improved, please advise in the comments 🤓

You can go further with optimizing your cluster costs by picking generic advice from the article [FinOps EKS: 10 tips to reduce the bill up to 90% on AWS managed Kubernetes clusters](https://dev.to/zenika/eks-10-tips-to-reduce-the-bill-up-to-90-on-aws-managed-kubernetes-clusters-epe).

Wonder if it is worthy to use OVHCloud managed cluster alongside your already existing AWS clusters ? You can have a look at [Managed Kubernetes: Our dev is on AWS, our prod is on OVHCloud](https://dev.to/zenika/managed-kubernetes-our-dev-is-on-aws-our-prod-is-on-ovh-3nbf).

![(((purple excavator))), smoke, large white clouds, ((blue night sky)), stars, moon, ((dark))](https://raw.githubusercontent.com/bcouetil/articles/main/images/terraform/terraform-ovh-4.jpg)

_Illustrations generated locally by Automatic1111 using Lyriel model_

# Further reading

{% embed https://dev.to/zenika/why-managed-kubernetes-is-a-viable-solution-for-even-modest-but-actively-developed-applications-357g %}

{% embed https://dev.to/zenika/kubernetes-from-your-docker-compose-file-to-a-cluster-with-kompose-1gn0 %}

{% embed https://dev.to/zenika/kubernetes-a-pragmatic-kubectl-aliases-collection-17oc %}

{% embed https://dev.to/zenika/web-application-on-kubernetes-a-tutorial-to-observability-with-the-elastic-stack-2p8a %}

{% embed https://dev.to/zenika/kubernetes-nginx-ingress-controller-10-complementary-configurations-for-web-applications-ken %}

{% embed https://dev.to/zenika/kubernetes-awesome-maintained-links-you-will-keep-using-next-year-48o8 %}

{% embed https://dev.to/zenika/managed-kubernetes-our-dev-is-on-aws-our-prod-is-on-ovh-3nbf %}

{% embed https://dev.to/zenika/how-to-deploy-a-cost-efficient-awseks-kubernetes-cluster-using-terraform-in-2023-3903 %}

{% embed https://dev.to/zenika/eks-10-tips-to-reduce-the-bill-up-to-90-on-aws-managed-kubernetes-clusters-epe %}