Bootstrapping Label Studio on Google Cloud with Terraform
Tags: machinelearning, googlecloud, terraform, django
Published: 2021-08-29 22:04:43

Say you need to quickly label tons of data for an ML learning pipeline. You have people on your team or university lab ready to start labeling, but you are struggling to quickly put a tool up and running for them to label concurrently.

Label Studio is a great tool for this. In this post, my goal is to show you how to deploy label-studio as well as show how to create terraform files to quickly bootstrap a Django application running on Google Run connected to a Postgres instance on Google SQL.

The [code can be found in this pull request](https://github.com/heartexlabs/label-studio/pull/1374).

1. First login to your google cloud account.

```zsh
gcloud auth application-default login
```

2. Build the docker image and push it to Google Build with this cloudbuild.yaml script:

```yaml
steps:
  - id: "build image"
    name: "gcr.io/cloud-builders/docker"
    args:
      [
        "build",
        "-t",
        "gcr.io/${PROJECT_ID}/${_SERVICE_NAME}",
        "${_SERVICE_FOLDER}",
      ]

  - id: "push image"
    name: "gcr.io/cloud-builders/docker"
    args: ["push", "gcr.io/${PROJECT_ID}/${_SERVICE_NAME}"]

substitutions:
  _SERVICE_FOLDER: .
  _SERVICE_NAME: label-studio

images:
  - "gcr.io/${PROJECT_ID}/${_SERVICE_NAME}"
```
I used `direnv` to setup the PROJECT_ID env variable to be the Google Project created.

3. Set up the terraform files to bootstrap the project. 
Below are the terraform I created to get this all up, but you will be needing to run the following commands to let the resources be created:

```zsh
terraform init
terraform apply -var="database_password=<DB_PASSWORD>" \
                -var="project=<PROJECT_ID>" \
                -var="database_user=<DB_USER>" \
                -var="domain_name=<DOMAIN_NAME>"
```

After about 10-15 min, all the resources will be created, and you will have Label Studio up and running to create users and start labeling!


### main.tf 

```tf
terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 3.80"
    }
  }
}

provider "google" {
  project = var.project
}

locals {
  service_folder = "service"
  service_name   = "label-studio"

  deployment_name        = "label-studio"
  label_studio_worker_sa = "serviceAccount:${google_service_account.label_studio_worker.email}"
}
```

### project.tf
Creating the service account that will be used to run the application as well as enable the APIs and services that will be used.

```tf
# Enable services
resource "google_project_service" "run" {
  service            = "run.googleapis.com"
  disable_on_destroy = false
}

resource "google_project_service" "iam" {
  service            = "iam.googleapis.com"
  disable_on_destroy = false
}

resource "google_project_service" "cloudbuild" {
  service            = "cloudbuild.googleapis.com"
  disable_on_destroy = false
}

# Create a service account
resource "google_service_account" "label_studio_worker" {
  account_id   = "label-studio-worker"
  display_name = "Label Studio SA"
}

# Set permissions
resource "google_project_iam_binding" "service_permissions" {
  for_each = toset([
    "logging.logWriter", "cloudsql.client"
  ])

  role       = "roles/${each.key}"
  members    = [local.label_studio_worker_sa]
  depends_on = [google_service_account.label_studio_worker]
}
```
### database.tf
```tf
# The Cloud SQL postgres
resource "google_sql_database_instance" "label-studio-postgres" {
  name             = "label-studio-sql"
  database_version = "POSTGRES_13"
  region           = var.region

  settings {
    tier = var.database_tier
  }
}

resource "google_sql_user" "users" {
  name     = var.database_user
  instance = google_sql_database_instance.label-studio-postgres.name
  password = var.database_password
}
```

### service.tf
Run the service, setup IAM policy and create the domain mapping.
```tf
# The Cloud Run service
resource "google_cloud_run_service" "label-studio" {
  name                       = local.service_name
  location                   = var.region
  autogenerate_revision_name = true

  template {
    spec {
      service_account_name = google_service_account.label_studio_worker.email
      containers {
        image = "gcr.io/${var.project}/${local.service_name}"
        env {
          name  = "DEBUG"
          value = "True"
        }
        env {
          name  = "LOG_LEVEL"
          value = "DEBUG"
        }
        env {
          name  = "DJANGO_DB"
          value = "default"
        }
        env {
          name  = "POSTGRE_USER"
          value = "postgres"
        }
        env {
          name  = "POSTGRE_PASSWORD"
          value = google_sql_user.users.password
        }
        env {
          name  = "POSTGRE_NAME"
          value = "postgres"
        }
        env {
          name  = "POSTGRE_HOST"
          value = "/cloudsql/${google_sql_database_instance.label-studio-postgres.connection_name}"
        }
        env {
          name  = "POSTGRE_PORT"
          value = "5432"
        }
        env {
          name  = "GOOGLE_LOGGING_ENABLED"
          value = "True"
        }
      }
    }

    metadata {
      annotations = {
        "autoscaling.knative.dev/maxScale"      = "1000"
        "run.googleapis.com/cloudsql-instances" = google_sql_database_instance.label-studio-postgres.connection_name
        "run.googleapis.com/client-name"        = "terraform"
      }
    }
  }

  traffic {
    percent         = 100
    latest_revision = true
  }

  depends_on = [google_project_service.run, google_sql_database_instance.label-studio-postgres]
}

# Set service public
data "google_iam_policy" "noauth" {
  binding {
    role = "roles/run.invoker"
    members = [
      "allUsers",
    ]
  }
}

resource "google_cloud_run_service_iam_policy" "noauth" {
  location = google_cloud_run_service.label-studio.location
  project  = google_cloud_run_service.label-studio.project
  service  = google_cloud_run_service.label-studio.name

  policy_data = data.google_iam_policy.noauth.policy_data
  depends_on  = [google_cloud_run_service.label-studio]
}

resource "google_cloud_run_domain_mapping" "default" {
  location = var.region
  name     = var.domain_name

  metadata {
    namespace = var.project
  }

  spec {
    route_name = google_cloud_run_service.label-studio.name
  }
}
```

### variables.tf
```tf
variable "project" {
  type        = string
  description = "Google Cloud Platform Project ID"
}

variable "region" {
  default = "us-central1"
  type    = string
}

variable "database_user" {
  default     = "postgres"
  type        = string
  description = "PostgresSQL user."
}

variable "database_password" {
  type        = string
  description = "PostgresSQL database user password."
}

variable "database_tier" {
  default     = "db-f1-micro"
  type        = string
  description = "PostgresSQL database tier."
}

variable "domain_name" {
  type        = string
  description = "Domain name where service will be served from."
}
```