Simple GCP Authentication with Service Accounts
By: Yitaek Hwang
Tags: Security, Google Cloud Platform, Programming, Software Development, Coding

A practical guide for using GCP Service Accounts to authenticate and use Google Cloud APIs easily and securely.

Photo by Morning Brew on Unsplash

In order to use Google Cloud Products, applications must first authenticate to Google and check IAM permissions. While Google’s IAM documentation page goes into meticulous detail about security best practices, as a non-security engineer, it’s hard to parse through all the text and apply Google’s recommendations for each use case. I know that I’m supposed to enforce least privilege for my service accounts, but what’s the best way to authenticate with Google SDKs while developing locally vs. giving access on GKE? In this post, I’ll summarize the recommendations for common use cases when interacting with Google Cloud.

Using the CLI (gcloud, terraform)

If you are mostly interacting with GCP via CLI (either invoking gsutil , gcloud , or creating GCP components via terraform ), create a service account with respective roles, and use the service account impersonation feature. Until recently, the GCP console provided users with the option to create and download keys when creating a service account. Now that option is hidden in the default creation workflow, and users must click on the service account options to manually create keys.

All Google Cloud APIs authenticate using OAuth2. When you generate a service account key, it creates a public/private key pair that is used to sign a JWT token to authentical GCP credentials. While this is convenient, downloading keys are considering dangerous as it could be accidentally checked into version control and does not expire (default expiration date is Dec 31, 9999). GCP supports key rotation, but centrally enforcing that is really difficult unless you have tools like Vault set up to automate that process.

The good news is that you can impersonate a service account to authenticate without needing to download keys. First, you need the serviceAccountTokenCreator role and run --impersonate-service-accouunt=<sa-name>@project.iam.gservicaccount.com with regular gcloud commands. You can also set your config to avoid passing in the command every time:

gcloud config set auth/impersonate_service_account \

<sa-name>@project.iam.gserviceaccount.com

As for Terraform, set the GOOGLE_OAUTH_ACCESS_TOKEN variable to pass an OAuth2 token:

export GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token)

Now you can run terraform commands with a short-lived token instead of downloading keys that you have to securely manage. Ryan Canty from Google wrote up a good article about this topic with example bash scripts to switch between multiple service accounts:

Local/Non-GCP Development

All Google Cloud Client libraries use an underlying auth library called Application Default Credentials (ADC) to automatically find and set service account credentials. It looks for credentials in the following order:

It checks if env variable GOOGLE_APPLICATION_CREDENTIALS is set. If it is, it uses the service account file that the env points to. If the variable is not set, then it uses the default service account. If ADC can’t use any of the above, it errors out.

This gives developers two choices to authenticate:

Create a service account, download the keys to a secure location, and set GOOGLE_APPLICATION_CREDENTIALS Use the gcloud auth application-default login to set your user credentials to ~/.config/gcloud/application_default_credentials.json

While the second approach is more convenient for initial testing, it is not the recommended approach since it is using your user’s credentials as opposed to the service account’s permission scope, which most likely will be more restricted. Also, some SDK features such as Firebase Custom Tokens and GCS Signed URLs require the client_email field, which is not part of the application-default login credentials.

The general recommendation from the Google team is to create and download the JSON credential file and set the path to GOOGLE_APPLICATION_CREDENTIALS . Given the security concerns listed above with these long-lasting keys, make sure to store it in a safe place and test on development GCP projects. For more information, check out Theodore Siu’s article on local development with GCP:

Note: This section also applies when calling GCP APIs on hybrid cloud architectures. For example, you may be using Google’s ML APIs from AWS or using Firebase for mobile development while keeping other compute workloads on different clouds.

Applications on Google Cloud

On Google Cloud, ADC automatically searches for default service account when running on Compute Engine, App Engine, Kubernetes Engine, Cloud Run, and Cloud Functions. The default service account has the Editor role and sets default auth scopes for various GCP products. For example, a GKE cluster created with the default service account gives read-only access to Storage and write access to Stackdriver Logging and Monitoring. If you need more granular access, you can create a new service account and attach it to use its IAM roles instead of the default service accounts.

As for giving containers access to Google Cloud resources, the recommended approach is to use Workload Identity. Workload Identity is similar to Amazon EKS IAM Roles for Service Accounts (IRSA) in that it gives Kubernetes service accounts (KSA) the ability to act as Google service accounts (GSA) when accessing Google Cloud APIs. Once Workload Identity is enabled on GKE, an identity pool called <project-id>.svc.id.goog is created.

To make a connection between KSA and GSA, first create both service accounts:

$ kubectl create sa <ksa_name> -n <k8s_namespace>

$ gcloud iam service-accounts create <gsa-name>

Bind the IAM policy with the roles/iam.workloadIdentityUser role:

$ gcloud iam service-accounts add-iam-policy-binding \

--role roles/iam.workloadIdentityUser \

--member "serviceAccount:<project_id>.svc.id.goog[k8s_namespace/ksa_name]" \

<gsa_name>@<project_id>.iam.gserviceaccount.com

Finally, add an annotation to KSA:

$ kubectl annotate serviceaccount \

--namespace <k8s_namespace> \

<ksa_name> \

iam.gke.io/gcp-service-account=<gsa_name>@<project_id>.iam.gserviceaccount.com

Workload Identity does have some limitations such as no support for GKE on-prem and Windows nodes. In that case, you can mount service account keys from secret and set GOOGLE_APPLICATION_CREDENTIALS , or use a tool like Vault secret injector.

Final Words

I’m by no means a security expert, but you can follow these simple service account recommendations to securely access Google Cloud APIs even without a sophisticated secrets management setup. For the next steps, you can also check out the GCP CIS Benchmark Inspec Profile tool to make sure the appropriate IAM sections are covered in your project.