GCP Cloud APIs with Service Account
By: Kunwar Agrawal
Tags: Service Account, Rest Api, Gcp, Gcp Cloud Api, Cloud Api

Authorised GCP Cloud API request, directly from API clients. No Libraries!

GCP Cloud APIs are very helpful when it comes to direct interaction with GCP services and applications. For example, GCP Cloud Storage APIs can be used to easily put objects in a GCP bucket.

GCP offers these APIs in JSON and XML and can be used via many coding languages (Java, Python, etc.) with existing libraries. But here we will discuss, making these requests without writing any code or using any libraries. We can use API clients like Postman to test this. This article will also help understand the background functioning of the libraries.

First, let’s understand the overall flow. Service Account authorisation for Cloud APIs follows the OAuth 2.0 pattern, which means its a two-step process. The OAuth 2.0 flows something like this:

OAuth 2.0 Flow (Courtesy: Google Reference Docs)

We first prepare a JWT (JSON Web Token) Token. Then we use it to get the access token from the Google Auth Token Endpoint. We then make a request to any Google Cloud API.

First, let’s make a Service Account

In some organisations the GCP platform is managed by a separate team, in such cases, one can ask the team to make a Service Account and share the API Keys. But, If you are controlling the console:

Go to GCP Console > IAM & Admin > Service Accounts, and click on create a Service Account.

2. Give a name to the Service Account. You can make your own naming standards for your team/organization. For example, If am going to use my service account for GCP Cloud Storage API calls, I can name it “dev_cloud_storage_api_ac”. You can visit here for more information.

For this article, I am using “test_medium”

3. Next, we have to grant the required permissions (access) to the Service Account. There are a lot of predefined roles that can be used. As I will be using this service account for putting and getting objects to/from a bucket, I am giving my service account the below roles:

Storage Object Creator Storage Object Viewer

You can add multiple roles to a service account by simply clicking on “add another role”.

4. Last step is to create keys for the Service Account. Go to the Service Account Settings > Keys > Add Key > Create new key. Choose the key type as “JSON” and click on “Create”. It will now download the JSON key to your computer.

Making the JWT (JSON Web Token)

Creating the JWT sounds complicated, but in fact is very simple. As seen in the OAuth2.0 flow, the JWT is used to get the access token which is required to make a request to the GCP Cloud API.

GCP expects the JWT token in the below format:

The JWT is made up of three parts, concatenated with a period (.) symbol.

{Base64url encoded header}.{Base64url encoded claim set}.{Base64url encoded signature}

And the input for the signature is:

{Base64url encoded header}.{Base64url encoded claim set}

Header

The header part of the token is a static JSON Array, consisting of the signing algorithm and the token type:

{"alg":"RS256","typ":"JWT"}

Since GCP only offers RSA with SHA 256 for now, this is mostly going to be constant.

Base 64 Encoded:

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9

You can use notepad++ to encode the strings or an online tool.

Claim Set

Just like Header, the Claim Set is a JSON array with few attributes :

{

"iss": "<SERVICE ACCOUNT EMAIL ADDRESS>",

"scope": "<PERMISSIONS THAT YOU NEED>",

"aud": "https://oauth2.googleapis.com/token",

"exp": <EXPIRY TIME IN UNIX FORMAT>,

"iat": <CREATED TIME IN UNIX FORMAT>

}

Let’s clear a few things here,

Service Account Email Address can be taken from the GCP console, where we made it, or we can also find it in the JSON Key as the “client_email”. Scope or Permissions required can be found in the JWT Playground. As I am going to use this Service Account for Google Cloud Bucket APIs, I am using the below-highlighted scope.

3. Audience or “aud” defines the target for this JWT token. In our case it will be https://oauth2.googleapis.com/token. This is because our JWT token is for OAuth2.0 Authentication (to fetch the access token). This URI is provided to us by google. You can refer to the “token_uri” in the JSON key as well.

4. Expiry and the Creation Dates (“exp” and “iat”) have to be added in the UNIX Format. The expiry time can’t be more than one hour in the future. And the creation date obviously can’t be a future date/time (especially to when we are making the request). So in simple terms, you can give current time as the creation time (“iat”) and one-hour future time as the expiry time (“exp”).

Unix Epoch time is, literally speaking, the number of seconds passed till January 1, 1970 (midnight UTC/GMT). Epoch technically means 0th Unix Time. You can use Epoch Converter to convert any time into the Unix Time.

Tip: If you have the “created time” in UNIX format, you can just add “3600” to get the expiry time.

Now that you have the Claim Set ready, you can Base64url encode it, much like the Header:

ewogICJpc3MiOiAiPFNFUlZJQ0UgQUNDT1VOVCBFTUFJTCBBRERSRVNTPiIsCiAgInNjb3BlIjogIjxQRVJNSVNTSU9OUyBUSEFUIFlPVSBORUVEPiIsCiAgImF1ZCI6ICJodHRwczovL29hdXRoMi5nb29nbGVhcGlzLmNvbS90b2tlbiIsCiAgImV4cCI6IDxFWFBJUlkgVElNRSBJTiBVTklYIEZPUk1BVD4sCiAgImlhdCI6IDxDUkVBVEVEIFRJTUUgSU4gVU5JWCBGT1JNQVQ+Cn0=

Signature

The next step is to combine the Base64url encoded Header and Claim Set with a period (.) symbol.

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.ewogICJpc3MiOiAiPFNFUlZJQ0UgQUNDT1VOVCBFTUFJTCBBRERSRVNTPiIsCiAgInNjb3BlIjogIjxQRVJNSVNTSU9OUyBUSEFUIFlPVSBORUVEPiIsCiAgImF1ZCI6ICJodHRwczovL29hdXRoMi5nb29nbGVhcGlzLmNvbS90b2tlbiIsCiAgImV4cCI6IDxFWFBJUlkgVElNRSBJTiBVTklYIEZPUk1BVD4sCiAgImlhdCI6IDxDUkVBVEVEIFRJTUUgSU4gVU5JWCBGT1JNQVQ+Cn0=

We are now ready to sign! Use the private key given in the JSON KEY under “private_key” to sign the above the string. Do remember that as the key is given as a string, the line breaks are escaped as ‘

’ symbols. We need to convert these ‘

’ characters into real line breaks. We can use online tools like this to unescape our private key.

Save this private key with “.pem” extension and then proceed with signing.

You can use OpenSSL in Mac or Linux to sign. In windows, you can use Git Bash.

openssl dgst -sha256 -sign <PRIVATE KEY FILE NAME> -out <OUTPUT FILE NAME> <INPUT FILE NAME>

You can also use some online tool. But just remember we need to use SHA256withRSA.

{•bkÚtq}Œ‡‰µΩóh»¥9ûd©o”ˆˆÄFºˆ√ÒÏsf∆‚ÿ�|W3i“ C8¥;(s·?Õ¶û„–ïaå’˜¶óÈ;*«òl@∆÷´

æ+<^ÇÕGr¥Ω&:ÈkjU+©äáÑ‹8jîﬂ¬∞@‡4Ó@ç≥\∞XÔÕX”o�à≈då4PntﬂıÙFX^oÓtsß]Mâ/æÊÖ ™\ueq.Áz¥Ïí

‘ö´s — (Ë¶≠P™Œ?V(r+È¿6•«øøÕH≥éQH˙E4L7ˇdòÚ`,G˝ûµÁ\Ìµ≥ÍˇE◊:ái˘k›˚V∏5Ù∂Qñ∑à

Once the signing is done we will get some gibberish. Base64url encode this and you will get the final part of the JWT Token.

Final JWT

Now just concatenate the Base64url encoded Header, Claim Set and the Signature with a period (.) symbol, to get the final JWT.

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.ewogICJpc3MiOiAiPFNFUlZJQ0UgQUNDT1VOVCBFTUFJTCBBRERSRVNTPiIsCiAgInNjb3BlIjogIjxQRVJNSVNTSU9OUyBUSEFUIFlPVSBORUVEPiIsCiAgImF1ZCI6ICJodHRwczovL29hdXRoMi5nb29nbGVhcGlzLmNvbS90b2tlbiIsCiAgImV4cCI6IDxFWFBJUlkgVElNRSBJTiBVTklYIEZPUk1BVD4sCiAgImlhdCI6IDxDUkVBVEVEIFRJTUUgSU4gVU5JWCBGT1JNQVQ+Cn0=.e+KAomJ/a8OadHF9xZIe4oChGeKAsMK1zqnDs2jCu8KlOcO7ZMKpbyLLhsuGw4RGwrrLhuKImsOSw48Zcxpm4oiG4oCaGhXDv++/vXxXM2kiIEM4wqU7FChzwrc/w5UewrbDu+KAnuKAk8OvYcOlJ8ucwrbDs8OIOxgqwqvDsmxA4oiGw7fCtArDpis8XsOHw5VHcsKlzqkmOsOIa2pVBivCqcOkw6HDkeKAuThqw67vrILCrOKIng5A4oChNMOTQMOn4omlXOKInljDlMOVWCJv77+9w6DiiYhkw6U0F1BudO+sgsSxEcOZRlheb8OTdBVzw59dTcOiL8Omw4rDliDihKJcdWUbcRouw4F6GcKlw4/DrQonw7bCtHMgLSAow4vCtuKJoFDihKLFkj9WKHIrw4jCvzbigKLCq8O4w7jDlUjiiaXDqVFIy5lFNEw3y4dkf8Oyw5pgLEfLncO7wrXDgVzDjMK14omlw43Lh0Xil4o6w6Fpy5hr4oC6y5p/VuKIjzXDmeKIglHDseKIkcOg

Making the JWT (JSON Web Token), The Easy Way!!

Now that we know the background of how the JWT is made, we can look at an easy (only manual though) way to do that. The easiest way will be to use the JWT playground jwt.io. Let us look at the steps:

Header:

{"alg":"RS256","typ":"JWT"}

ClaimSet:

{

"iss": "<SERVICE ACCOUNT EMAIL ADDRESS>",

"scope": "<PERMISSIONS THAT YOU NEED>",

"aud": "https://oauth2.googleapis.com/token",

"exp": <EXPIRY TIME IN UNIX FORMAT>,

"iat": <CREATED TIME IN UNIX FORMAT>

}

Once the entries are filled, the JWT will automatically appear in the left dialogue box.

Hitting the Token Endpoint (Get the Access Token)

Once we have the JWT ready, we can go ahead and make a request to the GCP Token Endpoint to get an access token.

The request will be an HTTPS POST request to https://oauth2.googleapis.com/token

Host: oauth2.googleapis.com

Content-Type: application/x-www-form-urlencoded

Headers:

grant_type: urn:ietf:params:oauth:grant-type:jwt-bearer

assertion: <JWT TOKEN>

Hitting GCP Cloud APIs

Now that we have the access token, we can go ahead and use any GCP Cloud API. We just need to add the below Header to the request:

Authorization: Bearer <Access Token>

In the below example am hitting the “Object List” Endpoint: Which lists all the objects in the given bucket.

You can refer to the Google Cloud APIs to get more information about all the available endpoints.

Conclusion

In this article, we learnt about making REST Request to GCP Cloud APIs with Service Accounts. We saw how this can be done without using any libraries.

We can use this knowledge to make GCP Cloud API requests from any API Client like Postman or from some proprietary coding languages (with higher restrictions or no libraries).

Thanks for reading!!