Machine Learning in Production
By: Asjad Anis
Tags: Deployment, TensorFlow, Google Cloud Platform, Deep Learning, Machine Learning

Let’s get started, there are a couple of steps in order to deploy our model, each of these can either be performed via the Cloud-Console or the Clouds-SDK.

Uploading your Model to a Cloud Storage Bucket

First, we need to upload our model to a cloud storage bucket, for this, you can simply head over to your GCP-project and navigate to the storage service. From there you can create a bucket and give it a meaningful name, I have called mine mnist-digit.

Screenshot of Google Cloud Platform by the Author

Now that our bucket is created we can simply upload our model to the bucket using the below command.

gsutil cp -r <path-to-tf-js-model-dir> gs://<bucket-name>/

Please note that a good convention for organizing your models is to use a sub-folder in your model's directory which highlights the model version that can be seen below.

Screenshot of Google Cloud Platform by the Author

If you re-train your model or fine-tune the model architecture and export a different version you can create a separate folder here named 2 which will contain the updated model. The contents of this directory will be the entire saved model-directory as seen below.

Screenshot of Google Cloud Platform by the Author

Note: Your bucket must be in the same region as the AI-platform otherwise you will have to explicitly grant access to the AI-Platform.

Perfect we are done with the first step in getting our model uploaded to a cloud storage bucket, next comes

Creating an AI-Platform Model Resource

Before deploying the model first head over to your GCP project APIs & Services → Enable APIs & Services → AI Platform Training & Prediction API and enable this API.

Screenshot of Google Cloud Platform by the Author

Next head over to the AI-Platform service from the left side-bar in GCP and navigate to the Models tab.

Screenshot of Google Cloud Platform by the Author

Click on New model to create a model resource and give your model a unique name within the project, select a region this will be the region where the jobs will be running, once selected hit Create. I have called mine mnist_digit_recog

Screenshot of Google Cloud Platform by the Author

Now if you go to the Models tab you should see your model resource. Next, we will be creating a version resource for our model resource that we just created.

Creating an AI-Platform Version Resource

Now we will be creating a version with the trained model we uploaded to the cloud-storage bucket in the first step. For this click on the model resource we just created and hit create a version.

Screenshot of Google Cloud Platform by the Author

Next, select a version for your model and set the python version and framework. I trained my model on python version 3.7.4. To check your python version you can run the below command in terminal

python --version (for python-2)

python3 --version (for python-3)

To check tensorflow version you can run

import tensorflow as tf

tf.__version__

Next, configure the path to your cloud storage bucket where you uploaded the model make sure you configure the path to the folder containing the model files which in our case was the directory named “1”.

Screenshot of Google Cloud Platform by the Author

Once you have configured it click save, this will take up some time and once it's finished you should see a green checkmark next to your deployed version as shown below

Screenshot of Google Cloud Platform by the Author

Now we are all set to make predictions with our deployed instance. Btw if you are a fan of command-line then all these steps can be performed via the google cloud sdk. To explore the command line options please visit the official-docs.

Inference using AI-Platform

Now that we have our model deployed let's make some inferences, click on your model version and you will be taken to this page with additional details about the model and traffic.

Screenshot of Google Cloud Platform by the Author

Here we can see that our deployed model is expecting an object with an instances key containing the test data. To learn more about getting online predictions do give this link a visit which highlights how you should input your data and get predictions in response.

We will be using the google api client for python to get inference from our deployed model. If you are working in some other programming-language you can check out this page to see if the language is supported.

To get predictions we first need access to make calls to the model for which we need a credentials file. To set up the credentials in google cloud console, head over to APIs & Services and go to the credentials tab. From there click create new credentials and select service account. Give your service account a name and assign an appropriate role. I assigned my service account ML Engine Admin and Storage Object Admin. Once done click create.

Screenshot of Google Cloud Platform by the Author

Once created, click your service account and scroll down to Add Key click the button and select create a new key and set the format as JSON, and hit create.

Download the JSON on disk and rename it for your own convenience. I have renamed it as [project-name]_credentials.json.

Next we will install the python client libraries using the below command

pip3 install google-api-python-client oauth2client

For more details you can visit the official docs for python client here

Once installed we can use the below code snippet to request our model.

prediction from AI-Platform

Note that I am specifying the api_endpoint with the specific region in which I created my model resource. You can check that out quickly by going to your model resource in AI-Platform.

Screenshot of Google Cloud Platform by the Author

And that’s it now you can deploy your machine learning models on AI-Platform whether they were trained using AI-Platform or not, and consume them from any application. In the last article of this series, we will look at how we can use cloud-functions for serving our models.

If you find this article useful do share it in your circle and leave a clap. For any queries or suggestions, you can leave a comment or you can connect with me on LinkedIn.

Happy Coding.