Develop, Train and Deploy TensorFlow Models using Google Cloud AI Platform | by MA Raza, Ph.D. | Towards Data Science

The TensorFlow ecosystem has become very popular for developing applications involving deep learning. One of the reasons is that it has a strong community and a lot of tools have been developed around the core library to support developers. In this tutorial, I will guide you through how to prototype models in google colab, train it on Google Cloud AI Platform, and deploy the finalized model on Google Cloud AI Platform for production. I will include the working Google colab notebooks to recreate the work.
Google colab is a free resource for prototyping models in TensorFlow and comes with various runtime. Preparing a machine with GPU or TPU could be costly to start with however users can start with free GPU with google colab. Bear in mind, colab has limited resources and might not be suitable for properly training models requiring large compute resources. Nonetheless, colab is a perfect tool for prototyping your models and some initial experimentation.
Once you are satisfied with your model pipeline, it is time to train the model with the proper number of EPOCHS and full datasets. As you might know, training deep learning models requires a long time and a large cluster of CPU’s GPUs or TPU’s. One option is that users set up their own computing cluster which is costly and time-consuming most of the time. Another option is to use cloud computing to training the model and pay as you go. TensorFlow team has released a package called Tensorflow Cloud to let users train the models on the Google Cloud platform without any hassle. I have followed steps from Train your TensorFlow model on Google Cloud using TensorFlow Cloud blog and will share some issues I have faced to make it work. Some of the pre-requisites are ad defined in the project guidelines for submitting a training job to the GCP platform.
Python >= 3.5
A Google Cloud project
An authenticated GCP account
Google AI platform APIs enabled for your GCP account. We use the AI platform for deploying docker images on GCP.
Either a functioning version of docker if you want to use a local docker process for your build, or create a cloud storage bucket to use with Google Cloud build for docker image build and publishing.
After creating the GCP project, follow the below steps to config the environment with Google cloud authentication.
# Authenticate from google.colab import authauth.authenticate_user()
Set PROJECT_ID in the environment
os.environ['PROJECT_ID']='gcpessentials-rz'!gcloud config set project $PROJECT_ID
Create the Service Account and set some permissions needed for tensorflow-cloud the package. Download the service account key and add as an environment variable as GOOGLE_APPLICATION_CREDENTIALS. These credentials will be needed to submit the job to the google cloud platform.
os.environ['PROJECT_ID']='gcpessentials-rz'!gcloud config set project $PROJECT_IDos.environ['SA_NAME']='gcpessentials-rz'!gcloud iam service-accounts create $SA_NAME!gcloud projects add-iam-policy-binding $PROJECT_ID \--member serviceAccount:$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com \--role 'roles/editor'!gcloud iam service-accounts keys create key.json --iam-account $SA_NAME@$PROJECT_ID.iam.gserviceaccount.comos.environ['GOOGLE_APPLICATION_CREDENTIALS']='key.json'
Even after following the above steps, I have to enable service Accounts and cloud_build Status under Cloud Build settings. Below is the snapshot of my project settings.
The next step is to create GCP_BUCKET to store the data needed for submitting the job to Google Cloud.
BUCKET = 'tf2-model-training'!gsutil mb gs://$BUCKET
Once we have set up the environment in our google colab and created the GCP_BUCKET, we have to prepare the notebook with training the model. Below are some key points to consider while preparing a notebook.
Test the code in a notebook using google colab for a small number of EPOCHSMake sure, there are no errors in the notebook and remove any unnecessary codePrepare the requirements.txt and upload to google colab environment.Save the trained model in GCP_BUCKET (will be used for Deployment)
Test the code in a notebook using google colab for a small number of EPOCHS
Make sure, there are no errors in the notebook and remove any unnecessary code
Prepare the requirements.txt and upload to google colab environment.
Save the trained model in GCP_BUCKET (will be used for Deployment)
Once we are ready with the notebook, we submit the training on Google Cloud platform using tensorflow-cloud package.
import tensorflow_cloud as tfcBUCKET = 'tf2-model-training'labels= {'phase': 'test','owner': 'raza',}tfc.run(requirements_txt="requirements.txt",distribution_strategy="auto",chief_config='auto',docker_image_bucket_name=BUCKET,job_labels=labels,)
The above code will convert the notebook into identifier-Fg45-a.py and submit the training on the Google Cloud platform in the docker form. After you submit the job, you can see the message like the below figure.
Wait for a few minutes before the job starts actually training
By clicking on the provided link, you will see a page like below figure
You can see the logs by clicking the view logs and will look like the below figure. Logs are helpful to see what sort of Exceptions happened in your code.
Before submitting the huge job, test your pipeline submission script for few EPOCHS and then submit the complete job
If all goes well, your model will be trained using the Google AI platform and saved in GCP_BUCKET along with other resources.
Below is the working notebook to run the training on Google AI Platform using the TensorFlow-Cloud package.
# Install tensorflow
# !pip install tensorflow==2.3.0
# !pip install tensorflow_cloud

# Import libraries
# from __future__ import absolute_import, division, print_function, unicode_literals
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
import os
import numpy as np
from datetime import datetime
import tensorflow as tf
from tensorflow.keras.layers import Dense, Flatten, Conv2D
from tensorflow.keras import Model
from tensorflow.keras.models import Sequential, load_model
import tensorflow_cloud as tfc

#Load the tensorboard extension
# Load tensorboard 
# %load_ext tensorboard

# Upload requirements.txt file
# from google.colab import files
# uploaded = files.upload()

Saving requirements.txt to requirements (1).txt

BUCKET = 'tf2-model-training'
labels= {
       'phase': 'test',
       'owner': 'User',
    }
tfc.run(
    requirements_txt="requirements.txt",
    distribution_strategy="auto",
    chief_config='auto',
    docker_image_bucket_name=BUCKET,
    job_labels=labels,
)

INFO:tensorflow_cloud.core.containerize:Uploading files to GCS.
INFO:tensorflow_cloud.core.containerize:Building and publishing docker image using Google Cloud Build: gcr.io/gcpessentials-rz/tf_cloud_train:f96196a6_a0c1_44f1_a9f4_ec99a6880da6
INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/cloudbuild/v1/rest
INFO:googleapiclient.discovery:URL being requested: POST https://cloudbuild.googleapis.com/v1/projects/gcpessentials-rz/builds?alt=json
INFO:googleapiclient.discovery:URL being requested: GET https://cloudbuild.googleapis.com/v1/projects/gcpessentials-rz/builds/f6f3e56d-e615-4ee1-b5d2-4c3108470858?alt=json
INFO:googleapiclient.discovery:URL being requested: GET https://cloudbuild.googleapis.com/v1/projects/gcpessentials-rz/builds/f6f3e56d-e615-4ee1-b5d2-4c3108470858?alt=json
INFO:googleapiclient.discovery:URL being requested: GET https://cloudbuild.googleapis.com/v1/projects/gcpessentials-rz/builds/f6f3e56d-e615-4ee1-b5d2-4c3108470858?alt=json
INFO:googleapiclient.discovery:URL being requested: GET https://cloudbuild.googleapis.com/v1/projects/gcpessentials-rz/builds/f6f3e56d-e615-4ee1-b5d2-4c3108470858?alt=json
INFO:googleapiclient.discovery:URL being requested: GET https://cloudbuild.googleapis.com/v1/projects/gcpessentials-rz/builds/f6f3e56d-e615-4ee1-b5d2-4c3108470858?alt=json
INFO:googleapiclient.discovery:URL being requested: GET https://cloudbuild.googleapis.com/v1/projects/gcpessentials-rz/builds/f6f3e56d-e615-4ee1-b5d2-4c3108470858?alt=json
INFO:googleapiclient.discovery:URL being requested: GET https://cloudbuild.googleapis.com/v1/projects/gcpessentials-rz/builds/f6f3e56d-e615-4ee1-b5d2-4c3108470858?alt=json
INFO:googleapiclient.discovery:URL being requested: GET https://cloudbuild.googleapis.com/v1/projects/gcpessentials-rz/builds/f6f3e56d-e615-4ee1-b5d2-4c3108470858?alt=json
INFO:googleapiclient.discovery:URL being requested: GET https://cloudbuild.googleapis.com/v1/projects/gcpessentials-rz/builds/f6f3e56d-e615-4ee1-b5d2-4c3108470858?alt=json
INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/ml/v1/rest
INFO:googleapiclient.discovery:URL being requested: POST https://ml.googleapis.com/v1/projects/gcpessentials-rz/jobs?alt=json

Job submitted successfully.
Your job ID is:  tf_cloud_train_28b2567c_c6e8_4e63_b86d_1cddc2a4a5bd
Please access your job logs at the following URL:
https://console.cloud.google.com/mlengine/jobs/tf_cloud_train_28b2567c_c6e8_4e63_b86d_1cddc2a4a5bd?project=gcpessentials-rz

# Traing and Test data from github
data_path= 'https://raw.githubusercontent.com/amjadraza/blogs-data/master/deep_leraning_tensorflow/'
output_dir =  'tensorflow_out/'
os.makedirs(output_dir, exist_ok=True)
# Reading the traing and test data
print('Loading the training data')
training_data_df = pd.read_csv(data_path + 'sales_data_training.csv')
test_data_df = pd.read_csv(data_path + 'sales_data_test.csv')

scalar = MinMaxScaler(feature_range=(0,1))
scaled_training = scalar.fit_transform(training_data_df)
scaled_test = scalar.fit_transform(test_data_df)

scaled_training_df = pd.DataFrame(data = scaled_training, columns=training_data_df.columns)
scaled_test_df = pd.DataFrame(data = scaled_test, columns=training_data_df.columns)
# Splitting the feature and target columns. Using this dataset, we aim to predict total_earnings
X = scaled_training_df.drop('total_earnings', axis=1).values
Y = scaled_training_df['total_earnings'].values

# Define the Model
print('Building the Model')
model = Sequential()

model.add(Dense(50, input_dim=9, activation = 'relu'))

model.add(Dense(100, activation = 'relu'))

model.add(Dense(50, activation = 'relu'))

model.add(Dense(1, activation = 'linear'))

model.compile(loss = 'mean_squared_error', optimizer = 'adam')

X_test = scaled_test_df.drop('total_earnings', axis=1).values
Y_test = scaled_test_df['total_earnings'].values

# if not tfc.remote():
#   from google.colab import auth
#   auth.authenticate_user()
#   # GCP project name
#   os.environ['PROJECT_ID']='gcpessentials-rz'
#   !gcloud config set project $PROJECT_ID
#   os.environ['SA_NAME']='gcpessentials-rz'
#   !gcloud iam service-accounts create $SA_NAME
#   !gcloud projects add-iam-policy-binding $PROJECT_ID \
#     --member serviceAccount:$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com \
#     --role 'roles/editor'
#   !gcloud iam service-accounts keys create key.json --iam-account $SA_NAME@$PROJECT_ID.iam.gserviceaccount.com
#   os.environ['GOOGLE_APPLICATION_CREDENTIALS']='key.json'
#   # Create the storage bucket for tf2 Models
#   # !gsutil mb $BUCKET
#   # !gcloud auth login
#   # os.environ['PROJECT_ID']='gcpessentials-rz'
#   # !gcloud config set project $PROJECT_ID
#   !gsutil mb gs://$BUCKET

if tfc.remote():
  # Collecting the logs in Google Cloud Storage
  MODEL_PATH = "TF"
  checkpoint_path = os.path.join("gs://", BUCKET, MODEL_PATH, "save_at_{epoch}")
  tensorboard_path = os.path.join(
      "gs://", BUCKET, "logs", datetime.now().strftime("%Y%m%d-%H%M%S")
  )
  gcp_callbacks = [
      tf.keras.callbacks.ModelCheckpoint(checkpoint_path),
      tf.keras.callbacks.TensorBoard(log_dir=tensorboard_path, histogram_freq=1),
      tf.keras.callbacks.EarlyStopping(monitor="val_loss", patience=3),
  ]
  epochs = 50
  callbacks = gcp_callbacks
else:
    # Setup for tensorboard
  tensorboard_path="logs/scalars/" + datetime.now().strftime("%Y%m%d-%H%M%S")
  checkpoint_path = os.path.join("logs","ckp", "save_at_{epoch}")
  tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=tensorboard_path)

  local_callbacks = [
      tf.keras.callbacks.ModelCheckpoint(checkpoint_path),
      tf.keras.callbacks.TensorBoard(log_dir=tensorboard_path, histogram_freq=1),
      tf.keras.callbacks.EarlyStopping(monitor="val_loss", patience=3),
  ]
  epochs = 1
  callbacks = local_callbacks
 
training_history = model.fit(X, Y,
                             epochs = epochs,
                             shuffle=True,
                             verbose=1,
                             validation_data=(X_test, Y_test),
                             callbacks=[callbacks])
print("Average test loss: ", np.average(training_history.history['loss']))

if tfc.remote():
  MODEL = 'earnings-prediction'
  MODEL_SAVE_BUCKET = 'gs://gcpessentials-rz-tf2-models-serving'
  model.save(MODEL_SAVE_BUCKET  + f'/{MODEL}', save_format='tf')

 1/32 [..............................] - ETA: 0s - loss: 0.0078WARNING:tensorflow:Callbacks method `on_train_batch_end` is slow compared to the batch time (batch time: 0.0044s vs `on_train_batch_end` time: 0.0181s). Check your callbacks.

WARNING:tensorflow:Callbacks method `on_train_batch_end` is slow compared to the batch time (batch time: 0.0044s vs `on_train_batch_end` time: 0.0181s). Check your callbacks.

12/32 [==========>...................] - ETA: 0s - loss: 0.0081WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.

INFO:tensorflow:Assets written to: logs/ckp/save_at_1/assets

INFO:tensorflow:Assets written to: logs/ckp/save_at_1/assets

32/32 [==============================] - 1s 26ms/step - loss: 0.0056 - val_loss: 0.0048
Average test loss:  0.005643493961542845

Once the model is trained and finalized, users would like to deploy the model on a scale-able infrastructure. Google Cloud also provides the necessary infrastructure to deploy TensorFlow models on its platform without a lot of modifications. I will show you how to deploy the model on a google cloud platform. At the end of the training, we save the finalized model in Google Cloud Bucket. Below are the steps to deploy the model.
Configure the models serving using AI Platform
Perform the prediction on deployed model
Use the below set of commands to configure the model for deployment withing Google colab. I have explained in detail how to deploy the model on Google Cloud in Model with TensorFlow and Serve on Google Cloud Platform.
I have configured the model to serve as v2 following the same steps in the above tutorial.
Once the model is deployed and you can see the green tick, it is time to test the model for predictions. Some important points to consider.
Test the model prediction using Test & Use tab under your version or model tab.
Input JSON data depends on how you define your model
# [Reference](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/ml_engine/online_prediction/predict.py)import googleapiclient.discoverydef predict_json(project, model, instances, version=None):    service = googleapiclient.discovery.build('ml', 'v1')    name = 'projects/{}/models/{}'.format(project, model)    if version is not None:        name += '/versions/{}'.format(version)    response = service.projects().predict(        name=name,        body={'instances': instances}    ).execute()    if 'error' in response:        raise RuntimeError(response['error'])    return response['predictions']
Load the unseen data for predictions
df_new_products = pd.read_csv(data_path + 'proposed_new_product.csv')tdata_instances = {'dense_input':df_new_products.values[0].tolist()}predictions_gcloud = predict_json(CLOUD_PROJECT, 'earnings_prediction', tdata_instances, version='v2')predictions_gcloud = predictions_gcloud[0]['dense_3'][0] + 0.1159predictions_gcloud = predictions_gcloud/0.0000036968print('Earnings predictions for Proposed product - ${}'.format(predictions_gcloud))Earnings predictions for Proposed product - $259671.15201209177
Below is the working notebook to perform prediction on the deployed model
In this guide, we have learned about training the deep learning models with TensorFlow 2.3.0 using the Google Cloud AI platform with the help of tensorflow-cloud package and deployed the trained model on the Google Cloud AI platform too. Below are some key lessons
Use Google Colab to Building TensorFlow models
Train the Model on Google Cloud AI Platform using TensorFlow-Cloud
Save the trained model in GCP Bucket after training
Set up Model serving on GCP
Make Prediction using deployed model in the cloud