Model with TensorFlow and Serve on Google Cloud Platform | by MA Raza, Ph.D. | Towards Data Science

In this guide, we learn how to develop a TensorFlow model and serve it on the Google Cloud Platform (GCP). We consider a regression problem of predicting the earnings of products using a three-layer neural network implemented with TensorFlow and Keras APIs.
The key learning outcomes of this guide are
Build, compile and fit Model in TensorflowSet up tensorboard in Google colabSave, load and predict on unseen dataDeploy the TensorFlow model on Google Cloud platform
Build, compile and fit Model in Tensorflow
Set up tensorboard in Google colab
Save, load and predict on unseen data
Deploy the TensorFlow model on Google Cloud platform
In this guide, we will use the TensorFlow 2.1.0 and Google colab runtime environment. We will be using the google colab for training the models using TensorFlow and Keras APIS. Starting from TensorFlow 2.0, Keras is now part of the TensorFlow package. Google colab offers the training of machine learning models on free GPU and TPU. Therefore, it is recommended the environment to get hands-on experience of training your deep learning models on GPU and TPU. Moreover, we can set up tensorboard to get a better understanding of the model training. This guide comes with a working copy of Google colab to get you started quickly.
# Install TensorFlow >2.0!pip install tensorflow==2.1.0
Import the necessary python libraries along with TensorFlow and Keras functions
# Import librariesimport pandas as pdfrom sklearn.preprocessing import MinMaxScalerimport osimport numpy as npfrom __future__ import absolute_import, division, print_function, unicode_literalsimport tensorflow as tffrom tensorflow.keras.layers import Dense, Flatten, Conv2Dfrom tensorflow.keras import Modelfrom tensorflow.keras.models import Sequential, load_model
Load the tensorboard extension
# Load tensorboard %load_ext tensorboard
To access the data in Google colab environment, there are various ways to do it. One of my favourite is to upload data to your Google Drive and mount Google Drive in colab.
You can download the data for training and testing from data_deep_learning. You can try other open-source datasets to try.
We need to mount the google drive to read the data stored in google drive. Below is the simplest way to mount it. You will be asked to enter the token generated by your access procedure. You can consult the article “Using Google Colab GPU VM + Drive as persistent storage for long Deep Learning training runs” for more information on using Google drive as data storage for colab.
I have already uploaded the data to my google drive for easy access. You may follow the same.
from google.colab import drivedrive.mount('/content/drive')Go to this URL in a browser: Enter your authorization code:··········Mounted at /content/drive
I have uploaded the training and validation data to my google drive. You can try with any other open-source data for the sake of practice. For this tutorial, data is already prepared into training and testing sets however in practical problems, most likely data has to be cleaned and prepared on task by task basis. In my opinion, almost 50% of the work for Machine Learning Scientists is to prepare the data compatible with machine learning models
Make sure no information leakage is happening across train, valid and test splits.
Recently, Jason Brownlee has published a series of articles on Data Preparation on his website https://machinelearningmastery.com/blog/.
# I have already uploaded the data to my google drivedata_path= '/content/drive/My Drive/data_deep_learning_tensorflow/'output_dir =  '/content/drive/My Drive/Deep_Learning_Course/Ex_Files_Building_Deep_Learning_Apps/'
Reading the training and test data for our problem. The target is to predict the total_earnings of given products given below listed features.
# Reading the traing and test datatraining_data_df = pd.read_csv(data_path + 'sales_data_training.csv')test_data_df = pd.read_csv(data_path + 'sales_data_test.csv')training_data_df.head()
Normalizing is recommend the practice for most machine learning models and has proven to be working. Scikit-learn comes with a handful of industry-trusted transformations to be applied for preprocessing the data.
MinMaxScaler is one of the frequently used transformations implemented in the Scikit-learn transformation module. It scales the data between 0 and 1. It is advisable to normalize the training and test data before feeding into the TensorFlow model training.
scalar = MinMaxScaler(feature_range=(0,1))scaled_training = scalar.fit_transform(training_data_df)scaled_test = scalar.fit_transform(test_data_df)scaled_training_df = pd.DataFrame(data = scaled_training, columns=training_data_df.columns)scaled_test_df = pd.DataFrame(data = scaled_test, columns=training_data_df.columns)
let us have a look on scaled data
scaled_training_df.head()
For a supervised machine learning model, we split the features and target into X and y NumPy arrays. TensorFlow models are no different and consume the data in this format.
# Splitting the feature and target columns. Using this dataset, we aim to predict total_earningsX = scaled_training_df.drop('total_earnings', axis=1).valuesY = scaled_training_df['total_earnings'].values
To define the deep learning model, we use the Keras API module shipped with TensorFlow 2.1.0 We start with a basic model with two intermediate relu layers. The input dimension is 9 and the output dimension is one.
# Define the Modelmodel = Sequential()model.add(Dense(50, input_dim=9, activation = 'relu'))model.add(Dense(100, activation = 'relu'))model.add(Dense(50, activation = 'relu'))model.add(Dense(1, activation = 'linear'))model.compile(loss = 'mean_squared_error', optimizer = 'adam')
Display the defined model summary
model.summary()Model: "sequential_1"_________________________________________________________________Layer (type)                 Output Shape              Param #   =================================================================dense_4 (Dense)              (None, 50)                500       _________________________________________________________________dense_5 (Dense)              (None, 100)               5100      _________________________________________________________________dense_6 (Dense)              (None, 50)                5050      _________________________________________________________________dense_7 (Dense)              (None, 1)                 51        =================================================================Total params: 10,701Trainable params: 10,701Non-trainable params: 0_________________________________________________________________
Prepare the test features and target similar to training data
X_test = scaled_test_df.drop('total_earnings', axis=1).valuesY_test = scaled_test_df['total_earnings'].values
Set up the callbacks for tensorboard
# Setup for tensorboardfrom datetime import datetimelogdir="logs/scalars/" + datetime.now().strftime("%Y%m%d-%H%M%S")tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=logdir)
Train the model with verbose=0 and epochs=50 and testing data as a validation set.
# Train the networktraining_history = model.fit(X, Y,                             epochs = 50,                             shuffle=True,                             verbose=0,                             validation_data=(X_test, Y_test),                             callbacks=[tensorboard_callback])print("Average test loss: ", np.average(training_history.history['loss']))Average test loss:  0.0003906410974522077
Let us display the training/validation errors in tensorboard
# Click on the view output full screen from dropdown cell menu if tensorboard does not show up itself%tensorboard --logdir logs/scalars
Validation of the training is one of the very important steps in any Machine Learning framework. Here, we use validation/Test error interchangeably.
test_error_rate = model.evaluate(X_test,                                 Y_test,                                 verbose=0)print('The mean squared error for test data is {}'.format(test_error_rate))The mean squared error for test data is 0.09696226567029953
Let us read the sample input data and test our model to predict and save the model locally for future use.
# Read the values of features for new product and test on our traioned model df_new_products = pd.read_csv(data_path + 'proposed_new_product.csv').values# Use our trained moel to predictpredictions = model.predict(df_new_products)# Scaling up the earnings prediction using our Normalisation Parameterspredictions = predictions + 0.1159predictions = predictions/0.0000036968print('Earnings predictions for Proposed product - ${}'.format(predictions))Earnings predictions for Proposed product - $[[270639.12]]# Save the Modelmodel.save(output_dir + 'trained_model.h5')print('Trained Model saved at {}'.format(output_dir))Trained Model saved at /content/drive/My Drive/Deep_Learning_Course/Ex_Files_Building_Deep_Learning_Apps/
Once we are satisfied with the model, we can save this model locally and can be used to make predictions for unseen data. This model can be hosted on the cloud and you can send the input data to get the predictions.
# Importing saved modelmodel_trained = load_model(output_dir + 'trained_model.h5')# Read the values of features for new product and test on our traioned model df_new_products = pd.read_csv(data_path + 'proposed_new_product.csv').valuespredictions = model_trained.predict(df_new_products)predictions = predictions + 0.1159predictions = predictions/0.0000036968print('Earnings predictions for Proposed product - ${}'.format(predictions))Earnings predictions for Proposed product - $[[270639.12]]
Once, we have the trained model the next task is to deploy it to serve the clients. There are various deployment options available however in this section we focus on deploying it on Google Cloud AI platform.
The pre-requisites to deploy the TensorFlow models on google cloud are
Familiarity with Google Cloud Projects
Basic understanding of storage buckets and it gsutils command-line tool
Basic Understanding of gcloud command-line tool to interact with Google Cloud
A Trained Tensorflow 2.1.0 Model
Read Below Guide for details on how to deploy machine learning model
How-to deploy TensorFlow 2 Models on Cloud AI Platform
Authentication for your google account through colab
from google.colab import authauth.authenticate_user()
Create a Google cloud project if not done before and bucket for storage of model.
# GCP project nameCLOUD_PROJECT = 'gcpessentials-rz'BUCKET = 'gs://' + CLOUD_PROJECT + '-tf2-models'
Set the desired project as a default environment variable to be used in colab environment.
# Set the gcloud consol to $CLOUD_PROJECT 9Environment Variable for your Desired Project)!gcloud config set project $CLOUD_PROJECTUpdated property [core/project].
Create the Bucket using gsutil command-line utility. You can use the python API to create a storage bucket. Let us stick to the command line method for now.
# Create the storage bucket for tf2 Models!gsutil mb $BUCKETprint(BUCKET)Creating gs://gcpessentials-rz-tf2-models/...gs://gcpessentials-rz-tf2-models
Save our TensorFlow model into a storage bucket just created. You can confirm the model files are uploaded from the google console graphical user interface.
MODEL = 'earnings-prediction'model.save(BUCKET + f'/{MODEL}', save_format='tf')WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.Instructions for updating:If using Keras pass *_constraint arguments to layers.INFO:tensorflow:Assets written to: gs://gcpessentials-rz-tf2-models/earnings-prediction/assets
There are two ways to set up a model serving on google cloud. The first one is using the GCP graphical console and the other one is using gcloud command-line tool as described in How-to deploy TensorFlow 2 Models on Cloud AI Platform. I have followed the GCP console for this deployment. Some key points
Make sure your Model data is upload in the Storage Bucket
It is recommended to use the same region for your model and storage bucket (sometimes, it creates problems).
Be patient, creating model/version might take a few minutes
When doing the first time, carefully read all the instructions
!gcloud ai-platform versions create $VERSION \  --model $MODEL \  --origin $MODEL_DIR \  --runtime-version=2.1 \  --framework='tensorflow' \  --python-version=3.7
Once the model is deployed and you can see the green tick, it is time to test the model for predictions. Some important points to consider.
Test the model prediction using Test & Use tab under your version or model tab.
Input JSON data depends on how you define your model
# [Reference](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/ml_engine/online_prediction/predict.py)import googleapiclient.discoverydef predict_json(project, model, instances, version=None):    service = googleapiclient.discovery.build('ml', 'v1')    name = 'projects/{}/models/{}'.format(project, model)    if version is not None:        name += '/versions/{}'.format(version)    response = service.projects().predict(        name=name,        body={'instances': instances}    ).execute()    if 'error' in response:        raise RuntimeError(response['error'])    return response['predictions']
Load the unseen data to predict
df_new_products = pd.read_csv(data_path + 'proposed_new_product.csv')df_new_products
Conver the data into JSON format to be compatible with predict_json routine written above.
tdata_instances = {'dense_input':df_new_products.values[0].tolist()}predictions_gcloud = predict_json(CLOUD_PROJECT, 'earnings_prediction', tdata_instances, version='v1')predictions_gcloud = predictions_gcloud[0]['dense_3'][0] + 0.1159predictions_gcloud = predictions_gcloud/0.0000036968print('Earnings predictions for Proposed product - ${}'.format(predictions_gcloud))Earnings predictions for Proposed product - $261848.21778936993
Follow the below Google colaboratory notebook to reproduce and practice this guide.
In this guide, we have learned about training the deep learning models with TensorFlow 2.1.0 and deploying the trained model on the google cloud platform. Below are the highlights of the guide.
Install the TensorFlow 2.1..0 in google colab
Setup tensorboard in google colab
Build, Compile and Train a model using TensorFlow
Evaluate and save the trained model
Load saved model to predict on unseen data
Deploy the Model on Google Cloud
Make Prediction using deployed Model in the cloud