Machine Learning on GCP — Choosing GPUs to Train Your Models | by Komal Venkatesh Ganesan | Towards Data Science

Google cloud platform offers a wide range of GPU options to choose from. GPUs can boost your ML processing, especially when it comes to matrix computations. The CPU on our machine, however, isn’t optimized for this. They struggle with image or video processing and can often delay the model training process.
In Tensorflow/Keras, the convolutional networks with NCHW (channels first) input require GPUs to run.
Most of my ML experiments revolve around AI gaming using NEAT, Deep-Q reinforcement learning or similar techniques. These involve intense matrix manipulations of game frames which CPUs struggle with.
After making the switch to GCP GPUs, the gaming models that took around weeks to train on CPU finished within a few days on GPU. And because they run on the cloud, it makes it easier to monitor your models without logging in by simply exposing your Tensorboard using an external IP.
Here are the GPU options available at the moment (Aug 10th, 2020) and how much they cost: —
NVIDIA® Tesla® T4: $255.50/month NVIDIA® Tesla® K80: $328.50/monthNVIDIA® Tesla® P4: $438.00/monthNVIDIA® Tesla® P100: $1,065.80/monthNVIDIA® Tesla® V100: $1,810.40/month
More information here: https://cloud.google.com/compute/docs/gpus
This assessment covers only Tesla T4, K80 and P4. The P100 and V100 have been excluded simply because they are overkill and too expensive for small projects and hobbyists.
Note: Not all GPUs are available in all GCP regions. You might want to change the region depending on the GPU you are after.
Tesla T4 is the holy grail — it’s both cheap and efficient. It has become my first choice while setting up GCP environments for any ML models.
Here’s the Tensorboard output from running DDQN on Breakout using Tensorflow 2.2.0-rc0 (on Tesla T4): Max Score Achieved: 406
Model: "sequential"_________________________________________________________________Layer (type)                 Output Shape              Param #   =================================================================conv2d (Conv2D)              (None, 32, 20, 20)        8224      _________________________________________________________________conv2d_1 (Conv2D)            (None, 64, 9, 9)          32832     _________________________________________________________________conv2d_2 (Conv2D)            (None, 64, 7, 7)          36928     _________________________________________________________________flatten (Flatten)            (None, 3136)              0         _________________________________________________________________dense (Dense)                (None, 512)               1606144   _________________________________________________________________dense_1 (Dense)              (None, 4)                 2052      =================================================================
Link to Tensorboard:https://tensorboard.dev/experiment/1yrW70A0QHertoJDVZk1ig/#scalars&tagFilter=rew
You can observe that the training process achieved scores nearing 300 in the game of breakout in just ~2 days.
Another great aspect to Tesla T4 is that you can afford to run them a little longer to grab sustained discounts because they are cheap(discussed later).
The Tesla T4 is a relatively new model compared to both K80, P4.
Launch dates: Tesla T4 — September 2018 Tesla P4— September 2016Tesla K80s — November 2014
How these would have performed under Tensorflow distributed training on multiple GPUs has not been assessed here.
But as far as the performance goes, the DDQN for models for Atari Breakout took a similar time to complete on all 3 GPUs with no major difference in performance.
Github Link to my project: https://github.com/komal-SkyNET/ai-neural-networks/tree/master/ai-atari
Sustained discounts offer a significant discount just to keeping your instances running a little longer. This is calculated monthly.
In the image shown, notice that you can score almost 80$ sustained discount just for running a VM with Tesla T4.
More information here: https://cloud.google.com/compute/docs/sustained-use-discounts
Here’s a comparison with/without sustained discounts from running GPU workloads for a month. Notice the charts.
Note that GPU options are based on regions. If you find a particular GPU option missing in the drop-down, you might want to try a different region/zone.
Tesla T4 is both cheap and efficient and would easily be the first choice, followed by K80 and P4. However, their performance in distributed training (using multiple GPUs) might differ and is not covered in this assessment.
Sustained discounts: You can score huge discounts for just running your workloads a little longer. A forecast estimate is usually shown while provisioning the virtual machines.