GPU-Accelerated Machine Learning on MacOS | by Riccardo Di Sipio | Towards Data Science

Massively parallel programming is very useful to speed up calculations where the same operation is applied multiple times on similar inputs. If your code involves a number of if or case statements, you may want to run on a CPU using e.g. OpenMPI. If your code involves the generation of random numbers, parallel programming may not be the best solution (however, see here). Otherwise, you are probably in the right place, so keep reading!
Training a neural network involves a very large number of matrix multiplications. This is the quintessential massively parallel operation, which constitutes one of the main reasons why GPUs are vital to Machine Learning. One rule of thumb to remember is that 1K CPUs = 16K cores = 3GPUs, although the kind of operations a CPU can perform vastly outperforms those of a single GPU core. For GPUs, strength is in numbers!
iMac and MacBook Pro computers are equipped with an AMD Radeon GPU card. Unfortunately, this kind of hardware can not be used directly to speed-up calculations that are typical in Machine Learning applications, such as training a CNN. While there isn’t a solution for all possible applications, there is still a workaround for simple architectures based on a parallel programming language called OpenCL.
The two most popular ML frameworks Keras and PyTorch support GPU acceleration based on the general-purpose GPU library NVIDIA CUDA. CUDA only works with NVIDIA GPU cards.
NVIDIA external GPU cards (eGPU) can be used by a MacOS systems with a Thunderbolt 3 port and MacOS High Sierra 10.13.4 or later. Follow this guide to install the eGPU.
To install CUDA on MacOs follow the official documentation. Drivers can be found here.
The project PyOpenCL is probably the easiest way to get started with GP-GPU on a Mac. See also its sister project PyCUDA.
$ pip install pyopencl
The centrepiece of OpenCL is a kernel, which is a function (written in a C-like language) that can be applied to chunks of input data.
For an introduction to parallel programming follow this online course based on CUDA, or buy this book by Tim Mattson et al.
OpenCL is more flexible than CUDA to allow programs to be executed on different architectures. This comes with a price: it is necessary to write some “boilerplate” code to define a context (i.e. what kind of hardware is available), a queue (i.e. a sequence of commands) and a set of memory flags (e.g. READ_ONLY, WRITE_ONLY, etc.). The typical workflow is as follows:
Input data is copied to the GPU memoryGPU buffer memory is reserved to host the result of the calculationsA kernels are executedThe result is copied from the GPU memory to the host memory
Input data is copied to the GPU memory
GPU buffer memory is reserved to host the result of the calculations
A kernels are executed
The result is copied from the GPU memory to the host memory
NB: OpenCL kernels can be executed on both CPUs and GPUs, but if the code is highly optimized for a certain GPU architecture (e.g. with large memory) it may not be completely portable.
Though GPUs are generally faster, the time taken to transfer huge amounts of data from CPU to GPU can lead to higher overhead time depending on the architecture of the processors.
Here’s an example of a fully-functional OpenCL program that calculates the sum of two matrices:
#!/usr/bin/env python# -*- coding: utf-8 -*-from __future__ import absolute_import, print_functionimport numpy as npimport pyopencl as cla_np = np.random.rand(50000).astype(np.float32)b_np = np.random.rand(50000).astype(np.float32)ctx = cl.create_some_context()queue = cl.CommandQueue(ctx)mf = cl.mem_flagsa_g = cl.Buffer(ctx, mf.READ_ONLY | mf.COPY_HOST_PTR, hostbuf=a_np)b_g = cl.Buffer(ctx, mf.READ_ONLY | mf.COPY_HOST_PTR, hostbuf=b_np)kernel = """__kernel void sum(    __global const float *a_g, __global const float *b_g, __global float *res_g){  int gid = get_global_id(0);  res_g[gid] = a_g[gid] + b_g[gid];}"""prg = cl.Program(ctx, kernel).build()res_g = cl.Buffer(ctx, mf.WRITE_ONLY, a_np.nbytes)prg.sum(queue, a_np.shape, None, a_g, b_g, res_g)res_np = np.empty_like(a_np)cl.enqueue_copy(queue, res_np, res_g)# Check on CPU with Numpy:print(res_np - (a_np + b_np))print(np.linalg.norm(res_np - (a_np + b_np)))assert np.allclose(res_np, a_np + b_np)
Keras is one of the most popular deep learning frameworks. It’s very easy to define the architecture of a network using Keras’ functional APIs, run the training and execute inference. However, Keras does not perform the actual calculations by itself, but rather deploys other software libraries to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays. The most common ones are Theano and TensorFlow. In turn, these libraries execute parallel calculations on GPUs using CUDA. As previously states, this comes with the very strong hardware limitation that it works only on NVIDIA cards.
This problem can be partially evaded by using instead the PlaidML library:
$ pip install plaidml-keras
Following installation, execute the setup script (select default unless you know what you’re doing):
$ plaidml-setup
You should see something like this:
PlaidML Setup (0.6.4)Thanks for using PlaidML!Some Notes:  * Bugs and other issues: https://github.com/plaidml/plaidml  * Questions: https://stackoverflow.com/questions/tagged/plaidml  * Say hello: https://groups.google.com/forum/#!forum/plaidml-dev  * PlaidML is licensed under the Apache License 2.0Default Config Devices:   metal_amd_radeon_r9_m380.0 : AMD Radeon R9 M380 (Metal)Experimental Config Devices:   llvm_cpu.0 : CPU (LLVM)   opencl_amd_radeon_r9_m380_compute_engine.0 : AMD AMD Radeon R9 M380 Compute Engine (OpenCL)   opencl_cpu.0 : Intel CPU (OpenCL)   metal_amd_radeon_r9_m380.0 : AMD Radeon R9 M380 (Metal)Using experimental devices can cause poor performance, crashes, and other nastiness.Enable experimental device support? (y,n)[n]:Selected device:    metal_amd_radeon_r9_m380.0Almost done. Multiplying some matrices...Tile code:  function (B[X,Z], C[Z,Y]) -> (A) { A[x,y : X,Y] = +(B[x,z] * C[z,y]); }Whew. That worked.Save settings to /Users/user/.plaidml? (y,n)[y]:Success!
This library supports many but not all Keras layers. If your architecture involves only Dense, LSTM, CNN and Dropout layers, you’re certainly good, otherwise check the documentation.
In principle, all you have to do is to pre-pend the following lines of code to your program to activate the PlaidML backend:
import osos.environ["KERAS_BACKEND"] = "plaidml.keras.backend"
During the execution, you should see some printouts like these:
Using plaindml.keras.backend backendINFO:plaidml:Opening device "metal_amd_radeon_r9_m380.0"
This is an example, adapted from from Keras’ official documentation, that should work out-of-the-box:
#!/usr/bin/env pythonimport osos.environ["KERAS_BACKEND"] = "plaidml.keras.backend"import kerasfrom keras.datasets import mnistfrom keras.models import Sequentialfrom keras.layers import Dense, Dropout, Flattenfrom keras.layers import Conv2D, MaxPooling2Dfrom keras import backend as Kbatch_size = 128num_classes = 10epochs = 12# input image dimensionsimg_rows, img_cols = 28, 28# the data, split between train and test sets(x_train, y_train), (x_test, y_test) = mnist.load_data()if K.image_data_format() == 'channels_first':    x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)    x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)    input_shape = (1, img_rows, img_cols)else:    x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)    x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)    input_shape = (img_rows, img_cols, 1)x_train = x_train.astype('float32')x_test = x_test.astype('float32')x_train /= 255x_test /= 255print('x_train shape:', x_train.shape)print(x_train.shape[0], 'train samples')print(x_test.shape[0], 'test samples')# convert class vectors to binary class matricesy_train = keras.utils.to_categorical(y_train, num_classes)y_test = keras.utils.to_categorical(y_test, num_classes)model = Sequential()model.add(Conv2D(32, kernel_size=(3, 3),                 activation='relu',                 input_shape=input_shape))model.add(Conv2D(64, (3, 3), activation='relu'))model.add(MaxPooling2D(pool_size=(2, 2)))model.add(Dropout(0.25))model.add(Flatten())model.add(Dense(128, activation='relu'))model.add(Dropout(0.5))model.add(Dense(num_classes, activation='softmax'))model.compile(loss=keras.losses.categorical_crossentropy,              optimizer=keras.optimizers.Adadelta(),              metrics=['accuracy'])model.fit(x_train, y_train,          batch_size=batch_size,          epochs=epochs,          verbose=1,          validation_data=(x_test, y_test))score = model.evaluate(x_test, y_test, verbose=0)print('Test loss:', score[0])print('Test accuracy:', score[1])
On a 2015 iMac (3.2 GHz Intel Core i5, 16 GB DDR RAM, AMD Radeon R9 M380 2 GB GPU) it took 1m50s to train with PlaidML/OpenCL GPU backend, and 5m06s using the TensorFlow-2.0/CPU backend.
Note that the instruction from keras import backend as K would return an error if there is no available backend. For example, if you did not install TensorFlow (default), you would see:
Using TensorFlow backend.Traceback (most recent call last):  File "./test_keras.py", line 8, in <module>    import keras  File "/Users/Riccardo/development/venv_opencl/lib/python3.7/site-packages/keras/__init__.py", line 3, in <module>    from . import utils  File "/Users/Riccardo/development/venv_opencl/lib/python3.7/site-packages/keras/utils/__init__.py", line 6, in <module>    from . import conv_utils  File "/Users/Riccardo/development/venv_opencl/lib/python3.7/site-packages/keras/utils/conv_utils.py", line 9, in <module>    from .. import backend as K  File "/Users/Riccardo/development/venv_opencl/lib/python3.7/site-packages/keras/backend/__init__.py", line 89, in <module>    from .tensorflow_backend import *  File "/Users/Riccardo/development/venv_opencl/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py", line 5, in <module>    import tensorflow as tfModuleNotFoundError: No module named 'tensorflow'
On a final note, TensorFlow >=2.0 includes Keras API. If your program is written so that layers are defined from TF, and not Keras, you cannot just change the Keras backend to run on the GPU with OpenCL support, because TF2 does not support OpenCL. To be more specific:
This will not use the GPU (assuming you have installed TensorFlow >=2.0)
from tensorflow import kerasfrom tensorflow.keras import layers
This will work just fine (assuming you have installed Keras)
import kerasfrom keras import layers
It turns out that PlaidML can do much more than just being a Keras backend. In fact, it comes with a programming language called Tile to help users writing optimized kernels without requiring an in-depth knowledge of C and most of the quirks of OpenCL. Tile instructions look more like mathematical functions. For example, a matrix multiplication
looks like this in Tile
function (A[M, L], B[L, N]) -> (C) {    C[i, j: M, N] = +(A[i, k] * B[k, j]);}
where the “+” operator stands for the “sum” in the mathematical expression.
Many more examples can be found in the documentation, including some of the most common operations such as matrix multiplication, min/max, max pool, convolution, cumulative sum. When the kernel is synthesized, Tile flattens the tensors and transforms indices into pointer offsets. The calculation is split into tiles that fit into memory, with a size that is optimized based on the available hardware. The tiles are then laid out in the GPU memory to optimize SIMDs instructions such as multiplications and accumulations. Finally, the kernels are written out as if they were created by a human user, and passed on to OpenCL for the actual execution.
PlaidML is now part of Intel’s AI group Vertex.ai. It may finally make AMD cards (possibly more vendors to come) a viable option in ML. In fact, Intel is also set to enter the GPU market in 2020 with hardware designed for ML in data centres which could further reduce the price of computation through competition. Intel CEO Bob Swan said that “ in 2020, we’ll continue to expand our 10-nanometer portfolio with exciting new products including an AI Inference Accelerator, 5G base station SOC, Xeon CPUs for server storage, and network and a discrete GPU. This quarter, we’ve achieved power-on exit for our first discrete GPU, DG1, an important milestone.”. So stay tuned!