Here’s How to Use CuPy to Make Numpy Over 10X Faster | by George Seif | Towards Data Science

Want to be inspired? Come join my Super Quotes newsletter. 😎
Numpy has been a gift to the Python community. It’s allowed Data Scientists, Machine Learning Practitioners, and Statisticians to process huge amounts of data in matrix format in a way that’s easy and efficient.
Even on its own, Numpy is already a significant step up from Python in terms of speed. Whenever you find your Python code running slow, especially if you see a lot of for-loops, it’s always a good idea to move the data processing into Numpy and let its vectorization do the work at top speed!
Still, even with that speedup ,Numpy is only running on the CPU. With consumer CPUs typically having 8 cores or less, the amount of parallel processing, and therefore the amount of speedup that can be achieved, is limited.
That’s where our new friend CuPy comes in!
CuPy is a library that implements Numpy arrays on Nvidia GPUs by leveraging the CUDA GPU library. With that implementation, superior parallel speedup can be achieved due to the many CUDA cores GPUs have.
CuPy’s interface is a mirror of Numpy and in most cases, it can be used as a direct replacement. Just replace your Numpy code with compatible CuPy code and boom you have GPU speedup. CuPy will support most of the array operations that Numpy has including indexing, broadcasting, math on arrays, and various matrix transformations.
You can also write custom Python code which leverages CUDA and GPU speedups if you have something specific that isn’t yet supported. All that’s required is a small snippet of your code in C++ format and CuPy will automatically do the GPU conversion, very similar to using Cython.
To get started with CuPy we can install the library via pip:
pip install cupy
For these benchmarks I will be using a PC with the following setup:
i7–8700k CPU
1080 Ti GPU
32 GB of DDR4 3000MHz RAM
CUDA 9.0
Once CuPy is installed we can import it in a similar way as Numpy:
import numpy as npimport cupy as cpimport time
For the rest of the coding, switching between Numpy and CuPy is as easy as replacing the Numpy np with CuPy’s cp. The code below creates a 3D array with 1 Billion 1’s for both Numpy and CuPy. To measure the speed of creating the arrays, I used Python’s native time library:
### Numpy and CPUs = time.time()x_cpu = np.ones((1000,1000,1000))e = time.time()print(e - s)### CuPy and GPUs = time.time()x_gpu = cp.ones((1000,1000,1000))cp.cuda.Stream.null.synchronize()e = time.time()print(e - s)
That was easy!
Notice how we added an extra line after the initialization of the cupy array. What this does is ensure that our code finishes executing on the GPU before going to the next line.
The incredible part was that even though this was only array creation, CuPy was still much faster. Numpy created the array of 1 Billion 1’s in 1.68 seconds while CuPy only took 0.16; that’s a 10.5X speedup!
But we can still do more.
Let’s try doing some mathematical operations on the arrays. This time we’ll multiply the entire array by 5 and again check the speed of Numpy vs CuPy.
### Numpy and CPUs = time.time()x_cpu *= 5e = time.time()print(e - s)### CuPy and GPUs = time.time()x_gpu *= 5cp.cuda.Stream.null.synchronize()e = time.time()print(e - s)
In this case, CuPy shreds Numpy. Numpy took 0.5845 while CuPy only took0.0575; that’s a 10.17X speedup!
Let’s now try working with multiple arrays and do a few operations. The code down below will do the following:
Multiple the array by 5Multiple the array by itselfAdd the array to itself
Multiple the array by 5
Multiple the array by itself
Add the array to itself
### Numpy and CPUs = time.time()x_cpu *= 5x_cpu *= x_cpux_cpu += x_cpue = time.time()print(e - s)### CuPy and GPUs = time.time()x_gpu *= 5x_gpu *= x_gpux_gpu += x_gpucp.cuda.Stream.null.synchronize()e = time.time()print(e - s)
In this case, Numpy performed the process in 1.49 seconds on the CPU while CuPy performed the process in 0.0922 on the GPU; a more modest but still great 16.16X speedup!
Using CuPy is a great way to accelerate Numpy and matrix operations on the GPU by many times. It’s important to note that the speedups you’ll get are highly dependant on the size of the array you’re working with. The table below shows the difference in speed when we change the size of the array we’re processing:
The speedup drastically kicks up once we get to about 10 million data points and gets much much faster once we cross that 100 million points mark. Below that, Numpy is actually faster. Also, keep in mind that more GPU memory will help you process more data, so it’s important to see if your GPU has enough memory to fit enough data where CuPy is worth it.
Follow me on twitter where I post all about the latest and greatest AI, Technology, and Science! Connect with me on LinkedIn too!