Use Cython to get more than 30X speedup on your Python code
By: George Seif
Tags: Programming, Artificial Intelligence, Python, Data Science, Technology

Cython will give your Python code super-car speed

Python is a community favorite programming language! It’s by far one of the easiest to use as code is written in an intuitive, human-readable way.

Yet you’ll often hear the same complaint about Python over and over again, especially from the C code gurus out there: Python is slow.

And they’re not wrong.

Relative to many other programming languages, Python is slow. Benchmark game has some solid benchmarks for comparing the speed of various programming languages on different tasks.

There’s a couple of different ways to speed things up that I’ve written about before:

(1) Use multi-processing libraries to use all the CPU cores

(2) If you’re using Numpy, Pandas, or Scikit-Learn, use Rapids to accelerate the processing on GPU.

Those are great if what you’re doing can, in fact, be parallelized, such as data pre-processing or matrix operations.

But what if your code is pure Python? What if you have a big for-loop that you just have to use and can’t put into a matrix because the data has to be processed in sequence? Is there a way to speedup Python itself?

That’s where Cython comes in to speed up our raw Python code.

What is Cython?

At its core, Cython is an intermediate step between Python and C/C++. It allows you to write pure Python code with some minor modifications, which is then translated directly into C code.

The only adjustment you make to your Python code is adding type information to every variable. Normally, we might declare a variable in Python like this:

x = 0.5

With Cython, we’re going to add a type to that variable:

cdef float x = 0.5

This tells Cython that our variable is floating point, just like we would do in C. With pure Python, the variable’s type is determined on the fly. The explicit declaration of the type in Cython is what makes the conversion to C possible, since explicit type declarations are required+.

Installing Cython requires just a single line of pip:

pip install cython

Types in Cython

When using Cython, there are two different sets of types, for variables and functions.

For variables we have:

cdef int a, b, c

a, b, c cdef char *s

*s cdef float x = 0.5 (single precision)

x = 0.5 (single precision) cdef double x = 63.4 (double precision)

x = 63.4 (double precision) cdef list names

names cdef dict goals_for_each_play

goals_for_each_play cdef object card_deck

Notice how all of these types come from C/C++! For the functions we have:

def — regular python function, calls from Python only.

— regular python function, calls from Python only. cdef — Cython only functions which can’t be accessed from python-only code i.e must be called within Cython

— Cython only functions which can’t be accessed from python-only code i.e must be called within Cython cpdef — C and Python. Can be accessed from both C and Python

With an understanding of Cython types in hand, we can go right ahead and implement our speedup!

How to speedup your code with Cython

The first thing we’ll do is set up a Python code benchmark: a for-loop used to compute the factorial of a number. The raw Python code is shown below:

Our Cython equivalent of the same function looks very similar. First, we’ll make sure that our Cython code file has a .pyx extension. The only change to the code itself is that we’ve declared the type of each and every variable and function.

Notice how the function has a cpdef to make sure we can call it from Python. Also see how even our looping variable i has a type. You’ll need to set the type for all of the variables in the function, so that the C compiler knows what type to use!

Next, create a setup.py file which will compile the Cython code into C code:

And perform the compilation:

python setup.py build_ext --inplace

Boom! Our C code has been compiled and is ready to use!

You’ll see that in the folder where your Cython code is, you have all the files needed to run C code, including the run_cython.c file. If you’re curious, take a look at it to see the C code that Cython generated!

Now we’re ready to test out our new, super fast C code! Check out the code below, which implements a speed test to compare the raw Python code to the Cython one.

The code is very straight forward. We’re importing our files in the same way as regular Python, and running the functions in the same way as regular Python!

Cython will get you good speedups on almost any raw Python code, without too much extra effort at all. The key thing to note is that the more loops you’re going through, and the more data you’re crunching, the more Cython can help.

Check out the table below which shows how much speed Cython gave us for different factorial values. We got over 36X speedup with Cython!