Numba: JIT Compilation, But For Python | by Emmett Boudreau | Towards Data Science

Statistical computing has always been a very difficult discipline for programmers and computer-scientists to provide for. There are certain attributes and characteristics that programmers look for in not only a statistical programming language, but also a general-purpose programming language. The rise of statistical computing with the popularity of machine-learning and data science applications has also exacerbated these critical issues in the development of the “ perfect language.”
Going back ten years, nobody could have anticipated what the rise of machine-learning in Python would have been like. Most scientists were using the R programming language for statistical analysis and things of that nature. MATLAB is of course also a very popular choice for the statistical computing world, as well as Scala. Python has recently become very popular in this regard for its strong statistical capabilities
Of course, Python is interpreted by C. This means that Python isn’t as fast as many as its competitors, and not as suitable for big data solutions. Python by nature is not a language with a compiler built to satisfy many of the needs of modern scientific computing. While Python is a great application in for most statistical applications, the primary factor in Python’s short-comings is usually its speed when working with large amounts of data and recursively training machine-learning models.
If Python has issues with processing large amounts of data, and is by nature an interpreted, high-level language, then why is it that many scientists are enjoying the spoil of the language today? The answer is quite simple, at this current time, Python is the best solution available. While Python is relatively slow compared to many other languages, it is also very high-level when compared with other languages, and has only been getting faster and more optimized in the past couple of years.
To add to that beauty, Python has one of the strongest ecosystems for statistical computing available today. Most of these packages are written in C, as well, which can make Python often be a simple interface to very fast code. Regardless of how great the ecosystem is, however, it cannot help with processing enormous levels of data and writing Pythonic algorithms to clean and edit data.
Unless you have been living under a programming turtle shell, you might be familiar with — or at least heard of a compiler concept called JIT. JIT, or Just In Time compilation is a compiler feature that allows a language to be interpreted and compiled during runtime, rather than at execution. This means that rather than preparing all of the code to work, deciding what the code is going to do, and then doing it, a JIT compiler will be compiling the language as it is executing the logic prior to the code it is compiling.
The benefit to such a compiler is of course speed. Less time spent on initial compilation means that the code can be interpreted a lot faster. For an analogy, pretend that you have a list of things cooking for dinner. Rather than cooking all of the things individually, you could take a multi-tasking approach where multiple things are being worked on at the same time.
The Numba JIT compiler, similar to the Julia JIT compiler, uses the standard LLVM compiler library. Although it does have some short-comings in that regard, in that the focus of LLVM isn’t necessarily JIT, it does mean that the compiler is incredibly fast and precise. Numba-compiled Pythonic algorithms can approach speeds that are often seen in lower-level languages like C.
This might sound complicated, and it is — but that doesn’t mean that Numba is hard to use. As a matter of a fact, Numba is incredibly easy to use! In order to try it out, you are of course going to need to add it with Python’s package manager, PIP.
sudo pip3 install numba
After installing Numba, you can access it via the jit function:
from numba import jitimport random@jit(nopython=True)def monte_carlo_pi(nsamples):    acc = 0    for i in range(nsamples):        x = random.random()        y = random.random()        if (x ** 2 + y ** 2) < 1.0:            acc += 1    return 4.0 * acc / nsamples
For most code, Numba does an incredible job at optimizing Python code. As Julia developers discussed at JuliaCon, however, in its current version, Numba still has a long way to go and presents [problems with certain code. Some issues can’t be fixed with a simple Python call, and although Numba has done a great job at creating an optimized compiler for Python that can be imported easily, there are certainly improvements to be made.
Regardless as to whether or not Python will always be the go-to programming language for scientific purposes and statistical computing, it is clear that Python is certainly that for now. While Python isn’t the quickest programming language out there and has its fair share of minor issues with scientific computing, it is still a great choice for those wishing to do statistics or machine-learning. This is mostly due to its ecosystem and overall popularity.
Fortunately, many of Python’s flaws are being directly targeted by talented and driven developers. Over time, there is a pretty good chance that the technology will be so far along that many of these issues might not even be felt anymore when using Python for Data Science.
One of Python’s biggest problems is its speed. Numba is one of a series of awesome tools that help counter this problem and bring Python up to snuff to counter other statistical languages, especially newer languages like Julia. What is exciting about Numba is how simple it is to use, like a light-switch that makes your code run faster. This, in turn, can be used to breathe new life into applications that otherwise might not even be able to run.