FuncTools: An Underrated Python Package | by Emmett Boudreau | Towards Data Science

Last month, I wrote an article about some modules in the Python standard library that I have found to be incredibly useful in my programming and Data Science career. The article had such a great reception with quality feedback that I decided to double down on the idea and write another article discussing the same topic with a lot more standard library tools that everyone should be familiar with. It turns out that the Python programming language’s base is actually quite inclusive, and includes a lot of great tools for various programming challenges. If you would like to read either one of those articles, you can check them out here:
towardsdatascience.com
towardsdatascience.com
When going over a lot of these tools, it seemed like some should have an entire article dedicated to them, rather than just a little overview that is required for most of the other modules that were presented with it. The tool that I think most reciprocated this idea to me is the functools module. This is an incredibly powerful module that can be used to improve nearly any function in Python by utilizing simple and classic methods, such as utilizing stack over processor speed. While in some circumstances I could see that being a big negative, there are certainly exceptions to that case.
notebook
Probably the coolest thing that the functools module provides is the ability to cache certain calculations inside of memory rather than throwing them away just to recalculate them later. This is a great way to save processing times, especially if you find yourself in a Python3 time-out scenario where you are unable to get your code interpreted. While this comes with the trade-off of using a lot more memory, it can certainly make sense to utilize in many different situations. The Python programming language itself is quite declarative, and usually the interpreter handles all of the memory management for us. While this is a less efficient method of programming, it also removes a lot of the hassle of allocating memory and things of that nature. Using functools, we can somewhat change this by determining some things about what lands in the stack and what will be recalculated ourselves.
What is great about the caching that functools provides is that it is both easy to use and allows one to take better control of the interpreter underneath their code. Taking advantage of this awesome feature is as simple as calling it above your function. Here is an example with factorial calculation that I really think takes advantage of what this can accomplish quite well:
def factorial(n):    return n * factorial(n-1) if n else 1
In order to use caching with this function, we will import lru_cache from functools and call it before our function:
@lru_cachedef factorial(n):    return n * factorial(n-1) if n else 1
Now let us assess the performance benefit that we might have endured just from doing this simple addition. Let us take a look at how fast that factorial function will calculate our factorials without it:
Now I am going to run restart the kernel in order to ensure there are no weird memory things going on and run our new function that uses lru_cache.
Already from this example, we can see a massive benefit from using this caching technique. Factorials are notoriously difficult to calculate by a computer. In most circumstances, this is because factorials are a natural mathematical example for recursion. The Cumulative Distribution Function, or CDF, for binomial distribution gives me nightmares as a result. The calculation is so intensive to make that often programming language’s base factorial function will use a look-up table, rather than calculate the number.
That being said, if you are going to be using recursion like this, it might be a great idea to start getting familiar with functools. This standard library tool can bring a lot of speed to problems that are typically very difficult for Python to solve. In a way, it really brings me back to the Numba Python compiler, where one simple call will instantly make your code faster. If you would like to read an article I wrote all about that a while back, you can check it out here:
towardsdatascience.com
Has there ever been a time where you really wanted to use a function from some really old Python code, but the function was compiled to be a comparison function? These types of functions aren’t as well-supported or even used anymore in modern Python 3, and it can be quite difficult to translate one function type into another. That being said, functools is here to easily fix that with another simple method call:
newfn = cmp_to_key(myfunction)
The partial function will return a new partial object which can then later be called using the same exact arguments and will behave exactly like our previous func. The function inside the code ends up looking a little something like this:
def partial(func, /, *args, **keywords):    def newfunc(*fargs, **fkeywords):        newkeywords = {**keywords, **fkeywords}        return func(*args, *fargs, **newkeywords)    newfunc.func = func    newfunc.args = args    newfunc.keywords = keywords    return newfunc
We can create a partial using our factorial function before:
from functools import partialfact = partial(factorial)fact(10)
The partial() is used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature. What this results in is an easy way to save memory and speed by creating a simplistic version of a function that is wrapped in an entirely new object.
The reduce function will apply the function of two arguments with cumulative iteration. That in mind, we will need our arguments to be iterables. For this example, I am going to be using a generator, range. This will make it super easy for us to construct a list datatype of essentially any length we desire.
from functools import reducereduce(lambda x, y: x+y, [1, 2, 3, 4, 5])
What will this do? This will reduce the iterable down into its simplist form. The reduce name comes from its mathematical equivalent. This can be handy for saving runtime performance, and like many others on this list is an easy call that there isn’t a good excuse for not using!
Okay — so this is really, really cool.
Readers of my blog will likely note that I am a huge fan of the Julia programming language. This is not only because Julia is an awesome, high-performance scientific programming language, but also because I have a particular attraction to multiple dispatch. The concept is something that I genuinely miss when it is not around, and it makes programming feel almost more natural. In a way, it is great to have the functions be more type-based, rather than just have different calls in order to handle different situations.
The functools module for Python actually provides a way that Python can effectively be made into a programming language with multiple dispatch. On top of that, it is as easy as adding a decorator as before. In order to do this, we are going to first decorate a function with no specific types required for any arguments with the singledispatch decorator:
@singledispatchdef fun(arg, verbose=False):        if verbose:            print("Let me just say,", end=" ")        print(arg)
Now we will use ourfunction.register in order to register dispatching for new types:
@fun.registerdef _(arg: int, verbose=False):    if verbose:        print("Strength in numbers, eh?", end=" ")    print(arg)@fun.registerdef _(arg: list, verbose=False):    if verbose:        print("Enumerate this:")    for i, elem in enumerate(arg):        print(i, elem)
Functools is a really awesome package! Furthermore, I think that the package will most certainly come in handy to just about any Python programmer. The package allows you to simplify your code and math with very minimal effort. Additionally, it is relatively easy to get a bit more in touch with the language and its cache, which can be very convenient to improve certain aspects of performance. Pretty much anything you might want to do that is interesting with functions can be done with this module, which is pretty awesome!