Introducing Python’s Functools Module | by Amayo Mordecai II | Towards Data Science

The functools module, part of Python’s standard Library, provides useful features that make it easier to work with high order functions (a function that returns a function or takes another function as an argument ). With these features, you can reuse or extend the utility of your functions or callable object without rewriting them. This makes the writing of reusable and maintainable code to be quite simple.
As per the current stable release i.e., Python 3.8 series, the functools module contains 11 funtions and some of these may not be available or work differently on earlier or later releases. They include:
reduce()lru_cache()partial()partialmethod()singledispatch()singledispatchmethod()cached_property()total_ordering()update_wrapper()wraps()cmp_to_key()
reduce()
lru_cache()
partial()
partialmethod()
singledispatch()
singledispatchmethod()
cached_property()
total_ordering()
update_wrapper()
wraps()
cmp_to_key()
We will discuss each one briefly and then, with the aid of examples, illustrate their usage and functionality.
First up we have a classic. The reduce(function, sequence) function receives two arguments, a function and an iterable. It applies the argument function cumulatively to all elements of the iterable from the left to the right and then returns a single value.
To put it simply, it first applies the argument function to the first two elements of the iterable and the value returned by this first call becomes the function’s first argument and the third element of the iterable becomes the second argument. This process is repeated until the iterable is exhausted.
For example, reduce() can be used to easily calculate the sum or the product of a list.
from functools import reduce

ls = [1,2,3,4]

list_sum = reduce(lambda a,b: a+b, ls)
list_product = reduce(lambda a,b: a*b, ls)

print(f"The sum of the list {ls} is {list_sum}")

The sum of the list [1, 2, 3, 4] is 10

print(f"The product of the list {ls} is {list_product}")

The product of the list [1, 2, 3, 4] is 24

 

In the first example reduce(lambda a, b: a + b, [1, 2, 3, 4]) calculates (((1+2)+3)+4) and returns 10 which is the sum of the list.
lru_cache() is a decorator, which wraps a function with a memoizing callable used for saving up to maxsize the results of a function call and returns the stored value if the function is called with the same arguments again. It can save time when an expensive or I/O bound function is periodically called with the same arguments.
Essentially it uses two data structures, a dictionary to map a function’s parameters to its result, and a linked list to keep track of the function’s call history.
In full LRU Cache stands for Least-Recently-Used Cache and refers to a cache which drops the least recently used element if the maximum size of entries is reached. The LRU feature is disabled if maxsize is set to None and caches arguments of different data types separately if typed is True e.g., f(3) and f(3.0) will be cached distinctly.
An example of the utility of lru_cache() can be shown in optimizing code that generates the factorial of a number
from functools import lru_cache

#function without lru_cache
def uncached_factorial(n):
    if n <= 1:
        return 1
    return n * uncached_factorial(n-1)

#evaluate the performance of the uncached function
%timeit uncached_factorial(10)

1.46 μs ± 16.6 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

#funtion with lru_cache 
@lru_cache(maxsize=None)
def cached_factorial(n):
    if n == 0:
        return 1
    if n == 1:
        return 1
    return n * uncached_factorial(n-1)

#evaluate the performance of the uncached function
%timeit cached_factorial(10)

158 ns ± 5.35 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

 

Without @lru_cache the factorial function takes around 1.46 μs to run while, on the other hand, with @lru_cache, the function only takes 158 ns. This equates to almost a 100,000 times improvement in performance- Amazing! Right?
In general, the LRU cache should only be used when you want to reuse previously computed values. Accordingly, it doesn’t make sense to cache functions that need to create distinct mutable objects on each call. Also, since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable.
Partial functions are derived functions that have some pre-assigned input parameters. For example, if a function takes in two parameters say “a” and “b”, a partial function can be created from it that has “a” as a prefilled argument and it can then be called with “b” as the only parameter. Functool’s partial() is used to create partial functions/objects and this is a useful feature as it allows for the:
Replication of existing functions with some arguments already passed in.Creation of newer version of the existing function in a well-documented manner.
Replication of existing functions with some arguments already passed in.
Creation of newer version of the existing function in a well-documented manner.
Let’s consider a simple example to illustrate this
from functools import partial
import math

permutation_of_nine = partial(math.perm, 9)

print(permutation_of_nine(2))

72

permutation_of_nine.__doc__ = '''Returns the number of ways to choose k items from 9 items without repetition and with order'''

permutation_of_nine.__name__ = "permutation_of_nine"

 

We first create a partial object based on math.perm() function. In this case, we set 9 as the first argument. Consequently, the newly created permutation_of_nine function behaves as if we call math.perm() with 9 set as the default parameter. In our example, permutation_of_nine(2) does the same thing as math.perm(9,2).
It is important to note that the __name__ and __doc__ attributes are to be specified by the programmer as they are not created automatically
The partial function also comes with important attributes that prove to be useful in tracking partial functions/objects. These include:
partial.args — Which returns the positional arguments preassigned to the partial function.
partial.keywords — Which returns the keyword arguments preassigned to partial function.
partial.func — Which returns the name of parent function along with its address.
Let us look at another that illustrates these features
from functools import partial

#parent function
def adder(x,y):
    return x+y

#partial functions
add2 = partial(adder, y=2)
add9 = partial(adder, 9)

print(f'2+3= {add2(3)}, 9+3 = {add9(3)}')
print(f'Parent function of add2 is: {add2.func}')
print(f'Keyword argument of add2 is: {add2.keywords}')
print(f'Argument of add9 is: {add9.args}')

2+3= 5, 9+3 = 12
Parent function of add2 is: <function adder at 0x0000022894E740D0>
Keyword argument of add2 is: {'y': 2}
Argument of add9 is: (9,)

 

Partials are incredibly useful. For example, in a pipe-lined sequence of function calls in which the returned value from one function is the argument passed to the next.
The partialmethod() returns a new partialmethod descriptor which behaves like partial except that it is designed to be used as a method definition rather than being directly callable. You can think of it as the partial() for methods.
Perhaps an example is best suited to illustrate this.
We first create an Animal class that has an attribute species and an instance method _set_species() that sets the animal’s species. Next, we create two partialmethod descriptors set_dog() and set_rabbit(), which call _set_species() with “Dog” or “Rabbit”, respectively. This allows us to create a new instance of the class Animal, call set_dog() to change the animal’s species to Dog and finally print the new attribute.
Before we discuss this function, it is important that we first gloss over two important concepts:
The first one is a generic function which is a function composed of multiple functions implementing the same operation for different types. The implementation to be used during a call is determined by the dispatch algorithm.
The second is the Single dispatch, which is a form of a generic function dispatch where the implementation is chosen based on the type of a single argument.
With this in mind, the functool’s singledispatch is a decorator that transforms a simple function into a generic function whose behaviour is dependent on the type of its first argument. In plain language, it is used for function overloading
Let see an example of it in action.
We first define a function divide() that takes two arguments a and b and returns the value of a/b. However, dividing strings will result in a TypeError and to deal with this we define the _ functions which specifies the behaviour of divide() if it is supplied with strings. Note that the overloaded implementations are registered using the register() attribute of the generic function
It is a decorator that does the exact thing as @singledispatch but it is specified for methods rather than functions.
Consider the following example.
The prod method of the Product class is overloaded to return the product of the elements of a list or a set but if supplied with a different type it, by default, raises a NotImplementedError.
As the name suggests the cached_property() is a decorator that transforms a class method into a property whose value is calculated only once and then cached as a normal attribute for the life of the instance. It is similar to @property except the for its caching functionality. It is useful for computationally expensive properties of instances that are otherwise effectively permanent.
In the example above, we have a DataSet class that holds a list of observations and implements methods to calculate the variance and standard deviation. The problem is that every time the methods are called the variance and standard deviations would have to be re-calculated and this might prove to be expensive especially for large datasets. @cached_property mitigates this problem by calculating and storing the value only once and returns it if the method is called again by the same instance.
Given a class defining one or more rich comparison ordering methods i.e., __lt__(), __le__(), __gt__(), __ge__() or __eq__() (corresponding to <, <=, >, >=, and ==). You can define a few of the comparison methods, and @total_ordering will automatically supply the rest as per the given definitions. It is important that the class should supply an __eq__() method.
For example, if you want to create a class that compares different numbers. You would probably need to implement all of the rich comparison methods. However, this might be quite tedious and redundant, to solve this you can only implement the __eq__ and the __gt__ method and use @total_ordering to automatically fill up the rest.
There is a limitation though, using @total_ordering adds overhead leading to slower execution. Furthermore, the stack traces for the derived comparison methods are more complex. So if you need efficient code, it would be wise to explicitly implement the comparison methods by yourself
It updates the metadata of a wrapper function to look like the wrapped function. For example, in the case of partial functions, update_wrapper(partial, parent) will update the documentation(__doc__) and name(__name__) of the partial function to match that of the parent function.
It is a convenience function for invoking update_wrapper() to the decorated function. It is equivalent to running partial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated).
For example,
It transforms an old-style comparison function to a key function. A comparison function is any callable that accepts two arguments, compares them, and returns a negative number for less-than, zero for equality, or a positive number for greater-than. Whereas a key function is a callable that accepts one argument and returns another value to be used as the sort key, an example is the operator.itemgetter() key function. Key functions are used in tools such as sorted(), min(), max() and itertools.groupby().
cmp_to_key() is majorly used as a transition tool for programs written in Python 2 that support comparison functions.
Let’s take an example of how we can use a comparison function to sort a list of strings according to the first letter to illustrate the use of cmp_to_key()
In this article, we have gone through the functools module and hopefully, you now have a glimpse of how you can use it to implement high order functions and write highly robust, readable and reusable code.
Good luck, happy coding and may the bugs tremble before your might.