10 Of My Favorite Python Decorators | by Emmett Boudreau | Towards Data Science

If there is one thing that really fascinates me in computing, it is programming languages. Programming languages are interesting to me because of what they actually are, merely just an abstraction layer from the hardware. And the way that us humans have found to interact with the hardware from these abstraction layers is truly amazing. We wanted to work with something very human: words, relationships, and things; we emulated our brain’s process of definition inside of a computer. While this is similar with hardware programming, I think that these layers of abstraction really make the language interesting because there are all sorts of ways to handle different scenarios and your types with those scenarios that it is very interesting to see the kind of ideas people come up with.
This fascination I think grows even more whenever we are talking about a very declarative scripting language — in this case, Python. The great thing about a language like Python is that it is declarative because of abstraction. The language is incredibly easy to use, and all of the underlying hardware and kernel calls are kept very far away from the programmer. But with that much abstraction from the pushes and pops happening underneath, it is really cool to see how syntax can be manipulated to work with those types.
In a previous article, I dove into multiple dispatch in Python. The multiple dispatch module uses a simple decorator to turn the language of Python into a multiple dispatch. If you would like to read more about that, the article is here:
towardsdatascience.com
By the way, I LOVE polymorphism.
But writing that article, and exploring that package a bit did get my brain started on decorators again — and I suppose that is for a good reason... Decorators are awesome! They are so useful in so many different scenarios, and they are incredibly easy to use. Another awesome article I wrote, which mirrors this one — and can be considered a precursor of sorts is an article where I detailed 10 awesome Python decorators I like to use, if you would like to read that as well, here is a link:
towardsdatascience.com
Given the awesome nature of Python decorators, and what they are capable of, I considered it very apt to go about writing a new story with some new incredible decorators to use in your Python code. A lot of these bend the Python paradigm, and can really work to take your Python code to the next level! The decorators on this list are not included on the last, so if you would like to double up, you could read them both for some pretty desirable decorators in your arsenal. Also, there will be a notebook with all of the code and decorators I used in this article on Github at this link:
github.com
No surprise here, I do have an article I just wrote where I went into the multiple dispatch module and what it is capable of. Needless to say, as I tend to voice my multiple dispatch opinion and my Julia opinion quite loudly, if there is one thing I love it is multiple dispatch. It seems like a very natural way to program that does not really get in the way. I often miss this very Julian programming concept whenever I write Python, so stumbling across this package was a very big delight.
Before I go into how to use this decorator, let us first discuss the what and why of multiple dispatch. Firstly, multiple dispatch is a function and type system that allows methods to be listed by their type counter-parts. Typically, in an object-oriented approach all Pythonic functions would be defined within the scope of a Python class. This is basically the definition of object-oriented programming. However, one of the things that makes Python so much more of a usable language is that it does not force one to use this paradigm. A lot of Data-Science code is very methodized in the Pythonic ecosystem, with a lot less of a focus on types in a lot of ways than someone might expect after first hearing the language was “ object-oriented.”
I think that polymorphism through multiple dispatch can really take that methodization to the next level. This is even the case within a class, as well, as we will see in my example. The reason why I recommend using this decorator in some instances is because it is just one simple call that can change the code substantially, and make for some pretty great syntax. Let us consider the following example:
from multipledispatch import dispatchclass Pasta:    def __init__(self, width, height, classification):        self.width, self.height = width, height        self.classification = classification    def classify():        if self.width > 3 and height < 2:            self.classification = "Linguine"        elif self.width == self.height and self.width < 3:            self.classification = "Spaghetti"        elif width > 2 and height == 1:            self.classification = "Fettuccine"        else:            self.classification = "Dough"class Bread:    def __init__(self, width, height, classification):        self.width, self.height = width, height        self.classification = classification    def classify():        if self.width > 3 and height < 2:            self.classification = "Pizza Crust"        elif self.width == self.height and self.width < 3:            self.classification = "White bread"        elif width > 2 and height == 1:            self.classification = "Flatbread"        else:            self.classification = "Dough"class Roller:    def __init__(self):        pass    def roll_thinner(x : Pasta):        x.height -= 1        x.width += 1        x.classify()    def cut(x : Pasta):        x.width -= 1        x.classify()    def fold(x : Pasta):        x.width += 1        x.classify()
We have 3 classes to work with here. The first object is Pasta, which is just pasta that can be rolled out using our rolling pin (Roller.) The next is Bread. Bread has many of the same attributes of Pasta. However, for this example, we have one thing that is missing from the bread class, the only problem is that if we include this value — then none of the Roller functions are going to work on the bread! In a normal event, there would have to either be new functions that work exclusively with this type, so we would have for example roll_thinnerbread() and roll_thinnerpasta() respectively, or there would need to be a second rolling pin that we only use for bread, say
class BreadRoller:
With multiple dispatch, however, these worries can be gone — we can use multiple dispatch to handle types differently from function to function. In this example, I am only going to be changing one function name in the Bread class. I am just going to change the name of the classify() function to now be bake(). There is also this function I made, which will spit out a new object for us with some preset “ dough” settings:
def make_dough(pasta = False):    if pasta:        return(Pasta(10, 10, "Dough"))    else:        return(Bread(10, 10, "Dough"))
Let’s make some dough!
dough = make_dough()print(dough.classification)'Dough'
We will also make some pasta dough:
pasta_dough = make_dough(pasta = True)
Now let us roll out our pasta into a nice linguine, which we will serve alongside our pizza crust dough that we just created.
rollingpin = Roller()while pasta_dough.height >= 3:    rollingpin.roll_thinner(pasta_dough)while pasta_dough.width > 2:    rollingpin.cut(pasta_dough)print(pasta_dough.classification)Linguine
Great! We made some linguine! But watch all of this fall apart because our other type has a slightly different structure:
rollingpin.roll_thinner(dough)---------------------------------------------------------------------------AttributeError                            Traceback (most recent call last)<ipython-input-132-b755ead4a785> in <module>----> 1 rollingpin.roll_thinner(dough)<ipython-input-118-1af4dc2d94f2> in roll_thinner(self, x)      5         x.height -= 1      6         x.width += 1----> 7         x.classify()      8     def cut(self, x : Pasta):      9         x.width -= 1AttributeError: 'Bread' object has no attribute 'classify'
Oh no!
I was really looking forward to that pizza... Fortunately, we can now show off multiple dispatch by altering the functions in our rolling pin class to take both types. We will do so by simply cloning the functions, changing what needs to be changed, then using the dispatch decorator to make this code only execute when the correct type is passed, like so:
class Roller:    def __init__(self):        pass    @dispatch (Pasta)    def roll_thinner(self, x : Pasta):        x.height -= 1        x.width += 1        x.classify()    @dispatch (Pasta)    def cut(self, x : Pasta):        x.width -= 1        x.classify()    @dispatch (Pasta)    def fold(self, x : Pasta):        x.width += 1        x.classify()    @dispatch (Bread)    def roll_thinner(self, x : Bread):        x.height -= 1        x.width += 1        x.bake()    @dispatch (Bread)    def cut(self, x : Bread):        x.width -= 1        x.bake()    @dispatch (Bread)    def fold(self, x : Bread):        x.width += 1        x.bake()rollingpin.roll_thinner(dough)
Now this code executes. Needless to say, this can be pretty convenient in many different types of scenarios! Personally, I find multiple dispatch to be a very obvious way to program. It can be very flexible and create some pretty brilliant syntax. What is better is that it is so easy to do with this decorator, given that — I would highly recommend!
Celery is a must-have when it comes to working with tasks in Python. Celery is essentially a task manager for Python. It can keep track of task queues, which are operations at which a series of computers or threads are to perform next. The word queue in the computing world should always apply a first in, first out mentality to the data. To put it into a more metaphorical since, we are stacking pancakes but we eat the one on bottom first.
The only problem with Celery is that it would be particularly hard to demonstrate. In order to work with this library, I would need to have a production server at my disposal for it to work with. Of course, I do not have this, but I did end up using Celery several times in the past. Here is a basic example of creating tasks with Celery, from the documentation:
from celery import Celeryapp = Celery('tasks', broker='pyamqp://guest@localhost//')@app.taskdef add(x, y):    return x + y
Needless to say, Celery is incredibly easy to use. Just make a Celery stalk class and then use decorators to queue up your tasks. Available servers from the broker will then take on the tasks.
Click is another module that I think is really awesome. Click helps Python developers build Command Line Interfaces, or CLI’s. The greatest thing about this is that it can be used to provide CLA’s, or command line arguments to Python. Of course, this is not going to land very well in my Notebook, but regardless, I thought maybe it would still be a substantial tool to add to one’s arsenal. The following is an example that will greet someone after parsing their name as a CLI:
import click@click.command()@click.option("--name", prompt="Your name", help="The person to greet.")def hello(count, name):    click.echo(f"Hello, {name}!")if __name__ == '__main__':    hello()
Click is pretty cool because in a lot of ways it works like bash code alongside your Python. However, the advantage to doing that with Click is also that the code will work on a lot more than just Unix-like operating systems.
Deprecations can be extremely annoying to carry out. You want to get rid of a function, but everyone is still trying to use it. In a normal case, you might have to type a little bit of code in order to create your deprecation warning. For this, we would use the module warn:
warnings.warn(            "this function is deprecated... etc.",            DeprecationWarning        )
However, using deprecated, we only really need to do one thing with one simple call. We call this over our function as a decorator and instantly deprecation warnings can be given.
from deprecated import deprecated@deprecated ("This function is deprecated, please do not make dough here")def make_dough(pasta = False):    if pasta:        return(Pasta(10, 10, "Dough"))    else:        return(Bread(10, 10, "Dough"))
Now when we try to run this function, we will get a deprecation warning:
z = make_dough()
Now everyone trying to use this function will know there is
no dough allowed!
Deco is a parallel computing module for Python. If you were around for the last list I did of decorators, I mentioned a package called Numba. Numba can both JIT-compile, and GPU-compile Python software. Deco is a lot like Numba, but allows for synchronous and concurrent processes. For now, we are taking a look specifically at the concurrent decorator. The concurrent option uses multiprocessing.pool to make things concurrent. This is then used with another decorator to run the function inside of a thread in the background while continuing to take care of the function. Let us take a look at a simple use of the concurrent function on our pasta roller from earlier.
from deco import concurrent, synchronizeddef roll_linguine(count : int):    rollingpin = Roller()    pastas = []    for i in range(0, count):        dough = make_dough(pasta = True)        while dough.height >= 3:            rollingpin.roll_thinner(dough)        while dough.width > 2:            rollingpin.cut(dough)        pastas.append(dough)    return(pastas)
I also made an equivalent for pizza!:
def roll_pizzas(count : int):    rollingpin = Roller()    pizzas = []    for i in range(0, count):        dough = make_dough()        while dough.height > 3:            rollingpin.roll_thinner(dough)        while dough.width > 2:            rollingpin.cut(dough)
In order to now make these functions concurrent, we simply add the decorator:
@concurrentdef roll_linguine(count : int):    rollingpin = Roller()    pastas = []    for i in range(0, count):        dough = make_dough(pasta = True)        while dough.height >= 3:            rollingpin.roll_thinner(dough)        while dough.width > 2:            rollingpin.cut(dough)        pastas.append(dough)    return(pastas)@concurrentdef roll_pizzas(count : int):    rollingpin = Roller()    pizzas = []    for i in range(0, count):        dough = make_dough()        while dough.height > 3:            rollingpin.roll_thinner(dough)        while dough.width > 2:            rollingpin.cut(dough)
I am afraid we have made a mistake in our pizza and linguine endeavors prior... Unfortunately, we are going to have over 500 guests this evening! We will need to make a lot of pizza and linguine if we want to ensure that every guest is going to get fed. Each guest needs 2 pizzas (I invited the pizza enthusiast convention,) and at least 20 linguine noodles (pizza lovers are also pasta lovers in most cases.) Fortunately, since the two functions I have used for those two abilities are concurrent, I can synchronize them using the synchronized decorator from deco!
# linguine e pizza? molta bellisima!@synchronizeddef prepare_meal(count : int):    roll_linguine(count * 200)    roll_pizzas(count * 2)%timeit prepare_meal(500)
The pizza/pasta factory has quite a bit of fan noise, but the good news is this all ran — albeit with my deprecation warning from earlier IN EACH CONSECUTIVE LOOP, completely ruining my output.
Cache tools is another great package for helping your Python performance. Packages like that are great because they really do make the Python experience a lot better. Really well written Python code can be pretty fast, but the reality is that we are in a scripting language. We need as much performance as we can get. Additionally, we cannot always write flawless code! That in mind, let us take a look at another decorator which will enhance the performance of our Python package! The decorator is called cached, and it can take a lot of different arguments to change what type of cache is used, and the possible sizes of those caches.
All of this will also come in handy for our little linguine/pizza business we have started. As this function can be used to store previous calculations for future calls. The good news is that this means we can now have our pizza rollers get really good at rolling pizzas, but they still get tired after they roll out 100 of them, or whatever the set amount will be.
from cachetools import cached, LRUCache, TTLCache
We can provide a cache key-word argument in order to determine what kind of cache to use. One thing to note is that these caches all behave in different ways. If you are using this code for a large application that will likely have a lot of these, but you do not want your cache to be overwritten for the time it is worked with, you might want to use TTL cache. This cache disappears after 10 minutes, but will not be overwritten. LRU cache on the other hand can be overwritten, but can last forever — it is better choice for general purpose use cases. There is also a version that does not have the arguments that will work similarly to LRU I believe, just might use some other algorithm — to confess, I am not entirely sure.
Tomorrow is yet another library that can be used to speed up the performance of the Python programming language. The decorator for this is called threads. It can be called with a positional argument to indicated how many threads we would like to dedicate to the task, and then we can set key-word arguments — for example, timeouts.
from tomorrow import threads@threads(5)def prepare_meal(count : int):    roll_linguine(count)    roll_pizzas(count)
And here is an example using the timeout key-word argument:
@threads(5, timeout = 1)def prepare_meal(count : int):    roll_linguine(count)    roll_pizzas(count)
Tenacity is a more unique package that can be used to try something again in the case that some code does not execute properly. It can be used with in-development software, or perhaps with requests if they do not get fulfilled. There are hundreds of uses for such a thing, and what is great is that because it is wrapped into this decorator, it can be used incredibly easily! Given the nature of this package, the only way I could think up a demonstration in a notebook was to have a throw with a print.
from tenacity import retry@retrydef ohno():    print("function ran once")    try:        2 == 3 - 5 == 6    except ValueError:        print("Idk what that was even meant to do")ohno()
Whenever I ran this, it ran once and then crashed my kernel, that being said — I then retried it outside of the notebook — so there is something to note about this package, this is not so usable in the notebook.
The final module decorator we are going to look at is going to be the property decorator. This one is from the standard library, and the decorator portion is used to turn a method into a “ getter” for a read-only attribute of the same name. In other words, we are simply making this attribute a read-only copy of the prior attribute.
class PowerLine:    def __init__(self):        self._voltage = 100000    @property    def voltage(self):        return self._voltage
Decorators are quite an awesome feature in the Python programming language. They are particularly awesome because they can be used to change the way that a given type interacts with the methods around it. I find this to be very exciting personally. Another great thing about decorators is that they can often speed up Python, which I believe to be quite desirable given Python’s scripted nature.
Hopefully armed with this list of decorators your Python functions will speed up and change! Thank you very much for reading my article, it means the world! I hope some of these decorators make you a better Pythonista!