Understanding Decorators in Python
Tags: python, functional
Published: 2022-02-24 17:10:56

## What are decorators

Decorators are *wrappers* around Python functions (or classes) that change how these classes work. A decorator abstracts its own functioning as far away as possible. The Decorator notation is designed to be as minimally invasive as possible. A developer can develop his code within his domain as he is used to and only use the decorator to extend the functionality. Because this sounds very abstract, let's look at some examples.

In Python, decorators are used primarily *to decorate* functions (or methods, respectively). Maybe, one of the most commonly used decorators is the `@property` decorator:

```python
class Rectangle:
    def __init__(self, a, b):
        self.a = a
        self.b = b

    @property
    def area(self):
        return self.a * self.b

rect = Rectangle(5, 6)
print(rect.area)
# 30
```

As you see in the last line, you can access the `area` of our `Rectangle` like an attribute, i.e., you don't have to *call* the `area` method. Instead, when accessing `area` like an attribute (without the `()`) the method is called implicitly because of the `@property` decorator.

### How does it work?

Writing `@property` in front of a function definition is the equivalent to writing `area = property(area)`. In other words: `property` is a function that takes another function as an argument and returns a third function. And this is exactly what decorators do.

As a result, decorators *change* the behaviour of the decorated function. <!--  Common use cases include TODO -->

## Writing Custom Decorators

### Retry Decorator

<!-- Now that we've seen how to use decorators, it's time to write them. -->
With that vague definition, let's write our own decorators to understand how they work.

Let's say we have a function that we want to retry if it fails. We need a function (our decorator) that calls our function once or twice (depending on whether it failed the first time).

According to our initial definition of a decorator, we could write this simple decorator like this:

```python
def retry(func):
    def _wrapper(*args, **kwargs):
        try:
            func(*args, **kwargs)
        except:
            time.sleep(1)
            func(*args, **kwargs)
    return _wrapper

@retry
def might_fail():
    print("might_fail")
    raise Exception

might_fail()
```

`retry` is the name of our decorator, which accepts any function as an argument (`func`). Inside the decorator, a new function (`_wrapper`) is defined and returned. It can look somewhat unfamiliar to define a function *inside* another function at first sight. However, this is syntactically perfectly fine and has the advantage that our `_wrapper` function is just valid inside the namespace of our `retry` decorator.

*Note* that in this example, we decorated our function just with `@retry`. There are no parentheses (`()`) after the `@retry` decorator. Thus, when calling our `might_fail()` function, the `retry` decorator is called with our function (`might_fail`) as a first argument.

In total, we handle three functions here:

- `retry` 
- `_wrapper`
- `might_fail`

In some cases, we need the decorator to accept arguments. In our case, we could make the number of retries a parameter. However, a decorator *must* take our decorated function as the first argument. Remember that we did not need to *call* our decorator when decorating a function with it, i.e. we just wrote `@retry` as opposed to `@retry()` in front of our decorated function definition.

- The *decorator* is nothing else than a function (*which accepts another function as argument*)
- The *decorator* is used by putting it in front of a function definition without *calling* it

Hence, we could introduce a fourth function which accepts the parameter we want as configuration and returns a function that actually is a decorator (*which accepts another function as argument*).

Let's try this:

```python
def retry(max_retries):
    def retry_decorator(func):
        def _wrapper(*args, **kwargs):
            for _ in range(max_retries):
                try:
                    func(*args, **kwargs)
                except:
                    time.sleep(1)
        return _wrapper
    return retry_decorator


@retry(2)
def might_fail():
    print("might_fail")
    raise Exception


might_fail()
```

Tearing that one apart:

- On the first level, we have a function called `retry`.
- `retry` accepts an arbitrary argument (`max_retries` in our case) and returns a function
- `retry_decorator` is the function returned by `retry` and is our actual decorator
- `_wrapper` works in the same way as before (it now just obeys the maximum number of retries)

That's for the definition of our decorator. 

- `might_fail` is decorated by a function **call** this time, i.e. `@retry(2)`.
- `retry(2)` cause the function `retry` to be called and it returns the actual decorator
- `might_fail` is eventually decorated by `retry_decorator` as this function is the result of the `retry(2)` **call**.


### Timer Decorator

Here is another example of a useful decorator: Let's create a decorator which measures the runtime of the functions decorated with it.

```python
import functools
import time

def timer(func):
    @functools.wraps(func)
    def _wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        runtime = time.perf_counter() - start
        print(f"{func.__name__} took {runtime:.4f} secs")
        return result
    return _wrapper

@timer
def complex_calculation():
    """Some complex calculation."""
    time.sleep(0.5)
    return 42

print(complex_calculation())
```

Output:

```
complex_calculation took 0.5041 secs
42
```

As we see, the `timer` decorator executes some code before and after the *decorated* function and works in the exact same way as in the last example.

#### `functools.wraps`

You might have noticed that the `_wrapper` function itself is decorated with  `@functools.wraps`. This does not in any way change the logic or functionality of our `timer` decorator. **You could as well decide to not use `functools.wraps`**.

However, since our `@timer` decorator could have as well been written as: `complex_calculation = timer(complex_calculation)`, the decorator necessarily changes our `complex_calculation` function. Insbesondere, it changes some of the magic reflection attributes:

- `__module__`
- `__name__`
- `__qualname__`
- `__doc__`
- `__annotations__`

When using `@functools.wraps`, these attributes are set back to their originals


**Without `@functools.wraps`**

```python
print(complex_calculation.__module__)       # __main__
print(complex_calculation.__name__)         # wrapper_timer
print(complex_calculation.__qualname__)     # timer.<locals>.wrapper_timer
print(complex_calculation.__doc__)          # None
print(complex_calculation.__annotations__)  # {}
```

**With `@functools.wraps`**

```python
print(complex_calculation.__module__)       # __main__#
print(complex_calculation.__name__)         # complex_calculation
print(complex_calculation.__qualname__)     # complex_calculation
print(complex_calculation.__doc__)          # Some complex calculation.
print(complex_calculation.__annotations__)  # {} 
```


## Class Decorators

So far, we have just looked at decorators for *functions*. It's, however, possible to decorate *classes*, too.

Let's take the `timer` decorator from the example above.
It's perfectly fine to wrap a class with this decorator like so:


```python
@timer
class MyClass:
    def complex_calculation(self):
        time.sleep(1)
        return 42

my_obj = MyClass()
my_obj.complex_calculation()
```


The result?

```
Finished 'MyClass' in 0.0000 secs
```

So, there is obviously no timing printed for our `complex_calculation` method. Remember that the `@` notation is just the equivalent for writing `MyClass = timer(MyClass)`, i.e., the decorator will get called only when you "call" the class. *Calling* a class means instantiating it, so the timer is only executed at the line `my_obj = MyClass()`.

**Class methods are *not* automatically decorated when decorating a class**. To put it simple, using a *normal* decorator to decorate a *normal* class decorates its constructor (`__init__` method), only.

However, you can change the behaviour of a class as a whole by using another form of a constructor. However, let's first see if decorators can work the other way around, i.e. whether we can decorate a function with a class. Turns out we can:


```python
class MyDecorator:
    def __init__(self, function):
        self.function = function
        self.counter = 0
    
    def __call__(self, *args, **kwargs):
        self.function(*args, **kwargs)
        self.counter+=1
        print(f"Called {self.counter} times")


@MyDecorator
def some_function():
    return 42


some_function()
some_function()
some_function()
```

Output:

```
Called 1 times
Called 2 times
Called 3 times
```

The way this works:
- `__init__` is *called* when decorating `some_function`. Again, remember that decorating is just like `some_function = MyDecorator(some_function)`.
- `__call__` is *called* when an instance of a class is used, like calling a function. As `some_function` is now an instance of `MyDecorator` but we still want to use it as a function, the DoubleUnderscore magic method `__call__` is responsible for this.


**Decorating a class** in Python, on the other hand works by changing the class from the outside (i.e., from the decorator).

Consider this:


```python
def add_calc(target):

    def calc(self):
        return 42

    target.calc = calc
    return target

@add_calc
class MyClass:
    def __init__():
        print("MyClass __init__")

my_obj = MyClass()
print(my_obj.calc())
```

Output:
```
MyClass __init__
42
```

Again, if we recap the definition of a decorator, everything which happens here follows the same logic:

- `my_obj = MyClass()` is calling the decorator first
- the `add_calc` decorator *patches* the `calc` method to the class
- eventually, the class is instantiated by using the constructor.


You can use decorators to change classes in a way inheritance would do. If this is a good choice or not heavily depends on the architecture of your Python project as a whole. The standard library's `dataclass` decorator is an excellent example of a sensible usage choosing decorators over inheritance. We'll discuss that in a second.


## Using decorators

### decorators in Python's standard library

In the following sections, we will get to know a few of the most popular and most useful decorators that are already included in the standard library.


### property

As already discussed, the `@property` decorator is probably one of the most commonly used decorators in Python.
It's purpose is that you can access the result of a method like an attribute. Of course, there is also a counterpart to `@property` so that you could call a method behind the scenes when performing an assignment operation.

```python
class MyClass:
    def __init__(self, x):
        self.x = x
    
    @property
    def x_doubled(self):
        return self.x * 2
    
    @x_doubled.setter
    def x_doubled(self, x_doubled):
        self.x = x_doubled // 2

my_object = MyClass(5) 
print(my_object.x_doubled)  #  10  
print(my_object.x)          #  5  
my_object.x_doubled = 100   #    
print(my_object.x_doubled)  #  100 
print(my_object.x)          #  50    
```

### staticmethod

Another familiar decorator is `staticmethod`. This decorator is used when you want to call a function defined inside a class without instantiating the class:

```python
class C:
    @staticmethod
    def the_static_method(arg1, arg2):
        return 42

print(C.the_static_method())
```

### functools.cache

When you deal with functions that carry on a complex calculation, you might want to cache its result.

You could do something like this:


```python
_cached_result = None
def complex_calculations():
    if _cached_result is None:
        _cached_result = something_complex()
    return _cached_result
```


Storing a global variable like `_cached_result`, checking it for `None`, and putting the actual result into that variable if not present are repetitive tasks. This makes an ideal candidate for a decorator. Luckily, there is a decorator in Python's standard library which does exactly this for us:


```python
from functools import cache

@cache
def complex_calculations():
    return something_complex()
```

Now, whenever you call `complex_calculations()`, Python will check for a cached result first before it calls `something_complex`. If there is a result in the cache, `something_complex` will not get called twice. 



### dataclasses

In the section about *class decorators* we saw that decorators can be used to modify the behaviour of classes in the same way inheritance would change it.

The dataclasses module in the standard library is a good example when using a decorator is preferable over using inheritance. Let's first see how to use `dataclasses` in action:


```python
from dataclasses import dataclass

@dataclass
class InventoryItem:
    name: str
    unit_price: float
    quantity: int = 0

    def total_cost(self) -> float:
        return self.unit_price * self.quantity


item = InventoryItem(name="", unit_price=12, quantity=100)
print(item.total_cost())    # 1200
```

On the first sight, the `@dataclass` decorator only added a constructor for us, so we avoided boiler plate code like this:

```python
...
    def __init__(self, name, unit_price, quantity):
        self.name = name
        self.unit_price = unit_price
        self.quantity = quantity
...
```

However, if you decide to build a REST-API for your Python project and need to convert your Python objects into JSON strings.

There is a package called [`dataclasses-json`](https://pypi.org/project/dataclasses-json/) (not in the standard library) which decorates dataclasses and provide the serialisation and deserialisation of objects to JSON strings and vice versa.

Let's see how that looks:

```python
from dataclasses import dataclass
from dataclasses_json import dataclass_json

@dataclass_json
@dataclass
class InventoryItem:
    name: str
    unit_price: float
    quantity: int = 0

    def total_cost(self) -> float:
        return self.unit_price * self.quantity


item = InventoryItem(name="", unit_price=12, quantity=100)

print(item.to_dict())
# {'name': '', 'unit_price': 12, 'quantity': 100}
```

There are two takeaways here:

1. decorators can be nested. The order of their appearance is important.
2. the `@dataclass_json` decorator added a method called `to_dict` to our class

Of course, we could have written a mixin class that does the heavy work of implementing a data type safe `to_dict` method and then let our `InventoryItem` class inherit from that mixin. 

In the present case, however, the decorator only adds a technical functionality (as opposed to an extension within the subject domain). As a result, we can simply switch the decorator on and off without our domain application changing its behaviour. Our "natural" class hierarchy is preserved and no changes need to be made to the actual code. We could also add the `dataclasses_json` decorator to a project without changing existing method bodies.

In such a case, changing a class with a decorator is much more elegant (because it is more modular) than inheriting or using mixins.