GOF inspired python decorators
Tags: designpattens, functional, decorators, python
Published: 2019-09-30 08:21:45

---
title: GOF inspired python decorators
published: true
description: Inspired by GOF patterns, implemented with methods and decorators
tags: design pattens, functional programming, decorators, python
---

This post was originally posted in https://www.nacnez.com/gof-inspired-decorators.html

### The back story
As part of day to day development work, I have been trying to apply functional concepts wherever it makes sense. I have long been a OO developer and hence some of the effects stay with you. Especially design patterns. In my opinion, design patterns - primarily [GOF patterns](https://en.wikipedia.org/wiki/Design_Patterns), though referenced in the context of OO design and languages, have some good ideas that can apply elsewhere too. With languages that support functional features, I believe some of these patterns can be applied at a method/function level instead of carrying around the cruft of classes for just doing behavior (OO enthusiasts bear with me - Classes are great in the right places, but in others they are just there because a language forces you to use [them](http://bit.ly/2mTnlZz).

I am also a fan of Python decorators. Decorators are a very cool and powerful feature of python. It is syntactic sugar to create higher order functions - a very important functional paradigm feature. We can combine multiple functions to get work done - composability. You can read a lot about decorators [here](http://bit.ly/2lmNNKn) - a highly recommended read. I will referring to it throughout this post.

So given my background and my current love (functional programming and decorators), I have had a chance to get inspired by GOF pattens and use decorators to create some clean code as part of my work. Here I am going to share that attempt with you.

### Ok, let us get in
When you look at the general use and applicability of python decorators they feel like a ready-made way to replace the traditional [Decorator pattern](https://en.wikipedia.org/wiki/Decorator_pattern) - and for the code lovers you can look [here](https://sourcemaking.com/design_patterns/decorator/python/1). This thought is not knew and you will see it mentioned in [other places](https://yos.io/2013/07/05/decorator-pattern/) too. Theoretically, this is not exactly correct (my opinion). A class level decorator is a structural pattern which allows me to decorate multiple behaviors of the object or component. But in practical terms, I have seen that this does not pan out and method level decorators (using python decorators) are more prevalent.

As I played more with python decorators, I realized that there is one more of my favorite patterns that I could implement using decorators. That is the [Chain or Responsibility](https://en.wikipedia.org/wiki/Chain-of-responsibility_pattern) pattern (CoR) - [code sample](https://sourcemaking.com/design_patterns/chain_of_responsibility/python/1). CoR is a behavioral pattern and behaviors are generally managed at method level. Hence decorators feel like a great fit for CoR.

### Enough talk!
It is really difficult to explain my thinking with english words. So let us get down to writing some python words/code. I need an example to illustrate what this.

#### The example
Here is a contrived example (of course) - a calculator. And since I love [TDD](https://en.wikipedia.org/wiki/Test-driven_development) and [py.test](https://docs.pytest.org/en/latest/) let us start with a test which can drive our code.

```python
    import pytest
    import app.natural_number_calc as calc

    @pytest.mark.parametrize("operator, arg1, arg2, output", [
        ('+',3,2,5),
        ('+',4,2,6),
        ('-',22,12,10),
        ('-',24,4,20),
        ('*',12,7,84),
        ('*',11,11,121),
        ('/',84,6,14),
        ('/',11,11,1),
        ('^',11,2,121),
        ('^',4,3,64),
    ])
    def test_calculator(operator, arg1, arg2, output):
        assert output == calc.do(operator, arg1, arg2)


    def test_calculator_operator_not_supported():
        with pytest.raises(ValueError) as ve:
            calc.do('%', 5, 10)

        assert 'Calc Error - Operator not supported' in str(ve.value)


    @pytest.mark.parametrize("operator, arg1, arg2", [
        ('+',3,0),
        ('+',4,-2),
        ('-',22,0),
        ('-',-24,4),
        ('*',12,0),
        ('*',11,-4),
        ('/',84,0),
        ('/',12,-2),
        ('^',11,-2),
        ('^',-4,2),
    ])
    def test_calculator_non_natural_numbers_not_supported(operator, arg1, arg2):
        with pytest.raises(ValueError) as ve:
            calc.do(operator, arg1, arg2)

        assert 'Calc Error - Natural numbers only supported' in str(ve.value)

```
 

The test suite covers everything which I want to implement with my calculator. So we have a start.

#### That is some c***!
For satisfying the test suite, here is some plain vanilla code.

```python
    def do(operator, arg1, arg2):
        if operator == '+':
            return _add(arg1, arg2)
        elif operator == '-':
            return _subtract(arg1, arg2)
        elif operator == '*':
            return _multiply(arg1, arg2)
        elif operator == '/':
            return _divide(arg1, arg2)
        elif operator == '^':
            return _power(arg1, arg2)
        else:
            raise ValueError('Calc Error - Operator not supported')

    def _power(arg1, arg2):
        if arg1 < 1 or arg2 < 1:
            raise ValueError('Calc Error - Natural numbers only supported')
        return arg1 ** arg2

    def _divide(arg1, arg2):
        if arg1 < 1 or arg2 < 1:
            raise ValueError('Calc Error - Natural numbers only supported')
        return arg1 / arg2

    def _multiply(arg1, arg2):
        if arg1 < 1 or arg2 < 1:
            raise ValueError('Calc Error - Natural numbers only supported')
        return arg1 * arg2

    def _subtract(arg1, arg2):
        if arg1 < 1 or arg2 < 1:
            raise ValueError('Calc Error - Natural numbers only supported')
        return arg1 - arg2

    def _add(arg1, arg2):
        if arg1 < 1 or arg2 < 1:
            raise ValueError('Calc Error - Natural numbers only supported')
        return arg1 + arg2

```


Yeah, I know. This is pathetic code... so many `if`s and `elif`s and all. That is on purpose so that we can improve it with some _decorator and pattern_ goodness. And it is code that works and hence our test passes! We are on _green_ mode & we can start refactoring.

#### Let us start decorating
If you look at the above code we see an obvious copy paste case. The check for natural numbers is repeated on every operation code and we can easily **decorate** each operation code using the Decorator pattern or in our case python decorators. Let us first look at the decorator code.

```python
    def only_natural(func):
        def wrapper(arg1, arg2):
            if arg1 < 1 or arg2 < 1:
                raise ValueError('Calc Error - Natural numbers only supported')

            return func(arg1, arg2)
    return wrapper
```

If you know your decorators, this is a [simple decorator](https://realpython.com/primer-on-python-decorators/#simple-decorators) in action. It takes the function that needs to be wrapped. It defines an inner wrapper function which take two arguments which checks the arguments are natural numbers. If they are then the wrapped function is called else raise an error. Let us now use this decorator on our calculator module.

```python
    def do(operator, arg1, arg2):
        if operator == '+':
            return _add(arg1, arg2)
        elif operator == '-':
            return _subtract(arg1, arg2)
        elif operator == '*':
            return _multiply(arg1, arg2)
        elif operator == '/':
            return divide(arg1, arg2)
        elif operator == '^':
            return _power(arg1, arg2)
        else:
            raise ValueError('Calc Error - Operator not supported')

    @only_natural # decorator applied
    def _power(arg1, arg2):
        return arg1 ** arg2     #1

    @only_natural
    def divide(arg1, arg2):
        return arg1 / arg2      #2

    @only_natural
    def _multiply(arg1, arg2):
        return arg1 * arg2      #3

    @only_natural
    def _subtract(arg1, arg2):
        return arg1 - arg2      #4

    @only_natural
    def _add(arg1, arg2):
        return arg1 + arg2      #5

```

You can see that our calculator is surely improved (note the numbers) using the _decorator_ pattern created with decorators. But the `do` method is still a eyesore. With just 5 operations, we have a huge `if/elif/else` clause and this will only grow further if we want to support more operations (I understand this is a toy example but you get the picture).

#### Enter CoR
So how can we improve this. Let us dig in. It is clear that for each operation, there is a corresponding method to handle it. The handling determination happens one after the other. It almost feels like a set of actions to do... a chain of things to do... a Chain of Responsibilities to complete (come on, that is not such a bad lead up!). Let us give it a shot once.

Before we get to the code let us understand a bit. A classic COR goes roughly like this. A request which needs to be processed is given to the first link in the chain of processors. That processor either processes it if it can or passes it on to the next processor. A given processor knows what it can process and also knows who is the next one in the chain. That is pretty much the essence of it. The class based style is already referred earlier, so let us try to do this with decorators which can work at method level.

First shot:

```python
    def cor(next):
        def wrapper(current):
            def inner(*args):
                output = current(*args)
                return output if output else next(*args) if next else None
            return inner
        return wrapper
```


This is a more involved decorator than the first one. Because we need to take an argument, we need this double nested structure. The outer most `cor` function is the visible annotation part of the decorator and it takes the function object that needs to be called next in the chain as argument. The `cor` function defines a `wrapper` function which is the one that accepts the actual function that is being decorated - the `current`. The `wrapper` in turn has the `inner` function where the actual work happens. It calls the current function and gets its response. If that response is valid (a simple _truthy_ response), then it means that request has been processed and the chain can be broken to return the result. If the response is not valid, it means that the current function is not the one to handle the request. Hence the control has to pass on to the next processor or function (which is available to `inner` because it is a [closure](https://en.wikipedia.org/wiki/Closure_(computer_programming)) - another functional feature of python. You can read more about decorators that take arguments [here](https://realpython.com/primer-on-python-decorators/#decorators-with-arguments).

For this to [nest](https://realpython.com/primer-on-python-decorators/#nesting-decorators) along with the already existing decorator for natural numbers, we need to do some changes to that decorator.

```python
    def only_natural_with_operator(func):
        def wrapper(operator, arg1, arg2):
            if arg1 < 1 or arg2 < 1:
                raise ValueError('Calc Error - Natural numbers only supported')

            return func(operator, arg1, arg2)
        return wrapper
```

The above one is just a simple tweak, so let us move on. Let us look at the calculator to figure out how its usage gets manifested.

#### What the CoR?

```python
    def do(operator, arg1, arg2):
        value = _add(operator, arg1, arg2)
        if value:
            return value
        else:
            raise ValueError('Calc Error - Operator not supported')

    @only_natural_with_operator
    def _power(operator, arg1, arg2):
        if operator == '^':
            return arg1 ** arg2
        else:
            return None

    @only_natural_with_operator
    @cor(next=_power)
    def _divide(operator, arg1, arg2):
        if operator == '/':
            return arg1 / arg2
        else:
            return None

    @only_natural_with_operator
    @cor(next=_divide)
    def _multiply(operator, arg1, arg2):
        if operator == '*':
            return arg1 * arg2
        else:
            return None

    @only_natural_with_operator
    @cor(next=_multiply)
    def _subtract(operator, arg1, arg2):
        if operator == '-':
            return arg1 - arg2
        else:
            return None

    @only_natural_with_operator
    @cor(next=_subtract)
    def _add(operator, arg1, arg2):
        if operator == '+':
            return arg1 + arg2
        else:
            return None
```


Hmm... Pretty underwhelming to put it nicely. But before we get into it, let me first explain what this is doing.

The `do` method just calls the first method in the chain, the `_add` method. It expects that the processing gets completed to return a valid result. If it gets back an invalid result (`None`), then it understands that this calculation cannot be processed and throws an Error. The `do` method has improved for sure.

The `_add` method declares the next operator/function to call as decorator parameter - (`@cor(next=_subtract)`). In the body it checks if it can process the request - _is it the right operator?_ . If it is then great, else it returns `None` denoting that it is not interested. This happens in each operator/function till we reach `_power` which has no next. So the chain stops here. We can of course extend the chain from this point onwards and for that we don't have to touch the `do` or the `_add` (or other) methods. All the methods are separated. This `cor` decorator could be used to create a chain of any set of functions as long as they follow the basic contract of returning a truthy value if they did the processing or a falsy value if they want to pass it on (provided if they have defined a next).

#### CoR 2.0?
Now back to that bad feeling we get on seeing the resultant code. In the process of introducing CoR to save the `do` method, the operation functions have lost their charm. They are crowded now and that is not what we want. And when we look closely, we see that each operation function does one common thing: it checks if the operation is what it can handle, before it actually does the real work. The error handling in the `do` function looks similar in some way too. All these seem to be common behavior which can be applied to these functions using a _wrapper/decorator_ - is it not? Let us get our decorator pattern back now. The improved cor (2.0) decorator looks like this.

```python
    def calc_cor(my_operator, next=None):
        def wrapper(current):
            def inner(*args):
                if args[0] == my_operator:
                    return current(*args)
                elif next:
                    return next(*args)
                else:
                    raise ValueError('Calc Error - Operator not supported')
            return inner
        return wrapper
```

This is no longer the very generic `cor` decorator we started with but we sort of expected that. This is a decorated-calculator-specific-CoR - `calc_cor`. This decorator takes two arguments. The first one is the operator supported by the current function and the second one is the next function. It also expects the the _current_ function (which is being decorated) to always takes the operator symbol as the first argument - the implicit contract. Since the operator is passed, the check of applicability can be done in the decorator itself. Also it does the error handling. This is the _decorated_ part of the new CoR. Now let us see how this changes our calculator code.

```python
    def do(operator, arg1, arg2):
        return _add(operator, arg1, arg2)


    @only_natural_with_operator
    @calc_cor(my_operator='^')
    def _power(operator, arg1, arg2):
        return arg1 ** arg2


    @only_natural_with_operator
    @calc_cor(my_operator='/', next=_power)
    def _divide(operator, arg1, arg2):
        return arg1 / arg2


    @only_natural_with_operator
    @calc_cor(my_operator='*', next=_divide)
    def _multiply(operator, arg1, arg2):
        return arg1 * arg2


    @only_natural_with_operator
    @calc_cor(my_operator='-', next=_multiply)
    def _subtract(operator, arg1, arg2):
        return arg1 - arg2


    @only_natural_with_operator
    @calc_cor(my_operator='+', next=_subtract)
    def _add(operator, arg1, arg2):
        return arg1 + arg2
```

That looks way better than what we had before. The `do` method just calls the first link in the chain. Each chain link just does its processing. Everything else is just declared as decorator arguments and we are done. This combination of COR and Decorator pattern using decorators seems to have produced the best results. Wouldn't you agree?

#### Centralized CoR -3.0!?!
I showed this result to Sathia, a [friend](https://twitter.com/sathia27) and colleague of mine (that was different production code but the concept is the same). He pointed out something. With this design somebody trying to add a new operator has to figure out where in the chain she needs to add it. Rather she has to figure out where the current chain ends and add the new one there. In the above example that is simple. In real world code this may or may not be easy. He wanted to see if there is a way for each operator function to just register itself and then the chain would execute them. Of course in all these cases we are talking with the underlying premise that the operator processing order does not matter.

So some more thinking is needed. Can we make the CoR satisfy this? For this, we probably have to give away the decentralized nature of the current CoR. We need some kind of centralization. We need some way to register operation functions to a common place and then _pull the chain_ to execute them all. Here goes another shot.

```python
    CHAIN = []

    def link_to_chain(predicate):
        def wrapper(operator):
            CHAIN.append((predicate,operator))
            return operator
        return wrapper


    def pull_chain(*args):
        for predicate,operator in CHAIN:
            if predicate(*args):
                return operator(*args)

        raise ValueError('Calc Error - Operator not supported')
```

The first function `link_to_chain` is a decorator function which registers (or links) the _operator_ and its _predicate_ into a registry or chain `CHAIN`. The predicate is nothing but a function or lambda which provides applicability check. In this case the decorator is not really doing any decoration (no pre or post processing). It is more a way to [plug things](https://realpython.com/primer-on-python-decorators/#registering-plugins) in.

The `pull_chain` function executes the CoR. It runs through the chain of handlers, uses the registered predicate to find the right one, executes them and breaks out of the chain. If there is none found then it _raises_.

With this idea now the calculator looks like this

```python
    def do(operator, arg1, arg2):
        return pull_chain(operator, arg1, arg2)


    @link_to_chain(lambda *args: args[0]=='^')
    @only_natural_with_operator
    def _power(operator, arg1, arg2):
        return arg1 ** arg2


    @link_to_chain(lambda *args: args[0]=='*')
    @only_natural_with_operator
    def _multiply(operator, arg1, arg2):
        return arg1 * arg2


    @link_to_chain(lambda *args: args[0]=='/')
    @only_natural_with_operator
    def _divide(operator, arg1, arg2):
        return arg1 / arg2


    @link_to_chain(lambda *args: args[0]=='+')
    @only_natural_with_operator
    def _add(operator, arg1, arg2):
        return arg1 + arg2


    @link_to_chain(lambda *args: args[0]=='-')
    @only_natural_with_operator
    def _subtract(operator, arg1, arg2):
        return arg1 - arg2
```


Not bad at all! Actually looks pretty good to me. Individual functions link to the chain and the `do` method pulls the chain. The single responsibility of the functions shine through. Given that the chain is centralized, the individual functions don't even care about who is next. All they do is register into the chain along with their predicate. This is probably the cleanest solution we can get as of now. Time to stop and take a break!

### Closing thoughts
Python is great language with support for useful functional features. Decorators are a sweet way of doing functional programming in python. The great thing I realized in this attempt is, when we try to do functional programming, your earlier learnings of GOF design patterns don't go waste. The ideas still make sense. All we need to do was to tweak them a bit to make them applicable in a functional context. And lo behold we have much better code than where we started ...the `if/elif/else` blob in the beginning... Let us keep learning and try to extract out the essence of the things we learn. Then we might actually be able use it in more than one places and in more than one ways.

Happy development to all in the festive season! Please chime in with your thoughts and comments. Your criticisms and improvements are most welcome since I  learn a lot from them. See you soon.


p.s: You can get all this code [here](http://bit.ly/2mMojaa)