The Correct Way to Overload Functions in Python
Tags: python, tutorial
Published: 2021-05-31 18:58:14


Function overloading is a common programming pattern which seems to be reserved to statically-typed, compiled languages. Yet there's an easy way to implement it in Python with help of _Multiple Dispatch_ or as it's called in Python _multimethods_.

## Overloading

First things first - you might be asking, how can we implement method overloading in Python when we all know that it's not possible? Well, even though Python is dynamically-typed language and therefore cannot have proper method overloading as that requires the language to be able to discriminate between types at compile-time, we can still implement it in a bit different way that is suitable for dynamically-typed languages.

This approach is called _Multiple Dispatch_ or _multimethods_, where the interpreter differentiates between multiple implementations of a function/method at runtime based on dynamically determined types. To be more precise, the language uses types of arguments passed to a function during its invocation to dynamically choose which one of the multiple function implementations to use (or dispatch).

Now you might be thinking: _"Do we really need this though? If it can't be implemented normally, maybe we shouldn't use it in Python..."_ Yea, valid point, but there are good reasons to want to implement some form of function/method overloading in Python. It's powerful tool that can make code more concise, readable and minimise its complexity. Without multimethods though, the _"obvious way"_ to do this is using type inspection with `isinstance()`. This is very ugly, brittle solution that is closed to extension and I would call it an anti-pattern.

Besides that, there already is method overloading in Python for operators and methods like `len()` or `new()` using so-called _dunder_ or _magic_ methods (see docs [here](https://docs.python.org/3/reference/datamodel.html#special-method-names)) and we all use that quite often, so why not use proper overloading for _all_ the function, right?

So, now we know that we can kind-of implement overloading in Python, so how exactly do we do that?

## Single Dispatch

Above we spoke about _Multiple Dispatch_, but Python doesn't support this out-of-the box, or in other words _Multiple Dispatch_ is not a feature of Python standard library. What is available to us however, is called _Single Dispatch_, so let's begin with this simpler case first.

The only actual difference between _multi_ and _single_ dispatch is number of arguments which we can overload. So, for this implementation in standard library it's just one.

The function (and decorator) that provides this feature is called `singledispatch` and can be found in `functools` [module](https://docs.python.org/3/library/functools.html#functools.singledispatch).

This whole concept is best explained with some examples. There are many _"academic"_ examples of overloading functions (geometric shapes, addition, subtraction...) that we've probably all seen already. Rather than going over that, let's see some practical examples. So, here's first example for `singledispatch` to format dates, times and datetimes:

```python
from functools import singledispatch
from datetime import date, datetime, time

@singledispatch
def format(arg):
    return arg

@format.register
def _(arg: date):
    return f"{arg.day}-{arg.month}-{arg.year}"

@format.register
def _(arg: datetime):
    return f"{arg.day}-{arg.month}-{arg.year} {arg.hour}:{arg.minute}:{arg.second}"

@format.register(time)
def _(arg):
    return f"{arg.hour}:{arg.minute}:{arg.second}"

print(format("today"))
# today
print(format(date(2021, 5, 26)))
# 26-5-2021
print(format(datetime(2021, 5, 26, 17, 25, 10)))
# 26-5-2021 17:25:10
print(format(time(19, 22, 15)))
# 19:22:15
```

We begin by defining the base `format` function that is going to be overloaded. This function is decorated with `@singledispatch` and provides base implementation, which is used if no better options is available. Next, we define individual functions for each type that we want to overload - in this case `date`, `datetime` and `time` - each of these have name `_` (underscore) because they will be called (dispatched) through the `format` method anyway, so no need to give them useful names. Each of them is also decorated with `@format.register` which attaches them to the previously mentioned `format` function. Then, to make it possible to differentiate between types, we have two options - we can use type annotations - as demonstrated in first two cases or explicitly add the type to decorator as with the last one from the example.

In some cases it might make sense to use same implementation for multiple types - for example for number types such as `int` and `float` - for these situations decorator stacking is allowed, meaning that you can list (stack) multiple `@format.register(type)` lines to associate a function with all the valid types.   

Besides ability to overload basic functions, `functools` module contains also [`singledispatchmethod`](https://docs.python.org/3/library/functools.html#functools.singledispatchmethod) that can be applied to methods of a class. Example of that could be the following:

```python
from functools import singledispatchmethod
from datetime import date, time

class Formatter:
    @singledispatchmethod
    def format(self, arg):
        raise NotImplementedError(f"Cannot format value of type {type(arg)}")

    @format.register
    def _(self, arg: date):
        return f"{arg.day}-{arg.month}-{arg.year}"

    @format.register
    def _(self, arg: time):
        return f"{arg.hour}:{arg.minute}:{arg.second}"

f = Formatter()
print(f.format(date(2021, 5, 26)))
# 26-5-2021
print(f.format(time(19, 22, 15)))
# 19:22:15
```

## Multiple Dispatch

Oftentimes _Single Dispatch_ won't be sufficient and you might need the proper _Multiple Dispatch_ functionality. This is available from `multipledispatch` module which can be found [here](https://pypi.org/project/multipledispatch/) and can be installed with `pip install multipledispatch`.

This module and it's decorator - `@dispatch`, behaves very similarly to the `@singledispatch` in the standard library. Only actual difference is that it can take multiple types as arguments:

```python
from multipledispatch import dispatch

@dispatch(list, str)
def concatenate(a, b):
    a.append(b)
    return a

@dispatch(str, str)
def concatenate(a, b):
    return a + b

@dispatch(str, int)
def concatenate(a, b):
    return a + str(b)


print(concatenate(["a", "b"], "c"))
# ['a', 'b', 'c']
print(concatenate("Hello", "World"))
# HelloWorld
print(concatenate("a", 1))
# a1
```

The above snippet shows how we can use `@dispatch` decorator to overload multiple arguments, for example to implement concatenation of various types. As you probably noticed, with `multipledispatch` library we didn't need to define and register base function, rather we created multiple functions with same name. If we wanted to provide base implementation, we could use `@dispatch(object, object)` which would catch any non-specific argument types.

The previous examples shows proof-of-concept, but if we wanted to really implement such `concatenate` function, we would need to make it much more generic. This can be solved with use of union types. In this specific example we could change the first function as follows:

```python
@dispatch((list, tuple), (str, int))
def concatenate(a, b):
    return list(a) + [b]

print(concatenate(["a", "b"], "c"))
# ['a', 'b', 'c']
print(concatenate(("a", "b"), 1))
# ['a', 'b', 1]
```

This would make it so that first argument of the function could be any of `list` or `tuple`, while second one would be `str` or `int`. This is already much better than the previous solution, but it can be further improved using abstract types. Instead of listing all the possible sequences, we can use `Sequence` abstract type (assuming that our implementation can handle it) which covers things like `list`, `tuple` or `range`:

```python
from collections.abc import Sequence

@dispatch(Sequence, (str, int))
def concatenate(a, b):
    return list(a) + [b]
```

If you want to take this approach, then it's good to take a look at `collections.abc` module and see which container data-type best suits your needs. Mostly to make sure that your function will be able to handle all the types that fall into the chosen container.

All this mixing and matching of argument types is convenient, but can also cause ambiguities when choosing suitable function for some specific set of parameters. Fortunately, `multipledispatch` provides `AmbiguityWarning` which is raised if ambiguous behaviour is possible:

```python
test_multipledispatch:10: AmbiguityWarning:
Ambiguities exist in dispatched function some_func

The following signatures may result in ambiguous behavior:
    [str, object], [object, str]


Consider making the following additions:

@dispatch(str, str)
def some_func(...)
```

## Closing Thoughts

In this article we went over a simple, yet powerful concept which I rarely see being used in Python, which is a shame considering that it can greatly improve code readability and get rid of anti-patters like type inspection using `isinstance()`. Also, I hope you would agree that this approach to function overloading should be considered the _"obvious way"_ and I hope that you will make use of it when needed.

If you want to dive deeper into this topic and get your hands dirty you can implement multimethods yourselves as shown in [Guido's article](https://www.artima.com/weblogs/viewpost.jsp?thread=101605) - this can be a good exercise to understand how multiple dispatch actually works.

Finally, I should also probably mention that this article omits examples of the well-known [operator overloading](https://docs.python.org/3/reference/datamodel.html#basic-customization) which I mentioned in the beginning as well as some approaches for overloading constructors for example using [factories](https://stackoverflow.com/a/141777). So, in case that's what you're looking for, go check out these links/resources, which give good overview on there topics.