Completely Type-Safe Dependency Injection in Python
Tags: functional, python, productivity, programming
Published: 2020-08-20 18:28:44

Simply put, [dependency injection](https://en.wikipedia.org/wiki/Dependency_injection) is a collection of programming techniques that enables software components to have their dependencies replaced, thereby increasing re-usability, for example by allowing a database module to depend on a connection string in order that it can connect to multiple databases. Dependency injection also improves testability because complicated dependencies such as database connections can be easily replaced with mocks.

In this post we'll study dependency injection in Python. We'll see how it can be made completely type safe using functional programming, specifically with a modern functional programming library called [pfun](https://pfun.dev/).

Dependency injection in Python is typically implemented using [monkey patching](https://en.wikipedia.org/wiki/Monkey_patch). In fact, monkey patching as a form of providing dependencies is so common that the `unittest.mock.patch` function was added to the standard library in Python 3.3. 

While monkey patching is a simple technique, it often leads to rather complex patching scenarios, where it's tricky to figure out what and how to patch. Moreover, it can be tricky to achieve complete type safety with monkey patching on both the dependency consumer and provider side. Finally, there are no straight-forward ways of making sure that the dependency provider provides all required dependencies, often leading to statements such as `if dependency is not None`.

An alternative (and potentially even simpler) method for dependency injection is the following: _function arguments_. In fact, this approach is so simple that using the term "dependency injection" to describe it seems almost pretentious. An attractive feature of implementing dependency injection with functions that take arguments is that it is completely type safe by default (provided that you use type annotations of course). Moreover, you can tell a function's dependencies simply by reading its signature, making it totally clear what needs to be provided, and in fact making it _impossible_ to not provide all required dependencies.

The main drawback from using function arguments as your "injection" mechanism, is that functions that call functions that have "injected dependencies" now need to take those dependencies as arguments themselves.

For example, consider a function `connect` that returns a connection to a database given a `connection_str` as argument:
```python
def connect(connection_str: str) -> Connection:
    ...
```
To achieve the promise of "dependency injection", every function that calls `connect` must now take a `connection_str` parameter in addition to its other parameters, in order that the calling function itself can be used against different databases:
```python
def get_user(connection_str: str, user_id: int) -> User:
    connection = connect(connection_str)
    return connection.get_user(user_id)
```
For functions with many dependencies, this quickly becomes _very_ tedious.

Thankfully, we can use functional programming to improve the situation by a margin: Notice how the only use `get_user` has of `connection_str` is to pass it to `connect`. With functional programming we can abstract this pattern of functions taking arguments only to pass them to other functions into some general 'plumbing' functions. This will alow us to write `get_user` without mentioning the `connection_str` at all!

To keep things nice and readable (and save us some typing), lets define a type-alias that represents functions that require dependencies:
```python
from typing import TypeVar, Callable


R = TypeVar('R')
A = TypeVar('A')
Depends = Callable[[R], A]
```
In words, `Depends[R, A]` is a function that depends on something of type `R` to produce something of type `A`. For example, our previous `connect` function is a `Depends[str, Connection]` value: it's a function that depends on a `str` to produce a `Connection`.

Simple enough, but how do we use the `Depends` represented by `connect` in `get_user` without explicitly taking `connection_str` as a parameter and calling `connect`? To do that, let's introduce our first plumbing function `map_`:
```python
B = TypeVar('B')


def map_(d: Depends[R, A], f: Callable[[A], B]) -> Depends[R, B]:
    def depends(r: R) -> B:
        return f(d(r))
    return depends
```
`map_` precisely implements the "passing of parameters" plumbing we were talking about earlier: it creates a new `Depends` value that calls `d`, and passes the result to a function `f`. This allows us to write `get_user` as follows:
```python
def get_user(user_id: int) -> Depends[str, User]:
    return map_(connect, lambda con: con.get_user(user_id))
```
And voila: we have type-safe dependency injection using only `Depends` (which is just a type-alias for functions of 1 argument) and `map_` (which is just a function that [composes](https://en.wikipedia.org/wiki/Function_composition) `Depends` values with functions). Realise that we could keep applying `map_` to `Depends` values to return a final `Depends` all the way back to where we make desicions about what concrete connection strings to pass in (probably in the main section of our program). The pattern we've discovered here seems simple enough that it would be surprising if we were the firsts to think of it, and sure enough it's in fact a common pattern in functional programming called the [reader effect](https://en.wikipedia.org/wiki/Monad_(functional_programming)#Environment_monad).

That's all well and good, but what happens when we want to `map_` functions that return new `Depends` values? For example, let's imagine that in addition to reading user data from databases, our application also needs to call an HTTP api with the user data. Doing so requires authentication credentials that we want to inject. The function that calls the api might look like this:
```python
from typing import Tuple


Auth = Tuple[str, str]


def call_api(user: User) -> Depends[Auth, bytes]:
    def depends(auth: Auth) -> bytes:
        ...
    return depends
```
We might try to use `map_` to pass the result of the `Depends` value returned by `get_user` to `call_api`. But in doing so we would end up with a `Depends[str, Depends[Auth, bytes]]`:
```python
if __name__ == '__main__':
    user_1: Depends[str, User] = get_user(user_id=1)
    call_api_w_user_1: Depends[str, Depends[Auth, bytes]] = map_(user_1, call_api)
```
When we want to call `call_api_w_user_1`, we need to first supply the connection string, and then the auth information:
```python
result: bytes = call_api_w_user_1('user@prod')(('prod_user', 'pa$$word'))
```
This might not seem like big issue in this example, but notice that we might apply `map_` over several functions that produces `Depends` values with the same dependency type, resulting in a situation where we have to pass in the same dependency many times. We might also have a varying number of dependencies if we use `map_` in a loop, leading to a situation where we don't know how many times we need to call the final `Depends`!

To fix this situation, let's introduce another plumbing function that can pass dependencies to both a `Depends` value, and a `Depends` value returned by a function that's being mapped. We'll call it `and_then` since it chains together results of `Depends` values with functions that return new `Depends` values:
```python
from typing import Any


R1 = TypeVar('R1')


def and_then(d: Depends[R, A], f: Callable[[A], Depends[R1, B]]) -> Depends[Any, B]:
    def depends(r: Any) -> B:
        return map_(d, f)(r)
    return depends
```
The observant reader will notice that there's a big problem with the typing of our `and_then` function: it returns a `Depends[Any, B]`! Here you might object: 'But wait a minute, I though you said this was "completely type safe"'! The reason for this epic fail is that the `r` parameter passed to the `depends` function must be passed to both `d`, which means it must be of type `R`, and the `Depends` returned by `f`, which means it must be of type `R1`. In other words, the dependency must be of _both_ type `R` and `R1` _at the same time_. In our example, using `and_then` to combine `get_user` and `call_api` _should_ result in a type that is both a `str` and a `Tuple[str, str]`:
```python
call_api_w_user_1: Depends[?, bytes] = and_then(get_user(user_id=1), call_api)
```
 Such a type is called an _intersection type_ and unfortunately it's not supported by the Python `typing` module ([yet](https://github.com/python/typing/issues/213)). So without introducing third party libraries, this is the best we can do.

This is where the library [pfun](https://pfun.dev/) comes into the picture. In `pfun`, `map_` and `and_then` are instance methods of a `Depends` type, but the idea is exactly the same: 

```python
from pfun import Depends


def connect() -> Depends[str, Connection]:
    ...

def get_user(user_id: int) -> Depends[str, User]:
    connect().map(lambda con: con.get_user(user_id))

def call_api(user: User) -> Depends[Auth, bytes]:
    ...

call_api_w_user_1 = get_user(user_id=1).and_then(call_api)
```



`pfun` provides mechanisms for combining `Depends` values with different dependency types when using [MyPy](http://mypy-lang.org/) with one small requirement: the dependency type must be a `typing.Protocol`. The reason is that intersections of protocol types are simply a new protocol that inherits from both:
```python
from typing import Protocol


class P1(Protocol):
    pass


class P2(Protocol):
    pass


class Intersection(Protocol, P1, P2):
    pass
```
This means that when using `and_then` with `Depends` values that have protocol types as dependencies, `pfun` can infer an intersection type automatically. Let's rewrite our example to take advantage of this feature:
```python
from typing import Protocol, Tuple


class HasConnectionStr(Protocol):
    connection_str: str


class HasAuth(Protocol):
    auth: Tuple[str, str]


def connect() -> Depends[HasConnectionStr, Connection]:
    ...


def get_user(user_id: int) -> Depends[HasConnectionStr, Connection]:
    connect().map(lambda con: con.get_user(1))


def call_api(user: User) -> Depends[HasAuth, bytes]:
    ...


call_api_w_user_1 = get_user(user_id=1).and_then(call_api)
```
The type of `call_api_w_user_1` in this example will be `Depends[pfun.Intersection[HasConnectionStr, HasAuth], bytes]`. The dependency type `pfun.Intersection` ensures that `call_api_w_user_1` must be
 called with an argument that fulfills both the `HasConnectionStr` protocol _and_ the `HasAuth` protocol. In other words, this would be a type error:
 ```python
class Dependency:
    def __init__(self):
        connection_str = 'user@prod'


call_api_w_user_1(Dependency())  # MyPy Error!
```
This compositional nature of `and_then` means that complex dependency types are built from simple dependency types _automatically_, which means you can always figure out which dependencies a part of your application has, simply by inspecting it's return type. To learn more about functional programming with `pfun` check out some of the other posts in the [series](https://dev.to/suned/series/6967), the [documentation](https://pfun.dev/) or the github repo.


{% github suned/pfun no-readme %}


<a href='https://www.freepik.com/vectors/background'>Background vector created by macrovector - www.freepik.com</a>