Python API Client Using Magic Methods
Tags: python, programming, api, tutorial
Published: 2021-10-29 11:30:09

Python "magic methods" allow for some very fun and powerful code.

Magic methods refers to all those special methods in Python classes that start and end with `__`. 

The documentation for [Special method names](https://docs.python.org/3/reference/datamodel.html#special-method-names) says:

>A class can implement certain operations that are invoked by special syntax (such as arithmetic operations or subscripting and slicing) by defining methods with special names.

For instance the `__init__` method is called when a new instance of a class is instantiated. `__str__` is a method that returns a string representation of the class. As stated in the docs, these are not methods that generally get invoked directly on the class. For example, you don't call `my_object.__str__()` but `__str__` would be called by `print(my_object)`.

In this post, I'm going to demonstrate using the `__getattr__` and `__call__` magic methods to build a dynamic API client.

Let's say you're building a client for an API with endpoints like:

- GET https://some-api.net/user
- GET https://some-api.net/user/permissions
- POST https://some-api.net/article
- GET https://some-api.net/article/<id>

There are a lot of ways to build a client for this. One might look like:

```python
import logging
import requests

class APIClient:
    BASE_URL = "https://some-api.net"

    def make_request(method, url, params=None, data=None):
        response = requests.request(
            method,
            f"{self.BASE_URL}/{url}"
            params=params,
            json=data
        )

        response_body = {}
        try:
            response_body = response.json()
        except ValueError:
            log.warning("Unexpected Response '%s' from '%s'",
                        response.content, response.url)
        return response_body

    def get_user(self):
        return self.make_request("GET", "user")

    def get_user_permissions(self):
        return self.make_request("GET", "user/permissions")

    def create_article(self, data):
        return self.make_request("POST", "article", data=data)

    def get_article(self, id):
        return self.make_request("GET", f"article/{id}")
```

We would then use the client:

```python
client = APIClient()

user = client.get_user()
permissions = client.get_user_permissions()
new_article = client.create_article({"some": "data"})
existing_article = client.get_article(123)
```

And this is fine. 

But what if the API is changing often? Or if there are dozens of endpoints and you don't know which you need to call? If later on you discover you need to call `GET https://some-api.net/user/roles` or `GET https://some-api.net/some_other_thing?some_param=foo`, you'll need to go back to you client and add matching methods.

It would be nice to have things a bit more dynamic, and one way to do that is with magic methods. Specifically, I'll be using:

- `__getattr__` which is "Called when the default attribute access fails with an AttributeError" (see [the docs](https://docs.python.org/3/reference/datamodel.html#object.__getattr__)). This means that if you call `my_object.some_missing_attr`, then `__getattr__` will get invoked with `"some_missing_attr"` as the `name` parameter.
- `__call__` which is "Called when the instance is "called" as a function" (see [the docs](https://docs.python.org/3/reference/datamodel.html#object.__call__)), eg. `my_object()`

Using these, we can build something like:

```python
import logging
import requests

log = logging.getLogger(__name__)

class APIClient:
    BASE_URL = "https://some-api.net"

    def __init__(self, base_url=None):
        self.base_url = base_url or BASE_URL

    def __getattr__(self, name):
        return APIClient(self._endpoint(name))

    def _endpoint(self, endpoint=None):
        if endpoint is None:
            return self.base_url
        return "/".join([self.base_url, endpoint])

    def __call__(self, *args, **kwargs):
        method = kwargs.pop("method", "GET")

        object_id = kwargs.pop("id"))
        data = kwargs.pop("data", None)

        response = requests.request(
            method,
            self._endpoint(object_id),
            params=kwargs,
            json=data
        )

        response_body = {}
        try:
            response_body = response.json()
        except ValueError:
            log.warning("Unexpected Response '%s' from '%s'",
                        response.content, response.url)
        return response_body
```

This allows us to call the API like so:

```python
client = APIClient()

user = client.user()
permissions = client.user.permissions()
new_article = client.article(method="POST", data={"some": "data"})
existing_article = client.article(id=123)
```

How does this work? 

Well in the case of `client.user.permissions()`, `.user` is an attribute and since the client doesn't have that attribute, it calls `__getattr__` with `name="user"`, which then returns an `APIClient` instance with the name appended to the endpoint URL. The same happens when `.permissions` is invoked on the `APIClient` instance that was returned by `.user`, in turn giving us another `APIClient` instance, now with a path of `https://some-api.net/user/permissions`. Finally this `APIClient` instance is called by `()`, which invokes the `__call__` method. This method makes the actually HTTP call to the constructed URL based in any parameteres passed in, but defaults to a `GET` request.

In the case of the article calls, `client.article(id=123)` works much the same way, but the `APIClient` is called with an id parameter. This id gets appended to the url by the same internal `_endpoint()` method that `__getattr__` uses, which results in a GET call to `https://some-api.net/article/<id>`.

For `client.article(method="POST", data={"some": "data"})`, we override the method and add a data payload to the POST.

As you can see, the client itself can handle a large variety of API calls without any significant changes, allowing the utilizing application a lot of flexibility. 

In the example of the newly required calls, `GET https://some-api.net/user/roles` and `GET https://some-api.net/some_other_thing?some_param=foo`, no changes to the client are needed:

```python
roles = client.user.roles()
thing = client.some_other_things(some_param="foo")
```