Rate-limiting API requests in Python with a decorator
Tags: python, tips, api, integration
Published: 2020-07-28 14:21:35

---
title: Rate-limiting API requests in Python with a decorator
published: true
date: 2020-07-28 12:00:00 UTC
tags: python,tips,api,integration
canonical_url: https://katiekodes.com/python-wrap-requests-functions/
---

The other day, I needed to make sure that a Python script I’m writing doesn’t call a REST API any faster than it’s allowed to.  
  
I wanted “[DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself),” readable code, so writing “if-else” code around every call to a function from the `requests` module was out of the question.

Thanks to [side-project work](https://dev.to/katiekodes/gatsby-for-novices-and-dabblers-19nn) in [TinaCMS](https://tinacms.org/packages/gatsby-tinacms-remark#3-the-higher-order-component-remarkform), I had some idea that the phrase I should Google when seeking out a way to “wrap” someone else’s code in additional functionality was “**higher-order**.”

That led me to these two tutorials:

- [Hacker Earth on higher-order functions and decorators in Python](https://www.hackerearth.com/practice/python/functional-programming/higher-order-functions-and-decorators/tutorial/)
- [Real Python on decorators](https://realpython.com/primer-on-python-decorators/#decorating-functions-with-arguments)

I put the following code at the top of my script:

```python
limitLeft = 500 # Although in theory some other code could be hitting the API, start with a presumption that we have all 500.

def __request_rate_limited(request_function):
    def limit_rate(*args, **kwargs):
        global limitLeft
        #if limitLeft < 500: # DEBUG LINE ONLY
            # Because we get our "actual rate limit" from API calls, it will never be 500 -- at most 499 -- in the wild.
            #print('API calls remaining before making call: ' + str(limitLeft)) # DEBUG LINE ONLY
        if limitLeft < 10:
            print('API calls remaining approaching zero (below ten); sleeping for 2 seconds to refresh.')
            sleep(2)
        response = request_function(*args, **kwargs)
        limitLeft = int(response.headers['X-Limit-Left'])
        #print('API calls remaining after making call: ' + str(limitLeft)) # DEBUG LINE ONLY
        return response
    return limit_rate

# Attach __request_rate_limited() to "requests.get" and "requests.post"    
requests.get = __request_rate_limited(requests.get)
requests.post = __request_rate_limited(requests.post)
```

After adding this chunk of code to the top of my script, **all invocations** of `requests.get()` and `requests.post()` **worked within the context of my additional code**.

Pretty neat.

The trick was remembering to attach the `__request_rate_limited()` wrapper function to both `requests.get()` and `requests.post()` before any code using them runs. At first, I only wrapped `requests.get()`, then couldn’t figure out why my code wasn’t running for certain parts of my script. Turns out those parts were making `post` requests.

This is probably an inelegant implementation of function wrapping / decorators in Python, but it definitely got the job done for a small command-line script.