Patiently - an API rate limit handler for (Node) JS. A legal way to wait for API rate limit resets.
Tags: showdev, node, javascript, webdev
Published: 2020-06-28 21:20:51

Hi, I am Daniel and I want to present to you one of my little minions I developed within the context of a side project of mine, the [Quip Exporter](https://mindactuate.github.io/quip-exporter/). The Quip Exporter is a bulk exporter for the popular web-based editor [Quip](https://quip.com/).

See also my other minion called "logmailer". You can find more info about logmailer [here](https://dev.to/mindactuate/log-mailer-for-node-js-aggregate-your-log-error-mail-from-all-over-your-app-and-send-it-when-you-want-1oh1) and [here](https://github.com/mindactuate/logmailer).

I would be so happy for a Github star on my repo. 😃 😍 I am looking very forward for discussions and comments. Thank you!

## Short description

Sometimes there is no other way to be patient and to wait for the API rate limit to reset. Just think of a personalized API that allows you to perform actions in a specific user context. You want to do like 10.000 reqs as fast as possible. What if there are rate limits like 50 reqs per minute and 750 reqs per hour? And what if those limits are not bound to an IP or a host but to your user? In this case there is no other way but to wait for a limit reset. Of course, you can also politely ask the API owner to increase the limits. But even then you need **patiently**. :)

## Where to find

[patiently @ github](https://github.com/mindactuate/patiently)

[patiently @ npm](https://www.npmjs.com/package/patiently)

## The "Why" and the "How"

I developed patiently for another tool which allows a bulk export of all documents including all images from a Quip account. [Quip](https://quip.com/) is a web based collaborative document and spreadsheet editor. I used Quip mainly for taking notes until I eventually switched to a Markdown based note taking system. You can find the exporter on my github page [mindactuate.github.io/quip-exporter](https://mindactuate.github.io/quip-exporter/).

The challenge I had while developing Quip Exporter was the **rate limit bound to a personal API token**. Even if I switch IPs regularly (think of a proxy scraper), there is **no chance to get around the API rate limits**. That's where patiently comes in.

Patiently implements a "waiter function" where you can wrap in another function like your API call.

- Each function you give to the waiter as a parameter is queued in an array (queue)
> ``` F -> E D C B A```
- The longest waiting function is first
> ``` F E D C B -> A```
- Before the function is called, it is checked whether we have to wait first or not
> ``` F E D C B -> A (wait?)```
- If yes the queue processing is *paused* and when the waiting time elapsed, function A will be called
> ``` F E D C B -> A (call)```

## Installing

Using npm:

```bash
$ npm install patiently
```

## How to use

### Use by setting limits manually

Perhaps you already know about the limits (maybe from the API docs).

```javascript
     import patiently from "patiently";
     // let patiently = require("patiently");

     let options = {
       startWaitingCallback: function(info){console.log(info)}, // default is function(){}, calls a function if waiting necessary
       endWaitingCallback: function(info){console.log(info)}, // default is function(){}, calls a function after waiting
       waitingTickCallback: function(info){console.log(info)}, // default is function(){}, calls a function every tick
       msBetweenTwoCalls: 1000, // default is 0 milliseconds (no waiting time between two calls)
       minutelyLimit: 50, // default is Infinity (no minutely limit set)
       hourlyLimit: 750, // default is Infinity (no hourly limit set)
       test: false // default is false (if true, max waiting time is 5 secs)
     }

     var waiter = new patiently.LimitWaiter(options);

     let myApiCallFunction = async (url, callback) => {
         waiter.wait(function(){
           // your api call
           axios.get(url)
             .then(res => {
                callback(res.data);
             })
             .error(err => {
                callback(null, err);
             })
         })
     }

     // you can call myApiCallFunction as often you want
     // patiently can handle asynchronous api calls :)
     let url = "https://www.npmjs.com/package/patiently";
     myApiCallFunction(url);
     myApiCallFunction(url);
     myApiCallFunction(url);
     myApiCallFunction(url);
     // ...
 ```

## License

MIT