Send multiple fetch requests simultaneously
Tags: javascript, webdev, tutorial, beginners
Published: 2020-07-05 13:24:29

First of all, sending multiple fetch requests simultaneously is not always the right solution and has some downsides. One downside is for example that you could end up being blocked if you send too many requests at the same time, to one server. I personally write a solution for sending requests simultaneously if I need to fetch a big list of URLs with unknown response times (> 100 URLs).

Okay, now let's get started. First of all we need to create a async `fetchSimultaneously` function and create the constants `queue` and `maxSimultaneouslyRequests` inside it. The `queue` constant exists just so we can keep control of which URLs have been fetched and which not. We also need to initialize a start index (`i`) and a variable for storing the value of the current open requests (`currentRequests`).
```javascript
async function fetchSimultaneously(urls) {
    const queue = urls;
    const maxSimultaneouslyRequests = 5;
    let currentRequests = 0;
    let i = 0;
}
```

To now send the requests simultaneously and handle when we want to stop the loop, `setInterval` is the best option.
To return the result array more easily, we are wrapping everything in a `new Promise` and call the `resolve` function when all requests have been finished.
```javascript
async function fetchSimultaneously(urls) {
    const queue = urls;
    const maxSimultaneouslyRequests = 5;
    let currentRequests = 0;
    let i = 0;

    return await new Promise(resolve => {
        const result = [];

        const fetcher = setInterval(async () => {
        }, 100);
    });
}
```
The first thing we need to check inside the `setInterval` function is if all results have been fetched and if so, we resolve the promise and clear the interval. The easiest way to find out if a queue array entry has already been fetched is to set the value of the entry to `undefined`. This allows us to filter for array entries with a value different than `undefined` to find out if any array entries haven't been fetched yet.
```javascript
if (queue.filter(url => url).length === 0) {
    clearInterval(fetcher);
    resolve(result);
}
```

The next thing we need to check is if the ongoing requests are exceeding the `maxSimultaneouslyRequests`.
Since we need to wait for all responses to arrive, we also need to check if the value of `i` is exceeding the length of our queue. (At least the last request will take longer to finish than the value of `i` to reach the length of the queue array.)
```javascript
if (currentRequests >= maxSimultaneouslyRequests || i > queue.length - 1) {
    return;
}
```

Now we can finally fetch the data and set it to the correct index of the result array.
```javascript
async function fetchSimultaneously(urls) {
    const queue = urls;
    const maxSimultaneouslyRequests = 5;
    let currentRequests = 0;
    let i = 0;

    return await new Promise(resolve => {
        const result = [];

        const fetcher = setInterval(async () => {
            if (queue.filter(url => url).length === 0) {
                clearInterval(fetcher);
                resolve(result);
            }

            if (currentRequests >= maxSimultaneouslyRequests || i > queue.length - 1) {
                return;
            }

            // Get current index and increase i
            const index = i++;
            const url = queue[index];

            currentRequests++;
            // Keep same index as of the passed urls array
            result[index] = await (await fetch(url)).text();
            currentRequests--;

            // Set value of index to empty (undefined)
            delete queue[index];
        }, 100);
    });
}
```