How To Make Parallel API calls in JSPython
Tags: api, http, programming, jspython
Published: 2022-10-13 07:00:21

It is quite a common scenario when you have to make multiple an API calls in parallel. Making parallel calls will reduce the latency and improve the performance of your solution. Either you are looking to speed up a data loading process or perform some sort of load/stress testing.
In Javascript, you can spin up multiple `Promises` and then use the function [Promise.all()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all) to wait for all of them to succeed. Or you can use RxJS, streams or controlled callbacks

[JSPython ](https://www.jspython.dev/)was designed to evaluate all instructions synchronously. Mainly because of simplicity, it looks like Python users - data engineers and analysts don't like the complexity that comes with Promises, Streams or callbacks. JSPython awaits/resolves all promises at its core. So, you won't have any Promise anywhere in the JSPython runtime (apart from some edge cases).

So, how to run code parallel, then?
The answer is [RxJs](https://rxjs.dev/guide/overview). You can use RxJs functions to create observable sequences, then combine response values they emitted (with forkJoin) and convert them back to Promise (which will be automatically evaluated in JSPython)

The following JSPython code shows this approach in action. This example makes five parallel requests and returns an array of responses along with the corresponding requestId

```python
from 'rxjs' import forkJoin, lastValueFrom, map

ids = [2, 7, 4, 9, 5]

httpRequests$ = ids
	.map(
		requestId => httpRequest$("GET", "https://jsonplaceholder.typicode.com/posts/" + requestId)
											.pipe(
												map(r => {requestId, response: r.data})
											)
	)

return lastValueFrom(forkJoin(httpRequests$))
	
	
```
Let's go through the code.

1. Import RxJs functions
2. Create an array of ids.
3. Use [map function](https://www.w3schools.com/jsref/jsref_map.asp) to transform an array of ids into an array of Observables.
4. Use the **httpRequest$** function to fetch data from a server. The asynchronous built-in function sends an HTTP request, and returns an Observable that emits the requested data when the response is received.
5. Then we use observable's [pipe(map(...))](https://www.learnrxjs.io/learn-rxjs/operators/transformation/map) and transform result, by adding a requestId and response
6. Then we wait for all observables to complete and combine responses in the array. We use [forkJoin ](https://www.learnrxjs.io/learn-rxjs/operators/combination/forkjoin) for this
7. Then convert observable into Promise with function [lastValueFrom](https://rxjs.dev/api/index/function/lastValueFrom). Then, JSPython natively resolves Promise into a synchronous result 

## Run
Press `Run` and you will see the result


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5zwo38rkgug3iocwouaq.png)


## Working example

A working example can be found here
[https://run.worksheet.systems/data-studio/app/guest/jspy-playground?file=http-calls%2Fparallel-http-requests.jspy](https://run.worksheet.systems/data-studio/app/guest/jspy-playground?file=http-calls%2Fparallel-http-requests.jspy)
or if you are using chrome extention
[chrome-extension://dkhnlgcpdiifkfjdjceogenclkdfbonh/index.html#/data-studio/app/guest/jspy-playground?file=http-calls%2Fparallel-http-requests.jspy](chrome-extension://dkhnlgcpdiifkfjdjceogenclkdfbonh/index.html#/data-studio/app/guest/jspy-playground?file=http-calls%2Fparallel-http-requests.jspy)

## Sequential vs. Parallel API Calls

In another example, I want to show two different functions where one of them run_Sequential is making https requests sequential and function run_Parallel is doing same things but in parallel

```python
from 'rxjs' import forkJoin, lastValueFrom, map

ids = [2, 7, 4, 9, 5]

async def run_Sequential():
	data = []

	for requestId in ids:
		response = httpGet("https://jsonplaceholder.typicode.com/posts/" + requestId)
		data.push({requestId, response})

	return data

async def run_Parallel():
	httpRequests$ = ids
		.map(
			requestId => httpRequest$("GET", "https://jsonplaceholder.typicode.com/posts/" + requestId)
												.pipe(
													map(r => {requestId, response: r.data})
												)
		)

	return lastValueFrom(forkJoin(httpRequests$))
   
if __env.entryFunction == '':
	return {
			sequential: run_Sequential(),
			parallel: run_Parallel()
		}
			
```

Working example is here 
[https://run.worksheet.systems/data-studio/app/guest/jspy-playground?file=http-calls%2Fsequential-vs-parallel-http-calls.jspy](https://run.worksheet.systems/data-studio/app/guest/jspy-playground?file=http-calls%2Fsequential-vs-parallel-http-calls.jspy).
Just Run this code and open chrome dev-tools - network tab. Where you can clearly see the difference


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hmdhd0vg9boqve2kt8zn.png)

## WORKSHEETS Data Studio
WORKSHEETS Data Studio is a Low Code Data Management Platform for Data Analysis, Data Processing, SQL Database Management and RAPID App Development.