Package: parallely
Parallely - Parallel Python made simpleInstallationTo install this library, all you have to do is writepip install parallely. The library is really simple with just 3 functions. However, if needed, you can find the docs atReadTheDocsOverviewDealing with multi-threading, parallel processes, and concurrent functions can be difficult in Python. A lot of boiler plate code is needed, and it is difficult to gauge whether it actually improved performance or not.In some cases, it is necessary to tailor the program to utilize the underlying computational resources. In most cases, we just want to do the same thing many times with small alterations. In these scenariosparallelycan make your life much easier.Multi ThreadingDue to theGILmulti threading is far from as useful in Python as it is in other langauges. However, when dealing with I/O intensive applications it can still be really usefull to have multiple threads waiting for a response in parallel instead of waiting for each response sequentially, if you are confused by this, there are plenty of tutorials etc. tohelp you outMost of the time we just want to make a series of webrequests. In this caseparallelyremoves all complexity of handling all the threads with a single decoratorthreaded. Which is easiest to explain with an example:importtimefromparallelyimportthreaded@threadeddefthread_function(name,duration,other_arg):print(f"Thread{name}: starting",other_arg)time.sleep(duration)print(f"Thread{name}: finishing",other_arg)print("Synchronous")thread_function(1,duration=2,other_arg="hello world")thread_function(2,duration=1,other_arg="hello world")# NOTICE: We can use the thread_function the exact way we would expect without any overheadprint()print("Parallel/Asynchrous")thread_function.map(name=[1,2],duration=[2,1],other_arg="hello world")# NOTICE: the constant given to 'other_arg' is repeated in all function calls# thread_function.map([1, 2], [2, 1], "hello world") would produce a similar resultAs can be seen, the decorated function can be used like one would expect, which makes it easiest to test the function etc. However, it allso gets a.map()attribute, which can be used to run the function in a parallel manner.Multi ProcessingWorking with multiple processes is one of the only ways to get around the GIL. However, this approach hasall sorts of problems on its own. In many cases transferring the data between processes takes more time than the actual calculations. However, some times it can dramatically speed things up.parallelymakes it just as easy to work with multiple processes as it does with threads. Simply use the decoratorparallelas can be seen in the example below.fromtimeimporttimefromrandomimportrandintfromparallelyimportparallel@paralleldefcount_in_range(size,search_minimum,search_maximum):"""Returns how many numbers lie within `maximum` and `minimum` in a random array"""rand_arr=[randint(0,10)for_inrange(int(size))]returnsum([search_minimum<=n<=search_maximumforninrand_arr])size=1e7print("Sequential")start_time=time()for_inrange(4):result=count_in_range(size,search_minimum=1,search_maximum=2)print(result,round(time()-start_time,2),"seconds")print()print("Parallel")start_time=time()result=count_in_range.map(size=[size,size,size,size],search_minimum=1,search_maximum=2)print(result,round(time()-start_time,2),"seconds")Asynchronousimportasyncioimporttimefromrandomimportrandintfromparallelyimportasyncedasyncdefecho(delay,start_time):awaitasyncio.sleep(randint(0,delay))print(delay,round(time.time()-start_time,1))@asyncedasyncdefmain(counts):start_time=time.time()print(f"started at{time.strftime('%X')}")corr=[]forcountinrange(counts):corr.append(echo(count,start_time))awaitasyncio.gather(*corr)print(f"finished at{time.strftime('%X')}")# The asynchronous function can now be called in a synchronous manner without awiting itmain(10)# It can also be called in a parallel mannermain.map([5,5])