Package: ejtraderTH
pipinstallejtraderTHThis tiny little python module is useful for creating multiple threads of any function in seconds.What's new in v1.0A quick launch mode is added, just typeejtraderTH.start(func_name, repeat=10)and it will execute the given function given number of times in parallel. A standard way of measuring elapsed time is added as well. see examples below to understand how to use quick launch mode.After using this library for a number of times in various projects i found that if you pass in lots of data say1000data in a list, it was creating1000threads to do it all once, how ever in the practical world most of the times cpu's are not capable of creating so many threads at once or worse it eats up all resources at once. To prevent this problem now you can pass inmax_threadsvalue by default if you don't pass in the value it will automatically be set equal to 4 just for the safety purpose, and passing a0value will throw an error while creating aejtraderTHobject this way it will only create specified number of threads at once and will wait untill the previously started threads has finished their job.Other than that to keep a track on how many threads are been created in real time you can push in a new log method inn you processing function so that whenever a new thread is created you can see it, there ar two methods of tracking them.just to print out the thread number which is being created, use:ejtraderTH.console_log(output=True)if you want to store it in some variable you can use:thread_number = ejtraderTH.console_log()in case some error occurs, the thread will keep on runningif you want to kill all the threads useejtraderTH.stop()inside your processing function while handling errors.Problem StatementLet say for example, we have alistof numbers from1 to 10and all we wanted to do ismultiply every number by 1000but the challenge isit takes 5 secfor multiplying a single number by 1000 from the list, I know its an arbitary condition, but we can create it withtime.sleep(5)function.Or you want to make a web request million times, without waiting for the server to respond you to make the next request.WorkingSo what this module does is, at the time of object initialization it takes in the function which is used for processing data and max number of thread which can be created at once, when running in multi threads, and as input it takes a list of arguments for which multiple threads will be created.Example 1importejtraderTHimporttime#the function for processing datadefmy_func(data):print(f'>> Running Thread{ejtraderTH.console_log()}')data=data*1000time.sleep(5)returndata#list of input data for processingraw_data=[1,2,3,4,5,6,7,8,9,10]#sending arguments for asynchronous multi thread processingprocessed_data=ejtraderTH.start(my_func,data=raw_data,max_threads=10)#printing the synchronised received resultsprint()print(f'>> Input:{raw_data}')print(f'>> Result:{processed_data}')print(f'>> Elapsed time:{ejtraderTH.elapsed()}sec')output>>RunningThread1>>RunningThread2>>RunningThread3>>RunningThread4>>RunningThread5>>RunningThread6>>RunningThread7>>RunningThread8>>RunningThread9>>RunningThread10>>Input:[1,2,3,4,5,6,7,8,9,10]>>Result:[1000,2000,3000,4000,5000,6000,7000,8000,9000,10000]>>Elapsedtime:5.0secNow you can clearly see, if we do it without multi threading it would have taken around50 Secondsfor processing the data while doing the task one by one and waiting for5 Secafter running the function each time. but since we are doing it with multithreading it will take only5 Secondsfor processing the same task with different data, in their individual threads.Example 2#the function for processing datadefmy_func(data):ejtraderTH.console_log(output=True)ip=req.request(method='GET',url="http://ipinfo.io/ip")returnip.content#sending arguments for asynchronous multi thread processingprocessed_data=ejtraderTH.start(my_func,repeat=20,max_threads=20)#printing the synchronised received resultsprint()print(f'>> Result:{processed_data}')ejtraderTH.elapsed(output=True)output>>CreatingThreads1>>CreatingThreads2>>CreatingThreads3>>CreatingThreads4>>CreatingThreads5>>CreatingThreads6>>CreatingThreads7>>CreatingThreads8>>CreatingThreads9>>CreatingThreads10>>CreatingThreads11>>CreatingThreads12>>CreatingThreads13>>CreatingThreads14>>CreatingThreads15>>CreatingThreads16>>CreatingThreads17>>CreatingThreads18>>CreatingThreads19>>CreatingThreads20>>Result:[b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37',b'200,234.182.37']>>Elapsedtime:0.31secIn this example we didn't used time.sleep() instead we make a request to the webserver and it took0.5 secondsto get the result back so we did it 20 times with multi threading and were able to receive the results in less time in a synchronous order.Lets try to do it without multithreading and see how it affects the processing time.So in this new updateejtraderTH v1.0we can specify that at once how many threads should be created so lets change the input parameter asmax_threads = 1this way it will only create one thread at a time and will wait until the previous thread has finished properly.output...>>Elapsedtime:6.43secIt is clear that every request to the server was taking approx.0.5 secondsso while making one request at a time it took6.43 secondsas expected.Example 3Quick Launch mode, a new feature is added where you can directly use ejtraderTH to pass in the repetitive function, input data for those functions and how many threads you want it to create at a time. other than that if you just want it to repeat the function without any inputs you can do that too.importejtraderTHimporttimeimportrandomnames=['April','May']#the function for processing datadefmy_func(data):ejtraderTH.console_log(output=True)time.sleep(1)name=random.choice(names)returnf'{name}says, Hello World!'processed_data=ejtraderTH.start(my_func,repeat=4)print(processed_data)ejtraderTH.elapsed(output=True)output>> Creating Threads 1
>> Creating Threads 2
>> Creating Threads 3
>> Creating Threads 4
['May says, Hello World!', 'April says, Hello World!', 'May says, Hello World!', 'April says, Hello World!']
>> Elapsed time: 1.0 secwe kept a time gap of 1 sec inside the function still it repeated the task 4 times in same time. since it can access the global variables we can assign certain tasks that don't need different inputs every time.Decorators supportApart from calling thestart()attribute we can also use decorators to explicitly make our functions for concurrent execution.Example 1importejtraderTHimporttime@ejtraderTH.run(repeat=5,max_threads=5)defmy_func(i):time.sleep(1)returni*2print(ejtraderTH.result['my_func'])Output[0,2,4,6,8]This will execute the function as soon as you run your python code, in this case we are trying to perform the same task five times in a row concurrently. The final output of all the function can be accessed byejtraderTH.result['function_name']. Notice if you set your function on repeat it will always receive a parameter which is it's thread number.Example 2importejtraderTHimporttime@ejtraderTH.run(data=[1,2,3,4,5,6],max_threads=5)defmy_func(i):time.sleep(1)returni*2print(ejtraderTH.result['my_func'])Output[2,4,6,8,10,12]In this case we are directly passing our arguments in a list via decorator and receiving the result same way as we did in previous example.Exampe 3importejtraderTHimporttime@ejtraderTH.set(max_threads=10)defmy_func(i):time.sleep(1)returni*2result=my_func(repeat=7)print(result)Output[0,2,4,6,8,10,12]This is an another cool way to first convert your function in concurrent function and then passing the argument as how many time you want to execute that function all in parallel.Example 4importejtraderTHimporttime@ejtraderTH.set()defmy_func(i):time.sleep(1)returni*2result=my_func([5,6,7])print(result)Output[10,12,14]Again we can also specify what arguments we want to pass to the function to process it concurrently. if in the@ejtraderTH.set()decorator you won't pass anymax_threadsargumentmax_threads=10will be set.Handling errors and killing all threadsSo, by default if any error occurs the threads will keep on running, in case if you want to ignore some errors but if you want to kill all the thread at once you can useejtraderTH.stop()while handling errors.#the function for processing datadefmy_func(data):thread_number=ejtraderTH.console_log(output=True)try:data=requests.get("http://httpbin.org/get")returndataexceptExceptionase:print(e)#printing other errors#killing all active threadsejtraderTH.stop()#use to kill all threadsif you don't useejtraderTH.stop()function then the threads will keep on running and fillingNonein place of returned data. if you used theejtraderTH.stop()it will kill all active threads immediately and will return the data that were processed by your function so far.