Package: threadgroup-lib
threadgroup-libThis package is used to define thread groups and register functions to run concurrently in that groupUsageUsing it is fairly simply and intuitive:importthreadgroupfirst=threadgroup.ThreadGroup()second=threadgroup.ThreadGroup()@first.register()deff1():pass@first.register()deff2():pass@second.register()defs1():pass@second.register()defs2():passfirst.execute()assertfirst.executedassertsecond.executedisFalsesecond.execute()assertsecond.executedWhat we do here is to define two separate groups,firstandsecond, then register which functions we want to run in each groupUsing with varvaultVarvaultcan be leveraged to create some very interesting code that runs concurrently while also being very flexible and easy to maintain:importtimeimportthreadgroupimportvarvaultclassKeyring(varvault.Keyring):k1=varvault.Key("k1",valid_type=str)k2=varvault.Key("k2",valid_type=int)k3=varvault.Key("k3",valid_type=float)vault=varvault.create_vault(Keyring,"vault")sleep=0.5init=threadgroup.ThreadGroup()use=threadgroup.ThreadGroup()@init.register()@vault.vaulter(return_keys=Keyring.k1)defset_k1():time.sleep(sleep)return"valid"@init.register()@vault.vaulter(return_keys=Keyring.k2)defset_k2():time.sleep(sleep)return1@init.register()@vault.vaulter(return_keys=Keyring.k3)defset_k3():time.sleep(sleep)return3.14@use.register()@vault.vaulter(input_keys=Keyring.k1)defuse_k1(k1=None):time.sleep(sleep)assertk1=="valid"returnk1@use.register()@vault.vaulter(input_keys=Keyring.k2)defuse_k2(k2=None):time.sleep(sleep)assertk2==1returnk2@use.register()@vault.vaulter(input_keys=Keyring.k3)defuse_k3(k3=None):time.sleep(sleep)assertk3==3.14returnk3start=time.time()init.execute()assertsleep-0.2<time.time()-start<sleep+0.2,"Took too long to run. Concurrency seems broken"assertKeyring.k1invaultandvault.get(Keyring.k1)=="valid"assertKeyring.k2invaultandvault.get(Keyring.k2)==1assertKeyring.k3invaultandvault.get(Keyring.k3)==3.14assertinit.executedassertuse.executedisFalsestart=time.time()use.execute()assertsleep-0.2<time.time()-start<sleep+0.2,"Took too long to run. Concurrency seems broken"results=use.get_results()We define a varvault object calledvaultwhich we set the variables to. The functionsset_k1,set_k2,set_k3set each of the keys to the vault and they all run in the thread groupinit.
We then use those variables inuse_k1,use_k2,use_k3.The benefit of using varvault together with thread-groups is that varvault already abstracts away input variables and return variables from functions,
allowing for a decoupled flow that works extremely well to run in thread-groups that are executed in this fashion. Since functions are registered and then executed whenexecute()is called,
supplying input variables to the functions and handling return variables from the functions is a bit of a hassle. Varvault handles this entirely,
allowing the user to focus on other things and running things concurrently (as much as Python will allow).Running without thread groupsIt's entirely possible to use the library to just run functions concurrently. This is how you may use it:importthreadgroupfromtypingimportListdeff1(a1):asserta1==1returna1deff2(a2):asserta2==2returna2deff3(a1,kw1=None):asserta1==1assertkw1isTruereturna1,kw1functions:List[threadgroup.Function]=[threadgroup.create_function(f1,1),threadgroup.create_function(f2,2),threadgroup.create_function(f3,1,kw1=True)]result:threadgroup.ResultList[threadgroup.ResultStruct]=threadgroup.threaded_execution(functions)assertf"{result}"=="[(function=f1; result=1), (function=f2; result=2), (function=f3; result=(1, True))]"What we do here is to define three different functions that each take different arguments. We then build a list of functions with their arguments. We then run those functions concurrently by callingthreadgroup.threaded_execution.It's also possible to run the same function multiple times with different input parameters. This is how you'd do it:importthreadgroupfromtypingimportListarg_values=[1,2,3,4,5]deff1(a1):asserta1inarg_values,f"{a1}not in{arg_values}"returna1functions:List[threadgroup.Function]=[threadgroup.create_function(f1,arg_value)forarg_valueinarg_values]result:threadgroup.ResultList[threadgroup.ResultStruct]=threadgroup.threaded_execution(functions)assertf"{result}"=="[(function=f1; result=1), (function=f1; result=2), (function=f1; result=3), (function=f1; result=4), (function=f1; result=5)]"In this example we also create a list of functions, but this time we do it based on the input parameters inarg_values, allowing it to scale easily based on an iterable.
We then do the same as previously and run the functions concurrently throughthreadgroup.threaded_execution.