Package: dev4py-utils
Dev4py-utilsA set of Python regularly used classes/functionsTable of contentsQuickstartProject templateProject linksDev4py-utils modulesdev4py.utils.AsyncJOptionaldev4py.utils.awaitablesdev4py.utils.collectorsdev4py.utils.dictsdev4py.utils.iterablesdev4py.utils.JOptionaldev4py.utils.listsdev4py.utils.objectsdev4py.utils.pipelinedev4py.utils.retrydev4py.utils.Streamdev4py.utils.tuplesdev4py.utils.typesQuickstartpipinstalldev4py-utilsProject templateThis project is based onpymsdl_templateProject linksDocumentationPyPi projectDev4py-utils modulesdev4py.utils.AsyncJOptionalAsyncJOptional documentationNote:AsyncJOptionalclass is designed in order to simplify
JOptional with async mapper.Note:AsyncJOptional support T or Awaitable[T] values. That's why some checks are done when terminal operation is
called withawait.Examples:importasynciofromdev4py.utilsimportAsyncJOptionaldefsync_mapper(i:int)->int:returni*2asyncdefasync_mapper(i:int)->str:returnf"The value is{i}"asyncdefasync_sample()->None:value:int=1awaitAsyncJOptional.of_noneable(value)\.map(sync_mapper)\.map(async_mapper)\.if_present(print)# The value is 2asyncio.run(async_sample())dev4py.utils.awaitablesAwaitables documentationNote:awaitablesmodule provides a set of utility functions to
simplify Awaitable operations.Examples:importasynciofromdev4py.utilsimportawaitables,JOptional# is_awaitable sampleawaitables.is_awaitable(asyncio.sleep(2))# Trueawaitables.is_awaitable(print('Hello'))# False# to_sync_or_async_param_function sampledefmapper(s:str)->str:returns+'_suffix'asyncdefasync_mapper(s:str)->str:awaitasyncio.sleep(1)returns+'_async_suffix'asyncdefasync_test():# Note: mapper parameter is str and async_mapper returns an Awaitable[str] so we have to manage it# Note: !WARNING! Since 3.0.0 see AsyncJOptional / JOptional to_async_joptional methodresult:str=awaitJOptional.of("A value")\.map(async_mapper)\.map(awaitables.to_sync_or_async_param_function(mapper))\.get()print(result)# A value_async_suffix_suffixasyncio.run(async_test())dev4py.utils.collectorsCollectors documentationNote:Thecollectorsclass is inspired byjava.util.stream.CollectorsExamples:fromdev4py.utilsimportStream,collectorsStream.of('a','b','c').collect(collectors.to_list())# ['a', 'b', 'c']dev4py.utils.dictsDicts documentationNote:dictsmodule provides a set of utility functions to
simplify dict operations.Examples:fromdev4py.utilsimportdictsfromdev4py.utils.typesimportSupplier# is_dict sampledicts.is_dict("A str")# Falsedicts.is_dict({'key':'A dict value'})# True# get_value sampleint_supplier:Supplier[int]=lambda:3dictionary:dict[str,int]={'key_1':1,'key_2':2}dicts.get_value(dictionary,'key_1')# 1dicts.get_value(dictionary,'key_3')# Nonedicts.get_value(dictionary,'key_3',int_supplier)# 3# get_value_from_path samplestr_supplier:Supplier[str]=lambda:"a3"deep_dictionary:dict[str,dict[int,str]]={\'a':{1:'a1',2:'a2'},\'b':{1:'b1',2:'b2'}\}dicts.get_value_from_path(deep_dictionary,["a",1])# 'a1'dicts.get_value_from_path(deep_dictionary,["c",1])# Nonedicts.get_value_from_path(deep_dictionary,["a",3])# Nonedicts.get_value_from_path(deep_dictionary,["a",3],str_supplier)# 'a3'dev4py.utils.iterablesIterables documentationNote:Theiterablesmodule provides a set of utility functions to simplify
iterables operations.Example:fromtypingimportIteratorfromdev4py.utilsimportiterablesvalues:range=range(0,10)chunks:Iterator[list[int]]=iterables.get_chunks(values,3)[chunkforchunkinchunks]# [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]dev4py.utils.JOptionalJOptional documentationNote:JOptionalclass is inspired byjava.util.Optionalclass with some adds (likepeekmethod).Examples:fromdev4py.utilsimportJOptionalvalue:int=1JOptional.of_noneable(value)\.map(lambdav:f"The value is{v}")\.if_present(print)# The value is 1dev4py.utils.listsLists documentationNote:listsmodule provides a set of utility functions to simplify lists
operations.Examples:fromdev4py.utilsimportlists# empty samplelst:list[int]=lists.empty_list()# []# append samplelst:list[int]=[1,2,3,4]app_lst:list[int]=lists.append(lst,5)# [1, 2, 3, 4, 5]# - Note: lst == app_lst# extend samplelst:list[int]=[1,2,3,4]lst2:list[int]=[5,6,7,8]ext_lst:list[int]=lists.extend(lst,lst2)# [1, 2, 3, 4, 5, 6, 7, 8]# - Note: lst == ext_lstdev4py.utils.objectsObjects documentationNote:Theobjectsmodule is inspired byjava.util.Objectsclass.Examples:fromdev4py.utilsimportobjects# non_none samplevalue=Noneobjects.non_none(value)# False# require_non_none samplevalue="A value"objects.require_non_none(value)# 'A value'# to_string samplevalue=Nonedefault_value:str="A default value"objects.to_string(value,default_value)# 'A default value'dev4py.utils.pipelinePipeline documentationNote:Thepipelinepackage provides a set of Pipeline class describing
different kind of pipelines.Examples:fromdev4py.utils.pipelineimportSimplePipeline,StepPipeline,StepResult# SimplePipeline samplepipeline:SimplePipeline[int,str]=SimplePipeline.of(lambdai:i*i)\.add_handler(str)\.add_handler(lambdas:f"Result:{s}| Type:{type(s)}")pipeline.execute(10)# "Result: 100 | Type: <class 'str'>"# StepPipeline sample# Note: StepPipeline can be stopped at each step by setting `go_next` value to Falsepipeline:StepPipeline[int,str]=StepPipeline.of(lambdai:StepResult(i*i))\.add_handler(lambdai:StepResult(value=str(i),go_next=i<150))\.add_handler(lambdas:StepResult(f"Result:{s}| Type:{type(s)}"))pipeline.execute(10)# StepResult(value="Result: 100 | Type: <class 'str'>", go_next=True)# - Note: When the pipeline is fully completed, `go_next` is Truepipeline.execute(15)# StepResult(value='225', go_next=False)# - Note: Even if the pipeline is not fully completed, the last StepResult is returned with `go_next=False`dev4py.utils.retryRetry documentationNote:Theretrymodule provides provides function to create retryable
callable from simple sync or async callables using exponential backoffUsage idea: network requests (HTTP, AMQP, MQTT, etc.) with retry on errorExamples:importasynciofromtimeimporttimefromtypingimportAwaitablefromdev4py.utils.retryimportRetryConfiguration,to_retryable,to_async_retryable,retryable,async_retryablefromdev4py.utils.typesimportBiFunction# RetryConfiguration:# Note: exponential backoff used formula is 'delay * (exponent^retry_number)'##   => Example: For the following RetryConfiguration, waiting times in case of error are:#       * first try:                    0 sec (always 0 for the first try)#       * second try (/first retry):    1 sec ('0.5 * (2^1)')#       * third try (/second retry):    2 sec ('0.5 * (2^2)')#       * max_tries=3 => no fourth try (/third retry)retry_config:RetryConfiguration=RetryConfiguration(delay=0.5,# the exponential backoff delay in second (default: 0.1)exponent=2,# the exponential backoff exponent to determine delay between each try (default: 2)max_tries=3# max try number (first try included) (default: 3, i.e.: first try and 2 retry))# to_retryable sample:# -> SUCCESSFUL CALL SAMPLEdefcallable_sample(j:int,start_time:float)->int:print("callable_sample - call time: '%.2f'"%(time()-start_time))returnj**2retryable_sample:BiFunction[int,float,int]=to_retryable(sync_callable=callable_sample,retry_config=retry_config)# Note: Since 3.5.0 you can also use `retryable(sync_callable=callable_sample, retry_config=retry_config)`result:int=retryable_sample(3,time())# result = 9# outputs:#  callable_sample - call time: '0.00'# -> IN ERROR CALL SAMPLEdefin_error_callable_sample(j:int,start_time:float)->int:print("in_error_callable_sample - call time: '%.2f'"%(time()-start_time))raiseValueError(j)in_error_retryable_sample:BiFunction[int,float,int]=\to_retryable(sync_callable=in_error_callable_sample,retry_config=retry_config)# Note: Since 3.5.0 you can also use `retryable(sync_callable=in_error_callable_sample, retry_config=retry_config)`# Note: By default the last raised exception is raised if max_tries is reach. You can change this behavior by setting#       the `on_failure` parameterresult:int=in_error_retryable_sample(3,time())# outputs:#  in_error_callable_sample - call time: '0.00'#  in_error_callable_sample - call time: '1.00'#  in_error_callable_sample - call time: '3.00'#  ValueError: 3## Note: By default the last raised exception is raised if max_tries is reached. You can change this behavior by setting#       the `on_failure` parameter# -> DECORATOR SAMPLE@retryable(retry_config=retry_config)defdecorated_in_error_callable_sample(j:int,start_time:float)->int:print("decorated_in_error_callable_sample - call time: '%.2f'"%(time()-start_time))raiseValueError(j)result:int=decorated_in_error_callable_sample(3,time())# outputs:#  decorated_in_error_callable_sample - call time: '0.00'#  decorated_in_error_callable_sample - call time: '1.00'#  decorated_in_error_callable_sample - call time: '3.00'#  ValueError: 3## Note: By default the last raised exception is raised if max_tries is reached. You can change this behavior by setting#       the `on_failure` parameter# to_async_retryable sample:# -> IN ERROR CALL ASYNC SAMPLEasyncdefin_error_async_callable_sample(j:int,start_time:float)->int:print("in_error_async_callable_sample - call time: '%.2f'"%(time()-start_time))raiseValueError(j)asyncdefasync_retryable_sample()->None:in_error_async_retryable_sample:BiFunction[int,float,Awaitable[int]]=\to_async_retryable(async_callable=in_error_async_callable_sample,retry_config=retry_config)# Note: Since 3.5.0 you can also use# `async_retryable(async_callable=in_error_async_callable_sample, retry_config=retry_config)`result:int=awaitin_error_async_retryable_sample(2,time())asyncio.run(async_retryable_sample())# outputs:#  in_error_async_callable_sample - call time: '0.00'#  in_error_async_callable_sample - call time: '1.00'#  in_error_async_callable_sample - call time: '3.00'#  ValueError: 2## Note: By default the last raised exception is raised if max_tries is reached. You can change this behavior by setting#       the `on_failure` parameter# -> DECORATOR ASYNC SAMPLE@async_retryable(retry_config=retry_config)asyncdefdecorated_in_error_async_callable_sample(j:int,start_time:float)->int:print("decorated_in_error_async_callable_sample - call time: '%.2f'"%(time()-start_time))raiseValueError(j)asyncdefasync_decorated_retryable_sample()->None:result:int=awaitdecorated_in_error_async_callable_sample(2,time())asyncio.run(async_decorated_retryable_sample())# outputs:#  in_error_async_callable_sample - call time: '0.00'#  in_error_async_callable_sample - call time: '1.00'#  in_error_async_callable_sample - call time: '3.00'#  ValueError: 2## Note: By default the last raised exception is raised if max_tries is reached. You can change this behavior by setting#       the `on_failure` parameterdev4py.utils.StreamStream documentationNote:Streamclass is inspired byjava.util.stream.Stream.Examples:fromconcurrent.futuresimportThreadPoolExecutor,ProcessPoolExecutorfromtimeimportsleepfromdev4py.utilsimportStream,ParallelConfiguration# Sequential sampleStream.of(1,2,3,4)\.map(str)\.peek(lambdas:sleep(0.5))\.map(lambdas:f"Mapped value:{s}")\.to_list()# ['Mapped value: 1', 'Mapped value: 2', 'Mapped value: 3', 'Mapped value: 4']# - Note: Execution time around 2 sec due to the sleep call# Multithreading samplewithThreadPoolExecutor(max_workers=2)asexecutor:Stream.of(1,2,3,4)\.parallel(parallel_config=ParallelConfiguration(executor=executor,chunksize=2))\.map(str)\.peek(lambdas:sleep(0.5))\.map(lambdas:f"Mapped value:{s}")\.to_list()# ['Mapped value: 3', 'Mapped value: 4', 'Mapped value: 1', 'Mapped value: 2']# - Note: Execution time around 1 sec due to the given ParallelConfiguration# - Note: Since this stream is (by default) unordered, results order is random# Multiprocessing sample# - Note: Due to use of Multiprocessing:#       * lambdas cannot be used since they cannot be pickled#       * This sample should be put in a python file in order to workdef_sleep(s:str)->None:# eq lambda s: sleep(0.5)sleep(0.5)def_mapper(s:str)->str:# eq lambda s: f"Mapped value: {s}"returnf"Mapped value:{s}"if__name__=='__main__':withProcessPoolExecutor(max_workers=2)asexecutor:Stream.of(1,2,3,4)\.parallel(parallel_config=ParallelConfiguration(executor=executor,chunksize=2))\.map(str)\.peek(_sleep)\.map(_mapper)\.to_list()# - Note: Execution time around 1 sec due to the given ParallelConfiguration#         (Reminder: Use Multiprocessing for CPU-bound tasks. In this case Multithreading is more appropriate)# - Note: Since this stream is (by default) unordered, results order is randomdev4py.utils.tuplesTuples documentationNote:tuplesmodule provides a set of utility functions to simplify
tuples operations.Examples:fromdev4py.utilsimporttuples# empty sampletpl:tuple[int,...]=tuples.empty_tuple()# ()# append sampletpl:tuple[int,...]=(1,2,3,4)app_tpl:tuple[int,...]=tuples.append(tpl,5)# (1, 2, 3, 4, 5)# extend sampletpl:tuple[int,...]=(1,2,3,4)tpl2:tuple[int,...]=(5,6,7,8)ext_tpl:tuple[int,...]=tuples.extend(tpl,tpl2)# (1, 2, 3, 4, 5, 6, 7, 8)dev4py.utils.typesTypes documentationNote:Thetypesmodule is inspired byjava.util.functionpackage.Examples:fromdev4py.utils.typesimportFunction,Predicate,Consumer# Function sampleint_to_str:Function[int,str]=lambdai:str(i)str_result:str=int_to_str(1)# '1'# Predicate samplestr_predicate:Predicate[str]=lambdas:s=="A value"pred_result:bool=str_predicate("Value to test")# False# Consumer sampledefsample(consumer:Consumer[str],value:str)->None:consumer(value)defmy_consumer(arg:str)->None:print(arg)sample(my_consumer,"My value")# My value