Package: execution-pipeline
python-pipelineThis library allows users to easily wrap functions with a series of decorators to form
an execution pipeline.
This is useful in scenarios where input needs to be cleaned and output needs to be processed in a systematic way.Installationpip install execution-pipelineUsageA pipeline consists of four optional segmentsPreThepreexecution segment allows you to modify any input parameters passed to the decorated function. Any functions
passed to thepresegment will always be executed first.frompipelineimportexecution_pipelinedefdo_thing_before(params):params['arg1']='okay'returnparams@execution_pipeline(pre=[do_thing_before])defdo_thing(arg1=5):returnarg1*10do_thing()# okayokayokayokayokayokayokayokayokayokayPostThepostexecution segment allows you to modify or handle the result after the decorated function has already run.defdo_thing_after(response):response['added']='yup'returnresponse@execution_pipeline(post=[do_thing_after])defdo_thing(**kwargs):return{**kwargs}# just make a new dictionary using the passed keyword argumentsdo_thing(apples=2,oranges=3,bananas=0)# {'apples': 2, 'oranges': 3, 'bananas': 0, 'added': 'yup'}Just like the other segments, you can use as many post processing functions as you need; they will be executed in the order
that they are passed.defdo_another_thing_after(response):assertresponse['added']=='yup'# the one that is first in the pipeline happens first.response['also_added']='also yup'returnresponse@execution_pipeline(post=[do_thing_after,do_another_thing_after])defdo_thing(**kwargs):return{**kwargs}do_thing()# {'apples': 2, 'oranges': 3, 'bananas': 0, 'added': 'yup', 'also_added': 'also yup'}ErrorTheerrorexecution segment allows you to pass error handling to log, modify, absorb, etc. any exceptions that are
raised by the wrapped function.classMyException(BaseException):passdefhandle_this_error(e,response=None):print(f"oh no, Bob!{e}")return"Don't worry, we handled a TypeError."defhandle_that_error(e,response=None):print(f"oh no, Bob!{e}")return"Don't worry, we handled MyException."defhandle_other_errors(e,response=None):print(f"?{e}")return"Other errors?"error_handlers=[{"exception_class":TypeError,"handler":handle_this_error},{"exception_class":MyException,"handler":handle_that_error},{"exception_classes":(Exception,BaseException),"handler":handle_other_errors},]@execution_pipeline(pre=[do_thing_before],post=[do_thing_after],error=error_handlers)deffun_boys(arg1,arg4,arg2,arg3,thing=None):raiseMyException('Something went wrong!')result=fun_boys(1,2,3,4,5)# oh no, Bob! Something went wrong!print(result)# Don't worry, we handled MyException.You can also use class instances instead of dictionaries to define your error handlers if you prefer.classErrorHandler:def__init__(self,exception_class,handler):self.exception_class=exception_classself.handler=handlererror_handlers=[ErrorHandler(TypeError,handle_this_error),ErrorHandler(MyException,handle_that_error),]CacheThecacheexecution segment will record all arguments (before and after thepresegment) and store the result
(after thepostanderrorsegments).frompipeline.cache.mockimportMockCache# MockCache is basically just a dict() with some expiration convenience methods.mock_cache=MockCache()changing_value=0@execution_pipeline(cache=mock_cache)deffun_boys(arg1,arg4,arg2,arg3,thing=None):returnchanging_valuefun_boys(1,2,3,4,5)# 0changing_value=100fun_boys(1,2,3,4,5)# 0 # ignores the changes ( ¯\_(ツ)_/¯ that's caching! )Supported Cache BackendsNote: if the appropriate backend is not installed, they will be replaced with aMockCacheinstance at runtime. This
is intended to improve portability of pipeline code.RedispipinstallredisAnd then same as above except withfrompipeline.cache.redisimportRedisCacheredis=RedisCache(host='localhost',port=6379)# defaultsMemCachedpipinstallmemcachedAnd then the same as above except withfrom pipeline.cache.mem_cache import MemCache 
mem_cache = MemCache(host='localhost', port=11211) # defaults