Package: rerun.me
A library for rerunning functions in the case of raised exceptions and specific return values with configurable delays.@rerun(on_delay=fibonacci(1000,3),on_error=[ConnectionTimeoutError,DeadlockVictimError],on_return=[None]on_retry=lambdad,r:log.info('Retrying connection again (#%s) in%sseconds'%(r,d)))defconnection(conn_str,params):conn=db(conn_str,params)returndb.open()InstallationThe latest version of rerun.me is available viapip:pipinstallrerun.meAlternatively, you can download and install from source:pythonsetup.pyinstallGetting StartedThererunfunction contains the following signature:defrerun(on_delay=None,on_error=None,on_return=None,on_retry=None,retry_after_delay=False):...It serves as both a function decorator, and a runnable wrapper and is configurable through it’s dynamic parameters. Most
of which are function callbacks which allow the user to highly configure the retrying behavior.This configurable nature is what sets this library apart from others with similar functionality. Many of which allow
basic configuration using defined retry limits and constant delays between requests, which may be OK for the most
simplistic of use cases. But most applications need more complex functionality which can delay with various common
algorithms such as exponential or fibonacci delays. This library provides a subset of the most common delay generators,
but is easily expandable to fit the application-specific needs.Delay GeneratorsDifferenton_delaygenerators can be used for increasing the delays between successive retries. Note that the values
for the delays are given in milliseconds.@rerun(on_delay=[1000,2000],on_error=KeyError)deffunc():...Generators and iterable items can be used to generate delays too.deffancy_generator():# yield delays...@rerun(on_delay=fancy_generator)deffunc():...If a single delay is desired, anintegerorfloatvalue can be given, like so.@rerun(on_delay=1000,on_error=KeyError)deffunc():...A couple of generator functions are provided in the library. These are the typical algorithms used in most systems, and
can serve as a baseline example for more complex delay systems.constant(delay, limit): yields a constant delay at each iterationlinear(start, increment, limit): yields a linearly increasing delay at each iterationexponential(base, multiplier, limit): yields an exponentially increasing delay at each iterationfibonacci(multiplier, limit): yields a delay following the fibonacci pattern at each iterationIf the function fails to yield a response that isn’t handled before running out of generated items by theon_delaygenerator, aMaxRetryExceptionis thrown.@rerun(on_delay=None,on_error=KeyError)# No retriesdeffunc():raiseKeyError# MaxRetryException is raisedError HandlingTheon_errorcan be used to determine if a raised exception should be handled and the function retried. A single
exception can be specified to be handled. If an exception is raised that isn’t handled, it will bubble up to the outer
scope without retrying the function.@rerun(on_delay=[1000],on_error=TypeError)deffunc():raiseKeyError# KeyError isn't handled, and is thus raisedMultiple errors can be given as a sequence to handle more than one.@rerun(on_delay=[1000],on_error=[ValueError,TimeoutError])deffunc():...A callable object (such as a function), can be used for more complex handling of errors. These should accept a single
value, the error raised, and return a boolean indicatingTrueto handle, orFalseto not.@rerun(on_delay=[1000],on_error=lambdax:notisinstance(ValueError,TimeoutError))deffunc():...Return Value HandlingLike raised exception, return values can also be handled in a similar manner. Return values that are handled cause the
function to be retried, and those that aren’t are simply return. A common use case for this is when interacting with
functions that yield a return value that indicates a failed state (like-1orNone), while other values indicate
a successful state (like0or anobject).@rerun(on_delay=[1000],on_return=-1)deffunc()return-1# Function is retried because -1 is handledOne note to make is that if a sequence is given, any value that is matched in the sequence is handled. If, however, the
return value is a sequence, either a function should be used to check for equality oron_returnshould be a sequence
of sequences, like so.# WRONG: checks if [-1, -1] is in the sequence [-1, -1]@rerun(on_delay=[1000],on_return=[-1,-1])deffunc():return[-1,-1]# Not handled# CORRECT: checks if [-1, -1] is the return value@rerun(on_delay=[1000],on_return=lambdax:x==[-1,-1])deffunc():return[-1,-1]# Is handled# CORRECT: checks if [-1, -1] is in the sequence [[-1, -1]]@rerun(on_delay=[1000],on_return=[[-1,-1]])deffunc():return[-1,-1]# Is handledEach time a retry takes place theon_retrycallback is called, if given, passing in the current delay and the number
of retries thus far. Logging is a common use-case for this, as shown below.deflog(delay,retry):logging.info('Retrying function again (#%s) in%sseconds'%(delay,retry))@rerun(on_delay=[1000,2000,3000],on_return=-1,on_retry=log)deffunc():...Theon_retrycallback is called prior to waiting for the delay in-between successive retries. If calling the
it after the delay, theretry_after_delayparameter can be specified.@rerun(on_delay=[1000],on_return=-1,on_retry=lambdad,r:print('Waited%sseconds for retry #%s'%(d,r)))deffunc():...Advanced UsageInstead of using as a decorator,reruncan be used as an instead for wrapping an arbitrary number of function
calls. This can be achieved via therunmethod.deffunc_a():...deffunc_b():...rerunner=rerun(on_delay=...,on_error=...,on_return=...,on_retry=...)# Using same configured rerun instancererun.run(func_a,args,kwargs)rerun.run(func_b,args,kwargs)Besides using the providedrunmethod, like any decorator functions can be locally wrapped, passed around, and
executed.deffunc():...rerunner=rerun(on_delay=...,on_error=...,on_return=...,on_retry=...)rerun_func=rerunner(func)rerun_func(args,kwargs)# Or as a one-off like sorerun(...)(func)(args,kwargs)Each of the function parameters that can be passed intorerun, can actually be configured to accepts different
number of parameters depending on the function. They can each either accept 0 parameters, the parameters that would be
typically passed in, or the wrapped function’s args and kwargs in addition to the parameters typically given.Optionally passing in the args and kwargs allows for building more complex callback functions. Each of the possible
function variations are shown below.defon_delay():...defon_delay(*args,**kwargs):...defon_error():...defon_error(error):...defon_error(error,*args,**kwargs):...defon_return():...defon_return(value):...defon_return(value,*args,**kwargs):...defon_retry():...defon_retry(delay,retries):...defon_retry(delay,retries,*args,**kwargs):...ContributionContributions or suggestions are welcome! Feel free toopen an issueif a bug is found or an enhancement is desired,
or even apull request.ChangelogAll changes and versioning information can be found in theCHANGELOG.LicenseCopyright (c) 2018 Jared Gillespie. SeeLICENSEfor details.