Package: backoff
backoffFunction decoration for backoff and retryThis module provides function decorators which can be used to wrap a
function such that it will be retried until some condition is met. It
is meant to be of use when accessing unreliable resources with the
potential for intermittent failures i.e. network resources and external
APIs. Somewhat more generally, it may also be of use for dynamically
polling resources for externally generated content.Decorators support both regular functions for synchronous code andasyncio’s coroutines
for asynchronous code.ExamplesSince Kenneth Reitz’srequestsmodule
has become a defacto standard for synchronous HTTP clients in Python,
networking examples below are written using it, but it is in no way required
by the backoff module.@backoff.on_exceptionTheon_exceptiondecorator is used to retry when a specified exception
is raised. Here’s an example using exponential backoff when anyrequestsexception is raised:@backoff.on_exception(backoff.expo,requests.exceptions.RequestException)defget_url(url):returnrequests.get(url)The decorator will also accept a tuple of exceptions for cases where
the same backoff behavior is desired for more than one exception type:@backoff.on_exception(backoff.expo,(requests.exceptions.Timeout,requests.exceptions.ConnectionError))defget_url(url):returnrequests.get(url)Give Up ConditionsOptional keyword arguments can specify conditions under which to give
up.The keyword argumentmax_timespecifies the maximum amount
of total time in seconds that can elapse before giving up.@backoff.on_exception(backoff.expo,requests.exceptions.RequestException,max_time=60)defget_url(url):returnrequests.get(url)Keyword argumentmax_triesspecifies the maximum number of calls
to make to the target function before giving up.@backoff.on_exception(backoff.expo,requests.exceptions.RequestException,max_tries=8,jitter=None)defget_url(url):returnrequests.get(url)In some cases the raised exception instance itself may need to be
inspected in order to determine if it is a retryable condition. Thegiveupkeyword arg can be used to specify a function which accepts
the exception and returns a truthy value if the exception should not
be retried:deffatal_code(e):return400<=e.response.status_code<500@backoff.on_exception(backoff.expo,requests.exceptions.RequestException,max_time=300,giveup=fatal_code)defget_url(url):returnrequests.get(url)By default, when a give up event occurs, the exception in question is reraised
and so code calling anon_exception-decorated function may still
need to do exception handling. This behavior can optionally be disabled
using theraise_on_giveupkeyword argument.In the code below,requests.exceptions.RequestExceptionwill not be raised
when giveup occurs. Note that the decorated function will returnNonein this
case, regardless of the logic in theon_exceptionhandler.deffatal_code(e):return400<=e.response.status_code<500@backoff.on_exception(backoff.expo,requests.exceptions.RequestException,max_time=300,raise_on_giveup=False,giveup=fatal_code)defget_url(url):returnrequests.get(url)This is useful for non-mission critical code where you still wish to retry
the code inside ofbackoff.on_exceptionbut wish to proceed with execution
even if all retries fail.@backoff.on_predicateTheon_predicatedecorator is used to retry when a particular
condition is true of the return value of the target function.  This may
be useful when polling a resource for externally generated content.Here’s an example which uses a fibonacci sequence backoff when the
return value of the target function is the empty list:@backoff.on_predicate(backoff.fibo,lambdax:x==[],max_value=13)defpoll_for_messages(queue):returnqueue.get()Extra keyword arguments are passed when initializing the
wait generator, so themax_valueparam above is passed as a keyword
arg when initializing the fibo generator.When not specified, the predicate param defaults to the falsey test,
so the above can more concisely be written:@backoff.on_predicate(backoff.fibo,max_value=13)defpoll_for_message(queue):returnqueue.get()More simply, a function which continues polling every second until it
gets a non-falsey result could be defined like like this:@backoff.on_predicate(backoff.constant,jitter=None,interval=1)defpoll_for_message(queue):returnqueue.get()The jitter is disabled in order to keep the polling frequency fixed.@backoff.runtimeYou can also use thebackoff.runtimegenerator to make use of the
return value or thrown exception of the decorated method.For example, to use the value in theRetry-Afterheader of the response:@backoff.on_predicate(backoff.runtime,predicate=lambdar:r.status_code==429,value=lambdar:int(r.headers.get("Retry-After")),jitter=None,)defget_url():returnrequests.get(url)JitterA jitter algorithm can be supplied with thejitterkeyword arg to
either of the backoff decorators. This argument should be a function
accepting the original unadulterated backoff value and returning it’s
jittered counterpart.As of version 1.2, the default jitter functionbackoff.full_jitterimplements the ‘Full Jitter’ algorithm as defined in the AWS
Architecture Blog’sExponential Backoff And Jitterpost.
Note that with this algorithm, the time yielded by the wait generator
is actually themaximumamount of time to wait.Previous versions of backoff defaulted to adding some random number of
milliseconds (up to 1s) to the raw sleep value. If desired, this
behavior is now available asbackoff.random_jitter.Using multiple decoratorsThe backoff decorators may also be combined to specify different
backoff behavior for different cases:@backoff.on_predicate(backoff.fibo,max_value=13)@backoff.on_exception(backoff.expo,requests.exceptions.HTTPError,max_time=60)@backoff.on_exception(backoff.expo,requests.exceptions.Timeout,max_time=300)defpoll_for_message(queue):returnqueue.get()Runtime ConfigurationThe decorator functionson_exceptionandon_predicateare
generally evaluated at import time. This is fine when the keyword args
are passed as constant values, but suppose we want to consult a
dictionary with configuration options that only become available at
runtime. The relevant values are not available at import time. Instead,
decorator functions can be passed callables which are evaluated at
runtime to obtain the value:deflookup_max_time():# pretend we have a global reference to 'app' here# and that it has a dictionary-like 'config' propertyreturnapp.config["BACKOFF_MAX_TIME"]@backoff.on_exception(backoff.expo,ValueError,max_time=lookup_max_time)Event handlersBoth backoff decorators optionally accept event handler functions
using the keyword argumentson_success,on_backoff, andon_giveup.
This may be useful in reporting statistics or performing other custom
logging.Handlers must be callables with a unary signature accepting a dict
argument. This dict contains the details of the invocation. Valid keys
include:target: reference to the function or method being invokedargs: positional arguments to funckwargs: keyword arguments to functries: number of invocation tries so farelapsed: elapsed time in seconds so farwait: seconds to wait (on_backoffhandler only)value: value triggering backoff (on_predicatedecorator only)A handler which prints the details of the backoff event could be
implemented like so:defbackoff_hdlr(details):print("Backing off{wait:0.1f}seconds after{tries}tries ""calling function{target}with args{args}and kwargs ""{kwargs}".format(**details))@backoff.on_exception(backoff.expo,requests.exceptions.RequestException,on_backoff=backoff_hdlr)defget_url(url):returnrequests.get(url)Multiple handlers per event typeIn all cases, iterables of handler functions are also accepted, which
are called in turn. For example, you might provide a simple list of
handler functions as the value of theon_backoffkeyword arg:@backoff.on_exception(backoff.expo,requests.exceptions.RequestException,on_backoff=[backoff_hdlr1,backoff_hdlr2])defget_url(url):returnrequests.get(url)Getting exception infoIn the case of theon_exceptiondecorator, allon_backoffandon_giveuphandlers are called from within the except block for the
exception being handled. Therefore exception info is available to the
handler functions via the python standard library, specificallysys.exc_info()or thetracebackmodule. The exception is also
available at theexceptionkey in thedetailsdict passed to the
handlers.Asynchronous codeBackoff supports asynchronous execution in Python 3.5 and above.To use backoff in asynchronous code based onasyncioyou simply need to applybackoff.on_exceptionorbackoff.on_predicateto coroutines.
You can also use coroutines for theon_success,on_backoff, andon_giveupevent handlers, with the interface otherwise being identical.The following examples useaiohttpasynchronous HTTP client/server library.@backoff.on_exception(backoff.expo,aiohttp.ClientError,max_time=60)asyncdefget_url(url):asyncwithaiohttp.ClientSession(raise_for_status=True)assession:asyncwithsession.get(url)asresponse:returnawaitresponse.text()Logging configurationBy default, backoff and retry attempts are logged to the ‘backoff’
logger. By default, this logger is configured with a NullHandler, so
there will be nothing output unless you configure a handler.
Programmatically, this might be accomplished with something as simple
as:logging.getLogger('backoff').addHandler(logging.StreamHandler())The default logging level is INFO, which corresponds to logging
anytime a retry event occurs. If you would instead like to log
only when a giveup event occurs, set the logger level to ERROR.logging.getLogger('backoff').setLevel(logging.ERROR)It is also possible to specify an alternate logger with theloggerkeyword argument.  If a string value is specified the logger will be
looked up by name.@backoff.on_exception(backoff.expo,requests.exceptions.RequestException,logger='my_logger')# ...It is also supported to specify a Logger (or LoggerAdapter) object
directly.my_logger=logging.getLogger('my_logger')my_handler=logging.StreamHandler()my_logger.addHandler(my_handler)my_logger.setLevel(logging.ERROR)@backoff.on_exception(backoff.expo,requests.exceptions.RequestException,logger=my_logger)# ...Default logging can be disabled all together by specifyinglogger=None. In this case, if desired alternative logging behavior
could be defined by using custom event handlers.