Package: opnieuw
OpnieuwOpnieuw is a general-purpose retrying library, written in Python, in
order to simplify the task of adding retry behavior to both synchronous as well
as asynchronous tasks. Opnieuw is easy and straightforward to use.Opnieuw makes it easy to add robust retries:There is a single retry strategy, exponential backoff with jitter, which
makes it impossible to forget to add jitter.It has just two parameters, to eliminate the possibility of invalid and
nonsensical combinations.Parameters are named clearly, to avoid confusing e.g. number of calls
(including the initial one) with number of retries (excluding the initial
call).The parameters are intuitive: instead of configuring the base delay for
exponential backoff, Opnieuw accepts a maximum number of calls to make, and
maximum time after the first call to still retry.Time units are clear from parameter names to make the decorated code obvious,
and readable without needing to refer to documentation.See ourannouncement postfor a bit more background on why we wrote
Opnieuw.ExampleSuppose we want to parsehttps://tech.channable.com/atom.xmland we want to
add a retry to handle a specific network Exception. We can add Opnieuw to our
network handler as follows:importrequestsfromrequests.exceptionsimportConnectionError,HTTPErrorfromopnieuwimportretry@retry(retry_on_exceptions=(ConnectionError,HTTPError),max_calls_total=4,retry_window_after_first_call_in_seconds=60,)defget_page()->str:response=requests.get('https://tech.channable.com/atom.xml')returnresponse.textretry_on_exceptionsspecifies which exceptions we want to retry on.max_calls_totalis the maximum number of times that the decorated function
gets called, including the initial call.retry_window_after_first_call_in_secondsis the maximum number of seconds
after the first call was initiated, where we would still do a new attempt.FeaturesGeneric decorator APISpecify retry exception (i.e. type of exception that we want retry)Exponential backoff with jitterPre-shipped list of popular exceptions, which can easily be expandedInstallationTo install Opnieuw, simply:$ pip install opnieuwMore examplesThe short example above provides a concise demonstration of how Opnieuw could
be used. Let's dig deeper into Opnieuw and add another exception toretry_on_exceptionsto do a retry on:fromurllib.errorimportURLErrorimportrequestsfromopnieuwimportRetryException,retry@retry(retry_on_exceptions=(ConnectionError,RetryException,URLError),max_calls_total=4,retry_window_after_first_call_in_seconds=60,)defget_page()->str:response=requests.get('https://tech.channable.com/atom.xml')ifresponse.status_code!=200:raiseRetryExceptionreturnresponse.textWe can pass the name of exception we want do retry for or a tuple of exceptions
to theretry_on_exceptions. As mentioned earlier Opnieuw was developed to
make it more convenient to add retry behavior to any task.Let's make it a little bit more generic and define a list of retry exceptions
that should trigger a retry of the function:STANDARD_HTTP_EXCEPTIONS=(ConnectionError,ProtocolError,RetryException,...)@retry(retry_on_exceptions=STANDARD_HTTP_EXCEPTIONS,max_calls_total=4,retry_window_after_first_call_in_seconds=60,)defget_page()->str:response=requests.post('https://tech.channable.com/atom.xml')returnresponse.textNow our retry is more generic, as exceptions raised which are inSTANDARD_HTTP_EXCEPTIONSwill be retried.If you want retry behavior for async tasks, then there is also an async retry
which basically work the same way, but for async tasks.Here is the example above but in async mode:fromopnieuwimportretry_asyncSTANDARD_HTTP_EXCEPTIONS=(ConnectionError,EOFError,RetryException,...)@retry_async(retry_on_exceptions=STANDARD_HTTP_EXCEPTIONS,max_calls_total=4,retry_window_after_first_call_in_seconds=60,)asyncdefget_page()->str:response=requests.get('https://tech.channable.com/atom.xml')returnresponse.text