Package: aiohttp-retry
Simple aiohttp retry clientPython 3.7 or higher.Install:pip install aiohttp-retry.Breaking API changesEverything between [2.7.0 - 2.8.3) is yanked.There is a bug with evaluate_response_callback, it led to infinite retries2.8.0 is incorrect and yanked.https://github.com/inyutin/aiohttp_retry/issues/79Since 2.5.6 this is a new parameter inget_timeoutfunc called "response".If you have defined your ownRetryOptions, you should add this param into it.
Issue about this:https://github.com/inyutin/aiohttp_retry/issues/59Examples of usage:fromaiohttp_retryimportRetryClient,ExponentialRetryasyncdefmain():retry_options=ExponentialRetry(attempts=1)retry_client=RetryClient(raise_for_status=False,retry_options=retry_options)asyncwithretry_client.get('https://ya.ru')asresponse:print(response.status)awaitretry_client.close()fromaiohttpimportClientSessionfromaiohttp_retryimportRetryClientasyncdefmain():client_session=ClientSession()retry_client=RetryClient(client_session=client_session)asyncwithretry_client.get('https://ya.ru')asresponse:print(response.status)awaitclient_session.close()fromaiohttp_retryimportRetryClient,RandomRetryasyncdefmain():retry_options=RandomRetry(attempts=1)retry_client=RetryClient(raise_for_status=False,retry_options=retry_options)response=awaitretry_client.get('/ping')print(response.status)awaitretry_client.close()fromaiohttp_retryimportRetryClientasyncdefmain():asyncwithRetryClient()asclient:asyncwithclient.get('https://ya.ru')asresponse:print(response.status)You can change parameters between attempts by passing multiple requests params:fromaiohttp_retryimportRetryClient,RequestParams,ExponentialRetryasyncdefmain():retry_client=RetryClient(raise_for_status=False)asyncwithretry_client.requests(params_list=[RequestParams(method='GET',url='https://ya.ru',),RequestParams(method='GET',url='https://ya.ru',headers={'some_header':'some_value'},),])asresponse:print(response.status)awaitretry_client.close()You can also add some logic, F.E. logging, on failures by using trace mechanic.importloggingimportsysfromtypesimportSimpleNamespacefromaiohttpimportClientSession,TraceConfig,TraceRequestStartParamsfromaiohttp_retryimportRetryClient,ExponentialRetryhandler=logging.StreamHandler(sys.stdout)logging.basicConfig(handlers=[handler])logger=logging.getLogger(__name__)retry_options=ExponentialRetry(attempts=2)asyncdefon_request_start(session:ClientSession,trace_config_ctx:SimpleNamespace,params:TraceRequestStartParams,)->None:current_attempt=trace_config_ctx.trace_request_ctx['current_attempt']ifretry_options.attempts<=current_attempt:logger.warning('Wow! We are in last attempt')asyncdefmain():trace_config=TraceConfig()trace_config.on_request_start.append(on_request_start)retry_client=RetryClient(retry_options=retry_options,trace_configs=[trace_config])response=awaitretry_client.get('https://httpstat.us/503',ssl=False)print(response.status)awaitretry_client.close()Look tests for more examples.Be aware: last request returns as it is.If the last request ended with exception, that this exception will be raised from RetryClient requestDocumentationRetryClienttakes the same arguments as ClientSession[docs]RetryClienthas methods:requestgetoptionsheadpostputpatchputdeleteThey are same as forClientSession, but take one possible additional argument:classRetryOptionsBase:def__init__(self,attempts:int=3,# How many times we should retrystatuses:Optional[Iterable[int]]=None,# On which statuses we should retryexceptions:Optional[Iterable[Type[Exception]]]=None,# On which exceptions we should retryretry_all_server_errors:bool=True,# If should retry all 500 errors or not# a callback that will run on response to decide if retryevaluate_response_callback:Optional[EvaluateResponseCallbackType]=None,):...@abc.abstractmethoddefget_timeout(self,attempt:int,response:Optional[Response]=None)->float:raiseNotImplementedErrorYou can specifyRetryOptionsboth forRetryClientand it's methods.RetryOptionsin methods overrideRetryOptionsdefined inRetryClientconstructor.Important: by default all 5xx responses are retried + statuses you specified asstatusesparam
If you will passretry_all_server_errors=Falsethan you can manually set what 5xx errors to retry.You can define your own timeouts logic or use:ExponentialRetrywith exponential backoffRandomRetryfor random backoffListRetrywith backoff you predefine by listFibonacciRetrywith backoff that looks like fibonacci sequenceJitterRetryexponential retry with a bit of randomnessImportant: you can proceed server response as an parameter for calculating next timeout.However this response can be None, server didn't make a response or you have set upraise_for_status=TrueLook here for an example:https://github.com/inyutin/aiohttp_retry/issues/59Additionally, you can specifyevaluate_response_callback. It receive aClientResponseand decide to retry or not by returning a bool.
It can be useful, if server API sometimes response with malformed data.Request Trace ContextRetryClientaddcurrent attempt numbertorequest_trace_ctx(see examples,
for more info seeaiohttp doc).Change parameters between retriesRetryClientalso has a method calledrequests. This method should be used if you want to make requests with different params.@dataclassclassRequestParams:method:strurl:_RAW_URL_TYPEtrace_request_ctx:Optional[Dict[str,Any]]=Nonekwargs:Optional[Dict[str,Any]]=Nonedefrequests(self,params_list:List[RequestParams],retry_options:Optional[RetryOptionsBase]=None,raise_for_status:Optional[bool]=None,)->_RequestContext:You can find an example of usage above or in tests.But basicallyRequestParamsis a structure to define params forClientSession.requestfunc.method,url,headerstrace_request_ctxdefined outside kwargs, because they are popular.There is also an old way to change URL between retries by specifyingurlas list of urls. Example:fromaiohttp_retryimportRetryClientretry_client=RetryClient()asyncwithretry_client.get(url=['/internal_error','/ping'])asresponse:text=awaitresponse.text()assertresponse.status==200asserttext=='Ok!'awaitretry_client.close()In this example we request/interval_error, fail and then successfully request/ping.
If you specify less urls thanattemptsnumber inRetryOptions,RetryClientwill request last url at last attempts.
This means that in example above we would request/pingonce again in case of failure.Typesaiohttp_retryis a typed project. It should be fully compatablie with mypy.It also introduce one special type:ClientType = Union[ClientSession, RetryClient]This type can be imported byfrom aiohttp_retry.types import ClientType