Package: aifail
AiFailPython Library for retrying openai/gcp API callsInstallationpipinstallaifailUsageTo get started, simply wrap your functions with@retry_if, specifying the condition to retry on.https://github.com/GooeyAI/aifail/blob/a540c05a2a9436c0b6b1caab8ed823387999d5f9/examples/basic_openai.py#L5https://github.com/GooeyAI/aifail/blob/a540c05a2a9436c0b6b1caab8ed823387999d5f9/examples/basic_openai.py#L13-L30Custom logicYou can use this with anything that needs retrying, e.g. google sheets -defsheets_api_should_retry(e:Exception)->bool:returnisinstance(e,HttpError)and(e.resp.statusin(408,429)ore.resp.status>500)@retry_if(sheets_api_should_retry)defupdate_cell(spreadsheet_id:str,row:int,col:int,value:str):get_spreadsheet_service().values().update(spreadsheetId=spreadsheet_id,range=f"{col_i2a(col)}{row}:{col_i2a(col)}{row}",body={"values":[[value]]},valueInputOption="RAW",).execute()Advanced UsageThis library is used by GooeyAI in production to handle thousands of API calls every day.
To save costs and handle rate limits, you can intelligently specify quick fallbacks (eg azure openai) -https://github.com/GooeyAI/aifail/blob/a540c05a2a9436c0b6b1caab8ed823387999d5f9/examples/azure_openai_fallback.py#L7-L16https://github.com/GooeyAI/aifail/blob/a540c05a2a9436c0b6b1caab8ed823387999d5f9/examples/azure_openai_fallback.py#L19-L35Traceable ErrorsAiFail comes with a built in logger, and outputs complete stack traces tracing the error back to the original call site.# python examples/azure_openai_fallback.py2023-11-1804:36:01.364|WARNING|aifail.aifail:try_all:63-[2/2]tyringnextfn,prev_exc=NotFoundError("Error code: 404 - {'error': {'code': 'DeploymentNotFound', 'message': 'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}")2023-11-1804:36:01.778|WARNING|aifail.aifail:wrapper:98-[1/1]capturederror,retry_delay=0.4117675457681431s,exc=NotFoundError("Error code: 404 - {'error': {'message': 'The model `gpt-4-x` does not exist', 'type': 'invalid_request_error', 'param': None, 'code': 'model_not_found'}}")2023-11-1804:36:02.483|WARNING|aifail.aifail:try_all:63-[2/2]tyringnextfn,prev_exc=NotFoundError("Error code: 404 - {'error': {'code': 'DeploymentNotFound', 'message': 'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}")2023-11-1804:36:04.093|WARNING|aifail.aifail:wrapper:98-[2/1]capturederror,retry_delay=0.9974197744911488s,exc=NotFoundError("Error code: 404 - {'error': {'message': 'The model `gpt-4-x` does not exist', 'type': 'invalid_request_error', 'param': None, 'code': 'model_not_found'}}")Traceback(mostrecentcalllast):File"/Users/dev/Projects/dara/aifail/aifail/aifail.py",line65,intry_allreturnfn()File"/Users/dev/Projects/dara/aifail/examples/azure_openai_fallback.py",line28,in<lambda>lambda:azure_client.chat.completions.create(...
openai.NotFoundError:Errorcode:404-{'error':{'code':'DeploymentNotFound','message':'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}Theaboveexceptionwasthedirectcauseofthefollowingexception:

Traceback(mostrecentcalllast):File"/Users/dev/Projects/dara/aifail/aifail/aifail.py",line86,inwrapperreturnfn(*args,**kwargs)File"/Users/dev/Projects/dara/aifail/examples/azure_openai_fallback.py",line26,inchad_gpt4response=try_all(File"/Users/dev/Projects/dara/aifail/aifail/aifail.py",line69,intry_allraiseprev_excFile"/Users/dev/Projects/dara/aifail/aifail/aifail.py",line65,intry_allreturnfn()File"/Users/dev/Projects/dara/aifail/examples/azure_openai_fallback.py",line34,in<lambda>lambda:openai_client.chat.completions.create(...
openai.NotFoundError:Errorcode:404-{'error':{'message':'The model `gpt-4-x` does not exist','type':'invalid_request_error','param':None,'code':'model_not_found'}}Theaboveexceptionwasthedirectcauseofthefollowingexception:

Traceback(mostrecentcalllast):File"/Users/dev/Projects/dara/aifail/aifail/aifail.py",line65,intry_allreturnfn()File"/Users/dev/Projects/dara/aifail/examples/azure_openai_fallback.py",line28,in<lambda>lambda:azure_client.chat.completions.create(...
openai.NotFoundError:Errorcode:404-{'error':{'code':'DeploymentNotFound','message':'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}Theaboveexceptionwasthedirectcauseofthefollowingexception:

Traceback(mostrecentcalllast):File"/Users/dev/Projects/dara/aifail/examples/azure_openai_fallback.py",line43,in<module>chad_gpt4(File"/Users/dev/Projects/dara/aifail/aifail/aifail.py",line102,inwrapperraiseprev_excFile"/Users/dev/Projects/dara/aifail/aifail/aifail.py",line86,inwrapperreturnfn(*args,**kwargs)File"/Users/dev/Projects/dara/aifail/examples/azure_openai_fallback.py",line26,inchad_gpt4response=try_all(File"/Users/dev/Projects/dara/aifail/aifail/aifail.py",line69,intry_allraiseprev_excFile"/Users/dev/Projects/dara/aifail/aifail/aifail.py",line65,intry_allreturnfn()File"/Users/dev/Projects/dara/aifail/examples/azure_openai_fallback.py",line34,in<lambda>lambda:openai_client.chat.completions.create(...
openai.NotFoundError:Errorcode:404-{'error':{'message':'The model `gpt-4-x` does not exist','type':'invalid_request_error','param':None,'code':'model_not_found'}}Processfinishedwithexitcode1Sentry too, will capture the entire retry loop