Package: fastapi-queue
fastapi-queueA python implementation of a task queue based onRedisthat can serve as a peak shaver and protect your app.What is fastapi-queue?Fastapi-queue provides a high-performance redis-based task queue that allows requests sent by clients to theFastAPIserver to be cached in the queue for delayed execution. This means that you don't have to worry about overwhelming your back-end data service, nor do you have to worry about requests being immediately rejected due to exceeding the load limit, when there is an influx of requests to your app in a very short period of time.Why fastapi-queue?This module is for people who want to use task queues but don't want to start too many dependencies to prevent increased maintenance costs. For example if you want to enjoy the benefits of queues but want to maintain a lightweight application and don't want to installRabbitMQ, then fastapi-queue is your choice, you just need to rely on python runtime andRedisenvironment.FeatureSeparate gateway and service nodes.Superior horizontal scalability.Fully asynchronous framework, ultra fast.Requirementsfastapiaioredis >= 2.0.0ThreadPoolExecutorPlus >= 0.2.2msgpack >= 1.0.0Installpip install fastapi-queueDocumentationhttps://fastapi-queue.readthedocs.io(on going)Response sequence descriptionExamplesGateway'''A gateway application made with FastAPI, which only handles whether or not to allow therequest, but no need to handle the exact request logic.'''fromtypingimportOptional,AnyfromfastapiimportFastAPI,Requestfromfastapi.responsesimportJSONResponsefromfastapi_queueimportDistributedTaskApplyManagerimportaioredisapp=FastAPI()redis=aioredis.Redis.from_url("redis://localhost")defget_response(success_status:bool,result:Any)->JSONResponse|dict:ifsuccess_status:return{"status":200,"data":result}ifresult==-1:returnJSONResponse(status_code=503,content="Service Temporarily Unavailable")else:returnJSONResponse(status_code=500,content="Internal Server Error")@app.get('/')asyncdefroot(request:Request):success_status:bool=FalseasyncwithDistributedTaskApplyManager(redis=redis,request_path=request.url.path,)asdtmanager:ifnotdtmanager.success():# Exceed the maximum capacity of the back-end queue, return 503 directly.returnJSONResponse(status_code=503,content="Service Temporarily Unavailable")success_status,result=awaitdtmanager.rclt(form_data={},task_level=0)returnget_response(success_status,result)@app.get('/sync-test')asyncdefsync_test(request:Request,x:int):success_status:bool=FalseasyncwithDistributedTaskApplyManager(redis=redis,request_path=request.url.path,)asdtmanager:ifnotdtmanager.success():returnJSONResponse(status_code=503,content="Service Temporarily Unavailable")success_status,result=awaitdtmanager.rclt(form_data={'x':x},task_level=0)returnget_response(success_status,result)@app.get('/async-test')asyncdefasync_test(request:Request,n:int):n=min(n,80)success_status:bool=FalseasyncwithDistributedTaskApplyManager(redis=redis,request_path=request.url.path,)asdtmanager:ifnotdtmanager.success():returnJSONResponse(status_code=503,content="Service Temporarily Unavailable")success_status,result=awaitdtmanager.rclt(form_data={'n':n},task_level=0)returnget_response(success_status,result)Service nodes'''The following code will create a pool of workers of 4 processes with 4 threads undereach process. They rely on redis for synchronization, so you can run other instancesas you like without worrying about creating conflicts.'''fromfastapi_queueimportQueueWorkerfromloguruimportloggerimportasyncioimportaioredisimportsignalimportsysimportosqueueworker=Noneasyncdefasync_root(*args):return"Hello world."defsync_prime_number(redis,mysql,x):# Example synchronous function to determine if the input x is a prime number.# redis and mysql clients are entered by default, starting from the third parameter# is your custom parameters, only keyword parameters are supported.importmath,timeifx==1:returnTruefornumeratorinrange(2,int(math.sqrt(x))):ifx%numerator==0:returnFalsetime.sleep(0.2)# Simulation of calculation timereturnTrueasyncdefasync_fibonacci(redis,mysql,n):# Example asynchronous function to calculate the nth position of the Fibonacci series.# redis and mysql clients are entered by default, starting from the third parameter# is your custom parameters, only keyword parameters are supported.# Be sure to note that all data to upload and download must be serializable by msgpack.# This means that if you transfer some custom object, or in this case a very large integer,# the request will be responsed with an internal server error (http 500).a,b=0,1for_inrange(n):a,b=b,a+bawaitasyncio.sleep(0.2)# Simulation of calculation timereturnaroute_table={'/':async_root,'/sync-test':sync_prime_number,'/async-test':async_fibonacci,}route_table_maximum_concurrency={'/':9999,'/sync-test':320,'/async-test':1000,}asyncdefmain(pid,logger):globalqueueworkerfirst_time_run=TruewhileTrue:run_startup,first_time_run=(Trueifpid!=0elseFalse)andfirst_time_run,Falseredis=aioredis.Redis.from_url("redis://localhost")try:worker=QueueWorker(redis,threads=4,route_table_maximum_concurrency=route_table_maximum_concurrency,allowed_type_limit=None,run_startup=run_startup,logger=logger,)queueworker=worker[worker.method_register(name,func)forname,funcinroute_table.items()]awaitworker.run_serve()ifworker.closeing():breakexceptExceptionase:debug=Trueifdebug:raiseeawaitredis.close()logger.info(f"Pid:{worker.pid}, shutdown")defsigint_capture(sig,frame):ifqueueworker:queueworker.graceful_shutdown(sig,frame)else:sys.exit(1)if__name__=='__main__':logger.remove()logger.add(sys.stderr,level="DEBUG",enqueue=True)signal.signal(signal.SIGINT,sigint_capture)# In order for the program to capture the `ctrl+c` close signalfor_inrange(3):pid=os.fork()ifpid==0:breakasyncio.run(main(pid,logger))PerformanceDue to the fully asynchronous support, complex interprocedural calls exhibit a very low processing latency.(Latency vs. number of request threads on going)(Maximum capability requests per second vs. number of service nodes on going)TipsThe code has been carefully debugged and functions reliably, but I haven't spent much time making it a generic module, which means that if you encounter bugs you'll need to modify the code yourself, and they're usually caused by oversights of detail somewhere.The service has undergone rigorous stress tests and can work for hours under concurrent requests from hundreds of clients, but for reliability of protection, you need to carefully set the upper limit of your load. WhereRateLimitercan provide you with a low consumptionroughlypre-intercepted function.For example,fromfastapi_queueimportRateLimiterfromfastapiimportFastAPI,Requestfromfastapi.responsesimportJSONResponseapp=FastAPI()...@app.on_event("startup")asyncdefstartup():RateLimiter().porter_run_serve()...@app.get("/")@RateLimiter(bucket=5000,limits_s=1000)asyncdefroot(request:Request):'''The two parameters of RateLimiter mean that this particular FastAPI instanceholds a total of 5000 tokens and takes one token each time a request is received.If there is a large influx of requests come in a short period of time, whenthe number of remaining tokens in the bucket decreases to 0, the server willsimply reject all requests without forwarding them to the queue-worker. Withcurrent parameters, this bucket now keeps a maximum of 5000 tokens and restores1000 tokens per second.'''asyncwithDistributedTaskApplyManager(...)