Package: rap
raprap(par[::-1]) is advanced and fast python async rpcrapachieves very fast communication throughmsgpackandPython asyncioand multiplexing conn, while supporting high concurrency.
Implement theprotobufofGrpcthrough Python functions and TypeHint.Note: The currentrapAPI may change significantly in subsequent versionsThe rap first version feature idea comes fromaiorpc中文文档1.Installationpipinstallrap2.Quick StartServerimportasynciofromtypingimportAsyncIteratorfromrap.serverimportServerdefsync_sum(a:int,b:int)->int:returna+basyncdefasync_sum(a:int,b:int)->int:awaitasyncio.sleep(1)#  mock ioreturna+basyncdefasync_gen(a:int)->AsyncIterator[int]:foriinrange(a):yieldiloop=asyncio.new_event_loop()rpc_server=Server()# init service# register funcrpc_server.register(sync_sum)rpc_server.register(async_sum)rpc_server.register(async_gen)# run serverloop.run_until_complete(rpc_server.create_server())try:loop.run_forever()exceptKeyboardInterrupt:# stop serverloop.run_until_complete(rpc_server.await_closed())ClientThe client supports to call the service byraw_callandcallmethods, but this can not fully use the functions of TypeHint, it is recommended to use@client.registerto register the function and then call it.Note: Forrap.clientthere is no distinction betweenasync defanddef, but functions registered with@client.registercan be used directly by the user, so functions decorated with@client.registershould be similar to:asyncdefdemo():passexample:importasynciofromtypingimportAsyncIteratorfromrap.clientimportClientclient:"Client"=Client()# init client# Declare a function with no function. The function name, function type and return type must be the same as the server side function (async def does not differ from def)defsync_sum(a:int,b:int)->int:pass# The decorated function must be an async def function@client.register()asyncdefsync_sum(a:int,b:int)->int:pass# The decorated function must be the async def function, because the function is a generator syntax, to `yield` instead of `pass`@client.register()asyncdefasync_gen(a:int)->AsyncIterator:yieldasyncdefmain():awaitclient.connect()# Call the call method; read the function name and then call `raw_call`.print(f"call result:{awaitclient.call(sync_sum,1,2)}")# Basic calls to rap.clientprint(f"raw call result:{awaitclient.raw_call('sync_sum',1,2)}")# Functions registered through `@client.register` can be used directly# await async_sum(1,3) == await client.raw_call('async_sum', 1, 2)# It is recommended to use the @client.register method, which can be used by tools such as IDE to determine whether the parameter type is wrongprint(f"decorator result:{awaitsync_sum(1,3)}")async_gen_result:list=[]# Example of an asynchronous generator, which by default opens or reuses the current session of the rap (about the session will be mentioned below)asyncforiinasync_gen(10):async_gen_result.append(i)print(f"async gen result:{async_gen_result}")asyncio.run(main())3.Function Introduction3.1.Register functionThe server side supportsdefandasync def, if it is adeffunction, it will be run with multiple threads. When registering, the TypeHints of the function's parameters and return value will be checked, and an error will be reported if the type does not match the type specified by json.The server comes with a registration library. If there are duplicate registrations in the same group, an error will be reported. You can use thegroupparameter to define the group to be registered or redefine the name of the registration with thenameparameter (you also need to specify the corresponding group when the client calls it).In addition, you can setis_privateto True when registering, so that the function can only be called by the local rap.client.importasynciofromtypingimportAsyncIteratorfromrap.serverimportServerdefdemo1(a:int,b:int)->int:returna+basyncdefdemo2(a:int,b:int)->int:awaitasyncio.sleep(1)returna+basyncdefdemo_gen(a:int)->AsyncIterator[int]:foriinrange(a):yieldiserver:Server=Server()server.register(demo1)# register def funcserver.register(demo2)# register async def funcserver.register(demo_gen)# register async iterator funcserver.register(demo2,name='demo2-alias')# Register with the value of `name`server.register(demo2,group='new-group')# Register and set the groups to be registeredserver.register(demo2,group='root',is_private=True)# Register and set the group to be registered, and set it to privateFor clients, it is recommended to useclient.registerinstead ofclient.call,client.raw_call.client.registeruses Python syntax to define function names, arguments, parameter types, and return value types,
It allows the caller to call the function as if it were a normal function, and the function can be checked through tools using the TypeHint feature.
Note: When usingclient.register, be sure to useasync def ....fromtypingimportAsyncIteratorfromrap.clientimportClientclient:Client=Client()# register func@client.register()asyncdefdemo1(a:int,b:int)->int:pass# register async iterator fun, replace `pass` with `yield`# Since `async for` will make multiple requests to the same conn over time, it will check if the session is enabled and automatically reuse the current session if it is enabled, otherwise it will create a new session and use it.@client.register()asyncdefdemo_gen(a:int)->AsyncIterator:yield# Register the general function and set the name to demo2-alias@client.register(name='demo2-alias')asyncdefdemo2(a:int,b:int)->int:pass# Register the general function and set the group to new-group@client.register(group='new-group')asyncdefdemo2(a:int,b:int)->int:pass3.2.sessionexamplerapclient support session function, after enabling the session, all requests will only be requested through the current session's conn to the corresponding server, while each request, the session_id in the header will set the current session id, convenient for the server to identify.rapsessions support explicit and implicit settings, each with its own advantages and disadvantages, without mandatory restrictions.fromtypingimportAsyncIteratorfromrap.clientimportClientclient=Client()defsync_sum(a:int,b:int)->int:pass@client.register()asyncdefasync_sum(a:int,b:int)->int:pass@client.register()asyncdefasync_gen(a:int)->AsyncIterator[int]:yieldasyncdefno_param_run():# The rap internal implementation uses the session implicitly via the `contextvar` moduleprint(f"sync result:{awaitclient.call(sync_sum,1,2)}")print(f"async result:{awaitasync_sum(1,3)}")# The asynchronous generator detects if a session is enabled, and if so, it automatically reuses the current session, otherwise it creates a sessionasyncforiinasync_gen(10):print(f"async gen result:{i}")asyncdefparam_run(session:"Session"):# By explicitly passing the session parameters inprint(f"sync result:{awaitclient.call(sync_sum,1,2,session=session)}")print(f"sync result:{awaitclient.raw_call('sync_sum',1,2,session=session)}")# May be a bit unfriendlyprint(f"async result:{awaitasync_sum(1,3,session=session)}")# The asynchronous generator detects if a session is enabled, and if so, it automatically reuses the current session, otherwise it creates a sessionasyncforiinasync_gen(10):print(f"async gen result:{i}")asyncdefexecute(session:"Session"):# The best way to call a session explicitly, using a method similar to the mysql cursor# execute will automatically recognize the type of callprint(f"sync result:{awaitsession.execute(sync_sum,arg_list=[1,2])}")print(f"sync result:{awaitsession.execute('sync_sum',arg_list=[1,2])}")print(f"async result:{awaitsession.execute(async_sum(1,3))}")# The asynchronous generator detects if a session is enabled, and if so, it automatically reuses the current session, otherwise it creates a sessionasyncforiinasync_gen(10):print(f"async gen result:{i}")asyncdefrun_once():awaitclient.connect()# init sessionasyncwithclient.sessionass:awaitno_param_run()awaitparam_run(s)awaitexecute(s)awaitclient.await_close()3.3.channelexamplechannel supports client-server interaction in a duplex manner, similar to Http's WebSocket, it should be noted that the channel does not support group settings.Only@client.registeris supported on the client side to register the channel function, which is characterized by a single argument of typeChannel.
The channel will maintain a session and will only communicate with the server via a conn from the time the channel is enabled to the time it is closed.
To avoid the use of 'while True', the channel supports the use of 'async for' syntax and the use of 'while await channel.loop()` syntax instead of 'while Truefromrap.clientimportChannel,Clientfromrap.client.modelimportResponseclient=Client()@client.register()asyncdefasync_channel(channel:Channel)->None:awaitchannel.write("hello")# send datacnt:int=0whileawaitchannel.loop(cnt<3):cnt+=1print(awaitchannel.read_body())# read data@client.register()asyncdefecho_body(channel:Channel)->None:awaitchannel.write("hi!")# Reads data, returns only when data is read, and exits the loop if it receives a signal to close the channelasyncforbodyinchannel.iter_body():print(f"body:{body}")awaitchannel.write(body)@client.register()asyncdefecho_response(channel:Channel)->None:awaitchannel.write("hi!")# Read the response data (including header data), and return only if the data is read, or exit the loop if a signal is received to close the channelasyncforresponseinchannel.iter_response():response:Response=response#  help IDE check type....print(f"response:{response}")awaitchannel.write(response.body)3.4.sslexampleDue to the high degree of encapsulation of thePython asynciomodule,rapcan be used very easily with ssl# Quickly generate ssl.crt and ssl.keyopensslreq-newkeyrsa:2048-nodes-keyoutrap_ssl.key-x509-days365-outrap_ssl.crtclient.pyfromrap.clientimportClientclient=Client(ssl_crt_path="./rap_ssl.crt")server.pyfromrap.serverimportServerrpc_server=Server(ssl_crt_path="./rap_ssl.crt",ssl_key_path="./rap_ssl.key",)3.5.eventThe server side supportsstart_eventandstop_eventfor event handling before start and after shutdown respectively.fromrap.serverimportServerasyncdefmock_start():print('start event')asyncdefmock_stop():print('stop event')# example 1server=Server(start_event_list=[mock_start()],stop_event_list=[mock_stop()])# example 2server=Server()server.load_start_event([mock_start()])server.load_stop_event([mock_stop()])3.6.middlewarerapcurrently supports 2 types of middleware::Conn middleware: Used when creating conn, such as limiting the total number of links, etc...
referenceblock.py,
Thedispatchmethod will pass in a conn object, and then determine whether to release it according to the rules (return await self.call_next(conn)) or reject it (await conn.close)Message middleware: only supports normal function calls (no support forChannel), similar to the use ofstarlettemiddleware
referenceaccess.pyMessage middleware will pass in 4 parameters: request(current request object), call_id(current call id), func(current call function), param(current parameter) and request to return call_id and result(function execution result or exception object)In addition, the middleware supportsstart_event_handleandstop_event_handlemethods, which are called when theServerstarts and shuts down respectively.example:fromrap.serverimportServerfromrap.server.middlewareimportAccessMsgMiddleware,ConnLimitMiddlewarerpc_server=Server()rpc_server.load_middleware([ConnLimitMiddleware(),AccessMsgMiddleware()])3.7.processorTherapprocessor is used to handle inbound and outbound traffic, whereprocess_requestis for inbound traffic andprocess_responseis for outbound traffic.The methods ofrap.clientandrap.serverprocessors are basically the same,rap.serversupportsstart_event_handleandstop_event_handlemethods, which are called whenServerstarts and shuts down respectivelyserver crypto processor exampleclient crypto processor exampleclient load processor examplefromrap.clientimportClientfromrap.client.processorimportCryptoProcessorclient=Client()client.load_processor([CryptoProcessor('key_id','xxxxxxxxxxxxxxxx')])server load processor examplefromrap.serverimportServerfromrap.server.processorimportCryptoProcessorserver=Server()server.load_processor([CryptoProcessor({'key_id':'xxxxxxxxxxxxxxxx'})])4.Pluginrap supports plug-in functionality throughmiddlewareandprocessor,middlewareonly supports the server side,processorsupports the client and server side4.1.Encrypted transmissionEncrypted transmission only encrypts the request and response body content, not the header etc. While encrypting, the nonce parameter is added to prevent replay, and the timestamp parameter is added to prevent timeout access.client example:fromrap.clientimportClientfromrap.client.processorimportCryptoProcessorclient=Client()# The first parameter is the id of the secret key, the server determines which secret key is used for the current request by the id of the secret key# The second parameter is the key of the secret key, currently only support the length of 16 bits of the secret key# timeout: Requests that exceed the timeout value compared to the current timestamp will be discarded# interval: Clear the nonce interval, the shorter the interval, the more frequent the execution, the greater the useless work, the longer the interval, the more likely to occupy memory, the recommended value is the timeout value is 2 timesclient.load_processor([CryptoProcessor("demo_id","xxxxxxxxxxxxxxxx",timeout=60,interval=120)])server example:fromrap.serverimportServerfromrap.server.processorimportCryptoProcessorserver=Server()# The first parameter is the secret key key-value pair, key is the secret key id, value is the secret key# timeout: Requests that exceed the timeout value compared to the current timestamp will be discarded# nonce_timeout: The expiration time of nonce, the recommended setting is greater than timeoutserver.load_processor([CryptoProcessor({"demo_id":"xxxxxxxxxxxxxxxx"},timeout=60,nonce_timeout=120)])4.2. Limit the maximum number of connServer-side use only, you can limit the maximum number of links on the server side, more than the set value will not handle new requestsfromrap.serverimportServerfromrap.server.middlewareimportConnLimitMiddleware,IpMaxConnMiddlewareserver=Server()server.load_middleware([# max_conn: Current maximum number of conn# block_timeout: Access ban time after exceeding the maximum number of connConnLimitMiddleware(max_conn=100,block_time=60),# ip_max_conn: Maximum number of conn per ip# timeout: The maximum statistics time for each ip, after the time no new requests come in, the relevant statistics will be clearedIpMaxConnMiddleware(ip_max_conn=10,timeout=60),])4.3.Limit ip accessSupport restrict single ip or whole segment ip, support both whitelist and blacklist mode, if whitelist is enabled, blacklist mode is disabled by defaultfromrap.serverimportServerfromrap.server.middlewareimportIpBlockMiddlewareserver=Server()# allow_ip_list: whitelist, support network segment ip, if filled with allow_ip_list, black_ip_list will be invalid# black_ip_list: blacklist, support network segment ipserver.load_middleware([IpBlockMiddleware(allow_ip_list=['192.168.0.0/31'],block_ip_list=['192.168.0.2'])])5.Advanced FeaturesTODO, This feature is not yet implemented6.Protocol DesignTODO, Document is being edited7.Introduction to the rap transportTODO, Document is being edited