Package: yarr
Yarr ain't RPC, Ritchie!A simple RPC implementation using a binary serialization protocol. It supports
the basic built-in types, numpy arrays work if available on both client and
server. Hence, yarr is well equipped to efficiently handle large binary booty.A friend asked me about RPC and large binary data in python. It sounded like a
fun exercise so I charged forth. I choose to write my own serialization
protocol to have more control and a bit more security than pickle. I also ran
my benchmarks which pickle serialization and surprisingly yarr is not that slow.
Lists and dict take the biggest hit, being ~ 2x slower compared to pickle.Example# serveryarr.yarr(('localhost',8000),[print])# clientyarr.call(('localhost',8000),'print','Ahoy Lads!')FeaturesCompared to other RPC frameworks yarr has better support for native python
types. It's speed depends on the connection. On localhost it appears to be 2-10
times faster than the other frameworks. Over LAN it's comparable to
jsonrpc2-zeromq. Over internet the ZeroMQ based frameworks are twice as fast,
but yarr is still on par with xmlrpc.Here is a table comparing it to other RPC frameworks. Partial support is marked
with p, usually due to type decay. The speed value is based on the benchmarks
below. Feel free to add more.yarrxmlrpcrpycjr2-zmqspyne0rpcspeed1.00.60.040.90.30.6dependencies001315recursion--x---boolxxxxxxintxxxxxxint64x-xxxxbignumx-xx--floatxxxxxxcomplexx-x---strxxxxxxpep383x--x--dictxpxpxxtupledictx-x-p-listxxxxpxtuplexpxpxpsetx-x---bytesxpx-xxndarrayx-x---Xmlrpc is part of python's stdlib. Good enough for simple use-cases.Rpyc is glacially slow but has almost perfect type support. Surprisingly it
stumbles over pep 383. It's also one of the few frameworks that can handle
recursion.Jsonrpc2-zeromq has decent speed, though it is limited by it's protocol. There
is no direct support for binary data. The base64 workaround works but impacts
speed.Spyne is a big project and can do a lot more than pure RPC. For this test
MessagePack and ZeroMQ were used.Zerorpc performance is below jsonrpc2-zeromq, which was unexpected given it's
choice of protocols. A dict containing tuple keys crashes the server.BenchmarksThe values below are in calls per second. This test was run over LAN with a
server ping of 0.2 ms. The most recent framework versions (May 2020) were used.
When running this benchmark over internet with a ping of 25 ms, yarr
performance is roughly half of jsonrpc2-zeromq.yarrxmlrpcrpycjr2-zmqspyne0rpcnone1291746561225457800bool1254746541253450774int1250740491263448769int641238-471188450765bignum1205-491165--float1211750521223448765complex1236-54---str1204770541131449740pep3831236--1134--dict1214741541040447757list1183743561141449759tuple1186714631142448778set1214-49---frozenset1189-54---bytes78550446484349533bytearray78050260481350534ndarray765-19---ProtocolThe main design goals were simplicity and performance. There is no clever
encoding scheme like MessagePack. Data is basically just dumped into the TCP
socket without any transformations. Values are transferred as little endian,
there really is no point in using network byte order these days.In some cases messages can actually be bigger than in json, for example when it
contains many short strings. However if the total size stays below the MTU
there is be no big performance hit. I tried adding zlib compression and a
persistent connection, but that did not have a big impact and made everything
more complicated.BugsNot enough pirate-speak in documentation.Malicious data can cause crash.Recursive containers cause crash.LicenseDo What The Fuck You Want To Public LicenseEveryone is permitted to copy and distribute verbatim or modified
copies of this license document, and changing it is allowed as long
as the name is changed.