Package: ormsgpack
ormsgpackormsgpack is a fast msgpack library for Python. It is a fork/reboot oforjsonIt serializes faster thanmsgpack-pythonand deserializes a bit slower (right now).
It supports serialization of:dataclass,datetime,numpy,pydanticandUUIDinstances natively.Its features and drawbacks compared to other Python msgpack libraries:serializesdataclassinstances natively.serializesdatetime,date, andtimeinstances to RFC 3339 format,
e.g., "1970-01-01T00:00:00+00:00"serializesnumpy.ndarrayinstances natively and faster.serializespydantic.BaseModelinstances nativelyserializes arbitrary types using adefaulthookormsgpack supports CPython 3.8, 3.9, 3.10, 3.11 and 3.12. ormsgpack does not support PyPy. Releases follow semantic
versioning and serializing a new object type without an opt-in flag is
considered a breaking change.ormsgpack is licensed under both the Apache 2.0 and MIT licenses. The
repository and issue tracker isgithub.com/aviramha/ormsgpack, and patches may be
submitted there. There is aCHANGELOGavailable in the repository.UsageInstallQuickstartSerializedefaultoptionDeserializeTypesdataclassdatetimeenumfloatintnumpyuuidpydanticLatencyQuestionsPackagingLicenseUsageInstallTo install a wheel from PyPI:pipinstall--upgrade"pip>=20.3"# manylinux_x_y, universal2 wheel supportpipinstall--upgradeormsgpackTo build a wheel, seepackaging.QuickstartThis is an example of serializing, with options specified, and deserializing:>>>importormsgpack,datetime,numpy>>>data={"type":"job","created_at":datetime.datetime(1970,1,1),"status":"🆗","payload":numpy.array([[1,2],[3,4]]),}>>>ormsgpack.packb(data,option=ormsgpack.OPT_NAIVE_UTC|ormsgpack.OPT_SERIALIZE_NUMPY)b'\x84\xa4type\xa3job\xaacreated_at\xb91970-01-01T00:00:00+00:00\xa6status\xa4\xf0\x9f\x86\x97\xa7payload\x92\x92\x01\x02\x92\x03\x04'>>>ormsgpack.unpackb(_){'type':'job','created_at':'1970-01-01T00:00:00+00:00','status':'🆗','payload':[[1,2],[3,4]]}Serializedefpackb(__obj:Any,default:Optional[Callable[[Any],Any]]=...,option:Optional[int]=...,)->bytes:...packb()serializes Python objects to msgpack.It natively serializesbytes,str,dict,list,tuple,int,float,bool,dataclasses.dataclass,typing.TypedDict,datetime.datetime,datetime.date,datetime.time,uuid.UUID,numpy.ndarray, andNoneinstances. It supports arbitrary types throughdefault. It
serializes subclasses ofstr,int,dict,list,dataclasses.dataclass, andenum.Enum. It does not serialize subclasses
oftupleto avoid serializingnamedtupleobjects as arrays. To avoid
serializing subclasses, specify the optionormsgpack.OPT_PASSTHROUGH_SUBCLASS.The output is abytesobject containing UTF-8.The global interpreter lock (GIL) is held for the duration of the call.It raisesMsgpackEncodeErroron an unsupported type. This exception message
describes the invalid object with the error messageType is not msgpack serializable: .... To fix this, specifydefault.It raisesMsgpackEncodeErroron astrthat contains invalid UTF-8.It raisesMsgpackEncodeErrorif adicthas a key of a type other thanstrorbytes,
unlessOPT_NON_STR_KEYSis specified.It raisesMsgpackEncodeErrorif the output ofdefaultrecurses to handling bydefaultmore than 254 levels deep.It raisesMsgpackEncodeErroron circular references.It raisesMsgpackEncodeErrorif atzinfoon a datetime object is
unsupported.MsgpackEncodeErroris a subclass ofTypeError.defaultTo serialize a subclass or arbitrary types, specifydefaultas a
callable that returns a supported type.defaultmay be a function,
lambda, or callable class instance. To specify that a type was not
handled bydefault, raise an exception such asTypeError.>>>importormsgpack,decimal>>>defdefault(obj):ifisinstance(obj,decimal.Decimal):returnstr(obj)raiseTypeError>>>ormsgpack.packb(decimal.Decimal("0.0842389659712649442845"))MsgpackEncodeError:Typeisnotmsgpackserializable:decimal.Decimal>>>ormsgpack.packb(decimal.Decimal("0.0842389659712649442845"),default=default)b'\xb80.0842389659712649442845'>>>ormsgpack.packb({1,2},default=default)ormsgpack.MsgpackEncodeError:Typeisnotmsgpackserializable:setThedefaultcallable may return an object that itself
must be handled bydefaultup to 254 times before an exception
is raised.It is important thatdefaultraise an exception if a type cannot be handled.
Python otherwise implicitly returnsNone, which appears to the caller
like a legitimate value and is serialized:>>>importormsgpack,json,rapidjson>>>defdefault(obj):ifisinstance(obj,decimal.Decimal):returnstr(obj)>>>ormsgpack.unpackb(ormsgpack.packb({"set":{1,2}},default=default)){'set':None}To serialize a type as a MessagePack extension type, return anormsgpack.Extobject. The instantiation arguments are an integer in
the range[0, 127]and abytesobject, defining the type and
value, respectively.>>>importormsgpack,decimal>>>defdefault(obj):ifisinstance(obj,decimal.Decimal):returnormsgpack.Ext(0,str(obj).encode())raiseTypeError>>>ormsgpack.packb(decimal.Decimal("0.0842389659712649442845"),default=default)b'\xc7\x18\x000.0842389659712649442845'optionTo modify how data is serialized, specifyoption. Eachoptionis an integer
constant inormsgpack. To specify multiple options, mask them together, e.g.,option=ormsgpack.OPT_NON_STR_KEYS | ormsgpack.OPT_NAIVE_UTC.OPT_NAIVE_UTCSerializedatetime.datetimeobjects without atzinfoas UTC. This
has no effect ondatetime.datetimeobjects that havetzinfoset.>>>importormsgpack,datetime>>>ormsgpack.unpackb(ormsgpack.packb(datetime.datetime(1970,1,1,0,0,0),))"1970-01-01T00:00:00">>>ormsgpack.unpackb(ormsgpack.packb(datetime.datetime(1970,1,1,0,0,0),option=ormsgpack.OPT_NAIVE_UTC,))"1970-01-01T00:00:00+00:00"OPT_NON_STR_KEYSSerializedictkeys of type other thanstr. This allowsdictkeys
to be one ofstr,int,float,bool,None,datetime.datetime,datetime.date,datetime.time,enum.Enum, anduuid.UUID.>>>importormsgpack,datetime,uuid>>>ormsgpack.packb({uuid.UUID("7202d115-7ff3-4c81-a7c1-2a1f067b1ece"):[1,2,3]},option=ormsgpack.OPT_NON_STR_KEYS,)>>>ormsgpack.packb({datetime.datetime(1970,1,1,0,0,0):[1,2,3]},option=ormsgpack.OPT_NON_STR_KEYS|ormsgpack.OPT_NAIVE_UTC,)These types are generally serialized how they would be as
values, e.g.,datetime.datetimeis still an RFC 3339 string and respects
options affecting it.This option has the risk of creating duplicate keys. This is because non-strobjects may serialize to the samestras an existing key, e.g.,{"1970-01-01T00:00:00+00:00": true, datetime.datetime(1970, 1, 1, 0, 0, 0): false}.
The last key to be inserted to thedictwill be serialized last and a msgpack deserializer will presumably take the last
occurrence of a key (in the above,false). The first value will be lost.This option is not compatible withormsgpack.OPT_SORT_KEYS.OPT_OMIT_MICROSECONDSDo not serialize themicrosecondfield ondatetime.datetimeanddatetime.timeinstances.>>>importormsgpack,datetime>>>ormsgpack.packb(datetime.datetime(1970,1,1,0,0,0,1),)>>>ormsgpack.packb(datetime.datetime(1970,1,1,0,0,0,1),option=ormsgpack.OPT_OMIT_MICROSECONDS,)OPT_PASSTHROUGH_BIG_INTEnables passthrough of big (Python) ints. By setting this option, one can set adefaultfunction for ints larger than 63 bits, smaller ints are still serialized efficiently.>>>importormsgpack>>>ormsgpack.packb(2**65,)TypeError:Integerexceeds64-bitrange>>>ormsgpack.unpackb(ormsgpack.packb(2**65,option=ormsgpack.OPT_PASSTHROUGH_BIG_INT,default=lambda_:{"type":"bigint","value":str(_)})){'type':'bigint','value':'36893488147419103232'}OPT_PASSTHROUGH_DATACLASSPassthroughdataclasses.dataclassinstances todefault. This allows
customizing their output but is much slower.>>>importormsgpack,dataclasses>>>@dataclasses.dataclassclassUser:id:strname:strpassword:strdefdefault(obj):ifisinstance(obj,User):return{"id":obj.id,"name":obj.name}raiseTypeError>>>ormsgpack.packb(User("3b1","asd","zxc"))b'\x83\xa2id\xa33b1\xa4name\xa3asd\xa8password\xa3zxc'>>>ormsgpack.packb(User("3b1","asd","zxc"),option=ormsgpack.OPT_PASSTHROUGH_DATACLASS)TypeError:Typeisnotmsgpackserializable:User>>>ormsgpack.packb(User("3b1","asd","zxc"),option=ormsgpack.OPT_PASSTHROUGH_DATACLASS,default=default,)b'\x82\xa2id\xa33b1\xa4name\xa3asd'OPT_PASSTHROUGH_DATETIMEPassthroughdatetime.datetime,datetime.date, anddatetime.timeinstances
todefault. This allows serializing datetimes to a custom format, e.g.,
HTTP dates:>>>importormsgpack,datetime>>>defdefault(obj):ifisinstance(obj,datetime.datetime):returnobj.strftime("%a,%d%b %Y %H:%M:%S GMT")raiseTypeError>>>ormsgpack.packb({"created_at":datetime.datetime(1970,1,1)})b'\x81\xaacreated_at\xb31970-01-01T00:00:00'>>>ormsgpack.packb({"created_at":datetime.datetime(1970,1,1)},option=ormsgpack.OPT_PASSTHROUGH_DATETIME)TypeError:Typeisnotmsgpackserializable:datetime.datetime>>>ormsgpack.packb({"created_at":datetime.datetime(1970,1,1)},option=ormsgpack.OPT_PASSTHROUGH_DATETIME,default=default,)b'\x81\xaacreated_at\xbdThu, 01 Jan 1970 00:00:00 GMT'This does not affect datetimes indictkeys if using OPT_NON_STR_KEYS.OPT_PASSTHROUGH_SUBCLASSPassthrough subclasses of builtin types todefault.>>>importormsgpack>>>classSecret(str):passdefdefault(obj):ifisinstance(obj,Secret):return"******"raiseTypeError>>>ormsgpack.packb(Secret("zxc"))b'\xa3zxc'>>>ormsgpack.packb(Secret("zxc"),option=ormsgpack.OPT_PASSTHROUGH_SUBCLASS)TypeError:Typeisnotmsgpackserializable:Secret>>>ormsgpack.packb(Secret("zxc"),option=ormsgpack.OPT_PASSTHROUGH_SUBCLASS,default=default)b'\xa6******'This does not affect serializing subclasses asdictkeys if using
OPT_NON_STR_KEYS.OPT_PASSTHROUGH_TUPLEPassthrough tuples todefault.>>>importormsgpack>>>ormsgpack.unpackb(ormsgpack.packb((9193,"test",42),))[9193,'test',42]>>>ormsgpack.unpackb(ormsgpack.packb((9193,"test",42),option=ormsgpack.OPT_PASSTHROUGH_TUPLE,default=lambda_:{"type":"tuple","value":list(_)})){'type':'tuple','value':[9193,'test',42]}OPT_SERIALIZE_NUMPYSerializenumpy.ndarrayinstances. For more, seenumpy.OPT_SERIALIZE_PYDANTICSerializepydantic.BaseModelinstances.OPT_SORT_KEYSSerializedictkeys in sorted order. The default is to serialize in an
unspecified order.This can be used to ensure the order is deterministic for hashing or tests.
It has a substantial performance penalty and is not recommended in general.>>>importormsgpack>>>ormsgpack.packb({"b":1,"c":2,"a":3})b'\x83\xa1b\x01\xa1c\x02\xa1a\x03'>>>ormsgpack.packb({"b":1,"c":2,"a":3},option=ormsgpack.OPT_SORT_KEYS)b'\x83\xa1a\x03\xa1b\x01\xa1c\x02'The sorting is not collation/locale-aware:>>>importormsgpack>>>ormsgpack.packb({"a":1,"ä":2,"A":3},option=ormsgpack.OPT_SORT_KEYS)b'\x83\xa1A\x03\xa1a\x01\xa2\xc3\xa4\x02'dataclassalso serialize as maps but this has no effect on them.OPT_UTC_ZSerialize a UTC timezone ondatetime.datetimeinstances asZinstead
of+00:00.>>>importormsgpack,datetime>>>ormsgpack.packb(datetime.datetime(1970,1,1,0,0,0,tzinfo=datetime.timezone.utc),)b'"1970-01-01T00:00:00+00:00"'>>>ormsgpack.packb(datetime.datetime(1970,1,1,0,0,0,tzinfo=datetime.timezone.utc),option=ormsgpack.OPT_UTC_Z)b'"1970-01-01T00:00:00Z"'Deserializedefunpackb(__obj:Union[bytes,bytearray,memoryview],/,ext_hook:Optional[Callable[[int,bytes],Any]]=...,option:Optional[int]=...,)->Any:...unpackb()deserializes msgpack to Python objects. It deserializes todict,list,int,float,str,bool,bytesandNoneobjects.bytes,bytearray,memoryviewinput are accepted.ormsgpack maintains a cache of map keys for the duration of the process. This
causes a net reduction in memory usage by avoiding duplicate strings. The
keys must be at most 64 bytes to be cached and 512 entries are stored.The global interpreter lock (GIL) is held for the duration of the call.It raisesMsgpackDecodeErrorif given an invalid type or invalid
msgpack.MsgpackDecodeErroris a subclass ofValueError.ext_hookTo deserialize extension types, specify the optionalext_hookargument. The value should be a callable and is invoked with the
extension type and value as arguments.>>>importormsgpack,decimal>>>defext_hook(tag,data):iftag==0:returndecimal.Decimal(data.decode())raiseTypeError>>>ormsgpack.packb(ormsgpack.Ext(0,str(decimal.Decimal("0.0842389659712649442845")).encode()))>>>ormsgpack.unpackb(_,ext_hook=ext_hook)Decimal('0.0842389659712649442845')optionunpackb()supports theOPT_NON_STR_KEYSoption, that is similar to original msgpack'sstrict_map_key=False.
Be aware that this option is considered unsafe and disabled by default in msgpack due to possibility of HashDoS.Typesdataclassormsgpack serializes instances ofdataclasses.dataclassnatively. It serializes
instances 40-50x as fast as other libraries and avoids a severe slowdown seen
in other libraries compared to serializingdict.It is supported to pass all variants of dataclasses, including dataclasses
using__slots__, frozen dataclasses, those with optional or default
attributes, and subclasses. There is a performance benefit to not
using__slots__.Dataclasses are serialized as maps, with every attribute serialized and in
the order given on class definition:>>>importdataclasses,ormsgpack,typing@dataclasses.dataclassclassMember:id:intactive:bool=dataclasses.field(default=False)@dataclasses.dataclassclassObject:id:intname:strmembers:typing.List[Member]>>>ormsgpack.packb(Object(1,"a",[Member(1,True),Member(2)]))b'\x83\xa2id\x01\xa4name\xa1a\xa7members\x92\x82\xa2id\x01\xa6active\xc3\x82\xa2id\x02\xa6active\xc2'Performance--------------------------------------------------------------------------------- benchmark 'dataclass': 2 tests --------------------------------------------------------------------------------
Name (time in ms)                 Min                 Max                Mean            StdDev              Median               IQR            Outliers       OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dataclass_ormsgpack       3.4248 (1.0)        7.7949 (1.0)        3.6266 (1.0)      0.3293 (1.0)        3.5815 (1.0)      0.0310 (1.0)          4;34  275.7434 (1.0)         240           1
test_dataclass_msgpack       140.2774 (40.96)    143.6087 (18.42)    141.3847 (38.99)    1.0038 (3.05)     141.1823 (39.42)    0.7304 (23.60)         2;1    7.0729 (0.03)          8           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------datetimeormsgpack serializesdatetime.datetimeobjects toRFC 3339format,
e.g., "1970-01-01T00:00:00+00:00". This is a subset of ISO 8601 and is
compatible withisoformat()in the standard library.>>>importormsgpack,datetime,zoneinfo>>>ormsgpack.packb(datetime.datetime(2018,12,1,2,3,4,9,tzinfo=zoneinfo.ZoneInfo('Australia/Adelaide')))>>>ormsgpack.unpackb(_)"2018-12-01T02:03:04.000009+10:30">>>ormsgpack.packb(datetime.datetime.fromtimestamp(4123518902).replace(tzinfo=datetime.timezone.utc))>>>ormsgpack.unpackb(_)"2100-09-01T21:55:02+00:00">>>ormsgpack.packb(datetime.datetime.fromtimestamp(4123518902))>>>ormsgpack.unpackb(_)"2100-09-01T21:55:02"datetime.datetimesupports instances with atzinfothat isNone,datetime.timezone.utc, a timezone instance from the python3.9+zoneinfomodule, or a timezone instance from the third-partypendulum,pytz, ordateutil/arrowlibraries.datetime.timeobjects must not have atzinfo.>>>importormsgpack,datetime>>>ormsgpack.packb(datetime.time(12,0,15,290))>>>ormsgpack.unpackb(_)"12:00:15.000290"datetime.dateobjects will always serialize.>>>importormsgpack,datetime>>>ormsgpack.packb(datetime.date(1900,1,2))>>>ormsgpack.unpackb(_)"1900-01-02"Errors withtzinforesult inMsgpackEncodeErrorbeing raised.To disable serialization ofdatetimeobjects specify the optionormsgpack.OPT_PASSTHROUGH_DATETIME.To use "Z" suffix instead of "+00:00" to indicate UTC ("Zulu") time, use the optionormsgpack.OPT_UTC_Z.To assume datetimes without timezone are UTC, use the optionormsgpack.OPT_NAIVE_UTC.enumormsgpack serializes enums natively. Options apply to their values.>>>importenum,datetime,ormsgpack>>>classDatetimeEnum(enum.Enum):EPOCH=datetime.datetime(1970,1,1,0,0,0)>>>ormsgpack.packb(DatetimeEnum.EPOCH)>>>ormsgpack.unpackb(_)"1970-01-01T00:00:00">>>ormsgpack.packb(DatetimeEnum.EPOCH,option=ormsgpack.OPT_NAIVE_UTC)>>>ormsgpack.unpackb(_)"1970-01-01T00:00:00+00:00"Enums with members that are not supported types can be serialized usingdefault:>>>importenum,ormsgpack>>>classCustom:def__init__(self,val):self.val=valdefdefault(obj):ifisinstance(obj,Custom):returnobj.valraiseTypeErrorclassCustomEnum(enum.Enum):ONE=Custom(1)>>>ormsgpack.packb(CustomEnum.ONE,default=default)>>>ormsgpack.unpackb(_)1floatormsgpack serializes and deserializes double precision floats with no loss of
precision and consistent rounding.intormsgpack serializes and deserializes 64-bit integers by default. The range
supported is a signed 64-bit integer's minimum (-9223372036854775807) to
an unsigned 64-bit integer's maximum (18446744073709551615).numpyormsgpack natively serializesnumpy.ndarrayand individualnumpy.float64,numpy.float32,numpy.int64,numpy.int32,numpy.int16,numpy.int8,numpy.uint64,numpy.uint32,numpy.uint16,numpy.uint8,numpy.uintp,numpy.intp, andnumpy.boolinstances.ormsgpack is faster than all compared libraries at serializing
numpy instances. Serializing numpy data requires specifyingoption=ormsgpack.OPT_SERIALIZE_NUMPY.>>>importormsgpack,numpy>>>ormsgpack.packb(numpy.array([[1,2,3],[4,5,6]]),option=ormsgpack.OPT_SERIALIZE_NUMPY,)>>>ormsgpack.unpackb(_)[[1,2,3],[4,5,6]]The array must be a contiguous C array (C_CONTIGUOUS) and one of the
supported datatypes.If an array is not a contiguous C array or contains an supported datatype,
ormsgpack falls through todefault. Indefault,obj.tolist()can be
specified. If an array is malformed, which is not expected,ormsgpack.MsgpackEncodeErroris raised.Performance---------------------------------------------------------------------------------- benchmark 'numpy float64': 2 tests ---------------------------------------------------------------------------------
Name (time in ms)                      Min                 Max                Mean             StdDev              Median                IQR            Outliers      OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_numpy_ormsgpack[float64]      77.9625 (1.0)       85.2507 (1.0)       79.0326 (1.0)       1.9043 (1.0)       78.5505 (1.0)       0.7408 (1.0)           1;1  12.6530 (1.0)          13           1
test_numpy_msgpack[float64]       511.5176 (6.56)     606.9395 (7.12)     559.0017 (7.07)     44.0661 (23.14)    572.5499 (7.29)     81.2972 (109.75)        3;0   1.7889 (0.14)          5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


------------------------------------------------------------------------------------- benchmark 'numpy int32': 2 tests -------------------------------------------------------------------------------------
Name (time in ms)                      Min                   Max                  Mean             StdDev                Median                IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_numpy_ormsgpack[int32]       197.8751 (1.0)        210.3111 (1.0)        201.1033 (1.0)       5.1886 (1.0)        198.8518 (1.0)       3.8297 (1.0)           1;1  4.9726 (1.0)           5           1
test_numpy_msgpack[int32]       1,363.8515 (6.89)     1,505.4747 (7.16)     1,428.2127 (7.10)     53.4176 (10.30)    1,425.3516 (7.17)     72.8064 (19.01)         2;0  0.7002 (0.14)          5           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


-------------------------------------------------------------------------------- benchmark 'numpy int8': 2 tests ---------------------------------------------------------------------------------
Name (time in ms)                   Min                 Max                Mean            StdDev              Median                IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_numpy_ormsgpack[int8]     107.8013 (1.0)      113.7336 (1.0)      109.0364 (1.0)      1.7805 (1.0)      108.3574 (1.0)       0.4066 (1.0)           1;2  9.1712 (1.0)          10           1
test_numpy_msgpack[int8]       685.4149 (6.36)     703.2958 (6.18)     693.2396 (6.36)     7.9572 (4.47)     691.5435 (6.38)     14.4142 (35.45)         1;0  1.4425 (0.16)          5           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


------------------------------------------------------------------------------------- benchmark 'numpy npbool': 2 tests --------------------------------------------------------------------------------------
Name (time in ms)                       Min                   Max                  Mean             StdDev                Median                IQR            Outliers      OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_numpy_ormsgpack[npbool]        87.9005 (1.0)         89.5460 (1.0)         88.7928 (1.0)       0.5098 (1.0)         88.8508 (1.0)       0.6609 (1.0)           4;0  11.2622 (1.0)          12           1
test_numpy_msgpack[npbool]       1,095.0599 (12.46)    1,176.3442 (13.14)    1,120.5916 (12.62)    32.9993 (64.73)    1,110.4216 (12.50)    38.4189 (58.13)         1;0   0.8924 (0.08)          5           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


--------------------------------------------------------------------------------- benchmark 'numpy uint8': 2 tests ---------------------------------------------------------------------------------
Name (time in ms)                    Min                 Max                Mean             StdDev              Median                IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_numpy_ormsgpack[uint8]     133.1743 (1.0)      134.7246 (1.0)      134.2793 (1.0)       0.4946 (1.0)      134.3120 (1.0)       0.4492 (1.0)           1;1  7.4472 (1.0)           8           1
test_numpy_msgpack[uint8]       727.1393 (5.46)     824.8247 (6.12)     775.7032 (5.78)     34.9887 (70.73)    775.9595 (5.78)     36.2824 (80.78)         2;0  1.2892 (0.17)          5           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------uuidormsgpack serializesuuid.UUIDinstances toRFC 4122format, e.g.,
"f81d4fae-7dec-11d0-a765-00a0c91e6bf6".>>>importormsgpack,uuid>>>ormsgpack.packb(uuid.UUID('f81d4fae-7dec-11d0-a765-00a0c91e6bf6'))>>>ormsgpack.unpackb(_)"f81d4fae-7dec-11d0-a765-00a0c91e6bf6">>>ormsgpack.packb(uuid.uuid5(uuid.NAMESPACE_DNS,"python.org"))>>>ormsgpack.unpackb(_)"886313e1-3b8a-5372-9b90-0c9aee199e5d"Pydanticormsgpack serializespydantic.BaseModelinstances natively.Performance-------------------------------------------------------------------------------- benchmark 'pydantic': 2 tests ---------------------------------------------------------------------------------
Name (time in ms)                Min                 Max                Mean            StdDev              Median               IQR            Outliers       OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pydantic_ormsgpack       4.3918 (1.0)       12.6521 (1.0)        4.8550 (1.0)      1.1455 (3.98)       4.6101 (1.0)      0.0662 (1.0)         11;24  205.9727 (1.0)         204           1
test_pydantic_msgpack       124.5500 (28.36)    125.5427 (9.92)     125.0582 (25.76)    0.2877 (1.0)      125.0855 (27.13)    0.2543 (3.84)          2;0    7.9963 (0.04)          8           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------LatencyGraphsData----------------------------------------------------------------------------- benchmark 'canada packb': 2 tests ------------------------------------------------------------------------------
Name (time in ms)                   Min                Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_packb[canada]     3.5302 (1.0)       3.8939 (1.0)      3.7319 (1.0)      0.0563 (1.0)      3.7395 (1.0)      0.0484 (1.0)         56;22  267.9571 (1.0)         241           1
test_msgpack_packb[canada]       8.8642 (2.51)     14.0432 (3.61)     9.3660 (2.51)     0.5649 (10.03)    9.2983 (2.49)     0.0982 (2.03)         3;11  106.7691 (0.40)        106           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


------------------------------------------------------------------------------- benchmark 'canada unpackb': 2 tests --------------------------------------------------------------------------------
Name (time in ms)                      Min                Max               Mean             StdDev             Median                IQR            Outliers      OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_msgpack_unpackb[canada]       10.1176 (1.0)      62.0466 (1.18)     33.4806 (1.0)      18.8279 (1.0)      46.6582 (1.0)      38.5921 (1.02)         30;0  29.8680 (1.0)          67           1
test_ormsgpack_unpackb[canada]     11.3992 (1.13)     52.6587 (1.0)      34.1842 (1.02)     18.9461 (1.01)     47.6456 (1.02)     37.8024 (1.0)           8;0  29.2533 (0.98)         20           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------- benchmark 'citm_catalog packb': 2 tests -----------------------------------------------------------------------------
Name (time in ms)                         Min               Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_packb[citm_catalog]     1.8024 (1.0)      2.1259 (1.0)      1.9487 (1.0)      0.0346 (1.0)      1.9525 (1.0)      0.0219 (1.0)         79;60  513.1650 (1.0)         454           1
test_msgpack_packb[citm_catalog]       3.4195 (1.90)     3.8128 (1.79)     3.6928 (1.90)     0.0535 (1.55)     3.7009 (1.90)     0.0250 (1.14)        47;49  270.7958 (0.53)        257           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


------------------------------------------------------------------------------ benchmark 'citm_catalog unpackb': 2 tests ------------------------------------------------------------------------------
Name (time in ms)                           Min                Max               Mean             StdDev            Median               IQR            Outliers      OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_unpackb[citm_catalog]     5.6986 (1.0)      46.1843 (1.0)      14.2491 (1.0)      15.9791 (1.0)      6.1051 (1.0)      0.3074 (1.0)           5;5  70.1798 (1.0)          23           1
test_msgpack_unpackb[citm_catalog]       7.2600 (1.27)     56.6642 (1.23)     16.4095 (1.15)     16.3257 (1.02)     7.7364 (1.27)     0.4944 (1.61)        28;29  60.9404 (0.87)        125           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------------- benchmark 'github packb': 2 tests -----------------------------------------------------------------------------------
Name (time in us)                     Min                 Max                Mean            StdDev              Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_packb[github]      73.0000 (1.0)      215.9000 (1.0)       80.4826 (1.0)      4.8889 (1.0)       80.3000 (1.0)      1.1000 (1.83)     866;1118       12.4250 (1.0)        6196           1
test_msgpack_packb[github]       103.8000 (1.42)     220.8000 (1.02)     112.8049 (1.40)     4.9686 (1.02)     113.0000 (1.41)     0.6000 (1.0)     1306;1560        8.8649 (0.71)       7028           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------------- benchmark 'github unpackb': 2 tests -----------------------------------------------------------------------------------
Name (time in us)                       Min                 Max                Mean            StdDev              Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_unpackb[github]     201.3000 (1.0)      318.5000 (1.0)      219.0861 (1.0)      6.7340 (1.0)      219.1000 (1.0)      1.2000 (1.0)       483;721        4.5644 (1.0)        3488           1
test_msgpack_unpackb[github]       289.8000 (1.44)     436.0000 (1.37)     314.9631 (1.44)     9.4130 (1.40)     315.1000 (1.44)     2.3000 (1.92)      341;557        3.1750 (0.70)       2477           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------- benchmark 'twitter packb': 2 tests ---------------------------------------------------------------------------------------
Name (time in us)                        Min                   Max                  Mean             StdDev                Median                IQR            Outliers         OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_packb[twitter]       820.7000 (1.0)      2,945.2000 (2.03)       889.3791 (1.0)      78.4139 (2.43)       884.2000 (1.0)      12.5250 (1.0)          4;76  1,124.3799 (1.0)         809           1
test_msgpack_packb[twitter]       1,209.3000 (1.47)     1,451.2000 (1.0)      1,301.3615 (1.46)     32.2147 (1.0)      1,306.7000 (1.48)     14.1000 (1.13)      118;138    768.4260 (0.68)        592           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


------------------------------------------------------------------------------ benchmark 'twitter unpackb': 2 tests -----------------------------------------------------------------------------
Name (time in ms)                      Min                Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_unpackb[twitter]     2.7097 (1.0)      41.1530 (1.0)      3.2721 (1.0)      3.5860 (1.03)     2.8868 (1.0)      0.0614 (1.32)         4;38  305.6098 (1.0)         314           1
test_msgpack_unpackb[twitter]       3.8079 (1.41)     42.0617 (1.02)     4.4459 (1.36)     3.4893 (1.0)      4.1097 (1.42)     0.0465 (1.0)          2;54  224.9267 (0.74)        228           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ReproducingThe above was measured using Python 3.7.9 on Azure Linux VM (x86_64) with ormsgpack 0.2.1 and msgpack 1.0.2.The latency results can be reproduced using./scripts/benchmark.shand graphs usingpytest --benchmark-histogram benchmarks/bench_*.QuestionsWhy can't I install it from PyPI?Probablypipneeds to be upgraded to version 20.3 or later to support
the latest manylinux_x_y or universal2 wheel formats.Will it deserialize to dataclasses, UUIDs, decimals, etc or support object_hook?No. This requires a schema specifying what types are expected and how to
handle errors etc. This is addressed by data validation libraries a
level above this.PackagingTo package ormsgpack requiresRust1.65
or newer and thematurinbuild
tool. The default featureunstable-simdenables the usage of SIMD
operations and requires nightly Rust. The recommended build command
is:maturinbuild--release--stripormsgpack is tested on Linux/amd64, Linux/aarch64, Linux/armv7, macOS/amd64 and Windows/amd64.There are no runtime dependencies other than libc.Licenseorjson was written by ijl <ijl@mailbox.org>, copyright 2018 - 2021, licensed
under both the Apache 2 and MIT licenses.ormsgpack was forked from orjson by Aviram Hassan and is now maintained by Emanuele Giaquinta (@exg), licensed
same as orjson.