Package: orjson-pydantic2
orjson-pydanticThis is a (maintained) fork of orjson that adds serialization of pydantic objects.orjson is a fast, correct JSON library for Python. Itbenchmarksas the fastest Python
library for JSON and is more correct than the standard json library or other
third-party libraries. It serializesdataclass,datetime,numpy, andUUIDinstances natively.Its features and drawbacks compared to other Python JSON libraries:serializesdataclassinstances 40-50x as fast as other librariesserializesdatetime,date, andtimeinstances to RFC 3339 format,
e.g., "1970-01-01T00:00:00+00:00"serializesnumpy.ndarrayinstances 4-12x as fast with 0.3x the memory
usage of other librariespretty prints 10x to 20x as fast as the standard libraryserializes tobytesrather thanstr, i.e., is not a drop-in replacementserializesstrwithout escaping unicode to ASCII, e.g., "好" rather than
"\\u597d"serializesfloat10x as fast and deserializes twice as fast as other
librariesserializes subclasses ofstr,int,list, anddictnatively,
requiringdefaultto specify how to serialize othersserializes arbitrary types using adefaulthookhas strict UTF-8 conformance, more correct than the standard libraryhas strict JSON conformance in not supporting Nan/Infinity/-Infinityhas an option for strict JSON conformance on 53-bit integers with default
support for 64-bitdoes not provideload()ordump()functions for reading from/writing to
file-like objectsorjson supports CPython 3.7, 3.8, 3.9, and 3.10. It distributes x86_64/amd64,
aarch64/armv8, and arm7 wheels for Linux, amd64 and aarch64 wheels for macOS,
and amd64 wheels for Windows. orjson does not support PyPy. Releases
follow semantic versioning and serializing a new object type
without an opt-in flag is considered a breaking change.orjson is licensed under both the Apache 2.0 and MIT licenses. The
repository and issue tracker isgithub.com/ijl/orjson, and patches may be
submitted there. There is aCHANGELOGavailable in the repository.UsageInstallQuickstartMigratingSerializedefaultoptionDeserializeTypesdataclassdatetimeenumfloatintnumpystruuidpydanticTestingPerformanceLatencyMemoryReproducingQuestionsPackagingLicenseUsageInstallTo install a wheel from PyPI:pipinstall--upgrade"pip>=20.3"# manylinux_x_y, universal2 wheel supportpipinstall--upgradeorjson-pydanticTo build a wheel, seepackaging.QuickstartThis is an example of serializing, with options specified, and deserializing:>>>importorjson_pydantic,datetime,numpy>>>data={"type":"job","created_at":datetime.datetime(1970,1,1),"status":"🆗","payload":numpy.array([[1,2],[3,4]]),}>>>orjson_pydantic.dumps(data,option=orjson_pydantic.OPT_NAIVE_UTC|orjson_pydantic.OPT_SERIALIZE_NUMPY)b'{"type":"job","created_at":"1970-01-01T00:00:00+00:00","status":"\xf0\x9f\x86\x97","payload":[[1,2],[3,4]]}'>>>orjson_pydantic.loads(_){'type':'job','created_at':'1970-01-01T00:00:00+00:00','status':'🆗','payload':[[1,2],[3,4]]}Migratingorjson version 3 serializes more types than version 2. Subclasses ofstr,int,dict, andlistare now serialized. This is faster and more similar
to the standard library. It can be disabled withorjson_pydantic.OPT_PASSTHROUGH_SUBCLASS.dataclasses.dataclassinstances
are now serialized by default and cannot be customized in adefaultfunction unlessoption=orjson_pydantic.OPT_PASSTHROUGH_DATACLASSis
specified.uuid.UUIDinstances are serialized by default.
For any type that is now serialized,
implementations in adefaultfunction and options enabling them can be
removed but do not need to be. There was no change in deserialization.To migrate from the standard library, the largest difference is thatorjson_pydantic.dumpsreturnsbytesandjson.dumpsreturns astr. Users withdictobjects using non-strkeys should specifyoption=orjson_pydantic.OPT_NON_STR_KEYS.sort_keysis replaced byoption=orjson_pydantic.OPT_SORT_KEYS.indentis replaced byoption=orjson_pydantic.OPT_INDENT_2and other levels of indentation are not
supported.Serializedefdumps(__obj:Any,default:Optional[Callable[[Any],Any]]=...,option:Optional[int]=...,)->bytes:...dumps()serializes Python objects to JSON.It natively serializesstr,dict,list,tuple,int,float,bool,dataclasses.dataclass,typing.TypedDict,datetime.datetime,datetime.date,datetime.time,uuid.UUID,numpy.ndarray,pydantic.BaseModel, andNoneinstances. It supports arbitrary types throughdefault. It
serializes subclasses ofstr,int,dict,list,dataclasses.dataclass, andenum.Enum. It does not serialize subclasses
oftupleto avoid serializingnamedtupleobjects as arrays. To avoid
serializing subclasses, specify the optionorjson_pydantic.OPT_PASSTHROUGH_SUBCLASS.The output is abytesobject containing UTF-8.The global interpreter lock (GIL) is held for the duration of the call.It raisesJSONEncodeErroron an unsupported type. This exception message
describes the invalid object with the error messageType is not JSON serializable: .... To fix this, specifydefault.It raisesJSONEncodeErroron astrthat contains invalid UTF-8.It raisesJSONEncodeErroron an integer that exceeds 64 bits by default or,
withOPT_STRICT_INTEGER, 53 bits.It raisesJSONEncodeErrorif adicthas a key of a type other thanstr,
unlessOPT_NON_STR_KEYSis specified.It raisesJSONEncodeErrorif the output ofdefaultrecurses to handling bydefaultmore than 254 levels deep.It raisesJSONEncodeErroron circular references.It raisesJSONEncodeErrorif atzinfoon a datetime object is
unsupported.JSONEncodeErroris a subclass ofTypeError. This is for compatibility
with the standard library.defaultTo serialize a subclass or arbitrary types, specifydefaultas a
callable that returns a supported type.defaultmay be a function,
lambda, or callable class instance. To specify that a type was not
handled bydefault, raise an exception such asTypeError.>>>importorjson_pydantic,decimal>>>defdefault(obj):ifisinstance(obj,decimal.Decimal):returnstr(obj)raiseTypeError>>>orjson_pydantic.dumps(decimal.Decimal("0.0842389659712649442845"))JSONEncodeError:TypeisnotJSONserializable:decimal.Decimal>>>orjson_pydantic.dumps(decimal.Decimal("0.0842389659712649442845"),default=default)b'"0.0842389659712649442845"'>>>orjson_pydantic.dumps({1,2},default=default)orjson_pydantic.JSONEncodeError:TypeisnotJSONserializable:setThedefaultcallable may return an object that itself
must be handled bydefaultup to 254 times before an exception
is raised.It is important thatdefaultraise an exception if a type cannot be handled.
Python otherwise implicitly returnsNone, which appears to the caller
like a legitimate value and is serialized:>>>importorjson_pydantic,json,rapidjson>>>defdefault(obj):ifisinstance(obj,decimal.Decimal):returnstr(obj)>>>orjson_pydantic.dumps({"set":{1,2}},default=default)b'{"set":null}'>>>json.dumps({"set":{1,2}},default=default)'{"set":null}'>>>rapidjson.dumps({"set":{1,2}},default=default)'{"set":null}'optionTo modify how data is serialized, specifyoption. Eachoptionis an integer
constant inorjson. To specify multiple options, mask them together, e.g.,option=orjson_pydantic.OPT_STRICT_INTEGER | orjson_pydantic.OPT_NAIVE_UTC.OPT_APPEND_NEWLINEAppend\nto the output. This is a convenience and optimization for the
pattern ofdumps(...) + "\n".bytesobjects are immutable and this
pattern copies the original contents.>>>importorjson_pydantic>>>orjson_pydantic.dumps([])b"[]">>>orjson_pydantic.dumps([],option=orjson_pydantic.OPT_APPEND_NEWLINE)b"[]\n"OPT_INDENT_2Pretty-print output with an indent of two spaces. This is equivalent toindent=2in the standard library. Pretty printing is slower and the output
larger. orjson is the fastest compared library at pretty printing and has
much less of a slowdown to pretty print than the standard library does. This
option is compatible with all other options.>>>importorjson_pydantic>>>orjson_pydantic.dumps({"a":"b","c":{"d":True},"e":[1,2]})b'{"a":"b","c":{"d":true},"e":[1,2]}'>>>orjson_pydantic.dumps({"a":"b","c":{"d":True},"e":[1,2]},option=orjson_pydantic.OPT_INDENT_2)b'{\n"a": "b",\n"c": {\n"d": true\n},\n"e": [\n1,\n2\n]\n}'If displayed, the indentation and linebreaks appear like this:{"a":"b","c":{"d":true},"e":[1,2]}This measures serializing the github.json fixture as compact (52KiB) or
pretty (64KiB):Librarycompact (ms)pretty (ms)vs. orjsonorjson0.060.071.0ujson0.180.192.8rapidjson0.22simplejson0.351.4921.4json0.361.1917.2This measures serializing the citm_catalog.json fixture, more of a worst
case due to the amount of nesting and newlines, as compact (489KiB) or
pretty (1.1MiB):Librarycompact (ms)pretty (ms)vs. orjsonorjson0.881.731.0ujson3.734.522.6rapidjson3.54simplejson11.7772.0641.6json6.7155.2231.9rapidjson is blank because it does not support pretty printing. This can be
reproduced using thepyindentscript.OPT_NAIVE_UTCSerializedatetime.datetimeobjects without atzinfoas UTC. This
has no effect ondatetime.datetimeobjects that havetzinfoset.>>>importorjson_pydantic,datetime>>>orjson_pydantic.dumps(datetime.datetime(1970,1,1,0,0,0),)b'"1970-01-01T00:00:00"'>>>orjson_pydantic.dumps(datetime.datetime(1970,1,1,0,0,0),option=orjson_pydantic.OPT_NAIVE_UTC,)b'"1970-01-01T00:00:00+00:00"'OPT_NON_STR_KEYSSerializedictkeys of type other thanstr. This allowsdictkeys
to be one ofstr,int,float,bool,None,datetime.datetime,datetime.date,datetime.time,enum.Enum, anduuid.UUID. For comparison,
the standard library serializesstr,int,float,boolorNoneby
default. orjson benchmarks as being faster at serializing non-strkeys
than other libraries. This option is slower forstrkeys than the default.>>>importorjson_pydantic,datetime,uuid>>>orjson_pydantic.dumps({uuid.UUID("7202d115-7ff3-4c81-a7c1-2a1f067b1ece"):[1,2,3]},option=orjson_pydantic.OPT_NON_STR_KEYS,)b'{"7202d115-7ff3-4c81-a7c1-2a1f067b1ece":[1,2,3]}'>>>orjson_pydantic.dumps({datetime.datetime(1970,1,1,0,0,0):[1,2,3]},option=orjson_pydantic.OPT_NON_STR_KEYS|orjson_pydantic.OPT_NAIVE_UTC,)b'{"1970-01-01T00:00:00+00:00":[1,2,3]}'These types are generally serialized how they would be as
values, e.g.,datetime.datetimeis still an RFC 3339 string and respects
options affecting it. The exception is thatintserialization does not
respectOPT_STRICT_INTEGER.This option has the risk of creating duplicate keys. This is because non-strobjects may serialize to the samestras an existing key, e.g.,{"1": true, 1: false}. The last key to be inserted to thedictwill be
serialized last and a JSON deserializer will presumably take the last
occurrence of a key (in the above,false). The first value will be lost.This option is compatible withorjson_pydantic.OPT_SORT_KEYS. If sorting is used,
note the sort is unstable and will be unpredictable for duplicate keys.>>>importorjson_pydantic,datetime>>>orjson_pydantic.dumps({"other":1,datetime.date(1970,1,5):2,datetime.date(1970,1,3):3},option=orjson_pydantic.OPT_NON_STR_KEYS|orjson_pydantic.OPT_SORT_KEYS)b'{"1970-01-03":3,"1970-01-05":2,"other":1}'This measures serializing 589KiB of JSON comprising alistof 100dictin which eachdicthas both 365 randomly-sortedintkeys representing epoch
timestamps as well as onestrkey and the value for each key is a
single integer. In "str keys", the keys were converted tostrbefore
serialization, and orjson still specifesoption=orjson_pydantic.OPT_NON_STR_KEYS(which is always somewhat slower).Librarystr keys (ms)int keys (ms)int keys sorted (ms)orjson1.532.164.29ujson3.075.65rapidjson4.29simplejson11.2414.5021.86json7.178.49ujson is blank for sorting because it segfaults. json is blank because it
raisesTypeErroron attempting to sort before converting all keys tostr.
rapidjson is blank because it does not support non-strkeys. This can
be reproduced using thepynonstrscript.OPT_OMIT_MICROSECONDSDo not serialize themicrosecondfield ondatetime.datetimeanddatetime.timeinstances.>>>importorjson_pydantic,datetime>>>orjson_pydantic.dumps(datetime.datetime(1970,1,1,0,0,0,1),)b'"1970-01-01T00:00:00.000001"'>>>orjson_pydantic.dumps(datetime.datetime(1970,1,1,0,0,0,1),option=orjson_pydantic.OPT_OMIT_MICROSECONDS,)b'"1970-01-01T00:00:00"'OPT_PASSTHROUGH_DATACLASSPassthroughdataclasses.dataclassinstances todefault. This allows
customizing their output but is much slower.>>>importorjson_pydantic,dataclasses>>>@dataclasses.dataclassclassUser:id:strname:strpassword:strdefdefault(obj):ifisinstance(obj,User):return{"id":obj.id,"name":obj.name}raiseTypeError>>>orjson_pydantic.dumps(User("3b1","asd","zxc"))b'{"id":"3b1","name":"asd","password":"zxc"}'>>>orjson_pydantic.dumps(User("3b1","asd","zxc"),option=orjson_pydantic.OPT_PASSTHROUGH_DATACLASS)TypeError:TypeisnotJSONserializable:User>>>orjson_pydantic.dumps(User("3b1","asd","zxc"),option=orjson_pydantic.OPT_PASSTHROUGH_DATACLASS,default=default,)b'{"id":"3b1","name":"asd"}'OPT_PASSTHROUGH_DATETIMEPassthroughdatetime.datetime,datetime.date, anddatetime.timeinstances
todefault. This allows serializing datetimes to a custom format, e.g.,
HTTP dates:>>>importorjson_pydantic,datetime>>>defdefault(obj):ifisinstance(obj,datetime.datetime):returnobj.strftime("%a,%d%b %Y %H:%M:%S GMT")raiseTypeError>>>orjson_pydantic.dumps({"created_at":datetime.datetime(1970,1,1)})b'{"created_at":"1970-01-01T00:00:00"}'>>>orjson_pydantic.dumps({"created_at":datetime.datetime(1970,1,1)},option=orjson_pydantic.OPT_PASSTHROUGH_DATETIME)TypeError:TypeisnotJSONserializable:datetime.datetime>>>orjson_pydantic.dumps({"created_at":datetime.datetime(1970,1,1)},option=orjson_pydantic.OPT_PASSTHROUGH_DATETIME,default=default,)b'{"created_at":"Thu, 01 Jan 1970 00:00:00 GMT"}'This does not affect datetimes indictkeys if using OPT_NON_STR_KEYS.OPT_PASSTHROUGH_SUBCLASSPassthrough subclasses of builtin types todefault.>>>importorjson_pydantic>>>classSecret(str):passdefdefault(obj):ifisinstance(obj,Secret):return"******"raiseTypeError>>>orjson_pydantic.dumps(Secret("zxc"))b'"zxc"'>>>orjson_pydantic.dumps(Secret("zxc"),option=orjson_pydantic.OPT_PASSTHROUGH_SUBCLASS)TypeError:TypeisnotJSONserializable:Secret>>>orjson_pydantic.dumps(Secret("zxc"),option=orjson_pydantic.OPT_PASSTHROUGH_SUBCLASS,default=default)b'"******"'This does not affect serializing subclasses asdictkeys if using
OPT_NON_STR_KEYS.OPT_SERIALIZE_DATACLASSThis is deprecated and has no effect in version 3. In version 2 this was
required to serializedataclasses.dataclassinstances. For more, seedataclass.OPT_SERIALIZE_NUMPYSerializenumpy.ndarrayinstances. For more, seenumpy.OPT_SERIALIZE_UUIDThis is deprecated and has no effect in version 3. In version 2 this was
required to serializeuuid.UUIDinstances. For more, seeUUID.OPT_SORT_KEYSSerializedictkeys in sorted order. The default is to serialize in an
unspecified order. This is equivalent tosort_keys=Truein the standard
library.This can be used to ensure the order is deterministic for hashing or tests.
It has a substantial performance penalty and is not recommended in general.>>>importorjson_pydantic>>>orjson_pydantic.dumps({"b":1,"c":2,"a":3})b'{"b":1,"c":2,"a":3}'>>>orjson_pydantic.dumps({"b":1,"c":2,"a":3},option=orjson_pydantic.OPT_SORT_KEYS)b'{"a":3,"b":1,"c":2}'This measures serializing the twitter.json fixture unsorted and sorted:Libraryunsorted (ms)sorted (ms)vs. orjsonorjson0.50.921ujson1.612.482.7rapidjson2.172.893.2simplejson3.565.135.6json3.594.595The benchmark can be reproduced using thepysortscript.The sorting is not collation/locale-aware:>>>importorjson_pydantic>>>orjson_pydantic.dumps({"a":1,"ä":2,"A":3},option=orjson_pydantic.OPT_SORT_KEYS)b'{"A":3,"a":1,"\xc3\xa4":2}'This is the same sorting behavior as the standard library, rapidjson,
simplejson, and ujson.dataclassalso serialize as maps but this has no effect on them.OPT_STRICT_INTEGEREnforce 53-bit limit on integers. The limit is otherwise 64 bits, the same as
the Python standard library. For more, seeint.OPT_UTC_ZSerialize a UTC timezone ondatetime.datetimeinstances asZinstead
of+00:00.>>>importorjson_pydantic,datetime,zoneinfo>>>orjson_pydantic.dumps(datetime.datetime(1970,1,1,0,0,0,tzinfo=zoneinfo.ZoneInfo("UTC")),)b'"1970-01-01T00:00:00+00:00"'>>>orjson_pydantic.dumps(datetime.datetime(1970,1,1,0,0,0,tzinfo=zoneinfo.ZoneInfo("UTC")),option=orjson_pydantic.OPT_UTC_Z)b'"1970-01-01T00:00:00Z"'OPT_SERIALIZE_PYDANTICSerialize apydantic.BaseModelinstance. For more seepydantic.Deserializedefloads(__obj:Union[bytes,bytearray,memoryview,str])->Any:...loads()deserializes JSON to Python objects. It deserializes todict,list,int,float,str,bool, andNoneobjects.bytes,bytearray,memoryview, andstrinput are accepted. If the input
exists as amemoryview,bytearray, orbytesobject, it is recommended to
pass these directly rather than creating an unnecessarystrobject. This has
lower memory usage and lower latency.The input must be valid UTF-8.orjson maintains a cache of map keys for the duration of the process. This
causes a net reduction in memory usage by avoiding duplicate strings. The
keys must be at most 64 bytes to be cached and 512 entries are stored.The global interpreter lock (GIL) is held for the duration of the call.It raisesJSONDecodeErrorif given an invalid type or invalid
JSON. This includes if the input containsNaN,Infinity, or-Infinity,
which the standard library allows, but is not valid JSON.JSONDecodeErroris a subclass ofjson.JSONDecodeErrorandValueError.
This is for compatibility with the standard library.Typesdataclassorjson serializes instances ofdataclasses.dataclassnatively. It serializes
instances 40-50x as fast as other libraries and avoids a severe slowdown seen
in other libraries compared to serializingdict.It is supported to pass all variants of dataclasses, including dataclasses
using__slots__, frozen dataclasses, those with optional or default
attributes, and subclasses. There is a performance benefit to not
using__slots__.Librarydict (ms)dataclass (ms)vs. orjsonorjson1.401.601ujsonrapidjson3.6468.4842simplejson14.2192.1857json13.2894.9059This measures serializing 555KiB of JSON, orjson natively and other libraries
usingdefaultto serialize the output ofdataclasses.asdict(). This can be
reproduced using thepydataclassscript.Dataclasses are serialized as maps, with every attribute serialized and in
the order given on class definition:>>>importdataclasses,orjson,typing@dataclasses.dataclassclassMember:id:intactive:bool=dataclasses.field(default=False)@dataclasses.dataclassclassObject:id:intname:strmembers:typing.List[Member]>>>orjson_pydantic.dumps(Object(1,"a",[Member(1,True),Member(2)]))b'{"id":1,"name":"a","members":[{"id":1,"active":true},{"id":2,"active":false}]}'Users may wish to control how dataclass instances are serialized, e.g.,
to not serialize an attribute or to change the name of an
attribute when serialized. orjson may implement support using the
metadata mapping onfieldattributes,
e.g.,field(metadata={"json_serialize": False}), if use cases are clear.datetimeorjson serializesdatetime.datetimeobjects toRFC 3339format,
e.g., "1970-01-01T00:00:00+00:00". This is a subset of ISO 8601 and is
compatible withisoformat()in the standard library.>>>importorjson_pydantic,datetime,zoneinfo>>>orjson_pydantic.dumps(datetime.datetime(2018,12,1,2,3,4,9,tzinfo=zoneinfo.ZoneInfo("Australia/Adelaide")))b'"2018-12-01T02:03:04.000009+10:30"'>>>orjson_pydantic.dumps(datetime.datetime(2100,9,1,21,55,2).replace(tzinfo=zoneinfo.ZoneInfo("UTC")))b'"2100-09-01T21:55:02+00:00"'>>>orjson_pydantic.dumps(datetime.datetime(2100,9,1,21,55,2))b'"2100-09-01T21:55:02"'datetime.datetimesupports instances with atzinfothat isNone,datetime.timezone.utc, a timezone instance from the python3.9+zoneinfomodule, or a timezone instance from the third-partypendulum,pytz, ordateutil/arrowlibraries.It is fastest to use the standard library'szoneinfo.ZoneInfofor timezones.datetime.timeobjects must not have atzinfo.>>>importorjson_pydantic,datetime>>>orjson_pydantic.dumps(datetime.time(12,0,15,290))b'"12:00:15.000290"'datetime.dateobjects will always serialize.>>>importorjson_pydantic,datetime>>>orjson_pydantic.dumps(datetime.date(1900,1,2))b'"1900-01-02"'Errors withtzinforesult inJSONEncodeErrorbeing raised.It is faster to have orjson serialize datetime objects than to do so
before callingdumps(). If using an unsupported type such aspendulum.datetime, usedefault.To disable serialization ofdatetimeobjects specify the optionorjson_pydantic.OPT_PASSTHROUGH_DATETIME.To use "Z" suffix instead of "+00:00" to indicate UTC ("Zulu") time, use the optionorjson_pydantic.OPT_UTC_Z.To assume datetimes without timezone are UTC, se the optionorjson_pydantic.OPT_NAIVE_UTC.enumorjson serializes enums natively. Options apply to their values.>>>importenum,datetime,orjson>>>classDatetimeEnum(enum.Enum):EPOCH=datetime.datetime(1970,1,1,0,0,0)>>>orjson_pydantic.dumps(DatetimeEnum.EPOCH)b'"1970-01-01T00:00:00"'>>>orjson_pydantic.dumps(DatetimeEnum.EPOCH,option=orjson_pydantic.OPT_NAIVE_UTC)b'"1970-01-01T00:00:00+00:00"'Enums with members that are not supported types can be serialized usingdefault:>>>importenum,orjson>>>classCustom:def__init__(self,val):self.val=valdefdefault(obj):ifisinstance(obj,Custom):returnobj.valraiseTypeErrorclassCustomEnum(enum.Enum):ONE=Custom(1)>>>orjson_pydantic.dumps(CustomEnum.ONE,default=default)b'1'floatorjson serializes and deserializes double precision floats with no loss of
precision and consistent rounding. The same behavior is observed in rapidjson,
simplejson, and json. ujson 1.35 was inaccurate in both serialization and
deserialization, i.e., it modifies the data, and the recent 2.0 release is
accurate.orjson_pydantic.dumps()serializes Nan, Infinity, and -Infinity, which are not
compliant JSON, asnull:>>>importorjson_pydantic,ujson,rapidjson,json>>>orjson_pydantic.dumps([float("NaN"),float("Infinity"),float("-Infinity")])b'[null,null,null]'>>>ujson.dumps([float("NaN"),float("Infinity"),float("-Infinity")])OverflowError:InvalidInfvaluewhenencodingdouble>>>rapidjson.dumps([float("NaN"),float("Infinity"),float("-Infinity")])'[NaN,Infinity,-Infinity]'>>>json.dumps([float("NaN"),float("Infinity"),float("-Infinity")])'[NaN, Infinity, -Infinity]'intorjson serializes and deserializes 64-bit integers by default. The range
supported is a signed 64-bit integer's minimum (-9223372036854775807) to
an unsigned 64-bit integer's maximum (18446744073709551615). This
is widely compatible, but there are implementations
that only support 53-bits for integers, e.g.,
web browsers. For those implementations,dumps()can be configured to
raise aJSONEncodeErroron values exceeding the 53-bit range.>>>importorjson_pydantic>>>orjson_pydantic.dumps(9007199254740992)b'9007199254740992'>>>orjson_pydantic.dumps(9007199254740992,option=orjson_pydantic.OPT_STRICT_INTEGER)JSONEncodeError:Integerexceeds53-bitrange>>>orjson_pydantic.dumps(-9007199254740992,option=orjson_pydantic.OPT_STRICT_INTEGER)JSONEncodeError:Integerexceeds53-bitrangenumpyorjson natively serializesnumpy.ndarrayand individualnumpy.float64,numpy.float32,numpy.int64,numpy.int32,numpy.int8,numpy.uint64,numpy.uint32,numpy.uint8,numpy.uintp, ornumpy.intp, andnumpy.datetime64instances.orjson is faster than all compared libraries at serializing
numpy instances. Serializing numpy data requires specifyingoption=orjson_pydantic.OPT_SERIALIZE_NUMPY.>>>importorjson_pydantic,numpy>>>orjson_pydantic.dumps(numpy.array([[1,2,3],[4,5,6]]),option=orjson_pydantic.OPT_SERIALIZE_NUMPY,)b'[[1,2,3],[4,5,6]]'The array must be a contiguous C array (C_CONTIGUOUS) and one of the
supported datatypes.numpy.datetime64instances are serialized as RFC 3339 strings and
datetime options affect them.>>>importorjson_pydantic,numpy>>>orjson_pydantic.dumps(numpy.datetime64("2021-01-01T00:00:00.172"),option=orjson_pydantic.OPT_SERIALIZE_NUMPY,)b'"2021-01-01T00:00:00.172000"'>>>orjson_pydantic.dumps(numpy.datetime64("2021-01-01T00:00:00.172"),option=(orjson_pydantic.OPT_SERIALIZE_NUMPY|orjson_pydantic.OPT_NAIVE_UTC|orjson_pydantic.OPT_OMIT_MICROSECONDS),)b'"2021-01-01T00:00:00+00:00"'If an array is not a contiguous C array, contains an supported datatype,
or contains anumpy.datetime64using an unsupported representation
(e.g., picoseconds), orjson falls through todefault. Indefault,obj.tolist()can be specified. If an array is malformed, which
is not expected,orjson_pydantic.JSONEncodeErroris raised.This measures serializing 92MiB of JSON from annumpy.ndarraywith
dimensions of(50000, 100)andnumpy.float64values:LibraryLatency (ms)RSS diff (MiB)vs. orjsonorjson194991.0ujsonrapidjson3,04830915.7simplejson3,02329715.6json3,13329716.1This measures serializing 100MiB of JSON from annumpy.ndarraywith
dimensions of(100000, 100)andnumpy.int32values:LibraryLatency (ms)RSS diff (MiB)vs. orjsonorjson1781151.0ujsonrapidjson1,5125518.5simplejson1,6065049.0json1,5065038.4This measures serializing 105MiB of JSON from annumpy.ndarraywith
dimensions of(100000, 200)andnumpy.boolvalues:LibraryLatency (ms)RSS diff (MiB)vs. orjsonorjson1571201.0ujsonrapidjson7103274.5simplejson9313985.9json9964006.3In these benchmarks, orjson serializes natively, ujson is blank because it
does not support adefaultparameter, and the other libraries serializendarray.tolist()viadefault. The RSS column measures peak memory
usage during serialization. This can be reproduced using thepynumpyscript.orjson does not have an installation or compilation dependency on numpy. The
implementation is independent, readingnumpy.ndarrayusingPyArrayInterface.strorjson is strict about UTF-8 conformance. This is stricter than the standard
library's json module, which will serialize and deserialize UTF-16 surrogates,
e.g., "\ud800", that are invalid UTF-8.Iforjson_pydantic.dumps()is given astrthat does not contain valid UTF-8,orjson_pydantic.JSONEncodeErroris raised. Ifloads()receives invalid UTF-8,orjson_pydantic.JSONDecodeErroris raised.orjson and rapidjson are the only compared JSON libraries to consistently
error on bad input.>>>importorjson_pydantic,ujson,rapidjson,json>>>orjson_pydantic.dumps('\ud800')JSONEncodeError:strisnotvalidUTF-8:surrogatesnotallowed>>>ujson.dumps('\ud800')UnicodeEncodeError:'utf-8'codec...>>>rapidjson.dumps('\ud800')UnicodeEncodeError:'utf-8'codec...>>>json.dumps('\ud800')'"\\ud800"'>>>orjson_pydantic.loads('"\\ud800"')JSONDecodeError:unexpectedendofhexescapeatline1column8:line1column1(char0)>>>ujson.loads('"\\ud800"')''>>>rapidjson.loads('"\\ud800"')ValueError:Parseerroratoffset1:Thesurrogatepairinstringisinvalid.>>>json.loads('"\\ud800"')'\ud800'To make a best effort at deserializing bad input, first decodebytesusing
thereplaceorlossyargument forerrors:>>>importorjson_pydantic>>>orjson_pydantic.loads(b'"\xed\xa0\x80"')JSONDecodeError:strisnotvalidUTF-8:surrogatesnotallowed>>>orjson_pydantic.loads(b'"\xed\xa0\x80"'.decode("utf-8","replace"))'���'uuidorjson serializesuuid.UUIDinstances toRFC 4122format, e.g.,
"f81d4fae-7dec-11d0-a765-00a0c91e6bf6".>>>importorjson_pydantic,uuid>>>orjson_pydantic.dumps(uuid.UUID('f81d4fae-7dec-11d0-a765-00a0c91e6bf6'))b'"f81d4fae-7dec-11d0-a765-00a0c91e6bf6"'>>>orjson_pydantic.dumps(uuid.uuid5(uuid.NAMESPACE_DNS,"python.org"))b'"886313e1-3b8a-5372-9b90-0c9aee199e5d"'pydanticorjson serializespydantic.BaseModelinstances based on the existence of__fields__attribute.:warning:The serialization doesn't behave likepydantic.BaseModel.json():It doesn't respect any of the Config attributes.It doesn't have any of the additional features of Pydantic (json_encoder, exclusions, inclusions, etc).TestingThe library has comprehensive tests. There are tests against fixtures in theJSONTestSuiteandnativejson-benchmarkrepositories. It is tested to not crash against theBig List of Naughty Strings.
It is tested to not leak memory. It is tested to not crash
against and not accept invalid UTF-8. There are integration tests
exercising the library's use in web servers (gunicorn using multiprocess/forked
workers) and when
multithreaded. It also uses some tests from the ultrajson library.orjson is the most correct of the compared libraries. This graph shows how each
library handles a combined 342 JSON fixtures from theJSONTestSuiteandnativejson-benchmarktests:LibraryInvalid JSON documents not rejectedValid JSON documents not deserializedorjson00ujson380rapidjson60simplejson130json170This shows that all libraries deserialize valid JSON but only orjson
correctly rejects the given invalid JSON fixtures. Errors are largely due to
accepting invalid strings and numbers.The graph above can be reproduced using thepycorrectnessscript.PerformanceSerialization and deserialization performance of orjson is better than
ultrajson, rapidjson, simplejson, or json. The benchmarks are done on
fixtures of real data:twitter.json, 631.5KiB, results of a search on Twitter for "一", containing
CJK strings, dictionaries of strings and arrays of dictionaries, indented.github.json, 55.8KiB, a GitHub activity feed, containing dictionaries of
strings and arrays of dictionaries, not indented.citm_catalog.json, 1.7MiB, concert data, containing nested dictionaries of
strings and arrays of integers, indented.canada.json, 2.2MiB, coordinates of the Canadian border in GeoJSON
format, containing floats and arrays, indented.Latencytwitter.json serializationLibraryMedian latency (milliseconds)Operations per secondRelative (latency)orjson0.591698.81ujson2.14464.33.64rapidjson2.39418.54.06simplejson3.15316.95.36json3.56281.26.06twitter.json deserializationLibraryMedian latency (milliseconds)Operations per secondRelative (latency)orjson2.28439.31ujson2.89345.91.27rapidjson3.85259.61.69simplejson3.66272.11.61json4.05246.71.78github.json serializationLibraryMedian latency (milliseconds)Operations per secondRelative (latency)orjson0.0715265.21ujson0.224556.73.35rapidjson0.263808.94.02simplejson0.372690.45.68json0.352847.85.36github.json deserializationLibraryMedian latency (milliseconds)Operations per secondRelative (latency)orjson0.185610.11ujson0.283540.71.58rapidjson0.333031.51.85simplejson0.293385.61.65json0.293402.11.65citm_catalog.json serializationLibraryMedian latency (milliseconds)Operations per secondRelative (latency)orjson0.991008.51ujson3.69270.73.72rapidjson3.55281.43.58simplejson11.7685.111.85json6.89145.16.95citm_catalog.json deserializationLibraryMedian latency (milliseconds)Operations per secondRelative (latency)orjson4.53220.51ujson5.67176.51.25rapidjson7.51133.31.66simplejson7.54132.71.66json7.8128.21.72canada.json serializationLibraryMedian latency (milliseconds)Operations per secondRelative (latency)orjson4.72198.91ujson17.7656.33.77rapidjson61.8316.213.11simplejson80.612.417.09json52.3818.811.11canada.json deserializationLibraryMedian latency (milliseconds)Operations per secondRelative (latency)orjson10.2897.41ujson16.4960.51.6rapidjson37.9226.43.69simplejson37.726.53.67json37.8727.63.68Memoryorjson's memory usage when deserializing is similar to or lower than
the standard library and other third-party libraries.This measures, in the first column, RSS after importing a library and reading
the fixture, and in the second column, increases in RSS after repeatedly
callingloads()on the fixture.twitter.jsonLibraryimport, read() RSS (MiB)loads() increase in RSS (MiB)orjson13.52.5ujson144.1rapidjson14.76.5simplejson13.22.5json12.92.3github.jsonLibraryimport, read() RSS (MiB)loads() increase in RSS (MiB)orjson13.10.3ujson13.50.3rapidjson140.7simplejson12.60.3json12.30.1citm_catalog.jsonLibraryimport, read() RSS (MiB)loads() increase in RSS (MiB)orjson14.67.9ujson15.111.1rapidjson15.836simplejson14.327.4json1427.2canada.jsonLibraryimport, read() RSS (MiB)loads() increase in RSS (MiB)orjson17.115.7ujson17.617.4rapidjson18.317.9simplejson16.919.6json16.519.4ReproducingThe above was measured using Python 3.8.3 on Linux (x86_64) with
orjson 3.3.0, ujson 3.0.0, python-rapidson 0.9.1, and simplejson 3.17.2.The latency results can be reproduced using thepybenchandgraphscripts. The memory results can be reproduced using thepymemscript.QuestionsWhy can't I install it from PyPI?Probablypipneeds to be upgraded to version 20.3 or later to support
the latest manylinux_x_y or universal2 wheel formats.Will it deserialize to dataclasses, UUIDs, decimals, etc or support object_hook?No. This requires a schema specifying what types are expected and how to
handle errors etc. This is addressed by data validation libraries a
level above this.Will it serialize tostr?No.bytesis the correct type for a serialized blob.Will it support PyPy?If someone implements it well.PackagingTo package orjson requiresRustand thematurinbuild tool.This is an example for x86_64 on the Rust nightly channel:exportRUSTFLAGS="-C target-cpu=k8"maturinbuild--release--strip--cargo-extra-args="--features=unstable-simd"To build on the stable channel, do not specify--features=unstable-simd.The project's own CI tests againstnightly-2022-02-13and stable 1.54. It
is prudent to pin the nightly version because that channel can introduce
breaking changes.orjson is tested for amd64, aarch64, and arm7 on Linux. It is tested for
amd64 on macOS and ships an aarch64 wheel also supporting aarch64. For Windows
is is tested on amd64.There are no runtime dependencies other than libc.orjson's tests are included in the source distribution on PyPI. The
requirements to run the tests are specified intest/requirements.txt. The
tests should be run as part of the build. It can be run withpytest -q test.Licenseorjson was written by ijl <ijl@mailbox.org>, copyright 2018 - 2022, licensed
under both the Apache 2 and MIT licenses.