Package: ducktools-jsonkit
ducktools: jsonkitDefault functions and default function generators to make JSON serialization
with the python standard library easier.MotivationThe documentation for the JSON module in the Python standard library (as of 3.11.1)
instructs the user to subclassJSONEncoderif you wish to serialize objects
that are not natively serializable. This is unnecessary. The serialization methodsdumpanddumpsprovide adefaultargument which achieves the same result
without needing to subclass.This module provides some functions and function generators that can be used as
values for thisdefaultargument to serialize some standard classes and custom
classes.UnlikeJSONEncodersubclasses,defaultfunctions are also supported as arguments
in some other libraries that implement their own JSON serialization such asorjsonorrapidjson.If you're using theencodemethod on aJSONEncoderclass directly you can provide
thedefaultfunction as an argument toJSONEncoderin the same way as todumps.
Ifdumpsis being called multiple times with a default, creating aJSONEncoderinstance
and calling theencodemethod directly will be faster asdumpscreates a new instance
each time it is called.Generated methods for field and dataclass serializationThe serializers for dataclasses and fields exist for cases where you need to
encode a large number of instances of the same dataclass (or other objects
with the same set of fields).While calling exec usually takes longer than a single naive serialization,
the resulting static functions are faster than their dynamic equivalents.
This is noticeable when serializing a large number of instances of the same class.
As the results are cached, the cost ofexecis only paid the first time.This is actually similar to the methodcattrsuses, although that module useseval(compile(...))to provide a 'fake' source
file for inspections. If you're already usingattrsyou should usecattrsfor serialization.MethodsThemethod_defaultfunction is provided to create adefaultfunction to pass
to json.dumps if you have classes with a method that is intended to prepare
them for serialization.Example:importjsonfromducktools.jsonkitimportmethod_defaultclassExample:def__init__(self,x,y):self.x,self.y=x,ydefasdict(self):return{'x':self.x,'y':self.y}example=Example("hello","world")# dumpsdata=json.dumps(example,default=method_default('asdict'))# encoderencoder=json.JSONEncoder(default=method_default('asdict'))encoder_data=encoder.encode(example)print(encoder_data==data)print(data)Output:True
{"x": "hello", "y": "world"}Merge defaultsThemerge_defaultsfunction combines multipledefaultfunctions into one.importjsonfrompathlibimportPathfromducktools.jsonkitimportmerge_defaultsdefpath_default(pth):ifisinstance(pth,Path):returnstr(pth)else:raiseTypeError()defset_default(s):ifisinstance(s,set):returnlist(s)else:raiseTypeError()new_default=merge_defaults(path_default,set_default)data={"Path":Path("usr/bin/python"),"versions":{'3.9','3.10','3.11'}}print(json.dumps(data,default=new_default))Output:{"Path": "usr/bin/python", "versions": ["3.11", "3.9", "3.10"]}RegisterThe module provides aJSONRegisterclass that provides methods
to add classes and their serialization methods to the register, these are
then used by providing theJSONRegisterinstancedefaulttojson.dumps.Example:fromducktools.jsonkitimportJSONRegisterimportjsonimportdataclassesfrompathlibimportPathfromdecimalimportDecimalregister=JSONRegister()@dataclasses.dataclassclassDemo:id:intname:strlocation:Pathnumbers:list[Decimal]@register.register_methoddefto_json(self):return{'id':self.id,'name':self.name,'location':self.location,'numbers':self.numbers,}register.register(Path,str)@register.register_function(Decimal)defunstructure_decimal(val):return{'cls':'Decimal','value':str(val)}numbers=[Decimal(f"{i}")/Decimal('1000')foriinrange(1,3)]pth=Path("usr/bin/python")demo=Demo(id=42,name="Demonstration Class",location=pth,numbers=numbers)print(json.dumps(demo,default=register.default,indent=2))Output:{
  "id": 42,
  "name": "Demonstration Class",
  "location": "usr/bin/python",
  "numbers": [
    {
      "cls": "Decimal",
      "value": "0.001"
    },
    {
      "cls": "Decimal",
      "value": "0.002"
    }
  ]
}FieldsThefield_defaultfunction is intended to be used to handle creating default for
objects where the serialization format is{name: item.name, ...}. This is used
for the dataclasses default provided.For example this could be used to serialize classes based on the field names defined
in__slots__(will not work on slots defined by a consumed iterable).importjsonfromfunctoolsimportlru_cachefromducktools.jsonkitimportfield_default@lru_cachedefslot_defaultmaker(cls):try:slots=cls.__slots__exceptAttributeError:raiseTypeError(f'Object of type{cls.__name__}is not JSON serializable')slot_tuple=tuple(slots)returnfield_default(slot_tuple)defslot_default(o):func=slot_defaultmaker(type(o))returnfunc(o)classSlotExample:__slots__=['x','y']def__init__(self,x,y):self.x,self.y=x,yexample=SlotExample("Hello","World")data=json.dumps(example,default=slot_default)print(data)Result:{"x": "Hello", "y": "World"}DataclassesDataclasses itself provides its ownasdictfunction, but unfortunately this
includes additional logic for deepcopying objects and performing recursive
serialization.For the purpose of basic serialization of dataclasses a basic non-recursive
default method will be faster thanasdict.Note: Theasdictmethod has been improved in Python 3.12+ so the difference
is less significant.
Seehttps://github.com/python/cpython/issues/103000.fromdataclassesimportis_dataclass,fieldsdefsimple_dc_default(o):ifis_dataclass(o)andnotisinstance(o,type):return{f.name:getattr(o,f.name)forfinfields(o)}else:raiseTypeError(f'Object of type{type(o).__name__}is not JSON serializable')Using: performance/dataclass_serializers_compared.pyComparingasdict,simple_dc_default(simple) anddataclass_default(cached).Python 3.11MethodTime /sTime /cachejson asdict4.4923.9json simple2.4002.1json cached1.1451.0Python 3.12MethodTime /sTime /cachejson asdict1.9912.2json simple1.9102.1json cached0.8961.0