Package: dataclass-mage
Full documentation is available atRead The Docs. (Installation)This library provides a set of simple, yet elegantwizardingtools for
interacting with the Pythondataclassesmodule.The primary use is as a fast serialization framework that enables dataclass instances to
be converted to/from JSON; this works well in particular with anested dataclassmodel.Behold, the power of the Dataclass Mage:>>> from __future__ import annotations
>>> from dataclasses import dataclass, field
>>> from dataclass_mage import JSONWizard
...
>>> @dataclass
... class MyClass(JSONWizard):
...     my_str: str | None
...     is_active_tuple: tuple[bool, ...]
...     list_of_int: list[int] = field(default_factory=list)
...
>>> string = """
... {
...   "my_str": 20,
...   "ListOfInt": ["1", "2", 3],
...   "isActiveTuple": ["true", false, 1]
... }
... """
...
>>> instance = MyClass.from_json(string)
>>> instance
MyClass(my_str='20', is_active_tuple=(True, False, True), list_of_int=[1, 2, 3])
>>> instance.to_json()
'{"myStr": "20", "isActiveTuple": [true, false, true], "listOfInt": [1, 2, 3]}'
>>> instance == MyClass.from_dict(instance.to_dict())
True—ContentsInstallationFeaturesWizard MixinsSupported TypesUsage and ExamplesJSON MarshallingNo Inheritance NeededCustom Key MappingsExtending fromMetaDate and Time with Custom PatternsDataclasses inUnionTypesSerialization OptionsField PropertiesContributingTODOsCreditsInstallationThe Dataclass Mage library is availableon PyPI, and can be installed withpip:$pipinstalldataclass-mageThedataclass-magelibrary officially supportsPython 3.9or higher.FeaturesHere are the supported features thatdataclass-magecurrently provides:JSON/YAML (de)serialization: marshal dataclasses to/from JSON, YAML, and Pythondictobjects.Field properties: support for using properties with default
values in dataclass instances.JSON to Dataclass generation: construct a dataclass schema with a JSON file
or string input.Wizard MixinsIn addition to theJSONWizard, here are a few extraMixinclasses that might prove quite convenient to use.JSONListWizard– ExtendsJSONWizardto returnContainer– instead oflist– objects where possible.JSONFileWizard– Makes it easier to convert dataclass instances from/to JSON files on a local drive.YAMLWizard– Provides support to convert dataclass instances to/from YAML, using the defaultPyYAMLparser.Supported TypesThe Dataclass Mage library provides inherent support for standard Python collections
such aslist,dictandset, as well as most Generics from the typing
module, such asUnionandAny. Other commonly used types such asEnum,defaultdict, and date and time objects such asdatetimeare also natively
supported.For a complete list of the supported Python types, including info on the
load/dump process for special types, check out theSupported Typessection
in the docs.Usage and ExamplesUsing the built-in JSON marshalling support for dataclasses:Note: The following example should work inPython 3.7+with the included__future__import.from__future__importannotations# This can be removed in Python 3.10+fromdataclassesimportdataclass,fieldfromdatetimeimportdatefromenumimportEnumfromdataclass_mageimportJSONWizard@dataclassclassData(JSONWizard):class_(JSONWizard.Meta):# Sets the target key transform to use for serialization;# defaults to `camelCase` if not specified.key_transform_with_dump='LISP'a_sample_bool:boolvalues:list[Inner]=field(default_factory=list)@dataclassclassInner:vehicle:Car|Nonemy_dates:dict[int,date]classCar(Enum):SEDAN='BMW Coupe'SUV='Toyota 4Runner'JEEP='Jeep Cherokee'defmain():my_dict={'values':[{'vehicle':'Toyota 4Runner','My-Dates':{'123':'2023-01-31'}},{'vehicle':None,'my_dates':{}}],'aSampleBool':'TRUE'}# De-serialize (a JSON string or dictionary data) into a `Data` instance.data=Data.from_dict(my_dict)print(repr(data))# > Data(a_sample_bool=True, values=[Inner(vehicle=<Car.SUV: 'Toyota 4Runner'>, ...)])# assert enums values are as expectedassertdata.values[0].vehicleisCar.SUVprint(data.to_json(indent=2))# {#   "a-sample-bool": true,#   "values": [#     {#       "vehicle": "Toyota 4Runner",#       "my-dates": {#         "123": "2023-01-31"#       },#   ...# Trueassertdata==data.from_json(data.to_json())if__name__=='__main__':main()… and with theproperty_wizard, which provides support forfield propertieswith default values in dataclasses:from__future__importannotations# This can be removed in Python 3.10+fromdataclassesimportdataclass,fieldfromtyping_extensionsimportAnnotatedfromdataclass_mageimportproperty_wizard@dataclassclassVehicle(metaclass=property_wizard):# Note: The example below uses the default value from the `field` extra in# the `Annotated` definition; if `wheels` were annotated as `int | str`,# it would default to 0, because `int` appears as the first type argument.## Any right-hand value assigned to `wheels` is ignored as it is simply# re-declared by the property; here it is simply omitted for brevity.wheels:Annotated[int|str,field(default=4)]# This is a shorthand version of the above; here an IDE suggests# `_wheels` as a keyword argument to the constructor method, though# it will actually be named as `wheels`.# _wheels: int | str = 4@propertydefwheels(self)->int:returnself._wheels@wheels.setterdefwheels(self,wheels:int|str):self._wheels=int(wheels)if__name__=='__main__':v=Vehicle()print(v)# prints:#   Vehicle(wheels=4)v=Vehicle(wheels=3)print(v)v=Vehicle('6')print(v)assertv.wheels==6,'The constructor should use our setter method'# Confirm that we go through our setter methodv.wheels='123'assertv.wheels==123… or generate a dataclass schema for JSON input, via thewiz-clitool:$echo'{"myFloat": "1.23", "Products": [{"created_at": "2021-11-17"}]}'|wizgs-my_file# Contents of my_file.pyfromdataclassesimportdataclassfromdatetimeimportdatefromtypingimportList,Unionfromdataclass_mageimportJSONWizard@dataclassclassData(JSONWizard):"""
    Data dataclass

    """my_float:Union[float,str]products:List['Product']@dataclassclassProduct:"""
    Product dataclass

    """created_at:dateJSON MarshallingJSONSerializable(aliased toJSONWizard) is aMixinclass which
provides the following helper methods that are useful for serializing (and loading)
a dataclass instance to/from JSON, as defined by theAbstractJSONWizardinterface.MethodExampleDescriptionfrom_jsonitem = Product.from_json(string)Converts a JSON string to an instance of the
dataclass, or a list of the dataclass instances.from_listlist_of_item = Product.from_list(l)Converts a Pythonlistobject to a list of the
dataclass instances.from_dictitem = Product.from_dict(d)Converts a Pythondictobject to an instance
of the dataclass.to_dictd = item.to_dict()Converts the dataclass instance to a Pythondictobject that is JSON serializable.to_jsonstring = item.to_json()Converts the dataclass instance to a JSON string
representation.list_to_jsonstring = Product.list_to_json(list_of_item)Converts a list of dataclass instances to a JSON string
representation.Additionally, it adds a default__str__method to subclasses, which will
pretty print the JSON representation of an object; this is quite useful for
debugging purposes. Whenever you invokeprint(obj)orstr(obj), for
example, it’ll call this method which will format the dataclass object as
a prettified JSON string. If you prefer a__str__method to not be
added, you can pass instr=Falsewhen extending from the Mixin class
as mentionedhere.Note that the__repr__method, which is implemented by thedataclassdecorator, is also available. To invoke the Python object
representation of the dataclass instance, you can instead userepr(obj)orf'{obj!r}'.To mark a dataclass as being JSON serializable (and
de-serializable), simply sub-class fromJSONSerializableas shown
below. You can also extend from the aliased nameJSONWizard, if you
prefer to use that instead.Check out amore complete exampleof using theJSONSerializableMixin class.No Inheritance NeededIt is important to note that the main purpose of sub-classing fromJSONWizardMixin class is to provide helper methods likefrom_dictandto_dict, which makes it much more convenient and easier to load or
dump your data class from and to JSON.That is, it’s meant tocomplementthe usage of thedataclassdecorator,
rather than to serve as a drop-in replacement for data classes, or to provide type
validation for example; there are already excellent libraries likepydanticthat
provide these features if so desired.However, there may be use cases where we prefer to do away with the class
inheritance model introduced by the Mixin class. In the interests of convenience
and also so that data classes can be usedas is, the Dataclass
Wizard library provides the helper functionsfromlistandfromdictfor de-serialization, andasdictfor serialization. These functions also
work recursively, so there is full support for nested dataclasses – just as with
the class inheritance approach.Here is an example to demonstrate the usage of these helper functions:NoteAs ofv0.18.0, the Meta config for the main dataclass will cascade down
and be merged with the Meta config (if specified) of each nested dataclass. To
disable this behavior, you can pass inrecursive=Falseto the Meta config.from__future__importannotationsfromdataclassesimportdataclass,fieldfromdatetimeimportdatetime,datefromdataclass_mageimportfromdict,asdict,DumpMeta@dataclassclassA:created_at:datetimelist_of_b:list[B]=field(default_factory=list)@dataclassclassB:my_status:int|strmy_date:date|None=Nonesource_dict={'createdAt':'2010-06-10 15:50:00Z','List-Of-B':[{'MyStatus':'200','my_date':'2021-12-31'}]}# De-serialize the JSON dictionary object into an `A` instance.a=fromdict(A,source_dict)print(repr(a))# A(created_at=datetime.datetime(2010, 6, 10, 15, 50, tzinfo=datetime.timezone.utc),#   list_of_b=[B(my_status='200', my_date=datetime.date(2021, 12, 31))])# Set an optional dump config for the main dataclass, for example one which# converts converts date and datetime objects to a unix timestamp (as an int)## Note that `recursive=True` is the default, so this Meta config will be# merged with the Meta config (if specified) of each nested dataclass.DumpMeta(marshal_date_time_as='TIMESTAMP',key_transform='SNAKE',# Finally, apply the Meta config to the main dataclass.).bind_to(A)# Serialize the `A` instance to a Python dict object.json_dict=asdict(a)expected_dict={'created_at':1276185000,'list_of_b':[{'my_status':'200','my_date':1640926800}]}print(json_dict)# Assert that we get the expected dictionary object.assertjson_dict==expected_dictCustom Key MappingsIf you ever find the need to add acustom mappingof a JSON key to a dataclass
field (or vice versa), the helper functionjson_field– which can be
considered an alias todataclasses.field()– is one approach that can
resolve this.Example below:fromdataclassesimportdataclassfromdataclass_mageimportJSONSerializable,json_field@dataclassclassMyClass(JSONSerializable):my_str:str=json_field('myString1',all=True)# De-serialize a dictionary object with the newly mapped JSON key.d={'myString1':'Testing'}c=MyClass.from_dict(d)print(repr(c))# prints:#   MyClass(my_str='Testing')# Assert we get the same dictionary object when serializing the instance.assertc.to_dict()==dExtending fromMetaLooking to change howdateanddatetimeobjects are serialized to JSON? Or
prefer that field names appear insnake casewhen a dataclass instance is serialized?The innerMetaclass allows easy configuration of such settings, as
shown below; and as a nice bonus, IDEs should be able to assist with code completion
along the way.NoteAs ofv0.18.0, the Meta config for the main dataclass will cascade down
and be merged with the Meta config (if specified) of each nested dataclass. To
disable this behavior, you can pass inrecursive=Falseto the Meta config.fromdataclassesimportdataclassfromdatetimeimportdatefromdataclass_mageimportJSONWizardfromdataclass_mage.enumsimportDateTimeTo@dataclassclassMyClass(JSONWizard):class_(JSONWizard.Meta):marshal_date_time_as=DateTimeTo.TIMESTAMPkey_transform_with_dump='SNAKE'my_str:strmy_date:datedata={'my_str':'test','myDATE':'2010-12-30'}c=MyClass.from_dict(data)print(repr(c))# prints:#   MyClass(my_str='test', my_date=datetime.date(2010, 12, 30))string=c.to_json()print(string)# prints:#   {"my_str": "test", "my_date": 1293685200}Other Uses forMetaHere are a few additional use cases for the innerMetaclass. Note that
a full list of available settings can be found in theMetasection in the docs.Debug ModeEnables additional (more verbose) log output. For example, a message can be
logged whenever an unknown JSON key is encountered whenfrom_dictorfrom_jsonis called.This also results in more helpful error messages during the JSON load
(de-serialization) process, such as when values are an invalid type –
i.e. they don’t match the annotation for the field. This can be particularly
useful for debugging purposes.NoteThere is a minor performance impact when DEBUG mode is enabled;
for that reason, I would personally advise against enabling
this in aproductionenvironment.Handle Unknown JSON KeysThe default behavior is to ignore any unknown or extraneous JSON keys that are
encountered whenfrom_dictorfrom_jsonis called, and emit a “warning”
which is visible whendebugmode is enabled (and logging is properly configured).
An unknown key is one that does not have a known mapping to a dataclass field.However, we can also raise an error in such cases if desired. The below
example demonstrates a use case where we want to raise an error when
an unknown JSON key is encountered in theload(de-serialization) process.importloggingfromdataclassesimportdataclassfromdataclass_mageimportJSONWizardfromdataclass_mage.errorsimportUnknownJSONKey# Sets up application logging if we haven't already done sologging.basicConfig(level='INFO')@dataclassclassContainer(JSONWizard):class_(JSONWizard.Meta):# True to enable Debug mode for additional (more verbose) log output.debug_enabled=True# True to raise an class:`UnknownJSONKey` when an unmapped JSON key is# encountered when `from_dict` or `from_json` is called. Note that by# default, this is also recursively applied to any nested dataclasses.raise_on_unknown_json_key=Trueelement:'MyElement'@dataclassclassMyElement:my_str:strmy_float:floatd={'element':{'myStr':'string','my_float':'1.23',# Notice how this key is not mapped to a known dataclass field!'my_bool':'Testing'}}# Try to de-serialize the dictionary object into a `MyClass` object.try:c=Container.from_dict(d)exceptUnknownJSONKeyase:print('Received error:',type(e).__name__)print('Class:',e.class_name)print('Unknown JSON key:',e.json_key)print('JSON object:',e.obj)print('Known Fields:',e.fields)else:print('Successfully de-serialized the JSON object.')print(repr(c))Date and Time with Custom PatternsAs ofv0.20.0, date and time strings in acustom formatcan be de-serialized
using theDatePattern,TimePattern, andDateTimePatterntype annotations,
representing patterneddate,time, anddatetimeobjects respectively.This will internally calldatetime.strptimewith the format specified in the annotation,
and also use thefromisoformat()method in case the date string is in ISO-8601 format.
All dates and times will continue to be serialized as ISO format strings by default. For more
info, check out thePatterned Date and Timesection in the docs.A brief example of the intended usage is shown below:fromdataclassesimportdataclassfromdatetimeimporttime,datetimefromtypingimportList# Note: in Python 3.9+, you can import this from `typing` insteadfromtyping_extensionsimportAnnotatedfromdataclass_mageimportfromdict,asdict,DatePattern,TimePattern,Pattern@dataclassclassMyClass:date_field:DatePattern['%m-%Y']dt_field:Annotated[datetime,Pattern('%m/%d/%y %H.%M.%S')]time_field1:TimePattern['%H:%M']time_field2:Annotated[List[time],Pattern('%I:%M %p')]data={'date_field':'12-2022','time_field1':'15:20','dt_field':'1/02/23 02.03.52','time_field2':['1:20 PM','12:30 am']}class_obj=fromdict(MyClass,data)# All annotated fields de-serialize as just date, time, or datetime, as shown.print(class_obj)# MyClass(date_field=datetime.date(2022, 12, 1), dt_field=datetime.datetime(2023, 1, 2, 2, 3, 52),#         time_field1=datetime.time(15, 20), time_field2=[datetime.time(13, 20), datetime.time(0, 30)])# All date/time fields are serialized as ISO-8601 format strings by default.print(asdict(class_obj))# {'dateField': '2022-12-01', 'dtField': '2023-01-02T02:03:52',#  'timeField1': '15:20:00', 'timeField2': ['13:20:00', '00:30:00']}# But, the patterned date/times can still be de-serialized back after# serialization. In fact, it'll be faster than parsing the custom patterns!assertclass_obj==fromdict(MyClass,asdict(class_obj))Dataclasses inUnionTypesThedataclass-magelibrary fully supports declaring dataclass models inUniontypes as field annotations, such aslist[Wizard | Archer | Barbarian].As ofv0.19.0, there is added support toauto-generatetags for a dataclass model
– based on the class name – as well as to specify a customtag keythat will be
present in the JSON object, which defaults to a special__tag__key otherwise.
These two options are controlled by theauto_assign_tagsandtag_keyattributes (respectively) in theMetaconfig.To illustrate a specific example, a JSON object such as{"oneOf": {"type": "A",...},...}will now automatically map to a dataclass
instanceA, provided that thetag_keyis correctly set to “type”, and
the fieldone_ofis annotated as a Union type in theA | Bsyntax.Let’s start out with an example, which aims to demonstrate the simplest usage of
dataclasses inUniontypes. For more info, check out theDataclasses in Union Typessection in the docs.from__future__importannotationsfromdataclassesimportdataclassfromdataclass_mageimportJSONWizard@dataclassclassContainer(JSONWizard):classMeta(JSONWizard.Meta):tag_key='type'auto_assign_tags=Trueobjects:list[A|B|C]@dataclassclassA:my_int:intmy_bool:bool=False@dataclassclassB:my_int:intmy_bool:bool=True@dataclassclassC:my_str:strdata={'objects':[{'type':'A','my_int':42},{'type':'C','my_str':'hello world'},{'type':'B','my_int':123},{'type':'A','my_int':321,'myBool':True}]}c=Container.from_dict(data)print(f'{c!r}')# Trueassertc==Container(objects=[A(my_int=42,my_bool=False),C(my_str='hello world'),B(my_int=123,my_bool=True),A(my_int=321,my_bool=True)])print(c.to_dict())# prints the following on a single line:# {'objects': [{'myInt': 42, 'myBool': False, 'type': 'A'},#              {'myStr': 'hello world', 'type': 'C'},#              {'myInt': 123, 'myBool': True, 'type': 'B'},#              {'myInt': 321, 'myBool': True, 'type': 'A'}]}# Trueassertc==c.from_json(c.to_json())Serialization OptionsThe following parameters can be used to fine-tune and control how the serialization of a
dataclass instance to a Pythondictobject or JSON string is handled.Skip DefaultsA common use case is skipping fields with default values - based on thedefaultordefault_factoryargument todataclasses.field- in the serialization
process.The attributeskip_defaultsin the innerMetaclass can be enabled, to exclude
such field values from serialization.Theto_dictmethod (or theasdicthelper
function) can also be passed anskip_defaultsargument, which should have the same
result. An example of both these approaches is shown below.fromcollectionsimportdefaultdictfromdataclassesimportfield,dataclassfromtypingimportDefaultDict,Listfromdataclass_mageimportJSONWizard@dataclassclassMyClass(JSONWizard):class_(JSONWizard.Meta):skip_defaults=Truemy_str:strother_str:str='any value'optional_str:str=Nonemy_list:List[str]=field(default_factory=list)my_dict:DefaultDict[str,List[float]]=field(default_factory=lambda:defaultdict(list))print('-- Load (Deserialize)')c=MyClass('abc')print(f'Instance:{c!r}')print('-- Dump (Serialize)')string=c.to_json()print(string)assertstring=='{"myStr": "abc"}'print('-- Dump (with `skip_defaults=False`)')print(c.to_dict(skip_defaults=False))Exclude FieldsYou can also exclude specific dataclass fields (and their values) from the serialization
process. There are two approaches that can be used for this purpose:The argumentdump=Falsecan be passed in to thejson_keyandjson_fieldhelper functions. Note that this is a more permanent option, as opposed to the one
below.Theto_dictmethod (or theasdicthelper function ) can be passed
anexcludeargument, containing a list of one or more dataclass field names
to exclude from the serialization process.Additionally, here is an example to demonstrate usage of both these approaches:fromdataclassesimportdataclassfromtypingimportAnnotatedfromdataclass_mageimportJSONWizard,json_key,json_field@dataclassclassMyClass(JSONWizard):my_str:strmy_int:intother_str:Annotated[str,json_key('AnotherStr',dump=False)]my_bool:bool=json_field('TestBool',dump=False)data={'MyStr':'my string','myInt':1,'AnotherStr':'testing 123','TestBool':True}print('-- From Dict')c=MyClass.from_dict(data)print(f'Instance:{c!r}')# dynamically exclude the `my_int` field from serializationadditional_exclude=('my_int',)print('-- To Dict')out_dict=c.to_dict(exclude=additional_exclude)print(out_dict)assertout_dict=={'myStr':'my string'}Field PropertiesThe Pythondataclasseslibrary has somekey limitationswith how it currently handles properties and default values.Thedataclass-magepackage natively provides support for using
field properties with default values in dataclasses. The main use case
here is to assign an initial value to the field property, if one is not
explicitly passed in via the constructor method.To use it, simply import
theproperty_wizardhelper function, and add it as a metaclass on
any dataclass where you would benefit from using field properties with
default values. The metaclass also pairs well with theJSONSerializablemixin class.For more examples and important how-to’s on properties with default values,
refer to theUsing Field Propertiessection in the documentation.ContributingContributions are welcome! Open a pull request to fix a bug, oropen an issueto discuss a new feature or change.Check out theContributingsection in the docs for more info.TODOsAll feature ideas or suggestions for future consideration, have been currently addedas milestonesin the project’s GitHub repo.CreditsThis package was created withCookiecutterand thernag/cookiecutter-pypackageproject template.