Package: serdataclasses
serdataclassesSerialize (serdataclasses.dump) from and deserialize (serdataclasses.load) to strongly-typed, native Python data structures.MotivationNo typing-focusedserdelibrary in Python satisfies me. Call me needy, but when I translate between loosely-typed data structures likelistanddictinto strongly-typed data structures likeNamedTupleanddataclasses.dataclass, I want the following capabilities:Effortlessl deserialization of unstructured Python types into structured, type-hinted Python types (dataclasses.dataclass,typing.NamedTuple)Effortless serialization of structured, type-hinted Python objects into unstructured Python types (eg, the reverse)Clear error messages when serde fails at runtimeNo inherited, non-standard types. dataclasses, NamedTuples, and other standard Python types are bread and butterEditor support: I like my autocompletion, so I jump through lots of hoops to make this library compatible with JediHandleoptional propertieswith a domain-specificserdataclasses.OptionalPropertyProvide option to automatically convert primitive types, but avoid converting ambiguous types (Union,TypeVar, etc). Handle ofOptionalandserdataclasses.OptionalPropertyRequire no 3rd party dependencies; Python 3.8+Installation# With pippipinstallserdataclasses# With poetrypoetryaddserdataclassesUsageSeeexamples folderif you'd like to get your hands dirty. Otherwise, keep reading for a complete, real-world example.Example: Libraries and BooksAssume that, from an external API, you receive aJSONlist of libraries containing the library name and a list of books that each library currently has.[{"name":"Clark County Library","books":[{"title":"Hello, World!","author":"Susy Smith","year":1929,"tags":["boring"]},{"title":"The great showman","author":"Beth John"},{"title":"My favorite pony","author":null}]},{"name":"Only 1 book here","books":[{"title":"The great fun time","author":"Smitty","year":1950,"tags":["swell"]}]}]Now you want to ingest this document into Python. Your first step is probably to deserialize the JSON string (or file) into Python data structures. Assuming the JSON is read from a file calledlibraries.py, the following script will print the following:importjsonfrompprintimportpprintwithopen("libraries.json","r")asinfile:libraries_raw=json.load(infile)pprint(libraries_raw)print(type(libraries_raw))print(type(libraries_raw[0]))[{'books': [{'author': 'Susy Smith',
             'tags': ['boring'],
             'title': 'Hello, World!',
             'year': 1929},
            {'author': 'Beth John', 'title': 'The great showman'},
            {'author': None, 'title': 'My favorite pony'}],
  'name': 'Clark County Library'},
 {'books': [{'author': 'Smitty',
             'tags': ['swell'],
             'title': 'The great fun time',
             'year': 1950}],
  'name': 'Only 1 book here'}]
<class 'list'>
<class 'dict'>Some observations:Python's nativejsonmodule deserializes the JSON string / document into Python's primitive (or primitive-like) typesnullis translated to Python'sNoneThe first list element is a dictionary. So Python appears to have translated the JSON into a list of dictionaries.There is little inherent structure to the Python objects deserialized by the JSON module. By this, I mean that we have no way of knowing whether the dictionaries contain keys that we expect or are structured improperly. Should books also have an"isbn"field? Does code we write that useslibraries_rawexpect an"isbn"field? What happens if there are missing tags?The first 3 items are merely facts;serdataclassesaccepts these facts and builds on them. The 4th item in this list is THE problem thatserdataclassesis designed to solve. If we take the above Python dictionary associate it with a Python variableLIBRARIES, we can define a container for use withserdataclassesto ingestLIBRARIESinto a strongly-typed Python container.fromdataclassesimportdataclass,fieldfrompprintimportpprintfromtypingimportList,OptionalimportserdataclassesfromserdataclassesimportOptionalProperty[{"books":[{"author":"Susy Smith","tags":["boring"],"title":"Hello, World!","year":1929,},{"author":"Beth John","title":"The great showman"},{"author":None,"title":"My favorite pony"},],"name":"Clark County Library",},{"books":[{"author":"Smitty","tags":["swell"],"title":"The great fun time","year":1950,}],"name":"Only 1 book here",},]@dataclassclassBook:author:Optional[str]title:stryear:OptionalProperty[int]tags:List[str]=field(default_factory=list)@dataclassclassLibrary:books:List[Book]name:strLIBRARIES_LOADED=serdataclasses.load(LIBRARIES,List[Library])print(LIBRARIES_LOADED[0].name)print(LIBRARIES_LOADED[0].books[1].author)pprint(serdataclasses.dump(LIBRARIES_LOADED))Running the above script, we get the following output to the terminal:[{'books': [{'author': 'Susy Smith',
             'tags': ['boring'],
             'title': 'Hello, World!',
             'year': 1929},
            {'author': 'Beth John', 'tags': [], 'title': 'The great showman'},
            {'author': None, 'tags': [], 'title': 'My favorite pony'}],
  'name': 'Clark County Library'},
 {'books': [{'author': 'Smitty',
             'tags': ['swell'],
             'title': 'The great fun time',
             'year': 1950}],
  'name': 'Only 1 book here'}]Notice how we have the following features:Data structures are loaded, recursively, without you needing to write anything more than a couple standard Python classes.For missing properties / dictionary keys (for example,Book.tags), we can set a default value in our dataclass using standard Python andserdataclassesadds the default value to our structureFor missing properties without default values, serdataclasses intelligently omits them when re-serializing the result. There is also an option toserdataclasses.loadthat allows you to convert missing values toNoneand keep the keys in the output. For all other desired default values, just usedataclasses.field; no need to re-invent the wheel!What about additional validation, you may ask? Again, just use dataclasses! Assume that, for some reason, no book can possibly be published before 1930, and that a book published before 1930 invalidates all data. No problem, just use the standard method__post_init__on the relevant dataclass!fromdataclassesimportdataclass,fieldfrompprintimportpprintfromtypingimportList,OptionalimportserdataclassesfromserdataclassesimportOptionalPropertyLIBRARIES=[{"books":[{"author":"Susy Smith","tags":["boring"],"title":"Hello, World!","year":1929,},{"author":"Beth John","title":"The great showman"},{"author":None,"title":"My favorite pony"},],"name":"Clark County Library",},{"books":[{"author":"Smitty","tags":["swell"],"title":"The great fun time","year":1950,}],"name":"Only 1 book here",},]@dataclassclassBook:author:Optional[str]title:stryear:OptionalProperty[int]tags:List[str]=field(default_factory=list)def__post_init__(self)->None:ifself.yearandself.year<1930:raiseValueError(f"Received illegal year{self.year}, cannot be before 1930")@dataclassclassLibrary:books:List[Book]name:strLIBRARIES_LOADED=serdataclasses.load(LIBRARIES,List[Library])Running this script should give you the following error message (traceback omitted because it can get somewhat long)serdataclasses.errors.DeserializeError: Received illegal year 1929, cannot be before 1930
  4. "<class '__main__.Book'>": "{'author': 'Susy Smith', 'tags': ['boring'], 'title': 'Hello, World!', 'year': 1929}"
  3. 'typing.List[__main__.Book]': "[{'author': 'Susy Smith', 'tags': ['boring'], 'title': 'Hello, World!', 'year': 1929}, {'author': 'Beth John', 'title': 'The great showman'}, {'author': None, 'title': 'My favorite pony'}]"
  2. "<class '__main__.Library'>": "{'books': [{'author': 'Susy Smith', 'tags': ['boring'], 'title': 'Hello, World!', 'year': 1929}, {'author': 'Beth John', 'title': 'The great showman'}, {'author': None, 'title': 'My favorite pony'}], 'name': 'Clark County Library'}"
  1. 'typing.List[__main__.Library]': "[{'books': [{'author': 'Susy Smith', 'tags': ['boring'], 'title': 'Hello, World!', 'year': 1929}, {'author': 'Beth John', 'title': 'The great showman'}, {'author': None, 'title': 'My favorite pony'}], 'name': 'Clark County Library'}, {'books': [{'author': 'Smitty', 'tags': ['swell'], 'title': 'The great fun time', 'year': 1950}], 'name': 'Only 1 book here'}]"The error message begins with the error message received, followed by a reverse presention container types paired with the data they attempted to deserialize. This structure makes it incredibly easy to see not only what your error is, but where exactly it occured in both the dataserdataclasses.loadreceives but also in the typesserdataclasses.loaduses to attempt to deserialize the received data.In serde, when working with resources external to your system, errors are inevitable. These error messages should hopefully make debugging your errors less annoying.Local DevelopmentLocal development for this project is simple.DependenciesInstall the following tools manually.PoetryGNU MakeRecommendedasdfSet up development environmentmakesetupRun TestsmaketestNotesInitially inspired byundictifyand a PR I helped with. serdataclasses's goals are different; it's focused on serde instead of general function signature overrides.I also notice some striking similarities with a library calledtypedload(great minds think alike, I guess :p). I renamed my top-level functions to "load" and "dump" in typedload's homage. Unfortunately, as of version1.20, typedload does not handle all types of dataclasses elegantly (mainly, InitVar). Since typedload supports Python 3.5+, it never will elegantly handle all dataclasses without lots of unfortunate conditionals in the codebase. If you must use Python 3.7-, I suggest looking into typedload.Written bySamuel Roecasamuel.roeca@gmail.com