Package: serializejson
serializejsonAuthorsBaptiste de La GorcePyPIhttps://pypi.org/project/serializejsonDocumentationhttps://smartaudiotools.github.io/serializejsonSourceshttps://github.com/SmartAudioTools/serializejsonIssueshttps://github.com/SmartAudioTools/serializejson/issuesNoncommercial licenseProsperity Public License 3.0.0Commercial licensePatron License 1.0.0⇒Sponsor me !orcontact me !serializejsonis a python library for fast serialization and deserialization
of python objects inJSONdesigned as a safe, interoperable and human-readable drop-in replacement for the Pythonpicklepackage.
Complex python object hierarchies are serializable, deserializable or updatable in once, allowing for example to save or restore a complete application state in few lines of code.
The library is build uponpython-rapidjson,pybase64andbloscfor optionalzstandardcompression.Some of the main features:supports Python 3.7 (maybe lower) or greater.serializes arbitrary python objects into a dictionary by adding__class__,and eventually__init__,__new__,__state__,__items__keys.calls the same objects methods as pickle. Therefore almost all pickable objects are serializable with serializejson without any modification.for not already pickable object, you will allways be able to serialize it by adding methodes to the object or creating plugins for pickle or serializejson.generally 2x slower than pickle for dumping and 3x slower than pickle for loading (on your benchmark) except for big arrays (optimisation will soon be done).serializes and deserializes bytes and bytearray very quickly in base64 thanks topybase64and losslessblosccompression.serialize properties and attributes with getters and setters if wanted (unlike pickle).json data will still be directly loadable if you have transform some attributes in slots or properties in your code since your last serialization. (unlike pickle)can serialize__init__(self,..)arguments by name instead of positions, allowing to skip arguments with defauts values and making json datas robust to a change of__init__parameters order.serialized objects take generally less space than when serialized with pickle: for binary data, the 30% increase due to base64 encoding is in general largely compensated using the losslessblosccompression.serialized objects are human-readable and easy to read. Unlike pickled data, your data will never become unreadable if your code evolves: you will always be able to modify your datas with a text editor (with find & replace for example if you change an attribut name).serialized objects are text and therefore versionable and comparable with versionning and comparaison tools.can safely load untrusted / unauthenticated sources if authorized_classes list parameter is set carefully with strictly necessary objects (unlike pickle).can update existing objects recursively instead of override them. serializejson can be used to save and restore in place a complete application state (⚠ not yet well tested).filters attribute starting with “_” by default (unlike pickle). You can keep them if wanted withfilter_ = False.numpy arrays can be serialized as lists with automatic conversion in both ways or in a conservative way.supports circular references and serialize only once duplicated objects, using “$ref” key an path to the first occurance in the json :{“$ref”: “root.xxx.elt”}(⚠ not yet if the object is a list or dictionary).accepts json with comment (// and /* */) ifaccept_comments = True.can automatically recognize objects in json from keys names and recreate them, without the need of__class__key, if passed inrecognized_classes.serializejson is easly interoperable outside of the Python ecosystem with this recognition of objects from keys names or with__class__translation between python and other language classes.dump and load support string path.can iteratively encode (with append) and decode (with iterator) a list in json file, which helps saving memory space during the process of serialization and deserialization and useful for logs.Warning⚠Do not load serializejson files from untrusted / unauthenticated sources without carefully setting the load authorized_classes parameter.⚠Never dump a dictionary with the__class__key, otherwise serializejson will attempt to reconstruct an object when loading the json.
Be careful not to allow a user to manually enter a dictionary key somewhere without checking that it is not__class__.
Due to current limitation of rapidjson we cannot we cannot at the moment efficiently detect dictionaries with the__class__key to raise an error.InstallationLast offical releasepip install serializejsonDeveloppement version unreleasedpip install git+https://github.com/SmartAudioTools/serializejson.gitExamplesSerialization with fonctions APIimportserializejson# serialize in stringobject1=set([1,2])dumped1=serializejson.dumps(object1)loaded1=serializejson.loads(dumped1)print(dumped1)>{>"__class__":"set",>"__init__":[1,2]>}# serialize in fileobject2=set([3,4])serializejson.dump(object2,"dumped2.json")loaded2=serializejson.load("dumped2.json")Serialization with classes based API.importserializejsonencoder=serializejson.Encoder()decoder=serializejson.Decoder()# serialize in stringobject1=set([1,2])dumped1=encoder.dumps(object1)loaded1=decoder.loads(dumped1)print(dumped1)# serialize in fileobject2=set([3,4])encoder.dump(object2,"dumped2.json")loaded2=decoder.load("dumped2.json")Update existing objectimportserializejsonobject1=set([1,2])object2=set([3,4])dumped1=serializejson.dumps(object1)print(f"id{id(object2)}:{object2}")serializejson.loads(dumped1,obj=object2,updatables_classes=[set])print(f"id{id(object2)}:{object2}")Iterative serialization and deserializationimportserializejsonencoder=serializejson.Encoder("my_list.json",indent=None)foreltinrange(3):encoder.append(elt)print(open("my_list.json").read())foreltinserializejson.Decoder("my_list.json"):print(elt)>[0,1,2]>0>1>2More examples and complete documentationhereLicenseCopyright 2020 Baptiste de La GorceFor noncommercial use or thirty-day limited free-trial period commercial use, this project is licensed under theProsperity Public License 3.0.0.For non limited commercial use, this project is licensed under thePatron License 1.0.0.
To acquire a license pleasecontact me, or justsponsor me on GitHubunder the appropriate tier ! This funding model helps me making my work sustainable and compensates me for the work it took to write this crate!Third-party contributions are licensed underApache License, Version 2.0and belong to their respective authors.
History
=======Version 0.3.4Date:2023-06-11Restore ducumentationVersion 0.3.3Date:2022-10-18Big speed improvement for bytes and numpy array serializationVersion 0.3.2Date:2022-10-01API changedadd better support for cicular reférences and duplicates with {“$ref”: …}Version 0.2.0Date:2021-02-18API changedcan serialize dict with no-string keysadd support for cicular reférences and duplicates with {“$ref”: …}Version 0.1.0Date:2020-11-28change description for pipyadd license for pipyenable load of tuple, time.struct_time, Counter, OrderedDict and defaultdictVersion 0.0.4Date:2020-11-24API changedadd plugins supportadd bytes, bytearray and numpy.array compression with blosc zstdfix itertive append and decode (not fully tested).fix dump of numpy types without conversion to python types(not yet numpy.float64)