Package: comserde
ComserdeComserde(compactserialization anddeserialization) is a Python library for serializing data structures into compact binary data.Installation$pipinstallcomserdeUsageSerializing dataclassesDecorating a dataclass with@serializablewill mark it as serializable and deserializable. Type annotations are used to infer the format of each field.fromcomserdeimportserializable# Add a decorator to make the dataclass serializable# The order of decorators matters, the reverse won't work.@serializable@dataclassclassUser:age:Optional[int]name:strTo produce and consume bytes from a serializable object, calldumps()andloads(). A major difference compared to other serialization algorithms (such asjson) is that you must provide the type of object you want to decode,Userin this example.fromcomserdeimportdumps,loadsuser=User(age=34,name="John Doe")data=dumps(user)user=loads(data,User)More complex types can be defined using most of the typical type hints, such as generics and union types.fromtypingimportNewTypeclassUser:id:UserIdfriends:list[Admin|User]images:dict[ImageId,tuple[str,Optional[Image]]]profile_image:bytesTo prevent a specific field from being serialized, assign it with Comserde'sfield()instead ofdataclasses.field()and setserialize=False. Other field attributes can be set as usual.fromcomserdeimportfieldclassUser:age:Optional[int]name:str_cache:Optional[int]=field(default=None,serialize=False)Serializing regular classesRegular classes can also be serialized, although the types of each attribute must be explicitly provided, which is why dataclasses are preferred.fromcomserdeimportserializable# Define serialization fields explicitly@serializable({'age':Optional[int],'name':str})classUser:def__init__(self,age:Optional[int],name:str):self.age=ageself.name=name# Not serializedself._cache:Optional[int]=NoneCustomizing deserializationIf present, the__init_deserialize__method is called upon deserialization of an object. This is similar to the__setstate__method called when unpickling.@serializable({'age':Optional[int],'name':str})classUser:def__init_deserialize__(self,*,age:Optional[int],name:str):self.age=ageself.name=nameCustomizing serialization formatsThe full serialization and deserialization process can be customized by adding the__serialize__and__deserialize__methods.fromtypingimportIOclassUser:def__serialize__(self,file:IO[bytes]):file.write(...)@classmethoddef__deserialize__(cls,file:IO[bytes]):returncls(file.read(...))Specifying an explicit formatFor every field in a class, a format can be explicitly selected by using thetyping.Annotatedtype introduced byPEP 593. The use of this type doesn't affect the behavior of type checkers while providing additional information to Comserde.fromcomserdeimportSerializationFormat,serializable@serializable@dataclassclassVec2d:x:Annotated[float,SerializationFormat('f64')]y:Annotated[float,SerializationFormat('f64')]The same syntax can be used on on regular classes, however the format can also be specified directly.@serializable({'x':'f64','y':'f64'})classVec2d:...Serializing any objectObjects that are not marked with@serializableand that do not implement__serializable__()are pickled when serialized, and a warning is emitted if the thepickleformat is not explicit. Keep in mind that pickling a large number of individual objects can significantly increase the output size due to the overhead of each pickled object.Handling exceptionsAny error caused by corrupt data raises aDeserializationErrorwhich can be caught if the data is not trusted.try:user=loads(data,User)exceptDeserializationError:passReferencePrimitive formatsThe following serialization formats are supported:Booleans and scalarsboolu8,u16,u32,u64i8,i16,i32,i64f32,f64v8,v16,v32,v64– unsigned variable-length quantity (VLQ) encoding of at least 1, 2, 4 and 8 bytesw8,w16,w32,w64– signed VLQ encodingBytes and stringsbytes– length-prefixed bytesnt-bytes– null-terminated bytesutf-8,utf-16Othersjsonobject– stores the full qualified name of the object's class and uses the class as the serialization format; useful when the class is not known in advance, e.g. because it might be a subclasspicklevoidComposite formatsdeque[T]NewType('X', T)PurePathand all of its subclassesboolbytearray,bytescomplex– as twof64dict[K, V]EllipsisType,NoneType,Literal[T],tuple[()]– no data producedfloat– asf64frozensetint– asw8list[T]Literal[T, S, etc.]set[T]strT | Stuple[T, ...]tuple[T, S, etc.]