Package: yaml-serde
yaml-serde - Simplified YAML serialization frameworkyaml-serdeprovides a framework for implementing custom conversions of
Python objects to and from the YAML serialization format.These transformations allow applications to use YAML as a simple way to persist
and share objects.Python to YAMLAfter the appropriate serializers have been defined (seeCustom YAML serialization),
objects can be transformed to a YAML string using functionyml():fromyaml_serdeimportYamlSerializer,yml,repr_ymlclassFoo(list):def__init__(self):super().__init__(Bar(1),Bar(2),Bar(3))class_YamlSerializer(YamlSerializer):defrepr_yml(self,py_repr,**kwargs):return{"foo":[repr_yml(b,**kwargs)forbinpy_repr]}classBar:def__init__(self,n):self.n=nclass_YamlSerializer(YamlSerializer):defrepr_yml(self,py_repr,**kwargs):return{"bar":py_repr.n}foo=Foo()yml_str=yml(foo)assertyml_str=="""---foo:- bar: 1- bar: 2- bar: 3..."""The result can also be saved directly into a file by specifying a path
withto_file:yml(foo,to_file="foo.yml")If you prefer, you can also convert objects to JSON using functionjson():fromyaml_serdeimportymlfoo=Foo()json_str=json(foo)assertjson_str=='{"foo": [{"bar": 1}, {"bar": 2}, {"bar": 3}]}'YAML to PythonGiven a YAML (or JSON) string, you can build a Python object out of it
using functionyml_obj(). This function takes a class object and the
input string, and will return an instance of the class built by its
serializer:importpathlibfromyaml_serdeimportYamlSerializer,yml,repr_yml,yml_obj,repr_pyclassFoo(list):def__init__(self,*args):ifnotargs:args=(Bar(1),Bar(2),Bar(3))super().__init__(*args)class_YamlSerializer(YamlSerializer):defrepr_yml(self,py_repr,**kwargs):return{"foo":[repr_yml(b,**kwargs)forbinpy_repr]}defrepr_py(self,yml_repr,**kwargs):returnFoo(*[repr_py(Bar,b,**kwargs)forbinyml_repr["foo"]])classBar:def__init__(self,n):self.n=nclass_YamlSerializer(YamlSerializer):defrepr_yml(self,py_repr,**kwargs):return{"bar":py_repr.n}defrepr_py(self,yml_repr,**kwargs):returnBar(yml_repr["bar"])withpathlib.Path("foo.yml").open("r")asinput:foo=yml_obj(Foo,input.read())Since loading YAML from a file is common enough,yml_obj()offers argumentfrom_fileto indicate that the input is the path
of a file from which to read the input string:foo=yml_obj(Foo,"foo.yml",from_file=True)Custom YAML serializationTheyaml-serdeframework is built on top ofPyYAML,
as way to easily call functionsyaml.safe_dump()andyaml.safe_load()on
objects of any user-defined class.The PyYAML functions only accept basic Python objects such as numbers, strings,
arrays, and dictionaries, and they are designed to reject any generic "object"
(which must be handled using their "unsafe" counterparts).For this reason,yaml_serdeallows users to implement the logic required to
convert instances of their classes into "YAML-safe" representations
compatible with the PyYAML functions.The conversion is implemented by a nested "serializer" class,
called_YamlSerializerand derived fromyaml_serde.YamlSerializer, which
must be manually defined for every class to convert, and provide two methods:YamlSerializer::repr_yml(self, py_repr, **kwargs):Take an object in its "Python representation" and return the equivalent
"YAML representation".The value returned by this function must be safe to pass toyaml.safe_dump().YamlSerializer::repr_py(self, yml_repr, **kwargs):Take an object's "YAML representation" and return its
"Python representation".The value returned by this function should be an instance of the associated
class.Implementations are free to map an objects state to YAML however they prefer.For example, in the case of a trivial class, a simple string might be used:fromyaml_serdeimportYamlSerializer,repr_ymlclassMyClass:def__init__(self,foo:str):self.foo=fooclass_YamlSerializer(YamlSerializer):defrepr_yml(self,py_repr,**kwargs):returnpy_repr.foodefrepr_py(self,yml_repr,**kwargs):returnMyClass(yml_repr)In most cases, a class will likely map to a dictionary, with entries for each of
its state attributes.Two convenience functions (repr_yml()andrepr_py()) can be used to
automatically invoke an object's serializer, and convert it between the two
formats. These functions can be useful to build a "recursive" serializer:fromyaml_serdeimportYamlSerializer,repr_yml,repr_pyclassFoo(list):def__init__(self,*args):ifnotargs:args=(Bar(1),Bar(2),Bar(3))super().__init__(*args)class_YamlSerializer(YamlSerializer):defrepr_yml(self,py_repr,**kwargs):return{"foo":[repr_yml(b,**kwargs)forbinpy_repr]}defrepr_py(self,yml_repr,**kwargs):returnFoo(*[repr_py(Bar,b,**kwargs)forbinyml_repr["foo"]])classBar:def__init__(self,n):self.n=nclass_YamlSerializer(YamlSerializer):defrepr_yml(self,py_repr,**kwargs):return{"bar":py_repr.n}defrepr_py(self,yml_repr,**kwargs):returnBar(yml_repr["bar"])The serializer class will be passed through all the extra keyword
arguments passed to theyml()andyml_obj()functions.This can be used to implement optional behavior in the serializer,
for example to exclude certain fields from serialization:fromyaml_serdeimportYamlSerializer,repr_ymlclassMyClassWithPrivateFields:def__init__(self,user:str,passwd:str):self.user=userself.passwd=passwdclass_YamlSerializer(YamlSerializer):defrepr_yml(self,py_repr,**kwargs):yml_repr={"user":py_repr.user}ifnotkwargs.get("public_only"):yml_repr["passwd"]=py_repr.passwdreturnyml_reprdefrepr_py(self,yml_repr,**kwargs):returnMyClass(yml_repr["user"],yml_repr.get("passwd",""))deftest_my_class_with_private_fields():fromyaml_serdeimportyml,yml_objobj=MyClassWithPrivateFields("foo","bar")yml_str=yml(obj)assert"passwd: bar"inyml_strobj=yml_obj(MyClassWithPrivateFields,yml_str)assertobj.passwd=="bar"yml_str=yml(obj,public_only=True)assert"passwd: bar"notinyml_strobj=yml_obj(MyClassWithPrivateFields,yml_str)assertobj.passwd==""ClassYamlSerializeroffers two methods which subclasses can override
to customize the serialization of data in files:YamlSerializer::file_format_out()is called when a YAML string is
about to be written to a file. It takes the YAML string
(along with any extra keyword arguments passed toyml()), and it
must return the actual string that will be written to the file system.YamlSerializer::file_format_in()is called whenever a YAML string is
loaded from a file. It take the file's contents (along with any extra
keywork arguments passed toyml_obj()) and it must return the
string to parse into an object.These function can be used to pre/post-process the contents of a file,
for example to encode it on serialization, and decoding it when the
value is read back from the file:fromyaml_serdeimportYamlSerializer,repr_ymlclassMyEncodedClass:def__init__(self,user:str,passwd:str):self.user=userself.passwd=passwdclass_YamlSerializer(YamlSerializer):defrepr_yml(self,py_repr,**kwargs):return{"user":py_repr.user,"passwd":py_repr.passwd}defrepr_py(self,yml_repr,**kwargs):returnMyClass(yml_repr["user"],yml_repr["passwd"])deffile_format_in(self,yml_str,**kwargs):encoder=kwargs["encoder"]returnencoder.encode(yml_str)deffile_format_out(self,yml_str,**kwargs):encoder=kwargs["encoder"]returnencoder.decode(yml_str)classMyEncoder:def__init__(self,prefix):self.prefix=prefixdefencode(self,contents):returnf"{self.prefix}{contents}{self.prefix}"defdecode(self,contents):returncontents[len(self.prefix):-len(self.prefix)]defverify(self,encoded):assertencoded.startswith(self.prefix)assertencoded.endswith(self.prefix)deftest_my_encoded_class():fromyaml_serdeimportyml,yml_objfrommy_encoder_packageimportMyEncoderobj=MyEncodedClass("foo","bar")encoder=MyEncoder("****\n")yml(obj,to_file="encoded.yml",encoder=encoder)importpathlibwithpathlib.Path("encoded.yml").open("r")asinput:assertencoder.verify(input.read())obj=yml_obj(MyEncodedClass,"encoded.yml",from_file=True,encoder=encoder)assertobj.user=="foo"assertobj.passwd=="bar"A similar result may also be achieved by having the serializer use a customFileSystemimplementation. By default, anyYamlSerializerwill rely on aLocalFileSyteminstance, which provides access to writing and storing files
in the local file system, and does not apply any transformation to file
contents.Overriding theFileSystemclass allows the same processing logic to be reused
by multipleYamlSerializerclasses:fromyaml_serdeimportLocalFileSystemclassMyEncoder:def__init__(self,prefix):self.prefix=prefixdefencode(self,contents):returnf"{self.prefix}{contents}{self.prefix}"defdecode(self,contents):returncontents[len(self.prefix):-len(self.prefix)]defverify(self,encoded):assertencoded.startswith(self.prefix)assertencoded.endswith(self.prefix)classMyEncodedFileSystem(LocalFileSystem):defformat_output(self,output,append=False,**kwargs):pfx=kwargs.get("pfx")ifpfxisnotNone:encoded=MyEncoder(pfx)returnself.encoder.encode(output)else:returnoutputdefformat_input(self,input,**kwargs):pfx=kwargs.get("pfx")ifpfxisnotNone:encoded=MyEncoder(pfx)returnself.encoder.decode(input)else:returninputencoded_fs=MyEncodedFileSystem()classMyEncodedClass:def__init__(self,user:str,passwd:str):self.user=userself.passwd=passwdclass_YamlSerializer(YamlSerializer):fs=encoded_fsdefrepr_yml(self,py_repr,**kwargs):return{"user":py_repr.user,"passwd":py_repr.passwd}defrepr_py(self,yml_repr,**kwargs):returnMyClass(yml_repr["user"],yml_repr["passwd"])classMyOtherEncodedClass:def__init__(self,foo:str):self.foo=fooclass_YamlSerializer(YamlSerializer):fs=encoded_fsdefrepr_yml(self,py_repr,**kwargs):returnpy_repr.foodefrepr_py(self,yml_repr,**kwargs):returnMyClass(yml_repr)deftest_my_encoded_classes():fromyaml_serdeimportyml,yml_objfrommy_encoder_packageimportMyEncoderobj=MyEncodedClass("foo","bar")other_obj=MyOtherEncodedClass("foo")pfx="****\n"yml(obj,to_file="encoded_obj.yml",pfx=pfx)yml(obj,to_file="encoded_other_obj.yml",pfx=pfx)encoder=MyEncoder(pfx)defcheck_encoded(f):importpathlibwithpathlib.Path(f).open("r")asinput:assertencoder.verify(input.read())check_encoded("encoded_obj.yml")check_encoded("encoded_other_obj.yml")obj=yml_obj(MyEncodedClass,"encoded_obj.yml",from_file=True,pfx=pfx)assertobj.user=="foo"assertobj.passwd=="bar"other_obj=yml_obj(MyOtherEncodedClass,"encoded_other_obj.yml",from_file=True,pfx=pfx)assertother_obj.foo=="foo"