Package: py-to-proto
PY To ProtoThis library holds utilities for converting in-memory data schema representations toProtobuf. The intent is to allow python libraries to leverage the power ofprotobufwhile maintaining the source-of-truth for their data in pure python and avoiding static build steps.Why?Theprotobuflangauge is a powerful tool for defining language-agnostic, composable datastructures.Protobufalso offers cross-language compatibility so that a given set of definitions can be compiled into numerous target programming languages. The downside is thatprotobufrequires_a static built step to perform thisproto->Xconversion step. Alternately, there are multiple ways of representing data schemas in pure python which allow a python library to interact with well-typed data objects. The downside here is that these structures can not easily be used from other programming languages. The pros/cons of these generally fall along the following lines:Protobuf:AdvantagesCompact serializationAuto-generatedgrpcclient and service librariesClient libraries can be used from different programming languagesDisadvantagesLearning curve to understand the full ecosystemNot a familiar tool outside of service engineeringStatic compilation step required to use in codePython schemas:AdvantagesCan be learned quickly using pure-python documentationCan be written inline in pure pythonDisadvantagesGenerally, no standard serialization beyondjsonNo automated service implementationsNo/manual mechanism for usage in other programming languagesThis project aims to bring the advantages of both types of schema representation so that a given project can take advantage of the best of both:Define your structures in pure python for simplicityDynamically creategoogle.protobuf.Descriptorobjects to allow forprotobufserialization and deserializationReverse render a.protofile from the generatedDescriptorso that stubs can be generated in other languagesNo static compiliation needed!Supported Python Schema TypesCurrently, objects can be declared using eitherpythondataclassesorJson TypeDef (JTD). Additional schemas can be added bysubclassingConverterBase.Dataclass To ProtoThe following example illustrates howdataclassesandenumscan be converted to proto:fromdataclassesimportdataclassfromenumimportEnumfromtypingimportAnnotated,Dict,List,Enumimportpy_to_proto# Define the Foo structure as a python dataclass, including a nested enum@dataclassclassFoo:classBarEnum(Enum):EXAM:0JOKE_SETTING:1foo:boolbar:List[BarEnum]# Define the Foo protobuf message classFooProto=py_to_proto.descriptor_to_message_class(py_to_proto.dataclass_to_proto(package="foobar",dataclass_=Foo,))# Declare the Bar structure as a python dataclass with a reference to the# FooProto type@dataclassclassBar:baz:FooProto# Define the Bar protobuf message classBarProto=py_to_proto.descriptor_to_message_class(py_to_proto.dataclass_to_proto(package="foobar",dataclass_=Bar,))# Instantiate a BarProtoprint(BarProto(baz=FooProto(foo=True,bar=[Foo.BarEnum.EXAM.value])))defwrite_protos(proto_dir:str):"""Write out the .proto files for FooProto and BarProto to the givendirectory"""FooProto.write_proto_file(proto_dir)BarProto.write_proto_file(proto_dir)JTD To ProtoThe following example illustrates how JTD schemas can be converted to proto:importpy_to_proto# Declare the Foo protobuf message classFoo=py_to_proto.descriptor_to_message_class(py_to_proto.jtd_to_proto(name="Foo",package="foobar",jtd_def={"properties":{# Bool field"foo":{"type":"boolean",},# Array of nested enum values"bar":{"elements":{"enum":["EXAM","JOKE_SETTING"],}}}},))# Declare an object that references Foo as the type for a fieldBar=py_to_proto.descriptor_to_message_class(py_to_proto.jtd_to_proto(name="Bar",package="foobar",jtd_def={"properties":{"baz":{"type":Foo.DESCRIPTOR,},},},),)defwrite_protos(proto_dir:str):"""Write out the .proto files for Foo and Bar to the given directory"""Foo.write_proto_file(proto_dir)Bar.write_proto_file(proto_dir)Similar ProjectsThere are a number of similar projects in this space that offer slightly different value:jtd-codegen: This project focuses on statically generating language-native code (includingpython) to represent the JTD schema.py-json-to-proto: This project aims to deduce a schema from an instance of ajsonobject.pure-protobuf: This project has a very similar aim topy-to-proto, but it skips the intermediatedescriptorrepresentation and thus is not able to produce nativemessage.Messageclasses.