Package: aristaproto
Arista Protobuf / Python gRPC bindings generator & libraryThis project is forked fromhttps://github.com/danielgtaylor/python-betterproto@b8a091ae7055dd949d193695a06c9536ad51eea8Changes in this project compared with the base project:Keep nanosecond precision forTimestamp.Subclassdatetimeto store the original nano-second value when converting fromTimestamptodatetime.On conversion from the subclass ofdatetimetoTimestampthe original nano-second value is restored.InstallationFirst, install the package. Note that the[compiler]feature flag tells it to install extra dependencies only needed by theprotocplugin:# Install both the library and compilerpipinstall"aristaproto[compiler]"# Install just the library (to use the generated code output)pipinstallaristaprotoBetterprotois under active development. To install the latest beta version, usepip install --pre aristaproto.Getting StartedCompiling proto filesGiven you installed the compiler and have a proto file, e.gexample.proto:syntax="proto3";packagehello;// Greeting represents a message you can tell a user.messageGreeting{stringmessage=1;}You can run the following to invoke protoc directly:mkdirlib
protoc-I.--python_aristaproto_out=libexample.protoor run the following to invoke protoc via grpcio-tools:pipinstallgrpcio-tools
python-mgrpc_tools.protoc-I.--python_aristaproto_out=libexample.protoThis will generatelib/hello/__init__.pywhich looks like:# Generated by the protocol buffer compiler.  DO NOT EDIT!# sources: example.proto# plugin: python-aristaprotofromdataclassesimportdataclassimportaristaproto@dataclassclassGreeting(aristaproto.Message):"""Greeting represents a message you can tell a user."""message:str=aristaproto.string_field(1)Now you can use it!>>>fromlib.helloimportGreeting>>>test=Greeting()>>>testGreeting(message='')>>>test.message="Hey!">>>testGreeting(message="Hey!")>>>serialized=bytes(test)>>>serializedb'\n\x04Hey!'>>>another=Greeting().parse(serialized)>>>anotherGreeting(message="Hey!")>>>another.to_dict(){"message":"Hey!"}>>>another.to_json(indent=2)'{\n"message": "Hey!"\n}'Async gRPC SupportThe generated ProtobufMessageclasses are compatible withgrpclibso you are free to use it if you like. That said, this project also includes support for async gRPC stub generation with better static type checking and code completion support. It is enabled by default.Given an example service definition:syntax="proto3";packageecho;messageEchoRequest{stringvalue=1;// Number of extra times to echouint32extra_times=2;}messageEchoResponse{repeatedstringvalues=1;}messageEchoStreamResponse{stringvalue=1;}serviceEcho{rpcEcho(EchoRequest)returns(EchoResponse);rpcEchoStream(EchoRequest)returns(streamEchoStreamResponse);}Generate echo proto file:python-mgrpc_tools.protoc-I.--python_aristaproto_out=.echo.protoA client can be implemented as follows:importasyncioimportechofromgrpclib.clientimportChannelasyncdefmain():channel=Channel(host="127.0.0.1",port=50051)service=echo.EchoStub(channel)response=awaitservice.echo(echo.EchoRequest(value="hello",extra_times=1))print(response)asyncforresponseinservice.echo_stream(echo.EchoRequest(value="hello",extra_times=1)):print(response)# don't forget to close the channel when done!channel.close()if__name__=="__main__":loop=asyncio.get_event_loop()loop.run_until_complete(main())which would outputEchoResponse(values=['hello','hello'])EchoStreamResponse(value='hello')EchoStreamResponse(value='hello')This project also produces server-facing stubs that can be used to implement a Python
gRPC server.
To use them, simply subclass the base class in the generated files and override the
service methods:importasynciofromechoimportEchoBase,EchoRequest,EchoResponse,EchoStreamResponsefromgrpclib.serverimportServerfromtypingimportAsyncIteratorclassEchoService(EchoBase):asyncdefecho(self,echo_request:"EchoRequest")->"EchoResponse":returnEchoResponse([echo_request.valuefor_inrange(echo_request.extra_times)])asyncdefecho_stream(self,echo_request:"EchoRequest")->AsyncIterator["EchoStreamResponse"]:for_inrange(echo_request.extra_times):yieldEchoStreamResponse(echo_request.value)asyncdefmain():server=Server([EchoService()])awaitserver.start("127.0.0.1",50051)awaitserver.wait_closed()if__name__=='__main__':loop=asyncio.get_event_loop()loop.run_until_complete(main())JSONBoth serializing and parsing are supported to/from JSON and Python dictionaries using the following methods:Dicts:Message().to_dict(),Message().from_dict(...)JSON:Message().to_json(),Message().from_json(...)For compatibility the default is to convert field names tocamelCase. You can control this behavior by passing a casing value, e.g:MyMessage().to_dict(casing=aristaproto.Casing.SNAKE)Determining if a message was sentSometimes it is useful to be able to determine whether a message has been sent on the wire. This is how the Google wrapper types work to let you know whether a value is unset, set as the default (zero value), or set as something else, for example.Usearistaproto.serialized_on_wire(message)to determine if it was sent. This is a little bit different from the official Google generated Python code, and it lives outside the generatedMessageclass to prevent name clashes. Note that itonlysupports Proto 3 and thus canonlybe used to check ifMessagefields are set. You cannot check if a scalar was sent on the wire.# Old way (official Google Protobuf package)>>>mymessage.HasField('myfield')# New way (this project)>>>aristaproto.serialized_on_wire(mymessage.myfield)One-of SupportProtobuf supports grouping fields in aoneofclause. Only one of the fields in the group may be set at a given time. For example, given the proto:syntax="proto3";messageTest{oneoffoo{boolon=1;int32count=2;stringname=3;}}You can usearistaproto.which_one_of(message, group_name)to determine which of the fields was set. It returns a tuple of the field name and value, or a blank string andNoneif unset.>>>test=Test()>>>aristaproto.which_one_of(test,"foo")["",None]>>>test.on=True>>>aristaproto.which_one_of(test,"foo")["on",True]# Setting one member of the group resets the others.>>>test.count=57>>>aristaproto.which_one_of(test,"foo")["count",57]>>>test.onFalse# Default (zero) values also work.>>>test.name="">>>aristaproto.which_one_of(test,"foo")["name",""]>>>test.count0>>>test.onFalseAgain this is a little different than the official Google code generator:# Old way (official Google protobuf package)>>>message.WhichOneof("group")"foo"# New way (this project)>>>aristaproto.which_one_of(message,"group")["foo","foo's value"]Well-Known Google TypesGoogle provides several well-known message types like a timestamp, duration, and several wrappers used to provide optional zero value support. Each of these has a special JSON representation and is handled a little differently from normal messages. The Python mapping for these is as follows:Google MessagePython TypeDefaultgoogle.protobuf.durationdatetime.timedelta0google.protobuf.timestampTimezone-awaredatetime.datetime1970-01-01T00:00:00Zgoogle.protobuf.*ValueOptional[...]Nonegoogle.protobuf.*aristaproto.lib.google.protobuf.*NoneFor the wrapper types, the Python type corresponds to the wrapped type, e.g.google.protobuf.BoolValuebecomesOptional[bool]whilegoogle.protobuf.Int32ValuebecomesOptional[int]. All of the optional values default toNone, so don't forget to check for that possible state. Given:syntax="proto3";import"google/protobuf/duration.proto";import"google/protobuf/timestamp.proto";import"google/protobuf/wrappers.proto";messageTest{google.protobuf.BoolValuemaybe=1;google.protobuf.Timestampts=2;google.protobuf.Durationduration=3;}You can do stuff like:>>>t=Test().from_dict({"maybe":True,"ts":"2019-01-01T12:00:00Z","duration":"1.200s"})>>>tTest(maybe=True,ts=datetime.datetime(2019,1,1,12,0,tzinfo=datetime.timezone.utc),duration=datetime.timedelta(seconds=1,microseconds=200000))>>>t.ts-t.durationdatetime.datetime(2019,1,1,11,59,58,800000,tzinfo=datetime.timezone.utc)>>>t.ts.isoformat()'2019-01-01T12:00:00+00:00'>>>t.maybe=None>>>t.to_dict(){'ts':'2019-01-01T12:00:00Z','duration':'1.200s'}Generating Pydantic ModelsYou can use python-aristaproto to generate pydantic based models, using
pydantic dataclasses. This means the results of the protobuf unmarshalling will
be typed checked. The usage is the same, but you need to add a custom option
when calling the protobuf compiler:protoc-I.--python_aristaproto_opt=pydantic_dataclasses--python_aristaproto_out=libexample.protoWith the important change being--python_aristaproto_opt=pydantic_dataclasses. This will
swap the dataclass implementation from the builtin python dataclass to the
pydantic dataclass. You must have pydantic as a dependency in your project for
this to work.DevelopmentRequirementsPython (3.9 or higher)poetryNeeded to install dependencies in a virtual environmentpoethepoetfor running development tasks as defined in pyproject.tomlCan be installed to your host environment viapip install poethepoetthen executed as simplepoeor run from the poetry venv aspoetry run poeSetup# Get set up with the virtual env & dependenciespoetryinstall-Ecompiler# Activate the poetry environmentpoetryshellCode styleThis project enforcesblackpython code formatting.Before committing changes run:poeformatTo avoid merge conflicts later, non-black formatted python code will fail in CI.TestsThere are two types of tests:Standard testsCustom testsStandard testsAdding a standard test case is easy.Create a new directoryaristaproto/tests/inputs/<name>add<name>.protowith a message calledTestadd<name>.jsonwith some test data (optional)It will be picked up automatically when you run the tests.See also:Standard Tests Development GuideCustom testsCustom tests are found intests/test_*.pyand are run with pytest.RunningHere's how to run the tests.# Generate assets from sample .proto files required by the testspoegenerate# Run the testspoetestTo run tests as they are run in CI (with tox) run:poefull-test(Re)compiling Google Well-known TypesBetterproto includes compiled versions for Google's well-known types atsrc/aristaproto/lib/google.
Be sure to regenerate these files when modifying the plugin output format, and validate by running the tests.Normally, the plugin does not compile any references togoogle.protobuf, since they are pre-compiled. To force compilation ofgoogle.protobuf, use the option--custom_opt=INCLUDE_GOOGLE.Assuming yourgoogle.protobufsource files (included with all releases ofprotoc) are located in/usr/local/include, you can regenerate them as follows:protoc\--plugin=protoc-gen-custom=src/aristaproto/plugin/main.py\--custom_opt=INCLUDE_GOOGLE\--custom_out=src/aristaproto/lib\-I/usr/local/include/\/usr/local/include/google/protobuf/*.protoLicenseCopyright 2023 Arista NetworksCopyright 2019-2023 Daniel G. TaylorThis software is free to use under the MIT license. See theLICENSEfile for license text.