Package: ubii-message-formats
ubii-msg-formatsHi. This is the repository for Ubii message formats. It contains everything aboutwhatubii nodes and devices say to each other.
If you are interested in how they communicate with each other, please seehttps://zeromq.org/,https://zguide.zeromq.org/docs/chapter3/#The-REQ-to-REP-Combination,https://zguide.zeromq.org/docs/chapter3/#The-DEALER-to-ROUTER-Combinationhttps://developer.mozilla.org/en-US/docs/Web/HTTP/Methods,https://developer.mozilla.org/en-US/docs/Web/API/WebSocketTable of Contentsubii-msg-formatsTable of ContentsCompilationCLIsUbii MessagesTopicDataServiceRequestServiceReplyProtobufLoading Proto FilesProto FilesRequired and ValidationOneOfUsage of Proto Files in JSTestingTipsCompilationrequired: python >=3.5 (optionally withmypyandmypy-protobufinstalled - seePython Static Typing)required: 'protoc' compiler =3.11.1. If compiler is not installed/available in system path, get the release matching your OS fromhttps://github.com/protocolbuffers/protobuf/releasesand copy the executable (/bin/protoc) to ubii-msg-formats/external/bin.run 'node scripts/full-compile.js' to compile all parts (if python3 executable not automatically found path needs to be edited manually in script)to compile just specific parts:run 'python scripts/compile.py --opt [language]' where [language] is one of py/python, j/java, js/javascript, cs/csharp, allrun 'node scripts/generate-protobuf-static.js'Contributiondon't commit /dist for feature branches. you can test compilation, just don't commit the compiled files. You may want to use $GIT_DIR/info/exclude functionality to avoid unintentional commits.CLIsRunnpm testto process all standard tests.If snapshot assertions fail you can runnpm run-script update-snapshotsto process all standard tests and update all snapshots if the changes are intentional.Python TypingThe python code in theubii-msg-formatsmodule includespython stub fileswhich can be interpreted by e.g.PyCharmormypy. If you want to build the python package yourself, you will need to choose a
python executable whith workingmypyandmypy-protobufwhen you are runningscripts/proto-compile.py.Attention!The node.js entry scriptscripts/full-compile.jsdefines the python executable
which it will use. You need to change this to the right executable when you don't installmypyandmypy-protobufglobally in your$PATH!Ubii MessagesNote:The messages are based on google's Protobuf. Check out theProtobufsection below for more information on how to handle these types of messages.
You can find some more information on the proto files itself describing the structure of the messages as well as sections on how to work with proto messages in relevant envrionments such as nodeJS.There are three first order messages:topicDataserviceRequestserviceReplyAll ubii related sub messages sent over the network are wrapped up in one of the first order messages. A first order message has submessages that contains further data that should be processed. Typically this is sepcified by atypeoneof field.Check out theoneof sectionbelow for how you can get the specified submessage by accessing theoneofproperties.The proto file of the first order messages imports all submessages. Thus the proto instance or a translator instance of a first order message can encode and decode all related message buffers.TopicDataThis message is used to communicate topic data from the client to the server and from the server to the client. It is used in a router-dealer / websocket communication pattern context.Beside general information this message contains a topic data pair. TheTopicDatacan contain the following data structures:https://github.com/SandroWeber/ubii-msg-formats/blob/develop/src/proto/topicData/topicDataRecord/topicDataRecord.protoYou can send aTopicDatamessage to a master node in order to publish topic data pairs to a master node.
In return, master nodes will sendTopicDatamessages to registered devices to inform them about changes of their subscribed topics.ServiceRequestThis message is used to formulate service request. It is sent from the client to the server. The message is used in a request-reply / HTTP(S) request communication pattern context.TheServiceRequestcan have the following submessages specifing the type of the reply:https://github.com/SandroWeber/ubii-msg-formats/blob/develop/src/proto/services/serviceRequest.protoServiceReplyThis message is used by the server to formulate service replies to previous service requests from clients. The message is used in a request-reply / HTTP(S) request communication pattern context.TheServiceReplycan have the following submessages specifing the type of the reply:https://github.com/SandroWeber/ubii-msg-formats/blob/develop/src/proto/services/serviceReply.protoProtobufThe messages are based on google's Protobuf. The following section contains some topics that are super relevant for this project.See theProtobuf Documentationand the usedprotobuf.js Github repository of a JS implementaionfor more details.Loading Proto FilesThe proto files can be loaded synchronously or asynchronously. The asynchronous method should be prefered. However it is only available in a nodeJS environment.Proto FilesRequired and ValidationAccording to Protocol Buffers documentation, the required field rule is not recommended:Required Is Forever You should be very careful about marking fields as required. If at some point you wish to stop writing or sending a required field, it will be problematic to change the field to an optional field – old readers will consider messages without this field to be incomplete and may reject or drop them unintentionally. You should consider writing application-specific custom validation routines for your buffers instead. Some engineers at Google have come to the conclusion that using required does more harm than good; they prefer to use only optional and repeated. However, this view is not universal. --Protobuf DocumentationOneOfThe use of oneofs in JS is very poorly documented. Therefore we have collected some snippets how you can use them in JS (and how they are used in this project).
If you have some improvements, please update the appropriate code snippets and update this section.Specifyoneofas documented in the protobuf documentation. Example:oneofavatar{stringimage_url=1;bytesimage_data=2;}You can then acces the currently set oneof type at runtime viamessage['oneof specifier']. Example:letcurrentOneofType=message.avatar;Themessage['oneof specifier']is a property of the protoMessageprototype. It is not visible if the message is converted to a plain js object ortoString()is used to convert it to a string.You can then access the explicit value viamessage[message['oneof specifier']]. Example:letcurrentOneofValue=message[message.avatar];Usage of Proto Files in JSThis repository providesmessageTranslatorclasses. These classes provide all relevant functionalities to work with the proto messages.The main aspect is on a validated translating between the three proto message states: buffer <-> message <-> payloadFlatbuffershttps://google.github.io/flatbuffers/index.htmlCompilationRelease version used to compile schema files: v1.11.0TestingThis module uses theAVAtest runner.You can add new tests to the test folder. Entry point for the test runner is test.js within the test folder. See theAVA Documentationfor more details on how to create new test cases for AVA.TipsThere are extensions availabe for Visual Studio Code that support Proto3 files (Sytntax highlighting, syntax validation, ...). Just search forext:protoin the marketplace.BSD 3-Clause LicenseCopyright (c) 2018, IN-FAR / Ubi-Interact, Sandro Weber
All rights reserved.Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.