Package: lifx-photons-messages-generator
This is the code that takes in the LIFX protocol definition and some adjustments
and outputs the code for thephotons_messagesmodule.Photons is a framework for interacting with LIFX products and can be found athttps://github.com/delfick/photons-coreChangelog0.6.8 - 27 July 2022Allow reserved fields in a union enum0.6.7 - 27 July 2022Updated the yaml library I use0.6.6 - 11 May 2021Support Union types0.6.5 - 10 May 2021Update dependencies0.6.4 - 23 November 2020Compatibility with python3.90.6.3 - 7 May 2020Fixed 1 bit reserved fields0.6.2 - 7 May 2020Can now understand size_bits: 1 in the source yaml as meaning T.Bool0.6.1 - 21 September 2019Formatted code with black and converted tests to pytestUpgraded todelfick_project0.6.0 - 21 September 2019I changed the many option in photons, it is now “multiple” instead0.5.4 - 1 September 2019Emit fmt: off and fmt: on stanzas so black doesn’t destroy the code0.5.3 - 28 June 2019Emit fields as lists instead of tuples to ensure it’s a list when there’s
only one item in it.0.5.2 - 23 Jan 2019Started using ruamel.yaml instead of PyYaml to load configuration0.5.1 - 12 Jan 2019Make it possible to say an enum value allows unknown values0.5 - 3 Jan 2019Initial releaseTestsCreate a virtualenv somewhere and run the following in that virtualenv from this
directory:$ pip install -e .
$ pip install -e ".[tests]"
$ ./test.shGenerating messagesOnce you have this installed in your virtualenv you can run thegenerate_photons_messagesscript in your PATH.This takes in the following options:–src or the SRC environment variableThe path to the yaml file containing the definition of all the messages–adjustments or the ADJUSTMENTS environment variableThe path to the adjustments.yml that contains all the customization–output-folder or the OUTPUT_FOLDER environment variableThe directory that is the root of the outputAuto customizationThe generator will make the following changes for you without you having to
specify them in the adjustments.ymlPacket namesPacket names in the source yaml are of the form <Namespace><PacketName>.
The code will strip the <Namespace>. So for example DeviceSetLabel is turned
into SetLabel under the DeviceMessages class.If the name is<Namespace>Getthen it’s turned intoGet<Namespace>.
Similar transformation is done for<Namespace>Set. A<Namespace>Statemessage is kept as is.Field namesAll field names are snake_case after transformation. For example if the field
was called AmazingField, it would appear as amazing_field.AdjustmentsThe main point of this code is thatphotons_messageshas some high level
concepts and changes to the source definition. The adjustments.yml looks like
this:---# If fields end up with these names then the generator complainsinvalid_field_names:["payload","count","index","fields","Meta"]# Where to start counting reserved fields from# So because we say 5 here, the first reserved field will be called reserved6# the next will be reserved7, etcnum_reserved_fields_in_frame:5# The packets are split up by namespace, this list will say the order of the# namespaces in the list. All other namespaces not in the list will be added# at the end in whatever order they've been definednamespace_order:["core","discovery","device","light"]# We can use this to rename an entire namespace# So here we're saying the namespace `some_namespace` should actually be# called `other_namespace`. This new name is used in output options and the# namespace_order option aboverename_namespaces:some_namespace:other_namespace# We can choose to ignore structs from being output# Here we're saying don't output either StructOne or StructTwo in fields.py# and replace StructOne with a `T.Bytes(size_bytes * 8)` where the size_bytes# comes from the field that is using this struct# And fields using StructTwo with the fields that are in StructTwo. Note that# StructOne doesn't need to actually be in the source yaml, but StructTwo does.ignore:StructOne:{}StructTwo:expanded:true# Output options define where we put our output# This is a list of options. You must specify printing enums, fields and all# the packets## Each item in the list has the following options:## create#   either "enums", "fields" or "packets"## dest#   either a string that is the name of the file under output_directory#   or a list of strings specifying the path. So saying ``["messages", "lan.py"]``#     would produce a file at ``<output_folder>/messages/lan.py``## static#   A string that is put at the top of that file## options#   If create is packet then this is a dictionary of ``include`` and ``exclude``#   These are either a string or a list of strings of globs to be applied to#   the namespaces. Include is applied first and then exclude is applied.#   To include all namespaces, say ``include: "*"``output:-create:enumsdest:"enums.py"static:|from enum import Enum-create:fieldsdest:"fields.py"static:|from photons_messages import enumsfrom photons_protocol.packets import dictobjfrom photons_protocol.messages import Tfrom lru import LRU-create:packetsdest:"messages.py"options:include:"*"static:|from photons_messages import enums, fieldsfrom photons_messages.frame import msgfrom photons_protocol.messages import T, Messages, MultiOptionsfrom photons_protocol.types import Optionaldef empty(pkt, attr):return pkt.actual(attr) in (Optional, sb.NotSpecified)# Types let's us specify special types that can then be used multiple times# by packets and structs. This let's us specify transformations in one place# rather than many.# They are of the form ``{<name>: <options>}`` and can be used by specifying# ``special_type: <name>`` in the options for a field (see "changes" below)# Note that we specify the type here so that you can only override a field# with the same type as this special type# So here we're defining a type called duration_type, it will appear in# fields.py like this:##  duration_type = T.Uint32.default(0).transform(#        lambda _, value: int(1000 * float(value))#      , lambda value: float(value) / 1000#      ).allow_float()#types:duration_type:type:uint32size_bits:32default:"0"extras:-|transform(lambda _, value: int(1000 * float(value)), lambda value: float(value) / 1000)-"allow_float()"# Clones let us create a clone of a struct that has different options for use# elsewhere. For example the clone here is the LightHsbk struct but where all# the fields are optional# The options for each field include ``more_extras`` and ``remove_default``# where more_extras adds more options to the type and remove_default makes it# so the type has no default even if one was set on the original struct.# Note that in this case LightHsbk has extras and defaults specified under# the "changes" section.clones:hsbk_with_optional:cloning:LightHsbkfields:Hue:more_extras:["optional()"]Saturation:more_extras:["optional()"]Brightness:more_extras:["optional()"]Kelvin:remove_default:truemore_extras:["optional()"]scaled_hue:...scaled_to_65535:...# The changes section lets us specify renames, different types, field renames# , namespace changes, many_options and using helper# Note that all names here are the original names in the source yaml# We are guaranteed that enums/structs/packets are all unique names and so# you don't need to specify what name is what type.changes:# First we're renaming LightHsbk as hsbk# Then we're saying that if it's used like ``[8]<LightHsbk>`` then we will# use the classname of Color and give it a cache amount of 8000# We also give special types to some fields. This produces:### hsbk = (#       ("hue", scaled_hue)#     , ("saturation", scaled_to_65535)#     , ("brightness", scaled_to_65535)#     , ("kelvin", T.Uint16.default(3500))#     )## class Color(dictobj.PacketSpec):#     fields = hsbk# Color.Meta.cache = LRU(8000)## Then if anything uses many of these then they will say# ``T.Bytes(size_bytes * 8).many(lambda pkt: fields.Color)``#LightHsbk:rename:hsbkmany_options:name:Colorcache_amount:8000fields:Hue:special_type:scaled_hueSaturation:special_type:scaled_to_65535Brightness:special_type:scaled_to_65535Kelvin:default:"3500"# Here we rename the enum DeviceService to ServicesDeviceService:rename:Services# Here we put the DeviceAcknowledgement packet in the "core" namespaceDeviceAcknowledgement:namespace:core# Here we're saying the Label field on the DeviceSetLabel packet is a string# This only works for fields that are bytes and will output# ``T.String(size_bytes * 8)`` instead of ``T.Bytes(size_bytes * 8)``DeviceSetLabel:fields:Label:string_type:true# Here we're saying DeviceStateLabel has the same fields as DeviceSetLabel# And will output ``StateLabel = SetLabel.using(pkt_type)`` where# pkt_type is the pkt_type for DeviceStateLabel from the source yaml.# This will complain if the fields are infact not the same.DeviceStateLabel:using:DeviceSetLabel# Here we're saying that GetService is under the discovery namespace and# has a multi option of -1# So it will output:##  GetService = msg(2#      , multi = -1#      )#DeviceGetService:namespace:discoverymulti:"-1"# Here we're renaming the Payload field on EchoRequest to be echoing# This is because payload is one of our fields we're not allowed to have.DeviceEchoRequest:fields:Payload:rename:echoing# Here we're giving the Version field a version_number() option# So it'll output##   StateHostFirmware = msg(15#       , ("build", T.Uint64)#       , ("install", T.Uint64)#       , ("version", T.Uint32.version_number())#       )#DeviceStateHostFirmware:fields:Version:extras:["version_number()"]# Here we give Duration the special type of duration_type# So it produces##  SetColor = msg(102#      , ("reserved6", T.Reserved(8))#      , *fields.hsbk#      , ("duration", fields.duration_type)#      )## Note that the *fields.hsbk means we are using the fields from hsbk here# inline.LightSetColor:rename:SetColorfields:Duration:special_type:duration_type# Here we use override_struct to use our hsbk_with_optional clone instead# of hsbk which is what would otherwise be usedLightSetWaveformOptional:fields:Color:override_struct:hsbk_with_optional# Apply is an enum here (as defined in the source yaml) and so the code# will make sure the default we specify is a valid value from that enum.MultiZoneSetColorZones:fields:Apply:default:"APPLY"# We can split up a field into a value for each of the bits in that field# So let's say we have a packet called ExamplePacket with a field Flags# that is a uint8, then the following will produce:##   ExamplePacket = msg(9001#     , ("option_one", T.Bool)#     , ("option_two", T.Bool)#     , ("option_three", T.Bool)#     , ("option_four", T.Bool)#     , ("option_five", T.Bool)#     , ("option_six", T.Bool)#     , ("option_seven", T.Bool)#     , ("option_eight", T.Bool)#     )## Note that the number of options must match the number of bits for that# field.ExamplePacket:fields:Flags:bits:-OptionOne-OptionTwo-OptionThree-OptionFour-OptionFive-OptionSix-OptionSeven-OptionEight