Package: generic-struct
This package allows quick and simple creation of protocol-handling classes. It consists of two maintypes -StructandParser- whereStructrepresent an object with fields and aParseris used to parse said
fields from and into binary buffers; the package also includes some basicParsers andStructs which can
be dynamically built to fit the developer's needs.UsageStructsThe classgeneric_struct.structs.Structis the basic protocol-handling class. it has members to represent the
different fields in the protocol and the API to write buffers from its fields and read its fields from buffers.
All protocol-handling classes created usinggeneric_structare derived fromStruct.The methods ofStruct:get_buffer()- create a binary buffer (abytesobject) representing the current fields of the structread_buffer(buffer)- read a binary buffer into the fields of the structget_buffer_size()- get the length of the corresponding binary bufferHow to create aStructclassThe current version includes two types ofStructs which can be createdgeneric_struct.structs.GenericStructThis type ofStructhas fields of varying types and generates buffers using each field's
correspondingParser.To create aStructof that type you can use the class-decoratorgeneric_struct.structs.GenericStruct.build_struct:fromgeneric_struct.structs.GenericStructimportbuild_struct@build_structclassMyStruct():<field_name>=<field_parser><field_name>=<field_parser>...object of this type ofStructcan be converted todictgeneric_struct.structs.FlagsThis type ofStructhandles flags fields. Each field is aboolobject and has a designated
bit in the serialized buffer.AStructof this type is created usnig the class-decoratorgeneric_struct.structs.Flags.build_flags:fromgeneric_struct.structs.Flagsimportbuild_flags@build_flagsclassMyFlags():FIELDS=['list','of','field','names']Where The first element in the FIELDS list is the most significant bit of the first byte in the buffer, and so on.To have more control over the positions of the bits in the buffer, you can insertgeneric_struct.structs.Flags.RESERVEDto represent one unused bit orgeneric_struct.structs.Flags.ReservedBits(n)to representnunused bits. unused bits in the buffer are Always 0.fromgeneric_struct.structs.Flagsimportbuild_flags,RESERVED,ReservedBits@build_flagsclassMyFlags():FIELDS=[ReservedBits(5),'some','bits','are',RESERVED,'bits']object of this type ofStructcan be converted todictParsersAParserclass is responsible to converting between a specifictypeand a binary buffer in a specific way;converting betweenintand an unsigned big-endian 4-bytes integer, for example.Parsers are mainly used in the creation ofStructclasses.The more complicatedParsers use otherParsers in order to do their work.This package contains a bunch of common usefulParsers:ParserSupported TypesBuffer typegeneric_struct.parsers.StaticField.StaticFieldParserint,bool,float,bytessimple conversion using thestructmodulegeneric_struct.parsers.StaticSizeBuffer.StaticSizeBufferParserstra buffer with a pre-determined sizegeneric_struct.parsers.DynamicSizeBuffer.DynamicSizeBufferParserstrlength of the buffer, then the buffergeneric_struct.parsers.StaticSizeList.StaticSizeListParserlistevery element of a pre-defined size list, parsed with aParserfor the elementsgeneric_struct.parsers.DynamicSizeList.DynamicSizeListParserlistlength of the list, then the elements of the listgeneric_struct.parsers.DelimitedBuffer.DelimitedBufferParserstra buffer with a delimiter buffer/byte at its endgeneric_struct.parsers.DelimitedList.DelimitedListParserstra list with a delimiter object at its endgeneric_struct.parsers.StructParserStructa parser for a specific type ofStructgeneric_struct.parsers.Union.UnionParserany one of a list ofStructsa number which defines the parsed type, then the parsed objectgeneric_struct.parsers.StaticField.StaticFieldParserThe most simpleParseris ``. ThisParserwraps the
functionality of `struct.pack()` and `struct.unpack_from()`, although meant to handle mainly `int` and `float`.Thegeneric_struct.parsers.StaticFieldmodule also contains wrappers for the different typesStaticFieldParsercan handle:SignedIntFormats,UnsignedIntFormats,FloatFormats,CHAR_FORMAT, andBOOL_FORMAT.Example: The parserStaticFieldParser(SignedIntFormats.BE_DWORD)would serialize11asb'\x00\x00\x00\x11'generic_struct.parsers.StaticSizeBuffer.StaticSizeBufferParserThis parser packs astrinto a buffer of a static size. Too short strings are padded withb'\x00'.
For example, the string'some txt'will be converted tob'some txt\x00\x00'.Note: trailingb'\x00's are not automatically removed byStaticSizeBufferParser.parse(). The bufferb'some txt\x00\x00'is parsed as the string'some txt\x00\x00'.generic_struct.parsers.DynamicSizeBuffer.DynamicSizeBufferParserThis parser packs astrinto a dynamic sized buffer. The buffer begins with the length of the string, serialized
by aParserthat can serialize the typeint, and then the string itself.Example: the parserDynamicSizeBufferParser(StaticFieldParser(UnsignedIntFormats.BYTE))serializes the string'some string'as the bufferb'\x0bsome string', while the parserDynamicSizeBufferParser(StaticFieldParser(UnsignedIntFormats.LE_DWORD))serializes it asb'\x0b\x00\x00\x00some string'generic_struct.parsers.StaticSizeList.StaticSizeListParserThis parser packs a static-sizedlistinto a buffer, with the elements packed by aParserthat can parse all the elements in the list.Example: the parserStaticSizeListParser(element_parser=StaticFieldParser(SignedIntFormats.LE_WORD), size=3)serializes the list[3,-1,5]asb'\x03\x00\xff\xff\x05\x00'generic_struct.parsers.DynamicSizeList.DynamicSizeListParserThis parser packs a dynamic-sized list into a buffer similarly togeneric_struct.parsers.StaticSizeList.StaticSizeListParser, except that the size of the list is added at the beginning
of the bufferExample: the parserDynamicSizeListParser(element_parser=StaticFieldParser(SignedIntFormats.LE_WORD),
                      size_parser=StaticFieldParser(SignedIntFormats.BYTE))serializes the list[3,-1,5]asb'\x03\x03\x00\xff\xff\x05\x00'generic_struct.parsers.DelimitedBuffer.DelimitedBufferParserThis parser packs a string as a buffer with a delimiter. The delimiter is abytesof any lengthExample: the parserDelimitedBufferParser(b'\x93)serializes the string'some_string'asb'some_string\x93'generic_struct.parsers.DelimitedList.DelimitedListParserThis parser packs alistas a buffer with a delimiter: The elements packed by aParserthat can parse all the elements in the list, and then a delimiter with its ownParserExample: the parserDelimitedListParser(element_parser=StaticFieldParser(SignedIntFormats.LE_WORD),
                    delimiter=700,
                    delimiter_parser=StaticFieldParser(UnsignedIntFormats.BE_DWORD))serializes the list[3,-1,5]asb'\x03\x00\xff\xff\x05\x00\x00\x00\x02\xbc'generic_struct.parsers.StructParserThis parser packs a child-class ofStruct. it is useful when one wishes one of the fields in theirStructto be
aStruct.generic_struct.parsers.Union.UnionParserThis parser packs one of a list of types, and adds a type identifier field packed with its own parser
at the beginning of the buffer.Example: The parserUnionParser(type_enum_parser=StaticFieldParser(UnsignedIntFormats.BYTE),
            type_parsers=[StaticFieldParser(FloatFormats.BE_QWORD),
            DelimitedBufferParser(b'\x00')])serializes:124.34asb'\x00@_\x15\xc2\x8f\\(\xf6''some string'asb'\x01some string\x00'Examplefromgeneric_struct.parsersimportStructParserfromgeneric_struct.parsers.DelimitedBufferimportDelimitedBufferParserfromgeneric_struct.parsers.DelimitedListimportDelimitedListParserfromgeneric_struct.parsers.DynamicSizeBufferimportDynamicSizeBufferParserfromgeneric_struct.parsers.StaticFieldimportStaticFieldParser,UnsignedIntFormats,CHAR_FORMATfromgeneric_struct.parsers.StaticSizeListimportStaticSizeListParserfromgeneric_struct.parsers.UnionimportUnionParserfromgeneric_struct.structs.FlagsimportReservedBits,build_flagsfromgeneric_struct.structs.GenericStructimportbuild_struct@build_flagsclassMyFlags(object):FIELDS=[ReservedBits(5),'bool_a','bool_b','bool_c']@build_structclassMyStruct(object):word_int=StaticFieldParser(UnsignedIntFormats.BE_WORD)byte_int=DynamicSizeBufferParser(StaticFieldParser(UnsignedIntFormats.BYTE))delim_list=DelimitedListParser(element_parser=DelimitedBufferParser(b'::'),delimiter=b'$',delimiter_parser=StaticFieldParser(CHAR_FORMAT))union=UnionParser(StaticFieldParser(UnsignedIntFormats.BYTE),[StructParser(MyFlags),StaticSizeListParser(StaticFieldParser(UnsignedIntFormats.BE_DWORD),4),DelimitedBufferParser(b';')])defmain():struct1=MyStruct(word_int=4,byte_int='some_ string',delim_list=['some','list','of','strings'],union=MyFlags(bool_a=False,bool_b=False,bool_c=True))print('struct1:',dict(struct1))print('struct1.union:',dict(struct1.union))buffer1=struct1.get_buffer()print('serialized:',buffer1)print()struct2=MyStruct()struct2.read_buffer(buffer1)struct2.union='this is a string now'buffer2=struct2.get_buffer()print('struct2:',dict(struct2))print('serialized:',buffer2)if__name__=='__main__':main()the output would be:struct1: {'word_int': 4, 'byte_int': 'some_ string', 'delim_list': ['some', 'list', 'of', 'strings'], 'union': <generic_struct.structs.Flags.flags.<locals>.Flags object at 0x03304810>}
struct1.union: {'bool_a': False, 'bool_b': False, 'bool_c': True}
serialized: b'\x00\x04\x0csome_ stringsome::list::of::strings::$\x00\x01'

struct2: {'word_int': 4, 'byte_int': 'some_ string', 'delim_list': ['some', 'list', 'of', 'strings'], 'union': 'this is a string now'}
serialized: b'\x00\x04\x0csome_ stringsome::list::of::strings::$\x02this is a string now;'