Package: beschi
BeschiThis is a custom bit-packing and unpacking code generator for C, C#, Go, Rust, Swift, and TypeScript. You feed it a data description and it generates source files for writing/reading buffers of that data, along the lines ofFlatBuffersorCap'n Proto, but with much less functionality for much simpler use cases. It was initially written for a larger project that was passing data back and forth between a Unity game, a Go server, and a web client, but I extracted it into its own thing. If all you need is a simple way to pack a data structure into a compact, portable binary form, this might be useful for you.I go into more explanation for why this existsin the documentation, but I'll be honest, too: itwaskind of fun to write a code generator. 😝DocumentationIntroductionProtocolsLanguage-Specific Documentation:CC#GoRustSwiftTypeScriptDev NotesHow to useYou can install fromPyPI:pip install beschiIt installs an executable that you can run directly, so long as however you invokedpipput it on your path. (This will work in an activated virtual environment, for instance.)Example:beschi --lang csharp --protocol ./messages.tomlBy default, it prints to standard output, but you can also write to a file with an output flag like--output ./Messages.cs.From the input protocol file (detailed below), you get a code file that you can integrate to a project allowing you encode messages as compact and portable binary buffers.ProtocolsThe protocol files are written inTOML. There'sa fuller example in the test suiteand amore through explanation in the documentation, but here's an annotated sample.[meta]# The namespace is optional; not all languages#  do something useful with it, but it's handy#  to have to help avoid clashesnamespace="AppMessages"# Messages are defined by a name and their data#  members. This will become a class or struct in#  the target languages with these pieces of data#  as accessible members.[[messages]]_name="Vector3Message"x="float"y="float"z="float"# There are a variety of different data member#  types that can be defined.[[messages]]_name="NewCharacterMessage"id="uint64"characterName="string"strength="uint16"intelligence="uint16"dexterity="uint16"goldInWallet="uint32"nicknames="[string]"# brackets indicate a list/array# You can also define structs, collections of data#  that go together, but are not themselves a message.[[structs]]_name="Color"red="float"green="float"blue="float"alpha="float"# Structs can contain other structs, and even lists of them.[[structs]]_name="Spectrum"defaultColor="Color"colors="[Color]"# Structs can then be used in messages.[[messages]]_name="CharacterJoinedTeam"characterID="uint64"teamName="string"teamColors="[Color]"Data MembersThese are the base types from which you can build up whatever structures and messages you need to, along with what they correspond to in the various languages.Protocol TypeCC#GoRustSwiftTypeScriptbyteuint8_tbytebyteu8UInt8numberboolboolboolboolboolBoolbooleanint16uint16_tshortint16i16Int16numberuint16int16_tushortuint16u16UInt16numberint32uint32_tintint32i32Int32numberuint32int32_tuintuint32u32UInt32numberint64uint64_tlongint64i64Int64bigintuint64int64_tulonguint64u64UInt64bigintfloatfloatfloatfloat32f32Float32numberdoubledoubledoublefloat64f64Float64numberstringchar*stringstringStringStringstringAll the numbers are stored as little-endian in the buffer, if that matters for you.Usage in codeBeschi does not generate any code to handle writing or reading from disk, pushing data across a network, or anything like that — it will turn a message into bytes and read that same message back from the bytes, but you are responsible for what you do with them otherwise.With the given protocol, though, you could create a message in C# and write it to a file:varmsg=newAppMessages.Vector3Message();msg.x=1.0f;msg.y=4096.1234f;msg.z=-42.56f;varfs=newFileStream("./vec3.msg",FileMode.Create);varbw=newBinaryWriter(fs);msg.WriteBytes(bw,false);And then read it back in TypeScript:constdata=fs.readFileSync("./vec3.msg");constdv=newDataView(newUint8Array(data).buffer);constmsg=AppMessages.Vector3Message.FromBytes(dv,0).val;if(msg.y==Math.fround(4096.1234)){console.log("Ready to go!");}Or Go:dat,_:=os.Open("./vec3.msg")deferdat.Close()msg:=AppMessages.Vector3MessageFromBytes(dat)ifmsg.X==1.0&&msg.Y==4096.1234&&msg.Z<0.0{print("Ready to go!\n")}For the most part, Beschi tries to keep behavior and structures consistent across the languages, but there are a few points of differenceoutlined on the various language pages. Notice in the example above, for instance, that in TypeScript you have to make a call toMath.froundif you want to do a straight comparison of float values because of how the underlying JavaScript engine treats all numbers as double-width floats. (Doing equality comparisons on floats is usually a bad idea, but in this instance wewantto check that they are actually bitwise identical.) Similarly, see how the data members are upper-cased in Go to match that language's export conventions, and the byte reading function is part of the namespace because Go doesn't have static functions for data types. The goal is to make working across languages feel as seamless as possible, but there are some differences that we adapt to as much as possible.There are more extensive examples inthe test harnesses.FutureBeschi makes code that I use "in production" on personal projects. It could probably stand to be better optimized, there are probably edge cases and bugs, etc. But for the most part, this is mature enough that I stopped thinking of it as a project in itself and now it's just a tool that I use.I will admit that part of me wants to make new writers, but that's not likely to get done until I have a specific project motivating it. If someone loves this system, though, and really wants to see a generator for Haskell or Lua or whatever, go for it. The existing writers should be decent starting points — they aren't terribly clever (no AST or interesting data structures), just iterating over the protocol and writing out serialization/deserialization code.Beschi?Constanzo Giuseppe Beschiwas an Italian Jesuit who worked in southern India during the early 18th century. He was noted as a talented linguist, able to tie concepts from multiple languages into a single form. At the same time, he was adept at the Jesuit principle of "inculturation," where foreign concepts are adapted for a new culture and the foreigner attempting the adaptation also respectfully adopts habits and ways of proceeding from the host culture.