Package: cain
CainA small yet powerful data format ✨IndexIndexPurposeComparisonJSONCainGetting StartedPrerequisitesInstallingOption 1: From PyPIOption 2: From GitUsagePythonEncodingDecodingHandling SchemasEncodingDecodingCustom EncoderCLIExamplesDeploymentContributingAuthorsLicensingPurposeCain is a new data interchange format which aims at providing the smallest possible size to encode data.It is based on pre-defined schemas which leverages the need to specify it within the final encoded data.NoteLook at theSPECIFICATIONSfile for more information on the purpose and idea behind this project.ComparisonFor example, we consider the following object:{"b":3,"c":5.5,"d":True,"e":{"f":False,# "g": b"Hello world""h":"HELLO WORLD","i":"Hi!","j":[1,2,3,1,1],"k":(1,"hello",True),"l":None,"m":"Yay","n":"Hi","o":2,"p":None}}JSONThis is the expected result from a minified JSON encoding:{"b":3,"c":5.5,"d":true,"e":{"f":false,"h":"HELLO WORLD","i":"Hi!","j":[1,2,3,1,1],"k":[1,"hello",true],"l":null,"m":"Yay","n":"Hi","o":2,"p":null}}CainThis is the expected result from the Cain data format:\x00\x00\x03\x00\x00\xb0@\x01\x00\x00HELLO WORLD\x00Hi!\x00\x00\x05\x00\x00\x00\x01\x00\x02\x00\x03\x00\x01\x00\x01\x00\x00\x01hello\x00\x01\x00\x01\x00Yay\x00\x00Hi\x00\x01\x00\x02NoteThis is 56.76% smaller than the JSON version ✨Moreover, objects which can't be encoded using JSON (bytes, set, range, etc.) or wrongly encoded using JSON (ex: tuple) are working out of the box with Cain!Getting StartedThese instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.PrerequisitesYou will need Python 3 to use this moduleMinimumrequiredversions:3.9
Incompatibleversions:2Always check if your Python version works withcainbefore using it in production.InstallingOption 1: From PyPIpipinstall--upgradecainThis will install the latest version from PyPIOption 2: From Gitpipinstall--upgradegit+https://github.com/Animenosekai/cain.gitThis will install the latest development version from the git repositoryYou can check if you successfully installed it by printing out its version:$cain--version1.1UsagePythonThe main entry point (cain.py) provides an API familiar to users of the standard libraryjsonmodule. The different datatype also present a very pythonic way of handling data to keep a nice and clean codebase.EncodingEncoding basic Python object hierarchies:>>>importcain>>>fromcain.typesimportObject,Optional>>>cain.dumps({"a":2},Object[{"a":int}])b'\x00\x00\x02'>>>classTestObject(Object):...bar:tuple[str,Optional[str],float,int]...>>>cain.dumps(['foo',{'bar':('baz',None,1.0,2)}],list[str,TestObject])b'\x00foo\x00\x00\x00baz\x00\x00\x00\x00\x80?\x00\x02'>>>print(cain.dumps("\"foo\bar",str))b'"foo\x08ar\x00'>>>print(cain.dumps('\u1234',str))b'\xe1\x88\xb4\x00'>>>print(cain.dumps('\\',str))b'\\\x00'>>>schema=list[str,Object[{"bar":tuple[str,Optional[str],float,int]}]]>>>withopen('test.cain','w+b')asfp:...cain.dump(['foo',{'bar':('baz',None,1.0,2)}],fp,schema)...>>>fromcain.typesimportInt>>>fromcain.types.numbersimportunsigned>>>Int[unsigned].encode(4)b'\x00\x04'You can also add a header using theinclude_headerparameter to add a header containing the schema for the encoding data. This gives a more portable output but increases its size.DecodingDecoding Cain:>>>importcain>>>fromcain.typesimportOptional,Object>>>schema=list[str,Object[{"bar":tuple[str,Optional[str],float,int]}]]>>>cain.loads(b'\x00foo\x00\x00\x00baz\x00\x00\x00\x00\x80?\x00\x02',schema)['foo',{'bar':('baz',None,1.0,2)}]>>>withopen('test.cain','r+b')asfp:...cain.load(fp,schema)...['foo',{'bar':('baz',None,1.0,2)}]>>>fromcain.typesimportInt>>>fromcain.types.numbersimportunsigned>>>Int[unsigned].decode(b'\x00\x04')4Handling SchemasIf you want to dynamically encode/decode data with the Cain format, it is also possible to encode/decode the schema.This is especially useful when developing a public API for example.Encoding>>>importcain>>>fromcain.typesimportObject,Optional>>>cain.encode_schema(Object[{"a":int}])b'\x00\x00\x01\x00\x00a\x00\x00\x01\x00\x00\x01\x03\x00\x01\x02\x00\x00\x00\x00\x06\x00\x00\x00\x00\x16'>>>classTestObject(Object):...bar:tuple[str,Optional[str],float,int]...>>>cain.encode_schema(list[str,TestObject])b'\x01\x02\x00\x01\x00\x00\x00\x00\x00\x02\x00\x00...\x00\x16\x01\x00TestObject\x00\x00\x00'Decoding>>>importcain>>>cain.decode_schema(b'\x00\x00\x01\x00\x00a\x00\x00\x01\x00\x00\x01\x03\x00\x01\x02\x00\x00\x00\x00\x06\x00\x00\x00\x00\x00\x16\x00')Object<{'a':Int}>>>>cain.decode_schema(b'\x01\x02\x00\x01\x00\x00\x00\x00\x00\x02\x00\x00...\x00\x16\x01\x00TestObject\x00\x00\x00')Array[String,TestObject]Custom EncoderYou can also create your own encoders:>>>importtyping>>>fromcain.modelimportDatatype>>>classMyObject(Datatype):...@classmethod# *args contains the args passed here : MyObject[args]...def_encode(cls,value:typing.Any,*args)->bytes:......#  your custom encoding...returnb'encoded data'...#...@classmethod...def_decode(cls,value:bytes,*args)->typing.Tuple[typing.Any,bytes]:......#  `value` contains more than just the value you should decode......#  try to only decode the first few bytes......#  your custom decoding...return'decoded data',value# the rest of the value that you didn't decode...# you can now use `MyObject` in your schemas and encode/decode from itWarningKeep in mind that custom datatypes outside of subclasses ofObjectwon't be able to be encoded by the Type encoder (used in schema headers for example)CLICain has a pretty complete command-line interface, which lets you manipulate and interact with the Cain data format easily.For more information, head over to your console and enter:cain--helpOrcain<action>--helpExamplesExample usage of the CLIPreparing the schema:# test.pyfromcainimportObjectclassTest(Object):username:strfavorite_number:intTrying to encode with a Python schema:cainencode'{"username": "Anise", "favorite_number": 2}'--schema="test.py"--schema-name="Test"--include-header--output="test.cain"Trying to decode the previous file:$caindecodetest.cain{"favorite_number":2,"username":"Anise"}Looking up at its schema:$cainschemalookuptest.cain--schema-header{"index":22,"name":"Test","annotations_keys":["username","favorite_number"],"annotations_values":[{"index":26,"name":null,"annotations_keys":[],"annotations_values":[],"arguments":[],"datatype":"String"},{"index":6,"name":null,"annotations_keys":[],"annotations_values":[],"arguments":[],"datatype":"Int"}],"arguments":[],"datatype":"Object"}Exporting its schema:cainschemaexporttest.cain--schema-header--outputtest.cainschemaTrying to encode another object with the exported schema:$cainencode'{"username": "yay", "favorite_number": 3}'--schema=test.cainschema\x00\x00\x03yay\x00Encoding "Hello world":$cainencode'"Hello world"'--schema="str"--schema-eval
Helloworld\x00$cainencode'["Hello", "world"]'--schema="list[str]"--schema-eval\x00\x02\x00\x00Hello\x00world\x00DeploymentThis module is currently in development and might contain bugs.This comes with a few disadvantages (for example, it takes a longer time to encode objects with Cain than with the standardjsonmodule) but this is expected to improve over time.Please verify and test the module thoroughly before releasing anything at a production stage.Feel free to report any issue you might encounter on Cain's GitHub page.ContributingPull requests are welcome. For major changes, please open a discussion first to discuss what you would like to change.Please make sure to update the tests accordingly.AuthorsAnimenosekai-Initial work-AnimenosekaiLicensingThis software is licensed under the MIT License. See theLICENSEfile for more information.