Package: basest
basest-pythonArbitrary base binary-to-text encoder (any base to any base), in Python.What?In short,basestisbased on(pundefinitelyintended :wink:) the concept of binary-to-text conversion, that is where binary or 8-bit data is converted or serialised into a text-based representation format that can be safely passed through a medium that would otherwise destroy or corrupt the meaning of the binary data.This concept is very commonly used in areas such as Email, the PDF format and Public Key Cryptography, to name but a few.There are many different formats and schemes for serilising binary data to text, employing different alphabet sizes and different printable ASCII characters used for various different reasons.It is also not just 8-bit binary data that could be serialised. Any collection of symbols declared to be in a given number base or alphabet size can be serialised into any other, provided an encoding ratio between the two symbols can be established and the input and output symbols defined.This library is my implementation of a generic, base-to-base converter which addresses this last point. An encoder and decoder for every binary-to-text format currently existing can be created and used with this library, requiring only for the details of the desired format to be given. Due to its flexibility, the library also makes it trivial to invent new wonderful and interesting base-to-base serialisation/conversion formats (I myself plan to work on and release one that translates binary files into a purely emoji-based format!).One limitation of the library is that it cannot encode data from a smaller input base to a larger output base with padding on the input (i.e. if you're encoding from base 2 to base 1000, you need to ensure that the number of input symbols exactly matches the encoding ratio you're using). This is an accepted limitation due to the complexities of implementing a padding system that works in the same manner as base-64 and others but which can be extended to any arbitrary base.So, I hope you find this library fun, useful or both!InstallationPython Versions SupportedThis library is designed to work withPython 2.7andPython 3.3or greater.It is tested against Python2.7,3.5,3.6andPyPy 3.5.:bulb:Help WantedIf you have tried or want to try this out on any other Python implementations, your feedback would be greatly appreciated!Open an issueif you are interested.Install from PyPi (preferred)pip install basestInstall from git repository (bleeding edge copy from develop branch)pip install git+git://github.com/saxbophone/basest-python@developUsageHere is a short overview of the interfaces defined in this library, where to import them from and how to use them.There is a functional interface and a class-based interface (the class-based one piggy-backs on the functional one, but will also be used to add additional features in the future).Class-based InterfaceTo use the class-based interface, you will need to create a subclass ofbasest.encoders.Encoderand override attributes of the class, as shown below (using base64 as an example):frombasest.encodersimportEncoderclassCustomEncoder(Encoder):input_base=256output_base=64input_ratio=3output_ratio=4# these attributes are only required if using decode() and encode()input_symbol_table=[chr(c)forcinrange(256)]output_symbol_table=[sforsin'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/']padding_symbol='='Note:You must subclassEncoder, you cannot use it directly!Subclasses ofEncoderhave the following public methods available:Encode from one base to anotherencode()will encode an iterable of symbols in the class'input symbol tableinto an iterable of symbols in the class'output symbol table, observing the chosen encoding ratios and padding symbol.encoder=CustomEncoder()encoder.encode(['c','a','b','b','a','g','e','s'])# -> ['Y', '2', 'F', 'i', 'Y', 'm', 'F', 'n', 'Z', 'X', 'M', '=']Encode Rawencode_raw()works just likeencode(), except that symbols are not interpreted. Instead, plain integers within range 0->(base - 1) should be used. the value of the base is used as the padding symbol.encoder=CustomEncoder()encoder.encode_raw([1,2,3,4,5,6,7])# -> [0, 16, 8, 3, 1, 0, 20, 6, 1, 48, 64, 64]Decode from one base to anotherdecode()works in the exact same way asencode(), but in the inverse.encoder=CustomEncoder()encoder.decode(['Y','2','F','i','Y','m','F','n','Z','X','M','='])# -> ['c', 'a', 'b', 'b', 'a', 'g', 'e', 's']Decode Rawdecode_raw()works just likedecode(), except that symbols are not interpreted. Instead, plain integers within range 0->(base - 1) should be used. the value of the base is used as the padding symbol.encoder=CustomEncoder()encoder.decode_raw([0,16,8,3,1,0,20,6,1,48,64,64])# -> [1, 2, 3, 4, 5, 6, 7]Functional InterfaceEncode from one base to another (where the encoding ratios to use are known)For a giveninput base,input symbol table,output base,output symbol table,output padding,input ratio,output ratioand theinput data(as an iterable composed of items which are defined ininput symbol table):
Return the input data, encoded into the specified base using the specified encoding ratio and symbol tables (and the suppliedoutput paddingsymbol used if needed).
Returns the output data as a list of items that are guaranteed to be in theoutput symbol table, or theoutput paddingsymbol.importbasestbasest.core.encode(input_base=256,input_symbol_table=[chr(c)forcinrange(256)],output_base=64,output_symbol_table=[sforsin'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'],output_padding='=',input_ratio=3,output_ratio=4,input_data='falafel')# -> ['Z', 'm', 'F', 's', 'Y', 'W', 'Z', 'l', 'b', 'A', '=', '=']Encode RawSimilar to the function above,basest.core.encode_rawwill encode one base into another, but only accepts and returns arrays of integers (e.g. bytes would be passed as integers between 0-255, not asbyteobjects). As such, it omits thepaddingandsymbol tablearguments, but is otherwise identical in function and form toencode.importbasestbasest.core.encode_raw(input_base=256,output_base=85,input_ratio=4,output_ratio=5,input_data=[99,97,98,98,97,103,101,115])# -> [31, 79, 81, 71, 52, 31, 25, 82, 13, 76]Decode from one encoded base to another.For a giveninput base,input symbol table,input padding,output base,output symbol table,input ratio,output ratioand theinput data(as an iterable composed of items which are defined ininput symbol table), return the input data, decoded from the base it was encoded into.
Returns the output data as a list of items that are guaranteed to be in theoutput symbol table, with no padding.This is essentially the inverse ofencode()importbasestbasest.core.decode(input_base=64,input_symbol_table=[sforsin'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'],input_padding='=',output_base=256,output_symbol_table=[chr(c)forcinrange(256)],input_ratio=4,output_ratio=3,input_data='YWJhY3VzIFpaWg==')# -> ['a', 'b', 'a', 'c', 'u', 's', ' ', 'Z', 'Z', 'Z']Decode RawSimilar to the function above,basest.core.decode_rawwill decode from one base to another, but only accepts and returns arrays of integers (e.g. base64 would be passed as integers between 0-65 (65 is for the padding symbol), not asstrobjects). As such, it omits thepaddingandsymbol tablearguments, but is otherwise identical in function and form todecode.importbasestbasest.core.decode_raw(input_base=85,output_base=256,input_ratio=5,output_ratio=4,input_data=[31,79,81,71,52,31,25,82,13,76])# -> [99, 97, 98, 98, 97, 103, 101, 115]Finding the best encoding ratio from one base to any base within a given rangeFor a giveninput base(e.g. base-256 / 8-bit Bytes), a given desiredoutput base(e.g. base 94)ORa given range of acceptableoutput basesand a range ofchunk sizesto consider using for the input (amount of bytes/symbols processed at once), return the most efficient output base and encoding ratio to use (in terms of input base to output base).Returns tuples containing an integer as the first item (representing the output base that is most efficient) and a tuple as the second, containing two integers representing the ratio ofinput basesymbols tooutput basesymbols.importbasestbasest.core.best_ratio(input_base=256,output_bases=[94],chunk_sizes=range(1,256))# -> (94, (68, 83))basest.core.best_ratio(input_base=256,output_bases=[94],chunk_sizes=range(1,512))# -> (94, (458, 559))basest.core.best_ratio(input_base=256,output_bases=range(2,95),chunk_sizes=range(1,256))# -> (94, (68, 83))basest.core.best_ratio(input_base=256,output_bases=range(2,334),chunk_sizes=range(1,256))# -> (333, (243, 232))