Package: tally-token
python-tally-tokenWhat is this?tally-token is a Python library for split data into tokens with same length.Tallyis a historical object for prove something by splitting wood into tokens and matching tokens.Medieval English split tally stick (front and reverse view). The stick is notched and inscribed to record a debt owed to the rural dean of Preston Candover, Hampshire, of a tithe of 20d each on 32 sheep, amounting to a total sum of £2 13s. 4d.UsageInstall$pipinstalltally-tokenCLI Usage$tally-token--help
usage:tally-token[-h]{split,merge}...

positionalarguments:{split,merge}Commands:split:splitafileintomultiplefilesmerge:mergemultiplefilesintoafileExample:tally-tokensplitexample.binexample.bin.1example.bin.2example.bin.3tally-tokenmergeexample-merged.binexample.bin.1example.bin.2example.bin.3

options:-h,--helpshowthishelpmessageandexitsplitYou can usesplitto split a file into multiple files.$tally-tokensplitsomething.binsplit-1.binsplit-2.binsplit-3.binmergeYou can usemergeto merge multiple files into a file.$tally-tokenmergemerged.binsplit-1.binsplit-2.binsplit-3.binLarge filesNothing special. You can split and merge large file.$ddif=/dev/urandomof=original.1g.binbs=1Gcount=1$tally-tokensplitoriginal.1g.binsplit-1.binsplit-2.binsplit-3.bin
$shasum-a256original.1g.bin
>736a344d99d27e2dcdab8bc37ca94c83eda26f812a3dee87ac98989f89b3f965original.1g.bin
$tally-tokenmergerecovery.1g.binsplit-1.binsplit-2.binsplit-3.bin
$shasum-a256recovery.1g.bin
>736a344d99d27e2dcdab8bc37ca94c83eda26f812a3dee87ac98989f89b3f965recovery.1g.binExamplesplitYou can usesplit_textto split text into tokens.split_textreturns list of random bytes.>>>fromtally_tokenimportsplit_text>>>split_text("Hello, World!")[b'qQ\xa5\x97\x84\x88\xd7U%\xfb(k\xa1',b'94\xc9\xfb\xeb\xa4\xf7\x02J\x89D\x0f\x80']mergeYou can usemerge_textto merge tokens into text.merge_textreturns cleartext.>>>fromtally_tokenimportmerge_text>>>merge_text([b'qQ\xa5\x97\x84\x88\xd7U%\xfb(k\xa1',b'94\xc9\xfb\xeb\xa4\xf7\x02J\x89D\x0f\x80'])'Hello, World!'split with custom length>>>fromtally_tokenimportsplit_text,merge_text>>>split_text("Hello, World!",5)[b'N&\xce\\\xbc6dxp\x87\xa8#z',b'\xa3D\\A\xf8\xd1KDX\x1cKx\x87',b'\xffZ\x03\xf5\x92Q\xf52\xc4\x1e\xf2\xf8\x06',b'\xaa\xdd:\x85F\xa1\xcdbp\xf3\xe6P\xe5',b'\xf0\x80\xc7\x01\xff;7;\xf3\x04\x9b\x97?']>>>merge_text([b'N&\xce\\\xbc6dxp\x87\xa8#z',b'\xa3D\\A\xf8\xd1KDX\x1cKx\x87',b'\xffZ\x03\xf5\x92Q\xf52\xc4\x1e\xf2\xf8\x06',b'\xaa\xdd:\x85F\xa1\xcdbp\xf3\xe6P\xe5',b'\xf0\x80\xc7\x01\xff;7;\xf3\x04\x9b\x97?'])'Hello, World!'split with custom encoding>>>fromtally_tokenimportsplit_text,merge_text>>>split_text("こんにちは",encoding="CP932")[b'g\xc3\x12\xeal?\xe5[\x03\xad',b'\xe5r\x90\x1b\xee\xf6g\xe4\x81`']>>>merge_text([b'g\xc3\x12\xeal?\xe5[\x03\xad',b'\xe5r\x90\x1b\xee\xf6g\xe4\x81`'],encoding="CP932")'こんにちは'bytes interfaceYou can usesplit_bytes_intoandmerge_bytes_intoto split and merge bytes.
This is useful for split binary data.>>>fromtally_tokenimportsplit_bytes_into,merge_bytes_into>>>split_bytes_into(b"Hello, World!",5)[b'\xc5b\xf4E)\xe1vO8\xff@\xf9\xdd',b'\x84\xb9X#\x85\xf5\xed\xbcM\xc4\xef\xf4\xd3',b'\xb47\xf6\xfa?\x14\xa8`\xc9\xe0\xe5\x87\x14',b'\x1cd\xb4o\xe8I:\xe5\xf6\x13\xe5\x93G',b'\xa1\xed\x82\x9f\x14e)!%\xba\xc3}|']>>>merge_bytes_into([b'\xc5b\xf4E)\xe1vO8\xff@\xf9\xdd',b'\x84\xb9X#\x85\xf5\xed\xbcM\xc4\xef\xf4\xd3',b'\xb47\xf6\xfa?\x14\xa8`\xc9\xe0\xe5\x87\x14',b'\x1cd\xb4o\xe8I:\xe5\xf6\x13\xe5\x93G',b'\xa1\xed\x82\x9f\x14e)!%\xba\xc3}|'])b'Hello, World!'ReferenceTally stickhttps://en.wikipedia.org/wiki/Tally_stickTessera or Symbolum (Hospitium token)https://en.wikipedia.org/wiki/Hospitium割符https://ja.wikipedia.org/wiki/%E5%89%B2%E7%AC%A6One-time padhttps://en.wikipedia.org/wiki/One-time_padLICENSEBSD 3-Clause License