Package: merkleasy
Merkle StructuresThis is a simple-to-use implementation of the concept of Merklized data
structures, e.g. the Merkle Tree and the Merkle Mountain Range. There is a
single main class,merkleasy.Tree, three error classes, and two configuration
functions. See the Usage section for details. This uses sha256 as the default
hash algorithm, but it can use any in theory.StatusTestsImplementationProofsUsage DocumentationPublish to pypiInstallationpipinstallmerkleasyTestingTo develop or test, fork or clone the repo and install the dependency.Windows Setuppython-mvenvvenv/sourcevenv/Scripts/activate
pipinstallpycelium-specifications*nix Setuppython-mvenvvenv/sourcevenv/bin/activate
pipinstallpycelium-specificationsRunning TestsThere are two test files: the first is a unit test suite; the second is a
specification compliance test. Run them with the following:pythontests/test_classes.py
pythontests/test_specification.pyThese files demonstrate all the intended behaviors of the class and rule out
many unintended behaviors. They userandintand many repetitions to ensure
that the test suite is thorough. The tests are also a form of technical
documentation; any questions about the code can likely be answered by reading
through them.ClassesImplementationError(BaseException)UsagePreconditionError(BaseException)SecurityError(BaseException)TreeThe Usage section describes for each method/function which (if any) of these
errors it can raise.Functions and Methodsset_hash_function(hash_function: Callable[[bytes], bytes]) -> Noneget_hash_function() -> CallableTree__init__(self, left: Tree | bytes, right: Tree | bytes) -> None__str__(self) -> str__repr__(self) -> str__eq__(self, other: object) -> bool__hash__(self) -> intto_dict(self) -> dictto_json(self, pretty: bool = False) -> str@classmethod from_leaves(cls, leaves: list[bytes]) -> Tree@classmethod from_dict(cls, data: dict) -> Tree@classmethod from_json(cls, data: str) -> Treeprove(self, leaf: bytes, verbose: bool = False) -> list[bytes]@staticmethod verify(root: bytes, leaf: bytes, proof: list[bytes]) -> NoneUsageUsage examples are shown below.Tree.from_leavesThe easiest way to use this to create a Merkle Tree is withfrom_leaves:frommerkleasyimportTreeleaves=[b'leaf1',b'leaf2',b'leaf3',b'leaf4',b'etc']tree=Tree.from_leaves(leaves)Note that all leaves are hashed by thefrom_leavesmethod.RaisesUsagePreconditionErrorupon invalid input.Tree.__init__To make custom Merklized data structures, use the__init__method:fromhashlibimportsha256frommerkleasyimportTreeleaf1=sha256(b'leaf1').digest()leaf2=sha256(b'leaf2').digest()leaf3=sha256(b'leaf3').digest()leaf4=sha256(b'leaf4').digest()leaf5=sha256(b'leaf5').digest()another_whole_tree=Tree.from_leaves([b'123',b'456',b'789'])tree=Tree(Tree(leaf1,Tree(Tree(leaf2,leaf3),Tree(leaf4,leaf5))),another_whole_tree)RaisesUsagePreconditionErrorupon invalid input.Tree.to_dictandTree.from_dictA Tree structure can be converted to a dict and back.frommerkleasyimportTreetree=Tree.from_leaves([b'leaf1',b'leaf2',b'leaf3'])serialized=tree.to_dict()deserialized=Tree.from_dict(serialized)assertdeserialized==treeTree.from_dictraises any of the following upon invalid input:UsagePreconditionErrorValueErrorSecurityErrorTree.to_jsonandTree.from_jsonSerialization and deserialization of a structure usesto_jsonandfrom_json:frommerkleasyimportTreetree=Tree.from_leaves([b'leaf1',b'leaf2',b'leaf3'])serialized=tree.to_json()pretty=tree.to_json(pretty=True)deserialized=Tree.from_json(serialized)assertdeserialized==Tree.from_json(pretty)Tree.from_jsonraises any of the following upon invalid input:json.decoder.JSONDecodeErrorUsagePreconditionErrorValueErrorSecurityErrorTree.proveInclusion proofs can be generated using theprovemethod:frommerkleasyimportTreetree=Tree.from_leaves([b'leaf1',b'leaf2',b'leaf3'])proof=tree.prove(b'leaf2')Each inclusion proof is a list of bytes, where the first byte in each item in
the list is one of the codes frominterfaces.ProofOp. An optional parameter,verbose, can be set toTruein the call toproveif the proof should
include checks for intermediate values; ifverboseis left to the defaultFalsevalue, a shorter proof that checks only the final hash will be produced.
There are no security advantages to using verbose proofs; it is primarily useful
for manual inspection by including intermediate, calculated values. However, the
functionality exists as a side-effect of preventing malicious proofs from
tricking the validator --test_Tree_verify_does_not_validate_malicious_proofcontains an example attack.RaisesUsagePreconditionErrorupon invalid input.Tree.verifyInclusion proofs can be verified usingTree.verify:frommerkleasyimportTree,UsagePreconditionError,SecurityErrortree=Tree.from_leaves([b'leaf1',b'leaf2',b'leaf3'])leaf=b'leaf1'proof1=tree.prove(leaf)proof2=tree.prove(b'leaf2')try:Tree.verify(tree.root,leaf,proof1)# expected resultprint(f'verified proof{[p.hex()forpinproof1]}for{leaf=}')exceptUsagePreconditionErrorase:print(f'invalid use of library:{e}')exceptValueErrorase:print(f'invalid proof supplied:{e}')exceptSecurityErrorase:print(f'error encountered:{e}')try:Tree.verify('some string',leaf,proof1)print(f'verified proof{[p.hex()forpinproof1]}for{leaf=}')exceptUsagePreconditionErrorase:# expected resultprint(f'invalid use of library:{e}')exceptValueErrorase:print(f'invalid proof supplied:{e}')exceptSecurityErrorase:print(f'error encountered:{e}')try:Tree.verify(tree.root,leaf,[b'\x99',*proof2])print(f'verified proof{[p.hex()forpinproof2]}for{leaf=}')exceptUsagePreconditionErrorase:print(f'invalid use of library:{e}')exceptValueErrorase:# expected resultprint(f'invalid proof supplied:{e}')exceptSecurityErrorase:print(f'error encountered:{e}')try:Tree.verify(tree.root,leaf,proof2)print(f'verified proof{[p.hex()forpinproof2]}for{leaf=}')exceptUsagePreconditionErrorase:print(f'invalid use of library:{e}')exceptValueErrorase:print(f'invalid proof supplied:{e}')exceptSecurityErrorase:# expected resultprint(f'error encountered:{e}')This static method parses the proof, interpreting the first byte in each proof
step as a code frominterfaces.ProofOp. It ensures that the proof starts with
the leaf and ends with the root, and then it follows the proof operations.RaisesUsagePreconditionErrororValueErrorwhen provided invalid parameters.
RaisesSecurityErrorwhen provided an invalid proof. If all checks pass, it
executes without error and returnsNone.get_hash_functionTo access the currently-set hash function, use the following:frommerkleasyimportget_hash_functionhash_function=get_hash_function()print(hash_function(b'abc').hex())# ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015adset_hash_functionThe package uses sha256 by default, but it can be used with any hash function.
This is accomplished by passing a Callable that takes a bytes parameter, applies
a hash algorithm, and returns a bytes value. For example, to use sha3_256:fromhashlibimportsha3_256frommerkleasyimportset_hash_function,get_hash_functionset_hash_function(lambdapreimage:sha3_256(preimage).digest())print(get_hash_function()(b'abc'))# 3a985da74fe225b2045c172d6bd390bd855f086e3e9d525b46bfe24511431532RaisesImplementationErrorif the Callable parameter passed in does not meet
the requirements.Note that callingset_hash_functionwill have no effect on anyTrees created
prior. However, itwillaffect any calls toTree.verifywith proofs from
thoseTrees. If you plan to use the library with a custom hash function, thenset_hash_functionshould be called during a setup routine.If you want to handle multipleTrees created with different hash algorithms,
then a context handler like the below might be useful:fromhashlibimportsha3_256frommerkleasyimportset_hash_function,get_hash_function,TreeclassHashAlgoSwitch:"""Context manager for switching out algorithms for Tree use."""def__init__(self,new_hash_function)->None:self.original_hash_function=get_hash_function()set_hash_function(new_hash_function)def__enter__(self)->None:returndef__exit__(self,__exc_type,__exc_value,__traceback)->None:set_hash_function(self.original_hash_function)defalt_create_tree(leaves)->Tree:withHashAlgoSwitch(lambdadata:sha3_256(data).digest()):returnTree.from_leaves(leaves)leaves=[b'one',b'two',b'three']tree1=Tree.from_leaves(leaves)tree2=alt_create_tree(leaves)asserttree1!=tree2Security / Usage NoteAny application/library that uses this package should use a schema for leaves
that is anything except exactly 32 bytes. This prevents the second-preimage
attack whereby the application is tricked into thinking that an intermediate
node in the tree is a leaf. It is hard to envision a scenario in which this
could actually become a serious security issue, but it is worth keeping in mind
during application development.So in addition to verifying an inclusion proof, verify that the data fits the
leaf schema. Preferably, leaf schema should not be bytes, and a serializer to
bytes from the schema should be used on the leaf before verifying the inclusion
proof.ISC LicenseCopyleft (c) 2023 k98kurzPermission to use, copy, modify, and/or distribute this software
for any purpose with or without fee is hereby granted, provided
that the above copyleft notice and this permission notice appear in
all copies.Exceptions: this permission is not granted to Alphabet/Google, Amazon,
Apple, Microsoft, Netflix, Meta/Facebook, Twitter, or Disney; nor is
permission granted to any company that contracts to supply weapons or
logistics to any national military; nor is permission granted to any
national government or governmental agency; nor is permission granted to
any employees, associates, or affiliates of these designated entities.THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE
AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR
CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.