Change:
This PR reorganizes the exposition of tree concept to make them read more clearly. A new Paths through the Ratchet Tree section is introduced, which covers direct paths, copaths, and filtered direct paths The Ratchet Tree Nodes section is moved forward, so that the definition of filtered direct paths can refer to the notion of resolutions (see NAME note on ) The path example expanded so that there are blanks in the tree, and so that it shows all three path types (direct, copath, and filtered direct) for every leaf. The section on verifying parent hashes uses the example tree from the path section to illustrate the concepts noted in
Latest commit adds coverage for the last part of . In addition to moving the figure forward, it refactors the explanation to be even more explicit about what must be consumed and what may be kept.
Currently defined only briefly in "RatchetTree Evolution". While that's where it's primarily used, it would fit better in the terminology section, especially if we expand it a little and add an illustration.
The problem here is that the definition of "filtered direct path" depends on the definition of "resolution" which isn't defined until a later section.
Indeed, so proposes moving up the definition of resolution as well.
Add illustration of filtered direct path Add illustration of parent hash validation, including: Leaves with no parent hash Nodes with invalid parent hash Skipping blank nodes Add corresponding derivations to Appendix A Move figure in Section 10.2 up to Section 10 as an overview
NAME Note that I have taken a stab at parts of this in because it came along with some other tree stuff.
Duplicate
Regarding the Tree Hash Including the parentnode in the tree hash which is supposed to be computed from the leaves, makes the definition appear circular to a new reader. Since the situation when you have blank nodes in the middle of a tree but not at the root is a rather complicated, the situation when you could calculate a tree hash using the parentnode is non-obvious. In the description of Tree Hash the document says that the ParentNode object is optional "depending on whether the node is blank or not." This isn't clear specification language. Saying the parentnode object is present if the node is blank or present if the node is not blank would be clear. What if the parent node is also blank? What if the root is blank? I think an example with and without blank nodes would be appropriate. Explaining the reason why the parentnode is included will help the reader establish a better mental model. Regarding the Parent Hash I still find this section super confusing. A few points Using a tree hash (siblingtreehash) inside the parent hash is non-obvious. There should be some explanation about what property this is providing and that it is a tree hash and not a parent hash. (Honestly it was the third time I read the section that I realized that it was really the tree hash and not the parent hash of the sibling.) Using P, V, and S for Parent, target node, and sibling node is introduced here and never used anywhere else in the document. P for parent and S for sibling make sense. I can't tell what V is supposed to stand for, and it unfortunately could cause people to confuse the vector notation used throughout, but shortly after introducing V for the target node. I propose using "N" for node or "X" instead. There is one reference to the filtered direct path of U. Perhaps this is V, or a child of V? The calculations are sufficiently complicated that I think this warrants a complete, non-trivial, worked example (possibly in an appendix). At some point we should consider providing test vectors in a future document, and this would be an ideal candidate.
Regarding the Tree Hash To be clear, the that is included in the tree hash is the node whose subtree is being hashed. The tree hash doesn't include only the leaves, it includes all the nodes in the tree. If you mean , then yes, I agree the connection is a little subtle. I've tried to explain this a little better. Updated this to be clear that the ParentNode is omitted if and only if the node is blank. Not clear what situation you mean here. I think the specific computations are pretty clearly specified for these cases. Something like this was added in : URL I extended the example and put it in an appendix. Assuming you mean here, hopefully this is addressed by the above notes. Added a note to clarify that this is the tree hash and why. Changed to "D" for "direct path", and explained the semantics. Actually it's the leaf, which is now called "L". I think the example linked above does a pretty good job. Feedback obviously welcome. Agreed that a test vectors document will be a useful follow-on.
In order to use tree hashes to represent siblings in parent hashes, we had to compute a different tree hash on the right edge of the tree, pretending that the tree is full, in the sense of having leaves. Thus we actually have two tree hashes in play, one used for group agreement on the tree and one used for parent hash inputs. We should consider specifying that the logical tree used by MLS is always full. This change to the specification would not change the computations done by the protocol much, if at all. Thanks to the fact that we no longer generate redundant nodes (), there is never anything non-blank to the right of the rightmost non-blank leaf anyway. So the main impact is to insert some blank nodes between a parent node on the right edge of the tree and its right child -- and all of the algorithms in the specification already skip over such nodes. Likewise, implementations' internal represntations of the tree wouldn't have to change much, since the extra nodes could be "virtual". The only time they would be noticed is when computing tree hashes. We would have to update the description of the extension to say that it only covers the tree up to the rightmost non-blank leaf. It would simplify the specification in a few ways, for example: The "original sibling tree hash" would just entail blanking unmerged leaves, not structure changes. The description of extending and truncating the tree would be much simpler. The main cost would be extra hashes when joining, or when updating a member close to the right edge of the tree. If you have a tree with leaves, then you have to compute tree hashes over blank leaves and their blank parents.
Thanks to NAME and NAME for helping inspire and develop this idea in discussions around
Virtual interim 2022-05-19 NAME notes that the current tree hash is quite tricky to implement, so this would be a useful simplification Doubling the hashes on a fresh tree doesn't sound too intimidating Folks would find more concreteness useful here, interested in PR Could we phrase this as only a change to TreeKEM? Would be premised on the equivalence of the hashed tree an the tree you operate on
Discussed at 19 May interim: NAME to draft a PR.
Holding a PR until after lands. Some notes on what would need to change: {{ratchet-tree-terminology}} would change from "the unique left-balanced tree with N leaves" to "the unique balanced binary tree with depth , where is the smallest number such that ". {{adding-and-removing-leaves}} would either be removed, or updated so that it talks about increasing or decreasing the depth of the tree (equivalently, doubling or halving the number of leaves). {{add}} would talk about increasing depth instead of adding leaves. {{remove}} would talk about decreasing depth instead of removing leaves. {{ratchet-tree-extension}} would be explicit that it only carries nodes up to the rightmost non-blank leaf. The encoding/decoding algorithms would have to be updated. {{array-based-trees}} and {{link-based-trees}} could probably be simplified. We would need to make sure all figures showed full trees.
I went ahead and filed for this, based on top of . Once that merges, we can update the PR to be a little easier to read. The changes are basically as described above. One additional note: We have actually already incurred the cost to a new joiner of hashing all the extra nodes, because they have to do it for parent hash verification anyway. So this approach is actually a net reduction in the amount of hashing that has to be done, since they don't have to also compute the non-full-tree hashes.
In , we introduced the idea of a "filtered direct path". It would be helpful to have an illustration of how a filtered direct path differs from a direct path.

Discussion:
used for the Merkle trees in the Certificate Transparency protocol I- D.ietf-trans-rfc6962-bis.) <del> The _direct path_ of a root is the empty list, and of any other node is the concatenation of that node's parent along with the parent's direct path.  The _copath_ of a node is the node's sibling concatenated with the list of siblings of all the nodes in its direct path, excluding the root. For example, in the below tree: The direct path of C is (W, V, X) The copath of C is (D, U, Z) A tree with "n" leaves has "2*n - 1" nodes.  For example, the above tree has 7 leaves (A, B, C, D, E, F, G) and 13 nodes. Each leaf is given an _index_ (or _leaf index_), starting at "0" from the left to "n-1" at the right. Finally, a node in the tree may also be _blank_, indicating that no value is present at that node (i.e. no keying material).  This is often the case when a leaf was recently removed from the tree. </del> There are multiple ways that an implementation might represent a ratchet tree in memory.  For example, left-balanced binary trees can be represented as an array of nodes, with node relationships computed

and associated algorithms, as long as they produce correct protocol messages. <ins> 5.1.1. Each leaf node in a ratchet tree is given an _index_ (or _leaf index_), starting at "0" from the left to "n-1" at the right (for a tree with "n" leaves).  A tree with "n" leaves has "2*n - 1" nodes, including parent nodes. Each node in a ratchet tree is either _blank_ (containing no value) or it holds an asymmetric key pair with some associated data: A public key (for the HPKE scheme in use, see ciphersuites) A private key (only within the member's direct path, see views) A credential (only for leaf nodes) An ordered list of "unmerged" leaves (see views) A hash of certain information about the node's parent, as of the last time the node was changed (see parent-hash). Every node, regardless of whether the node is blank or populated, has a corresponding _hash_ that summarizes the contents of the subtree below that node.  The rules for computing these hashes are described in tree-hashes. The _resolution_ of a node is an ordered list of non-blank nodes that collectively cover all non-blank descendants of the node.  The resolution of a node is effectively a depth-first, left-first enumeration of the nearest non-blank nodes below the node: The resolution of a non-blank node comprises the node itself, followed by its list of unmerged leaves, if any The resolution of a blank leaf node is the empty list The resolution of a blank intermediate node is the result of concatenating the resolution of its left child with the resolution of its right child, in that order For example, consider the following subtree, where the "_" character represents a blank node and unmerged leaves are indicated in square brackets: In this tree, we can see all of the above rules in play: The resolution of node Z is the list [Z, C] The resolution of leaf 1 is the empty list [] The resolution of top node is the list [A, Z, C] 5.1.2. The _direct path_ of a root is the empty list, and of any other node is the concatenation of that node's parent along with the parent's direct path. The _copath_ of a node is the node's sibling concatenated with the list of siblings of all the nodes in its direct path, excluding the root. The _filtered direct path_ of a leaf node L is the node's direct path, with any node removed whose child on the copath of L has an empty resolution (keeping in mind that any unmerged leaves of the copath child count toward its resolution).  The removed nodes do not need their own key pairs because encrypting to the nodes key pair would be equivalent to encrypting to its non-copath child. For example, consider the following tree (where blank nodes are indicated with "_", but also assigned a label for reference): In this tree, the direct paths, copaths, and filtered direct paths for the leaf nodes are as follows: </ins> 5.2. We generally assume that each participant maintains a complete and

Note how the tree invariant applies: Each member knows only their own leaf, and the private key AB is known only to A and B. <del> 5.3. A particular instance of a ratchet tree includes the same parameters that define an instance of HPKE, namely: A Key Encapsulation Mechanism (KEM), including a "DeriveKeyPair" function that creates a key pair for the KEM from a symmetric secret A Key Derivation Function (KDF), including "Extract" and "Expand" functions An AEAD encryption scheme Each non-blank node in a ratchet tree contains up to five values: A public key A private key (only within the member's direct path, see below) A credential (only for leaf nodes) An ordered list of "unmerged" leaves (see views) A hash of certain information about the node's parent, as of the last time the node was changed (see parent-hash). The _resolution_ of a node is an ordered list of non-blank nodes that collectively cover all non-blank descendants of the node.  The resolution of a non-blank node with no unmerged leaves is just the node itself.  More generally, the resolution of a node is effectively a depth-first, left-first enumeration of the nearest non-blank nodes below the node: The resolution of a non-blank node comprises the node itself, followed by its list of unmerged leaves, if any The resolution of a blank leaf node is the empty list The resolution of a blank intermediate node is the result of concatenating the resolution of its left child with the resolution of its right child, in that order For example, consider the following subtree, where the "_" character represents a blank node and unmerged leaves are indicated in square brackets: In this tree, we can see all of the above rules in play: The resolution of node Z is the list [Z, C] The resolution of leaf 1 is the empty list [] The resolution of top node is the list [A, Z, C] Every node, regardless of whether the node is blank or populated, has a corresponding _hash_ that summarizes the contents of the subtree below that node.  The rules for computing these hashes are described in tree-hashes. </del> 6. 6.1.

that allows group members to efficiently communicate the fresh secret keys to other group members, as described in update-paths. <del> Recall the definition of resolution from ratchet-tree-nodes.  To begin with, the generator of the UpdatePath updates its leaf and its leaf's _filtered direct path_ with new key pairs.  The filtered direct path of a node is obtained from the node's direct path by removing all nodes whose child on the nodes' copath has an empty resolution (keeping in mind that any unmerged leaves of the copath child count towards its resolution).  Such a removed node does not need its own key pair since, if the copath child's resolution is blank, then encrypting any data to the node's key pair would be equivalent to encrypting only to the non-copath child. </del> <ins> A member updates the nodes along its direct path as follows: </ins> Blank all the nodes on the direct path from the leaf to the root.

the nodes in the UpdatePath form a parent-hash chain from the committer's leaf to the root. <ins> For example, nodes the tree in full-tree might have been populated by full Commits from A, E, F, G, and B (in that order).  These Commits would create the following parent-hash validity relationship (where the value for a node is reset each time it is listed): Commit from A: A - U - X Commit from E: E - Y - Z - X Commit from F: F - Y - Z - X Commit from G: G - Z - X Commit from B: B - U - X After these commits, the tree will have the following parent-hash chains: A B - U - X E F - Y G - Z Since these chains collectively cover all non-blank parent nodes in the tree, the tree is parent-hash valid. Note that this tree, though valid, contains invalid parent-hash links.  If a client were checking parent hashes top-down from X, for example, they would find that Z has an invalid parent hash relative to X, but that U has valid parent hash.  Likewise, if the client were checking bottom-up, they would find that the chain from F ends in an invalid link from Y to Z.  These invalid links are the natural result of multiple clients having committed. </ins> 8.9. As described in commit, each Commit message may optionally contain an

via a "sender ratchet".  Each sender has their own sender ratchet, and each step along the ratchet is called a "generation". <ins> The following figure shows a secret tree for a four-member group, with the handshake and application ratchets that member D will use for sending and the first two application keys and nonces. </ins> A sender ratchet starts from a per-sender base secret derived from a Secret Tree, as described in secret-tree.  The base secret initiates a symmetric hash ratchet which generates a sequence of keys and

"application_ratchet_nonce_[L]_[j]" and "application_ratchet_key_[L]_[j]". <del> Concretely, suppose we have the following Secret Tree and ratchet for participant D: </del> <ins> Concretely, consider the Secret Tree shown in secret-tree-example. Client A, B, or C would generate the illustrated values on receiving a message from D with generation equal to 1, having not received a message with generation 0 (e.g., due to out-of-order delivery).  In such a case, the following values would be consumed: The key K1 and nonce N1 used to decrypt the message The application ratchet secrets AR1 and AR0 The tree secrets D, F, G (recall that G is the "encryption_secret" for the epoch) The "epoch_secret", "commit_secret", "psk_secret", and "joiner_secret" for the current epoch Other values may be retained (not consumed): K0 and N0 for decryption of an out-of-order message with generation 0 AR2 for derivation of further message decryption keys and nonces HR0 for protection of handshake messages from D </ins> <del> Then if a client uses key K1 and nonce N1 during epoch n then it must consume (at least) values G, F, D, AR0, AR1, K1, N1 as well as the key schedule secrets used to derive G (the "encryption_secret"), namely "init_secret" of epoch n-1 and "commit_secret", "joiner_secret", "epoch_secret" of epoch n.  The client MAY retain (not consume) the values K0 and N0 to allow for out-of-order delivery, and SHOULD retain AR2 for processing future messages. </del> <ins> E and C for deriving secrets used by senders A, B, and C </ins> 11.