Change:
The current draft implements the original incorrect version of the tree-hash-for-parent-hash mechanism proposed in . That version . This PR implements the corrected "full-extension + tree hash" mechanism proposed by NAME .
In order to use tree hashes to represent siblings in parent hashes, we had to compute a different tree hash on the right edge of the tree, pretending that the tree is full, in the sense of having leaves. Thus we actually have two tree hashes in play, one used for group agreement on the tree and one used for parent hash inputs. We should consider specifying that the logical tree used by MLS is always full. This change to the specification would not change the computations done by the protocol much, if at all. Thanks to the fact that we no longer generate redundant nodes (), there is never anything non-blank to the right of the rightmost non-blank leaf anyway. So the main impact is to insert some blank nodes between a parent node on the right edge of the tree and its right child -- and all of the algorithms in the specification already skip over such nodes. Likewise, implementations' internal represntations of the tree wouldn't have to change much, since the extra nodes could be "virtual". The only time they would be noticed is when computing tree hashes. We would have to update the description of the extension to say that it only covers the tree up to the rightmost non-blank leaf. It would simplify the specification in a few ways, for example: The "original sibling tree hash" would just entail blanking unmerged leaves, not structure changes. The description of extending and truncating the tree would be much simpler. The main cost would be extra hashes when joining, or when updating a member close to the right edge of the tree. If you have a tree with leaves, then you have to compute tree hashes over blank leaves and their blank parents.
Thanks to NAME and NAME for helping inspire and develop this idea in discussions around
Virtual interim 2022-05-19 NAME notes that the current tree hash is quite tricky to implement, so this would be a useful simplification Doubling the hashes on a fresh tree doesn't sound too intimidating Folks would find more concreteness useful here, interested in PR Could we phrase this as only a change to TreeKEM? Would be premised on the equivalence of the hashed tree an the tree you operate on
Discussed at 19 May interim: NAME to draft a PR.
Holding a PR until after lands. Some notes on what would need to change: {{ratchet-tree-terminology}} would change from "the unique left-balanced tree with N leaves" to "the unique balanced binary tree with depth , where is the smallest number such that ". {{adding-and-removing-leaves}} would either be removed, or updated so that it talks about increasing or decreasing the depth of the tree (equivalently, doubling or halving the number of leaves). {{add}} would talk about increasing depth instead of adding leaves. {{remove}} would talk about decreasing depth instead of removing leaves. {{ratchet-tree-extension}} would be explicit that it only carries nodes up to the rightmost non-blank leaf. The encoding/decoding algorithms would have to be updated. {{array-based-trees}} and {{link-based-trees}} could probably be simplified. We would need to make sure all figures showed full trees.
I went ahead and filed for this, based on top of . Once that merges, we can update the PR to be a little easier to read. The changes are basically as described above. One additional note: We have actually already incurred the cost to a new joiner of hashing all the extra nodes, because they have to do it for parent hash verification anyway. So this approach is actually a net reduction in the amount of hashing that has to be done, since they don't have to also compute the non-full-tree hashes.
Thanks for finding yet more parent-hash issues NAME :) I was initially concerned about this approach, because it seemed like it would be expensive to fill out the tree. However, (a) we only incur that cost for the nodes on the right edge of the tree, and (b) the extra nodes to the right are "virtual", in the sense that they don't take up any memory, they just cause some extra hashes. I and it was pretty straightforward (that also allowed me to verify that it fixes the pathological case that NAME notes). Overall, this looks good to merge once we get the noted nits fixed.

Discussion:
blank nodes that are not fixed by P's Parent Hash.  However, for each node that has an HPKE key, this key is fixed by P's Parent Hash. <del> Finally, "original_sibling_tree_hash" is the original tree hash of S. The original tree hash corresponds to the tree hash of S the last time P was updated.  It can be computed as the tree hash of S in the ratchet tree modified the following way: </del> <ins> Finally, "original_sibling_tree_hash" is the tree hash of S in the ratchet tree modified as follows: </ins> <del> reset the leaves in P.unmerged_leaves to blanks </del> <ins> Extend the subtree of S by adding blank leaves until it is full, i.e., until its number of leaves is a power of 2 (see adding-and- removing-leaves). </ins> <del> remove P.unmerged_leaves from all unmerged_leaves lists </del> <ins> For each leaf L in "P.unmerged_leaves", blank L and remove it from the "unmerged_leaves" sets of all parent nodes. </ins> <del> truncate the ratchet tree as described in </del> <ins> Observe that "original_sibling_tree_hash" does not change between updates of P.  This property is crucial for the correctness of the protocol. </ins> For example, in the following tree: With P = W and S = Y, "original_sibling_tree_hash" is the tree hash of the following tree: <del> Because "W.unmerged_leaves = [H]", H is removed from "Y.unmerged_leaves", then H is replaced with a blank leaf, then the tree is truncated removing the last two nodes. With P = W and S = U, "original_sibling_tree_hash" is the tree hash of the following tree: This time we have 4 leaf nodes because the truncation of the ratchet tree didn't remove the last leaf. </del> <ins> Because "W.unmerged_leaves" includes F, F is blanked and removed from "Y.unmerged_leaves". </ins> Note that no recomputation is needed if the tree hash of S is unchanged since the last time P was updated.  This is the case for

as long as the intersection of the parent's unmerged leaves with the subtree is the same as in the earlier computation. <del> Observe that "original_child_resolution" is equal to the resolution of S at the time the "UpdatePath" was generated, since at that point P's set of unmerged leaves was emptied.  (Observe also that "original_child_resolution" contains all unmerged leaves of S.) Therefore, P's Parent Hash fixes, for each node V on the path from P to the root, not only the HPKE public key of V, but also the set of HPKE public keys to which the corresponding HPKE secret key of V was encrypted by the generator of the "UpdatePath". </del> 8.8.1. The Parent Hash of P appears in three types of structs.  If V is