jiebi/CodeConvo huggingface dataset:
Code change:
This is a minimal section for MIME type registration of message/mls. It assumes that PR include wire_format in all MLS messages (making a format parameter unnecessary), and that all messages have the MLS version number encoded (making a version parameter unnecessary). These parameters can be added if needed.
When MLS messages are transmitted over a protocol which supports Content-Type / MIME Type negotiation, signal that the message is an MLS message. Because MLS messages are really a bag of formats today we should have a MIME type parameter which specifies the specific format included. message/mls;format={applicationgroupcontext}
This seems like a fine idea to me. Could go as a section in the protocol spec, or as a separate (small) spec.
Discussion on virtual interim: Section or separate doc? How do we handle versioning? Internal to the protocol (field in MLSMessage?) Field in the MIME type Different MIME type per version
I would prefer a short section in the protocol document. I think Internal to the protocol is cleanest. Likewise the format parameter becomes unnecessary if we add Welcome, PublicGroupState, and KeyPackage to wire_format (see issue )
NAME send a PR?
Asking for friend: would we want to define (or use) a structured syntax suffix ala ?
NAME If you mean for multiple formats, no, we have only one encoding. If someone wants to re-encode the protocol as XML or whatever, a MIME type will be the least of their troubles.
Fixed in

Related discussion:
text: be perceived as creating a conflict of interest for a particular MLS DE, that MLS DE SHOULD defer to the judgment of the other MLS DEs. <ins> 18.6. This document registers the "message/mls" MIME media type in order to allow other protocols (ex: HTTP RFC7540) to convey MLS messages. </ins> 19.  References 19.1.  URIs
Code change:
There are two changes in this PR: Pull the extension to the TLS syntax up into the Terminology section Add an extension that lets vectors have variable-length headers, allowing them to be efficient at small sizes while also accommodating large sizes All vectors are converted to the variable-length header representation. It seems like this should simplify implementation, since the encoder will no longer have to be told how many bytes of header to use for a given vector. This has been the source of several annoying interop bugs.
Looks great! I only have a small objection: there is now several valid ways to serialize a length, as the example shows, both and are valid ways to encode 37. This means that if you parse a structure, and later want to re-serialize it to check a signature for example, you need to remember how many bits were used to store the length. In short, we loose the property that . The fix is easy: can we add that "the length MUST be stored using the smallest amount of bits possible"?
Good point, updated!
I wasn't clear under which circumstances you would use a longer length with a number that would fit in a smaller range. If never, then the table should be:
DIscussion on virtual interim Some concern about being able to send gigantic stuff In adversarial cases, the 4-byte headers used for a lot of stuff might already be a problem Could also have length constraints decoupled from the syntax SIP experience shows that imposing arbitrary length limits isn't a good idea ... but applications should have a way to signal that things are too big TODO(NAME Advisory text "Beware, things can get big; consider imposing limits and clear failure cases for this" Consider streaming implementations, where you might not know that the whole message fits in memory Modulo the above TODO, clear to merge

Related discussion:
text: The tree hash of a tree is the tree hash of its root node, which we define recursively, starting with the leaves. <del> As some nodes may be blank while others contain data we use the following struct to include data if present. </del> The tree hash of a leaf node is the hash of leaf's "LeafNodeHashInput" object which might include a "LeafNode" object depending on whether or not it is blank.
Code change:
There are two changes in this PR: Pull the extension to the TLS syntax up into the Terminology section Add an extension that lets vectors have variable-length headers, allowing them to be efficient at small sizes while also accommodating large sizes All vectors are converted to the variable-length header representation. It seems like this should simplify implementation, since the encoder will no longer have to be told how many bytes of header to use for a given vector. This has been the source of several annoying interop bugs.
Looks great! I only have a small objection: there is now several valid ways to serialize a length, as the example shows, both and are valid ways to encode 37. This means that if you parse a structure, and later want to re-serialize it to check a signature for example, you need to remember how many bits were used to store the length. In short, we loose the property that . The fix is easy: can we add that "the length MUST be stored using the smallest amount of bits possible"?
Good point, updated!
I wasn't clear under which circumstances you would use a longer length with a number that would fit in a smaller range. If never, then the table should be:
DIscussion on virtual interim Some concern about being able to send gigantic stuff In adversarial cases, the 4-byte headers used for a lot of stuff might already be a problem Could also have length constraints decoupled from the syntax SIP experience shows that imposing arbitrary length limits isn't a good idea ... but applications should have a way to signal that things are too big TODO(NAME Advisory text "Beware, things can get big; consider imposing limits and clear failure cases for this" Consider streaming implementations, where you might not know that the whole message fits in memory Modulo the above TODO, clear to merge

Related discussion:
text: A GroupContextExtensions proposal is used to update the list of extensions in the GroupContext for the group. <del> "struct { Extension extensions<0..2^32-1>; } GroupContextExtensions; " </del> <ins> "struct { Extension extensions<V>; } GroupContextExtensions; " </ins> A member of the group applies a GroupContextExtensions proposal with the following steps:
Code change:
This change ensures that the "init" key pair associated to the key package is one-time-use, at least to the degree that KeyPackages are one-time-use. The init key pair is used to encrypt/decrypt the Welcome message, then never again. The main drawback is requiring the KeyPackage sender to generate an additional key pair, which requires more computation and more entropy.

Related discussion:
text: Verify that the signature on the KeyPackage is valid using the public key in "leaf_node.credential". <ins> Verify that the value of "leaf_node.public_key" is different from the value of the "init_key" field. </ins> 11.2. Within MLS, a KeyPackage is identified by its hash (see, e.g.,
Code change:
Also requires that padding be all-zero. Before, the sender was allowed to put arbitrary octets in there.
Looks good to me. NAME I think this covers your issue on the mailing list
Discussed at 19 May interim. Decision to merge.
As discussed , we should refactor so that the padding is not length-delimited, something like: In the MLSCiphertextContent struct we have a field for padding represented using a variable-size vector which helps hide information about the length of content; notably application messages. But because of the way we encode variable-size vectors it turns out you can't actually pad the rest of the content with certain number of bytes: namely 0, 65, 16386, 16387. (E.g. if you pad with a vector of 63 bytes the TLS encoding actually produces 64 bytes for padding: 63 bytes + 1 length byte. But if you pad with 64 bytes you end up with a 66 byte encoding as now you need 2 bytes for the length. Not being able to pad with 0 is OK. But 65 is kind of annoying. E.g. if you want to pad to a target content length X bytes then if the rest of the content ends up being X-65 bytes long you are out of luck. (Assuming I'm right about how this all works) it seems a bit silly to me for a padding scheme to have somewhat arbitrary gaps like that; especially at a not unreasonable number of bytes to want to pad with. If I didn't miss something, and you all agree that its better to avoid having gaps like that then how about represent padding as, say, a fixed length vector's? Joël Amazon Development Center Austria GmbH Brueckenkopfgasse 1 8020 Graz Oesterreich Sitz in Graz Firmenbuchnummer: FN 439453 f Firmenbuchgericht: Landesgericht fuer Zivilrechtssachen Graz

Related discussion:
text: The ciphertext content is encoded using the MLSCiphertextContent structure. <ins> The "padding" field is set by the sender, by first encoding the content (via the "select") and the "auth" field, then appending the chosen number of zero bytes.  A receiver identifies the padding field in a plaintext decoded from "MLSCiphertext.ciphertext" by first decoding the content and the "auth" field; then the "padding" field comprises any remaining octets of plaintext.  The "padding" field MUST be filled with all zero bytes.  A receiver MUST verify that there are no non-zero bytes in the "padding" field, and if this check fails, the enclosing MLSCiphertext MUST be rejected as malformed. </ins> In the MLS key schedule, the sender creates two distinct key ratchets for handshake and application messages for each member of the group. When encrypting a message, the sender looks at the ratchets it
Code change:
Also requires that padding be all-zero. Before, the sender was allowed to put arbitrary octets in there.
Looks good to me. NAME I think this covers your issue on the mailing list
Discussed at 19 May interim. Decision to merge.
As discussed , we should refactor so that the padding is not length-delimited, something like: In the MLSCiphertextContent struct we have a field for padding represented using a variable-size vector which helps hide information about the length of content; notably application messages. But because of the way we encode variable-size vectors it turns out you can't actually pad the rest of the content with certain number of bytes: namely 0, 65, 16386, 16387. (E.g. if you pad with a vector of 63 bytes the TLS encoding actually produces 64 bytes for padding: 63 bytes + 1 length byte. But if you pad with 64 bytes you end up with a 66 byte encoding as now you need 2 bytes for the length. Not being able to pad with 0 is OK. But 65 is kind of annoying. E.g. if you want to pad to a target content length X bytes then if the rest of the content ends up being X-65 bytes long you are out of luck. (Assuming I'm right about how this all works) it seems a bit silly to me for a padding scheme to have somewhat arbitrary gaps like that; especially at a not unreasonable number of bytes to want to pad with. If I didn't miss something, and you all agree that its better to avoid having gaps like that then how about represent padding as, say, a fixed length vector's? Joël Amazon Development Center Austria GmbH Brueckenkopfgasse 1 8020 Graz Oesterreich Sitz in Graz Firmenbuchnummer: FN 439453 f Firmenbuchgericht: Landesgericht fuer Zivilrechtssachen Graz

Related discussion:
text: The key and nonce provided to the AEAD are computed as the KDF of the first "KDF.Nh" bytes of the ciphertext generated in the previous section.  If the length of the ciphertext is less than "KDF.Nh", the <del> whole ciphertext is used without padding.  In pseudocode, the key and nonce are derived as: </del> <ins> whole ciphertext is used.  In pseudocode, the key and nonce are derived as: </ins> The Additional Authenticated Data (AAD) for the SenderData ciphertext is the first three fields of MLSCiphertext:
Code change:
Also requires that padding be all-zero. Before, the sender was allowed to put arbitrary octets in there.
Looks good to me. NAME I think this covers your issue on the mailing list
Discussed at 19 May interim. Decision to merge.
As discussed , we should refactor so that the padding is not length-delimited, something like: In the MLSCiphertextContent struct we have a field for padding represented using a variable-size vector which helps hide information about the length of content; notably application messages. But because of the way we encode variable-size vectors it turns out you can't actually pad the rest of the content with certain number of bytes: namely 0, 65, 16386, 16387. (E.g. if you pad with a vector of 63 bytes the TLS encoding actually produces 64 bytes for padding: 63 bytes + 1 length byte. But if you pad with 64 bytes you end up with a 66 byte encoding as now you need 2 bytes for the length. Not being able to pad with 0 is OK. But 65 is kind of annoying. E.g. if you want to pad to a target content length X bytes then if the rest of the content ends up being X-65 bytes long you are out of luck. (Assuming I'm right about how this all works) it seems a bit silly to me for a padding scheme to have somewhat arbitrary gaps like that; especially at a not unreasonable number of bytes to want to pad with. If I didn't miss something, and you all agree that its better to avoid having gaps like that then how about represent padding as, say, a fixed length vector's? Joël Amazon Development Center Austria GmbH Brueckenkopfgasse 1 8020 Graz Oesterreich Sitz in Graz Firmenbuchnummer: FN 439453 f Firmenbuchgericht: Landesgericht fuer Zivilrechtssachen Graz

Related discussion:
text: find out the plaintext by correlation between the question and the length. <del> The content and length of the "padding" field in "MLSCiphertextContent" can be chosen at the time of message encryption by the sender.  It is recommended that padding data is comprised of zero-valued bytes and follows an established deterministic padding scheme. </del> <ins> The length of the "padding" field in "MLSCiphertextContent" can be chosen at the time of message encryption by the sender.  Senders may use padding to reduce the ability of attackers outside the group to infer the size of the encrypted content. </ins> 16.2.
Code change:
This seems to be a layer of indirection that doesn't add any value to implementation.
The Credentials section claims there is an epoch id field in key packages. I don't think that actually exists. My apologies if this is a duplicate issue.
Addressed in but may need to be pulled out depending on the fate of that PR
contains all of the Proposals that were previously sent within the same epoch. This seems unnecessary, since all members should have them anyway. The list of in the actual message should be enough for members to parse and validate a message. We should discuss if we can de-duplicate this, since the list of Proposals might be quite long.
The current definition of is as follows; Maybe this issue is obsolete?
Seems it is now obsolete indeed.
I don't have a strong opinion on this question, but this change seems sensible to me. The use of LeafNodeRef requires either (a) a linear scan through the tree, which is time-expensive in large trees, or (b) keeping an index that maps LeafNodeRef values to indices, which is storage-intensive in large trees. IIRC, the change to LeafNodeRef was made in the context of NAME work to make the tree description less tied to the array representation. In general, that is the right thing to do, and it's good that we did it. But we still have leaf indices anyway, for . Since we still have the concept, we might as well use it where it makes sense. And it certainly seems to me like and make sense, since they are internal to the protocol. In case folks are worried about the application-facing API -- MLS stacks can certainly be more intelligent, e.g., exposing or or . But they can do that regardless of this PR.

Related discussion:
text: For a KeyPackageRef, the "value" input is the encoded KeyPackage, and the ciphersuite specified in the KeyPackage determines the KDF used. <del> For a LeafNodeRef, the "value" input is the LeafNode object for the leaf node in question.  For a ProposalRef, the "value" input is the MLSMessageContentAuth carrying the proposal.  In the latter two cases, the KDF is determined by the group's ciphersuite. </del> <ins> For a ProposalRef, the "value" input is the MLSMessageContentAuth carrying the proposal.  In the latter two cases, the KDF is determined by the group's ciphersuite. </ins> 6.3.
Code change:
This seems to be a layer of indirection that doesn't add any value to implementation.
The Credentials section claims there is an epoch id field in key packages. I don't think that actually exists. My apologies if this is a duplicate issue.
Addressed in but may need to be pulled out depending on the fate of that PR
contains all of the Proposals that were previously sent within the same epoch. This seems unnecessary, since all members should have them anyway. The list of in the actual message should be enough for members to parse and validate a message. We should discuss if we can de-duplicate this, since the list of Proposals might be quite long.
The current definition of is as follows; Maybe this issue is obsolete?
Seems it is now obsolete indeed.
I don't have a strong opinion on this question, but this change seems sensible to me. The use of LeafNodeRef requires either (a) a linear scan through the tree, which is time-expensive in large trees, or (b) keeping an index that maps LeafNodeRef values to indices, which is storage-intensive in large trees. IIRC, the change to LeafNodeRef was made in the context of NAME work to make the tree description less tied to the array representation. In general, that is the right thing to do, and it's good that we did it. But we still have leaf indices anyway, for . Since we still have the concept, we might as well use it where it makes sense. And it certainly seems to me like and make sense, since they are internal to the protocol. In case folks are worried about the application-facing API -- MLS stacks can certainly be more intelligent, e.g., exposing or or . But they can do that regardless of this PR.

Related discussion:
text: signature key depending on the sender's "sender_type": "member": The signature key contained in the Credential at the <del> leaf with the sender's "LeafNodeRef" </del> <ins> index indicated by "leaf_index" in the ratchet tree. </ins> "external": The signature key contained in the Credential at the <del> index indicated by the "sender_index" in the "external_senders" group context extension (see Section external-senders-extension). In this case, the "content_type" of the message MUST NOT be "commit", since only members of the group or new joiners can send Commit messages. </del> <ins> index indicated by "sender_index" in the "external_senders" group context extension (see Section external-senders-extension).  In this case, the "content_type" of the message MUST NOT be "commit", since only members of the group or new joiners can send Commit messages. </ins> "new_member": The signature key depends on the "content_type":
Code change:
This seems to be a layer of indirection that doesn't add any value to implementation.
The Credentials section claims there is an epoch id field in key packages. I don't think that actually exists. My apologies if this is a duplicate issue.
Addressed in but may need to be pulled out depending on the fate of that PR
contains all of the Proposals that were previously sent within the same epoch. This seems unnecessary, since all members should have them anyway. The list of in the actual message should be enough for members to parse and validate a message. We should discuss if we can de-duplicate this, since the list of Proposals might be quite long.
The current definition of is as follows; Maybe this issue is obsolete?
Seems it is now obsolete indeed.
I don't have a strong opinion on this question, but this change seems sensible to me. The use of LeafNodeRef requires either (a) a linear scan through the tree, which is time-expensive in large trees, or (b) keeping an index that maps LeafNodeRef values to indices, which is storage-intensive in large trees. IIRC, the change to LeafNodeRef was made in the context of NAME work to make the tree description less tied to the array representation. In general, that is the right thing to do, and it's good that we did it. But we still have leaf indices anyway, for . Since we still have the concept, we might as well use it where it makes sense. And it certainly seems to me like and make sense, since they are internal to the protocol. In case folks are worried about the application-facing API -- MLS stacks can certainly be more intelligent, e.g., exposing or or . But they can do that regardless of this PR.

Related discussion:
text: content.  Before being encrypted, the sender data is encoded as an object of the following form: <del> MLSSenderData.sender is assumed to be a "member" sender type.  When constructing an MLSSenderData from a Sender object, the sender MUST verify Sender.sender_type is "member" and use Sender.sender for MLSSenderData.sender. </del> <ins> When constructing an MLSSenderData from a Sender object, the sender MUST verify Sender.sender_type is "member" and use Sender.leaf_index for MLSSenderData.leaf_index. </ins> The "reuse_guard" field contains a fresh random value used to avoid nonce reuse in the case of state loss or corruption, as described in
Code change:
This seems to be a layer of indirection that doesn't add any value to implementation.
The Credentials section claims there is an epoch id field in key packages. I don't think that actually exists. My apologies if this is a duplicate issue.
Addressed in but may need to be pulled out depending on the fate of that PR
contains all of the Proposals that were previously sent within the same epoch. This seems unnecessary, since all members should have them anyway. The list of in the actual message should be enough for members to parse and validate a message. We should discuss if we can de-duplicate this, since the list of Proposals might be quite long.
The current definition of is as follows; Maybe this issue is obsolete?
Seems it is now obsolete indeed.
I don't have a strong opinion on this question, but this change seems sensible to me. The use of LeafNodeRef requires either (a) a linear scan through the tree, which is time-expensive in large trees, or (b) keeping an index that maps LeafNodeRef values to indices, which is storage-intensive in large trees. IIRC, the change to LeafNodeRef was made in the context of NAME work to make the tree description less tied to the array representation. In general, that is the right thing to do, and it's good that we did it. But we still have leaf indices anyway, for . Since we still have the concept, we might as well use it where it makes sense. And it certainly seems to me like and make sense, since they are internal to the protocol. In case folks are worried about the application-facing API -- MLS stacks can certainly be more intelligent, e.g., exposing or or . But they can do that regardless of this PR.

Related discussion:
text: is the first three fields of MLSCiphertext: When parsing a SenderData struct as part of message decryption, the <del> recipient MUST verify that the LeafNodeRef indicated in the "sender" field identifies a member of the group. </del> <ins> recipient MUST verify that the leaf index indicated in the "leaf_index" field identifies a non-blank node. </ins> 8.