Change:
We had longstanding issue to allow the participants in an MLS group to detect when the DS drops messages. That issue was closed because it had been open for a long time without a concrete proposed solution. More recently, however, NAME and NAME suggested using a Proposal for this, and it made a lot of sense to me when I looked into the details. So this PR should be considere joint work between, NAME NAME and me. This PR adds an Proposal that any member of the group can send to advertise the last generation it sent, and acknowledge the receipt of application messages from other members. As a Proposal, it is included in the transcript via Commits, providing a pretty strong defence against message suppression -- the DS basically has to stop the group progressing at all if it wants to drop applications without the group being aware of it.
BTW, there are a bunch of ways to go about ). Most of these schemes are designed for 1:1 connections like TCP or QUIC, but adapt in a straightforward way to groups. Which one is most efficient depends on what you assume about losses. Some examples: List of received messages (the most naïve approach; efficient only if losses dominate) List of received message ranges (implemented here, more efficient for infrequent losses) Highest received generation, plus list of generations not received (more efficient for even more infrequent losses) FWIW, scheme (2) is effectively . TCP does something like (3), but because it is synchronous, doesn't include generations not received. The option for TCP is basically the same as option (2), reporting ranges of received packets.
Great idea! Two comments: To make sure that the DS doesn't drop any Commit that has an AppAck extension (to cover up their dropping of Application messages), either one has to encrypt Commit messages or AppAcks need to be mandatory for Commits by policy. Otherwise the DS can just drop any Commit that has an AppAck in it (without impeding the sending of messages in the group). A cheaper, although functionally less powerful version would be to simply send a hash of the SecretTree. The other group members would then know if they received the same messages as the sender (but not more than that).
NAME good point about addressing suppression of AppAck itself. I added a paragraph. Sending a hash of the SecretTree seems both less powerful and way more complicated, since we would have to define a hashing scheme. So I'm gonna pass :)
There's a few implications of simply allowing forks: FS issues - clients would need to use initsecret twice, which means keeping around initsecret for a potential fork, which breaks FS. One resolution here is to use something from the ASTree for initsecret (maybe the nth app secret, or a new ratchet derived from astreenode[n]secret). Does detection yield any sort of meaningful resolution technique? Are we better off than we are now in terms of user experience? (What we have now, if the DS fails to order, is two (or N) groups of clients operating on different versions of the tree, which effectively partitions the group) I wrote about this here: URL suggesting merges as a resolution technique. The main downside is clients would need to keep around the list of proposals for a while. This isn't bad imo, it reduces complexity to some extent. I'm somewhat in favour of a non-pseudorandom tiebreaker if there's no other arguments against it. How many bits of sequence number? Have we considered using varints? Similar to the CTLS proposal - URL Other forking considerations, I think there's more involved than just changing the epoch identifier: We'll need merges as well, since forks can lead to inconsistent group membership: Suppose in epoch 1 a group has A, B, and C A removes C in epoch 2A B adds D in epoch 2B We now have two group states 2A with AB and 2B with ABCD - and it's unclear which group to use to send with. We could mandate that any client MUST send a "merge" commit on it's view of history before sending any message. This commit could just be the list of epochs that are being merged (+a usual DirectPath update?), and processing it would mean replaying all proposals up to the common ancestor for those epochs on the common ancestor. Forward secrecy issues. By allowing forks, devices must now keep the previous epoch around to derive possible future forks - but this means we'll be re-using initsecret. A possible solution could be to use the nth app secret from the ASTree instead of initsecret as the start for the new epoch secret. This ties nicely with using a non-pseudorandom tiebreaker. Ordering implications It's unclear what a sensible ordering strategy is for the DS if we allow forking. (simply ensuring that each commit is based on the "current" epoch isn't well defined since there can be multiple "current" epochs). By allowing forks the DS need not enforce ordering at all imo. This means that apps need to keep around old epochs, but this is already required per (2). A different question is how many we need to keep around - perhaps a TTL or the past N epochs after a topological sort would make sense. Generally I think getting rid of server ordering is good idea: Helps Matrix and general federation use cases A common consistent mutated state for group operations can cause server side DB contention in my experience Forks is one way to remove ordering, and if we need to support it anyways it's a nice benefit. Richard Barnes wrote: >

Discussion:
11.1.7. <ins> An AppAck proposal is used to acknowledge receipt of application messages.  Though this information implies no change to the group, it is structured as a Proposal message so that it is included in the group's transcript by being included in Commit messages. An AppAck proposal represents a set of messages received by the sender in the current epoch.  Messages are represented by the "sender" and "generation" values in the MLSCiphertext for the message.  Each MessageRange represents receipt of a span of messages whose "generation" values form a continuous range from "first_generation" to "last_generation", inclusive. AppAck proposals are sent as a guard against the Delivery Service dropping application messages.  The sequential nature of the "generation" field provides a degree of loss detection, since gaps in the "generation" sequence indicate dropped messages.  AppAck completes this story by addressing the scenario where the Delivery Service drops all messages after a certain point, so that a later generation is never observed.  Obviously, there is a risk that AppAck messages could be suppressed as well, but their inclusion in the transcript means that if they are suppressed then the group cannot advance at all. The schedule on which sending AppAck proposals are sent is up to the application, and determines which cases of loss/suppression are detected.  For example: The application might have the committer include an AppAck proposal whenever a Commit is sent, so that other members could know when one of their messages did not reach the committer. The application could have a client send an AppAck whenever an application message is sent, covering all messages received since its last AppAck.  This would provide a complete view of any losses experienced by active members. The application could simply have clients send AppAck proposals on a timer, so that all participants' state would be known. An application using AppAck proposals to guard against loss/ suppression of application messages also needs to ensure that AppAck messages and the Commits that reference them are not dropped.  One way to do this is to always encrypt Proposal and Commit messages, to make it more difficult for the Delivery Service to recognize which messages conatain AppAcks.  The application can also have clients enforce an AppAck schedule, reporting loss if an AppAck is not received at the expected time. 11.1.8. </ins> Add and Remove proposals can be constructed and sent to the group by a party that is outside the group.  For example, a Delivery Service might propose to remove a member of a group who has been inactive for