Draft: draft-ietf-quic-multipath

Commentary: This explanation of how to implement number spaces addresses issue . 
 Note that I still have some diffs when running . 
 I think this text is fine and is ready to merge . However , I still have one question : what 's about loss detection ? if you only change the CID but not the 4 - tuple / path , you do n't reset congestion control but as you reset your packet number space this might make loss detection more complicated . Do we need to provide some implementation guidance about this ? We can just merge this now and I can open an issue . Or there is anything we can / should add right now to this PR ? 
 I am a bit concerned that developing how and when variables like the RTT are updated will at best replicates the prescriptive text in the previous section , or worse conflicts with it . If we say anything , it will have to be " the processing and update of the per path variables is specified in section so and so " , but I am not sure that helps . I would rather check in as is . Of course , there is also a longer answer . If the path did not change , the acknowledgement based loss detection ( section 6.1 of RFC 9002 ) could take into account not just the number of " posterior " packets acknowledged with that old CID , but also the number of packets acknowledged in the " new " number space . That seems logical , but it implies that the acks for the " old " CID are not sent later than the acks for the " new " CID . This depends on implementation choices , and thus may not be reliable . We could have a discussion . So yes , leave it as is . 
 It seems that some implementers are confused about the relation between an abstract path , defined by a 4 - tuple , and the number spaces . We need to provide guidance . 
 Fixed by . 
 We need some description for how to init new paths in this section . 
 To me path validation and path initiation are very coupled together . In I merged both sections together . 
 That 's probably fine . I was keeping this as a placeholder in case we need anything in addition but probably not . 
 It 's ok .
The use of this multipath extension requires the use of non-zero
Connection IDs in both directions.
This specification requires the sender to use a non-zero connection
ID when opening an additional path.  Some deployments of QUIC use
zero-length connection IDs.  However, when a node selects to use
zero-length connection IDs, it is not possible to use different
connection IDs for distinguishing packets sent to that node over
different paths.
Packet Number Space Identifier (PN Space ID): An identifier that
is used to distinguish packet number spaces for different paths.
It is used in 1-RTT packets and ACK_MP frames.  Each node
maintains a list of "Received Packets" for each of the CID that it
provided to the peer, which is used for acknowledging packets
received with that CID.
If the parameter is set to 1, both endpoints MUST use non-zero
connection IDs.  If an enable_multipath parameter set to 1 is
received and the carrying packet does not contain a non-zero length
connection ID, the receiver MUST treat this as a connection error of
type TRANSPORT_PARAMETER_ERROR (specified in Section 20.1 of QUIC-
TRANSPORT) and close the connection.
If endpoint receives an unexpected value for the transport parameter
"enable_multipath", it MUST treat this as a connection error of type
TRANSPORT_PARAMETER_ERROR (specified in Section 20.1 of QUIC-
TRANSPORT) and close the connection.
To open a new path, an endpoint SHALL use different Connection IDs on
different paths.  Still, the receiver may observe the same Connection
ID used on different 4-tuples due to, e.g., NAT rebinding.  In such
case, the receiver reacts as specified in Section 9.3 of QUIC-
TRANSPORT.
An endpoint uses PATH_STATUS frames to inform that the peer should
send packets in the preference expressed by these frames.  Notice
that the endpoint might not follow the peer's advertisements, but the
PATH_STATUS frame is still a clear signal of suggestion for the
preference of path usage by the peer.
Each endpoint manages the set of paths that are available for
transmission.  At any time in the connection, each endpoint can
decide to abandon one of these paths, following for example changes
in local connectivity or changes in local preferences.  After an
endpoint abandons a path, the peer will not receive any more non-
probing packets on that path.  Non-probing packets are defined in
Section 9.1 of QUIC-TRANSPORT.
An endpoint that wants to close a path SHOULD use explicit request to
terminate the path by sending the PATH_ABANDON frame (see path-
abandon-close).  Note that while abandoning a path will cause
Connection ID retirement, only retiring the associated Connection ID
does not necessarily advertise path abandon (see retire-cid-close).
However, implicit signals such as idle time or packet losses might be
the only way for an endhost to detect path closure (see idle-time-
close).
Once a path is marked as "abandoned", it means that the resources
related to the path, such as the used connection IDs, can be
released.  However, information related to data delivered over that
path SHOULD not be released immediately as acknowledgments can still
be received or other frames that also may trigger retransmission of
data on another path.
The QUIC multipath extension uses different packet number spaces for
each path.  This also means that the same packet number can occur on
each path and the packet number is not a unique identifier anymore.
This requires changes to the ACK frame as well as packet protection
as described in the following subsections.
Not transmitting again ACK ranges that were present in an ACK
frame acknowledged by the peer.
Limit the total number of ranges sent in an ACK frame.
Limiting the number of transmissions of a specific ACK range, on
the assumption that a sufficient number of transmissions almost
certainly ensures reception by the peer.
Send multiple messages for a given path in a single socket
operation, so that a series of packets sent from a single path
uses a series of consecutive sequence numbers without creating
holes.
The Path ID for 1-RTT packets is the sequence number of the
Connection ID as specified in QUIC-TRANSPORT.  Section 19 of QUIC-
TRANSPORT encodes the Connection ID Sequence Number as a variable-
length integer, allowing values up to 2^62-1; in this specification,
a range of less than 2^32-1 values MUST be used before updating the
packet protection key.
fig-example-path-close1 illustrates an example of path closing when
both the client and the server use non-zero-length CIDs.  For the
first path, the server's 1-RTT packets use DCID C1, which has a
sequence number of 1; the client's 1-RTT packets use DCID S2, which
has a sequence number of 2.  For the second path, the server's 1-RTT
packets use DCID C2, which has a sequence number of 2; the client's
1-RTT packets use DCID S3, which has a sequence number of 3.  Note
that the paths use different packet number spaces.  In this case, the
client is going to close the first path.  It identifies the path by
the sequence number of the received packet's DCID over that path
(path identifier type 0x00), hence using the path_id 1.  Optionally,
the server confirms the path closure by sending an PATH_ABANDON frame
using the sequence number of the received packet's DCID over that
path (path identifier type 0x00) as path identifier, which
corresponds to the path_id 2.  Both the client and the server can
close the path after receiving the RETIRE_CONNECTION_ID frame for
that path.
As stated in introduction, when multipath is negotiated, each
Destination Connection ID is linked to a separate packet number
space.  This a big difference between implementations of QUIC as
specified in QUIC-TRANSPORT, which only have to manage three number
spaces for Initial, Handshake and Application packets.
Implementation of multipath capable QUIC will need to carefully model
the relations between paths and number spaces, as shown in fig-
number-spaces.
The path is defined by the 4-tuple through which packets are received
and sent.  Packets sent on the path will include the Destination
Connection ID currently used for that path, selected from the list of
CID provided by the peer.  Packets received on the path carry a
Destination CID selected by the peer from the list provided to that
peer.
The relation between CIDs and paths is not fixed.  A node may decide
to rotate the Destination CID it uses, a NAT may decide to change the
4-tuple over which packets from that path will be received.
Implementation will have to manage these evolving relations.
RTT measurements and congestion state are logically associated
with the 4-tuple.  They will remain unchanged if data starts being
received or sent through the same 4-tuple using new CIDs.
Implementations of loss recovery typically maintain lists of
packets sent and not yet acknowledged.  Such information, along
with the value of the next PN to use for sending, is logically
associated with the "Sender Number Space", and with the peer-
provided CID used for sending on the path.
Sending of acknowledgement requires keeping track of the PN of
received packets and of acknowledgements previously sent.  Such
information is logically associated with the "Receiver Number
Space", and with the CID used by the peer for sending on the path.
When the link between paths and CID changes, the information tied to
the now unused CID remains useful for some time.  For example, the
list of packet numbers to acknowledge maintained in the old receiver
number space could still be used to send ACK_MP frames for that
number space.  Similarly, the list of packets sent but not yet
acknowledged with an old sender number space can be used when
processing incoming ACK_MP frames for that number space.  Such data
should not be discarded immediately after a CID change, but only
later, for example when the CID is retired.
Path Identifier: An identifier of the path, which is formatted as
shown in fig-path-identifier-format.
Identifier Type: Identifier Type field is set to indicate the type
of path identifier.
Type 0: Refer to the connection identifier issued by the sender
of the control frame.  Note that this is the connection
identifier used by the peer when sending packets on the to-be-
closed path.  This method SHOULD be used if this connection
identifier is non-zero length.  This method MUST NOT be used if
this connection identifier is zero-length.
Type 1: Refer to the connection identifier issued by the
receiver of the control frame.  Note that this is the
connection identifier used by the sender when sending packets
on the to-be-closed path.  This method MUST NOT be used if this
connection identifier is zero-length.
Type 2: Refer to the path over which the control frame is sent
or received.
If the Identifier Type is 0x00 or 0x01, PATH_ABANDON frames MAY be
sent on any path, not only the path identified by the Path Identifier
Content field.  If the Identifier Type if 0x02, the PATH_ABANDON
frame MUST only be sent on the path that is intended to be abandoned.
PATH_STATUS Frame are used by endpoints to inform the peer of the
current status of one path, and the peer should send packets
according to the preference expressed in these frames.  PATH_STATUS
frames are formatted as shown in fig-path-status-format.
This document defines a new transport parameter for the negotiation
of enable multiple paths for QUIC, and two new frame types.  The
draft defines provisional values for experiments, but we expect IANA
to allocate short values if the draft is approved.
This document defines a new transport parameter for the negotiation
of enable multiple paths for QUIC, and three new frame types.  The
draft defines provisional values for experiments, but we expect IANA
to allocate short values if the draft is approved.