Draft: draft-ietf-masque-connect-ip

Commentary: Magnus , See [ BB ] [ BB ] I might be contradicting our ADs , but I do n't subscribe to the view held by some in the transport area that tunnels that encapsulate e2e connections within a transport header ( whether UDP , TCP or sthg else ) become responsible for congestion control over the extent of the tunnel . It is sufficient to only make them responsible for conveying the congestion signals ( whether loss or ECN ) to the inner at decap , so the e2e transport can do CC . Loss of a packet within the extent of the tunnel loses the inner and the outer , so it conveys the signal implicitly { Note 1 } ECN has to be conveyed explicitly , as in RFC6040 . { Note 1 } If the e2e transport does n't do CC , AFAIC , the buck stops at the e2e transport . Otherwise we end up with sanctimonious impossible requirements like the above that require a tunnel to know about the remote /behaviour/ of the senders originating all the flows that it is encapsulating . { Note 1 } :  recovery across the tunnel is OK , as long as : the egress does n't block waiting for in - order delivery , and it is partial , i.e. gives up after a certain time or a certain number of attempts ( which is OK , because the e2e loss recovery will be reliable if it needs to be ) . TCP certainly does n't do ( b ) and I 'm not sure whether any form of QUIC has been standardized that does ( b ) , altho I believe some implementations do . If the repair from such local loss recovery reaches the e2e destination before the e2e RACK deems there 's been a loss , it does n't even cause duplicate retransmissions . { Note 2 } : In one of my previous emails , I said you would n't need to propagate ECN from outer to inner . But that was because I thought you were running 2 nested control loops . If we have a choice , just one outer control loop is better . [ BB ] Also , ' underlying ' is n't the best choice of word to contrast against ' inside ' . Why not ' outer HTTP connection ' ? [ BB ] I 'm not sure what the first clause means : " a client or IP proxy with a connection containing an IP Proxying request stream " ? Does that include a connection that started out containing an IP Proxying request stream , but that is now solely transferring datagrams ? Assuming I understand it sufficiently , I do n't think it 's right to say that disabling CC requires disabling ECN over the extent of the tunnel . If tunnel CC is disabled ( which is good ) , it is better to propagate ECN signals from the outer to the forwarded packet ( as per draft - ietf - tsvwg - rfc6040update - shim ) , to be handled by the e2e ( inner ) congestion controllers . The only reason to disable ECN on the outer would be if ECN propagation is n't or ca n't be implemented at decap . There is however the problem of potentially mismatched packet boundaries , which makes ECN propagation less straightforward , but not impossible . [ BB ] IIUC { Note 3 } , I agree . BUT , I do n't agree with either premise . I.e. I do n't agree that either i ) reliable stream encap or ii ) datagram over a congestion controlled QUIC connection are sensible . { Note 3 } : My parser is having problems with the above sentence . I think it means " tunnel ingress will have to CE mark or drop packets to send an end - to - end congestion signal when its queue gets too big . " Otherwise , as written , it seems to mean the following , which I assume was n't intended : " the tunnel ingress will have to CE mark or drop packets when the queue gets so big that it can not send an end - to - end congestion signal " [ BB ] Correct . Unless it was made a requirement of the Proxying RFC that the receiving peer MUST perform RFC6040 recombining ( that 's what IPsec decided to do 'cos it 's simpler than having choices ) . [ BB ] Better , would be to define how ECN recombining is done if there are multiple DATAGRAM frames in a QUIC packet ( e.g. if the outer is CE - marked , and the inners are all ECT(0 ) or Not - ECT , then CE - mark all the inners that are ECT(0 ) , and drop any that are Not - ECT ) . That works for the standards track assumption that a CE mark is equivalent to a loss . But the L4S /experiment/ breaks that assumption . Then : you ca n't mix ECT(1 ) datagrams in the inner with ECT(0 ) on the outer . you ca n't mix ECT(0 ) or Not - ECT datagrams in the inner with ECT(1 ) on the outer . As long as the ingress classifies these into two streams before QUIC encap , the egress should be able to do ECN recombining OK . As I mentioned in my review , the ingress of the tunnel would have to follow RFC2983 regarding Diffserv as well . I.e. classify different DSCPs separately , so datagrams of different classes are not mixed into the same QUIC packet . The above ECN behaviour just extends that to have to classify on the last bit as well as the first 6 bits of the Traffic Class field ( or its equivalent in IPv4 ) . However , in addition , it depends on operator policy whether DSCPs are propagated from inner to outer and back , or not . [ BTW , ECN recombining can be done with mixed frames in each packet , but you have to be smarter . For an example , see ] [ BB ] As earlier , I would n't advise one CC loop within another anyway . [ BB ] Before I can agree or not , let 's go another round , to see to see if I 've understood correctly , and to see if the approaches I 've suggested are a simpler way forward : no CC loops within loops partial loss recovery , classifying datagrams at ingress separately based on bits 0 - 5 & 7 of the Traffic Class field ( or its equivalent in IPv4 ) . at egress , copying an outer ECN CE mark to all the datagram frames within . It would seem rather sad if one part of the IETF disabled both the low latency technologies contributed by other parts of the IETF across an HTTP/3 Proxying tunnel . Given the point of HTTP/3 is meant to be low latency , and most of these tunnels will span the access network part of the path , where most of the queue delay is .    everyone for the review . I think we 've reached a point where we all agree that this PR improves things . We can still tweak the text but I 'm going to merge this so that our ADs can see this during their IESG review because the telechat is in a week . 
  that we should discuss performance more in this document . I propose we add a Performance Considerations section based on 
 I will note that the ECN considerations for IP are different from them in UDP . If one disable congestion control and the outer QUIC connection will ignore the ECN marks then one MAY handle tunneled in datagram IP packets by copying the ECN bits from inner to outer and then one MUST recombine them in the egress using the rules in RFC 6040 . If one does n't disable congestion control in the QUIC connection then one MUST NOT copy the ECN field , Instead if the tunneled traffic enables ECN it can be either just ignored , or if one has an ingress QUEUE to the tunnel one MAY mark packets with CE rather than dropping them . 
 I do n't think it 's that simple . You ca n't copy the ECN bits if there are multiple packets in separate datagram frames coalesced into the same QUIC packet .
Paragraph (version B):
This document updates CONNECT-UDP to change the "masque" well-known
URI, see iana-uri.
Paragraph (version B):
Note that the IP forwarding tunnels described in this document are
not fully featured "interfaces" in the IPv6 addressing architecture
sense IPv6-ADDR.  In particular, they do not necessarily have IPv6
link-local addresses.  Additionally, IPv6 stateless autoconfiguration
or router advertisement messages are not used in such interfaces, and
neither is neighbor discovery.  Implementers need to ensure that they
do not forward any link-local traffic beyond the IP proxying
interface that it was received on.  IP proxying endpoints also need
to properly reply to packets destined to link-local multicast
addresses.
Paragraph (version B):
Note that it is possible for multiple proxied IP packets to be
encapsulated in the same outer packet, for example because a QUIC
packet can carry two QUIC DATAGRAM frames.  It is also possible for a
proxied IP packet to span multiple outer packets, because a DATAGRAM
capsule can be split across multiple QUIC or TCP packets.
Paragraph (version B):
Bursty traffic can often lead to temporally-correlated packet losses;
in turn, this can lead to suboptimal responses from congestion
controllers in protocols running inside the tunnel.  To avoid this,
endpoints SHOULD strive to avoid increasing burstiness of IP traffic;
they SHOULD NOT queue packets in order to increase batching beyond
the minimal amount required to take advantage of hardware offloads.
Paragraph (version B):
When the protocol running inside the tunnel uses congestion control
(e.g., TCP or QUIC), the proxied traffic will incur at least two
nested congestion controllers.  The outer HTTP connection MAY disable
congestion control if it knows that the inner packets belong to
congestion-controlled connections.
Paragraph (version B):
When the protocol running inside the tunnel uses loss recovery (e.g.,
TCP or QUIC), and the outer HTTP connection runs over TCP, the
proxied traffic will incur at least two nested loss recovery
mechanisms.  This can reduce performance as both can sometimes
independently retransmit the same data.  To avoid this, IP proxying
SHOULD be performed over HTTP/3 to allow leveraging the QUIC DATAGRAM
frame.
Paragraph (version B):
When using HTTP/3 with the QUIC Datagram extension DGRAM, IP packets
are transmitted in QUIC DATAGRAM frames.  Since these frames cannot
be fragmented, they can only carry packets up to a given length
determined by the QUIC connection configuration and the Path MTU
(PMTU).  If an endpoint is using QUIC DATAGRAM frames and it attempts
to route an IP packet through the tunnel that will not fit inside a
QUIC DATAGRAM frame, the IP proxy SHOULD NOT send the IP packet in a
DATAGRAM capsule, as that defeats the end-to-end unreliability
characteristic that methods such as Datagram Packetization Layer PMTU
Discovery (DPLPMTUD) depend on DPLPMTUD.  In this scenario, the
endpoint SHOULD drop the IP packet and send an ICMP Packet Too Big
message to the sender of the dropped packet; see ICMPv6.
Paragraph (version B):
If a client or IP proxy with a connection containing an IP Proxying
request stream disables congestion control, it cannot signal Explicit
Congestion Notification (ECN) ECN support on that outer connection.
That is, the QUIC sender MUST mark all IP headers with the Not-ECT
codepoint for QUIC packets which are outside of congestion control.
The endpoint can still report ECN feedback via QUIC ACK_ECN frames or
the TCP ECE bit, as the peer might not have disabled congestion
control.
Paragraph (version B):
Conversely, if congestion control is not disabled on the outer
congestion, the guidance in ECN-TUNNEL about transferring ECN marks
between inner and outer IP headers does not apply because the outer
connection will react correctly to congestion notifications if it
uses ECN.  The inner traffic can also use ECN, independently of
whether it is in use on the outer connection.