Code change:
I think there are two option here, 1) close/deny the connect request, and 2) use streams instead. I think we should consider both option, especially given stream are just there to use.
Isn't this a dup of ? That's fixed with . I guess the text could be generalized to talk about more than just IPv6 MTU issues.
No, this aspect on what you do when the MTU required is not present is not resolved by that PR. I would note that closing gives the endpoints one experience that is likely easier to handle in implementation and do avoid the issue with reordering and the varying packet treatment that moving to stream would result.
I would suggest adding a sentence saying that if an endpoint realizes that the IPv6 tunnel MTU goes lower than 1280, then it MUST close the stream.
IETF 114: Note that if something happens and your MTU gets smaller, you may need to destroy that link. Potentially make this option a SHOULD do PMTUD, otherwise you could end up in a situation where you're not a compliant IPv6 tunnel. Proposal:
Even if PR was an improvement to the description of the IPv6 minimal link MTU issue the text still leaves an open issue in that it doesn't make clear which of the endpoints that are responsible for HTTP intermediaries or if is truly both which has another implication. So a Connect-IP endpoint will be aware of the HTTP datagram supported MTU and if its HTTP/3 connection can support sending at least 1280 bytes of IPv6 packets without fragmentation. However, if there is two or more HTTP intermediaries in between the client and the server then there are HTTP intermediaries that might have an HTTP/3 connection which has less than 1280 bytes MTU for HTTP datagram payloads in-between the client and server. Thus, arises the issue of how to detect this. PR states: if HTTP intermediaries are in use, CONNECT-IP endpoints can enter in an out of band agreement with the intermediaries to ensure that endpoints and intermediaries pad QUIC INITIAL packets. From my perspective the big question is how a client or a server knows that there are intermediaries, or the most important aspect, that there are no unknown HTTP intermediaries. To my understanding both the client and the server can potentially bring in HTTP intermediaries unknown to the other part. Thus resulting in a HTTP intermediary to intermediary connection where neither endpoint knows if the requirement is fulfilled. Thus my interpretation is that the only way to be sure that it works it to actually execute on the third bullet in the PR: CONNECT-IP endpoints can also send ICMPv6 echo requests with 1232 bytes of data to ascertain the link MTU and tear down the tunnel if they do not receive a response. Unless endpoints have an out of band means of guaranteeing that one of the two previous techniques is sufficient, they MUST use this method. I would also note that this only verifies from the acting endpoint towards its peer, not the reverse path. For that to be verified the peer need to send ICMP probes the other direction. Thus, my conclusion is that Connect-IP still don't ensure that the minimal MTU is fulfilled.
Isn't the quick-fix to require that the ICMP reply also be 1232 byte sized?
Normally I would say now, as you want to ensure that the reply makes it back that shouldn't be padded. Also if the response is anything else than echo reply you also should not pad. However, I think the implication is the same either way. Both masque client and server need to send ICMP or modify its ICMP processing. Given that I would think having each endpoint doing independent processing of its forward path might actually be simpler and enables using a standard off the shelf ICMP implementation and not have to modify it.
Our implementation has userspace ICMP request/response handling for echo request/response, it wasn't too hard to write. I don't propose modifying anything, but the structures were ~50 lines of code to set up. I agree we shouldn't be modifying responses we did not generate.
I think the existing text already is generic to "endpoint", not client or server: So this implies that both sides need to send the PING if they haven't guaranteed the MTU. This seems like this is already handled?
If both endpoints are required to send ping it would be resolved. However, that is not what is written in the draft currently. What is written is the there are various techniques, and if an endpoint choses something else than verifying its forward path using ping, it becomes unclear who is actually responsible for any HTTP intermediaries. Which results in an unclear requirement and very likely failure of the MTU verification resulting in an MTU black hole.
Well, it does say that endpoints MUST use this method unless the already have other coordination. I could see some clarification text to say that the guarantee for the other methods must include knowing there isn't an intermediary — which is actually pretty easy to guarantee in many cases.
That is what I intended when writing , but perhaps I wasn't clear. Can one of you propose a PR that would clarify this?
As per , the IPv6 minimum link MTU is 1280 bytes. However, states that it is possible to have a functional QUIC connection on a path that only allows 1200 bytes of UDP payloads. Such a path would only allow HTTP Datagrams with payloads up to 1180 bytes. If we only send IPv6 packets over QUIC DATAGRAM frames, such a setup would result in violating the IPv6 minimum link MTU. We should describe this problem and how to react to it. Possible solutions include failing the CONNECT-IP request, using the DATAGRAM capsule, or different solutions altogether.
This is only a problem for the end-2-end QUIC handshake packets, right? I was assuming that you either have a QUIC tunnel that has a larger MTU in order to use datagrams or you have to use stream-based transmission for the e2e handshake and then need a way to switch to datagram later. In any case we need to say more in the draft!
From the most recent session, I think we may have consensus to address this by noting that CONNECT-IP over H3 requires a minimum link MTU of X bytes on every HTTP hop for IPv6 compliance, leaving support for smaller links to an extension.
I'm not sure the consensus was clear, but I think it's upon us to make a proposal and see if it flies =)
That sounds like the best path forward.
I propose a simple solution that should be enough for most use-cases: if any endpoint detects that the path MTU is too low, it MUST close the tunnel if the tunnel had IPv6 addresses or routes on it. Endpoints are not required to measure the path MTU. endpoints that wish to use IPv6 SHOULD ensure that their path MTU is sufficient. For example, that can be guaranteed by padding QUIC initial packets to 1320 bytes.
That seems reasonable to me as a proposal!
So how does the endpoint determine that the Path MTU is too low? In the scenario that looks like this C - I1 -I2 - S and it is the I1-I2 path that can't support 1280 byte datagrams. The I nodes are HTTP intermediaries. The client will not know that the datagram MTU size is too low on these nodes, to my understanding the datagram payloads will simply be dropped by the I nodes in either direction.
In that scenario, if the endpoint wants to check it can send an ICMPv6 echo request with 1232 bytes of data - if it receives an ICMPv6 echo reply that means that the path supports the minimum IPv6 MTU.
NAME then we are making it mandatory to send ICMPv6 echo to the external address assigned for the client, and for the server sending one to the client, this to determine that minimal 1280 bytes are working over MASQUE. Because to me there are no way the MASQUE endpoint can determine in all cases if there are HTTP intermediaries or not. Else we have to put requirements on the HTTP intermediaries to understand MASQUE, and to my understanding what was not desired. And maybe using ICMPv6 is the easiest method with the lowest implementation burden. The issue I see with this aspect is that many applications when establishing a Connect-IP path might not want to wait for the ICMPv6 response before sending other traffic so we have a situation where there is a small, but non-zero risk that these packets goes into a black hole, and then the ICMP response comes back and the MASQUE Connect-IP stream will be closed due to the response to indicate the failure. I think this is all manageable and not different from failures that can occur for other reasons.
I don't see why this should be mandatory. When I plug in an Ethernet cable into my computer, I don't know whether the IP stack on the next router spuriously drops packets above 1000 bytes long, and my OS has no requirement to measure the path MTU.
But for the said router dropping IPv6 packets below 1280 would be a protocol violation. The issue here is the interaction with the HTTP intermediaries and what is under the control of Connect-IP function. So Connect-IP basically provides a IPv6 capable link. Per IPv6 RFC that requires us to support 1280 at least. What have been the rough consensus so far is that we are not defining an fragmentation and reassembly mechanism, or rather how to ensure that one uses QUIC streams when the transport is HTTP/3 for below 1280 IPv6 packets and instead reject the service. Thus we are in the position of having to answer how does one know when to reject the service. The alternatives I see are: 1) Force the Connect-IP endpoints to ICMP ping each other to verify that the path supports 1280 IPv6 packets 2) Have the HTTP intermediaries be Connect-IP aware to the level that they can reject the HTTP request if the path does not support 1280 packets. Where 2) have the downside that we still likely have some black holes in cases where the HTTP intermediaries are not connect-IP aware and ignores the requirements. Where 1) have the downside that the endpoints request can't be rejected during the request response sequence and instead have to be closed with an error message indicating the paths failure to support the packet.
I'm opposed to requiring ICMP pings because the majority of use-cases will not have this problem. How about we add text like this: CONNECT-IP endpoints that enable IPv6 connectivity in the tunnel MUST ensure that the tunnel MTU is at least 1280 bytes to meet the IPv6 minimum required MTU. This can be accomplished multiple different ways, including: Endpoints padding QUIC initial packets when it is known that there are no intermediaries present Endpoints sending ICMPv6 echo requests with 1232 bytes of data an out of band agreement between one of the endpoints and any involved intermediaries to have them pad QUIC initial packets
So I understand your desire to not add complexity anywhere for a likely seldom arising case. However, I do fear that not addressing this with a clear assignment on whose responsibility it is to avoid the issue will result in that none does anything of it. Because an endpoint that uses HTTP/3 can verify its first hop. However, the complexities arise when you have HTTP intermediaries in the mix.
I get that, but since the complexity is added by intermediaries we just need to make it clear that the intermediary needs to not be silly
One part is to detect if the link is capable of 1232 bytes, however, if not, we also not to provide more guidance what to do. I think there are two options: either close the connection or use quic streams instead of data datagrams.

Related discussion:
text: an HTTP Datagram.  This prevents infinite loops in the presence of routing loops, and matches the choices in IPsec IPSEC. <ins> IPv6 requires that every link have an MTU of at least 1280 bytes IPv6.  Since CONNECT-IP conveys IP packets in HTTP Datagrams and those can in turn be sent in QUIC DATAGRAM frames which cannot be fragmented DGRAM, the MTU of a CONNECT-IP link can be limited by the MTU of the QUIC connection that CONNECT-IP is operating over.  This can lead to situations where the IPv6 minimum link MTU is violated. CONNECT-IP endpoints that support IPv6 MUST ensure that the CONNECT- IP tunnel link MTU is at least 1280 (i.e., that they can send HTTP Datagrams with payloads of at least 1280 bytes).  This can be accomplished using various techniques: if HTTP intermediaries are not in use, both CONNECT-IP endpoints can pad the QUIC INITIAL packets of the underlying QUIC connection that CONNECT-IP is running over. if HTTP intermediaries are in use, CONNECT-IP endpoints can enter in an out of band agreement with the intermediaries to ensure that endpoints and intermediaries pad QUIC INITIAL packets. CONNECT-IP endpoints can also send ICMPv6 echo requests with 1232 bytes of data to ascertain the link MTU and tear down the tunnel if they do not receive a response.  Unless endpoints have an out of band means of guaranteeing that one of the two previous techniques is sufficient, they MUST use this method. </ins> Endpoints MAY implement additional filtering policies on the IP packets they forward.