Code change:
Brought over the QUIC text about flow control performance. cc NAME
Thank you Cory. After the first sentence, I'd like to add the following because I'm pretty certain it is easily overlooked, especially in gateways where there's basically no way to figure the really available room:
Under what circumstances can that happen?
NAME basically any time on the forward path: if you only count on the local buffer you're using to receive a stream, the window will always be ridiculously low, while you know that you have plenty of buffers on the other side (e.g. forwarding to H1, which is dedicated to your stream, you can count on the H1 buffers and system socket buffers). When forwarding to H2 you can theoretically at least count on min(streamwindow,connectionwindow,buffer room+system buffers), though it can be quite hard to figure. The problem with a multiplexed protocol (when doing fctl-over-fctl like H2 does with TCP) is that your per-stream buffers are always smaller than the connection-level buffers (that include the system's huge socket buffers), which are usually automatically tuned to accommodate the BDP. So in the end relying on stream buffers only implies you can never reach the BDP and will be forced to limit the single-stream bandwidth. This problem doesn't happen with other protocols like QUIC simply because there's no need in-order buffering in the system that you have to rely on. In the end, a reasonable approach consists in having a configurable margin on top of the per-stream buffer room to figure what to advertise, but we need to warn about the risks of advertising too large a window.
Hmm, sorry NAME but I don't see how that relates to the original note you gave. Where is there a head-of-line blocking issue? Having small stream window sizes prevents a single stream reaching full network performance, sure, but that isn't a head-of-line blocking problem. It's a window management problem. If you have multiple streams, all with small windows but all of which can make progress, you can in principal avoid any issues with window management. What am I missing that brings HOL blocking into play?
Sorry Cory if my explanation was not clear. What I meant is that when you start to use reasonable (i.e. small) windows for your streams, you immediately see the network performance become horrible, despite all the buffering capacity around (kernel etc), only because of the streams window. I've had users complain about 5 Mbps uploads for example with 64kB windows :-/ As such it's extremely tempting to overprovision the stream window to make it cover some of the extra buffers available around that you're almost certain exists (i.e. the output buffer for where that frame is going to be forwarded), but when starting to play with this you risk HOL because it's often very hard to be certain about really available room (i.e. if you forward that over H2 and several streams consider that same room as available etc). Window-in-window is a very well-known anti-pattern and we knew it was going to hit one way or another and we have to live with it here. For most use cases where a browser is the sole owner of a connection, it's trivial to advertise huge windows and deal with it this way (no real risk that the browser blocks itself). When coalescing multiple agents it becomes quite visible a problem. I hope I managed to make it clearer this time, otherwise do not hesitate to let me know :-)
Thanks Willy, this is definitely clarifying things. So let me try to verify my understanding in bullets. The default per-stream window size (64kB), and other small per-stream window sizes, has performance problems over fat links. This is true. It is not necessarily true if you have multiple streams in flight and your connection window is larger, but it is definitely true for single streams. To that end it is tempting to advertise larger stream windows, to improve throughput. However that window may not be available in the target logical stream, such that if the user actually consumes it your application would have to buffer it. Possible reasons for this include the TCP send buffer not having room, the target H2 connection window not having room, the target H2 stream window not having room, etc. So this is definitely a resource management issue. Presumably the reason there's an HOL issue here is that, in response to needing to buffer on the outbound side, you stop reading from the H2 connection. I can think of a few ways to manage this, but this isn't really the forum for that kind of engineering chat. :wink: Instead, I'll say that I think the current text covers this. Specifically, the text says "bigger windows improve performance, but you need to bear in mind the resource exhaustion concerns elsewhere in this document". Should we be saying anything else more specific?
Well, I only partially agree, in that it's not exactly a resource exhaustion in the pure sense of the term as there can be plenty of resources available (socket buffers everywhere), but more a matter of difficulty/complexity to reliably measure available resources at any instant (i.e. the TCP buffer you have on one side is not necessarily dedicated to a single stream, so what is doable during a POST upload can suddenly change before you even have the option to advertise a window size change). I agree that this could be discussed elsewhere in the doc in more details, but my point was to at least put a short warning like the one I proposed to raise awareness of implementors against the risk of pushing this setting too far because "it seems to work much better". For the record, in haproxy what we've done to mitigate this on the download side is to avoid mixing multiple client connections over a single outgoing H2 connection. This way a slow client cannot block a fast one and it remains possible to advertise slightly larger stream windows that more realistically take into account available room on the path. For uploads, users generally consider that an uploading browser doesn't perform other operations in parallel over the same connection so it's okay to increase the advertised stream as well. But all these mitigations resulted from experience, were not obvious and will not necessarily apply to other implementations, which is why I thought a small warning could be useful.
Great, so do you think the current text covers it?
Looks great like this, thank you :-)
Thanks for the constructive discussion. No doubt we missed something (that's a constant risk here), but this looks like this change is a useful one. We'll track further improvements with new issues as the opportunity arises.
H2 strongly suffers from head-of-line blocking when streams do not progress at the same spead, typically when they come from coalesced connections from different clients. Per-stream flow control is generally ineffective against this as practical window sizes matching a BDP often require impractical buffering between the intermediary and the client. As such we should mention in the recommendations that intermediaries that coalesce connections either have very large buffers or try to group streams from a same client connection together on a server connection. Suggested by NAME
I'm not sure that I'm following the problem statement very well, so I'm not sure what sort of recommendation would go with it. Is the problem that when aggregating requests from multiple clients you expose all of those clients - and all of the requests on the corresponding connections - to head-of-line blocking on the shared connection? That doesn't seem quite right to me - though the requests that share that connection might suffer, the way that a stalled request on one connection might affect other requests on a different connection is limited. If a request affects others, is it because it would consume shared resources like flow control window and connection-level flow control? Keeping per-stream flow control relatively small relative to the connection limits would be the way to avoid that exposure, would it not? That said, larger connection-level buffers might reduce contention, but in general more buffering just means more latency. NAME am I chasing the right idea? Maybe you can provide a paragraph on the problem and I can try to massage it into shape for the draft.
The issue is that the intermediary that coalesces many client connections into a single H2 connection cannot have infinite resources to buffer store the response data for these clients, yet will have to advertise a significantly large stream window to the other end to maintain an acceptable bandwidth. In practice it most often goes well because your advertised window is lower than what can be stored into system buffers. But once a client stalls, all the chain stalls, and you quickly end up in a situation where the advertised stream window is ahead of what you're really able to store. Note that looking at how TCP buffers are filled in front is not always effective. You may very well be using H2 on the front with other streams making it hard to figure how much you could really push, you could have SSL under it having its own buffers, you could be applying compression in the middle, also making it hard to figure how much the output gives you on input, etc. In haproxy we've finally adressed this by deciding that by default multiple clients are not coalesced over a server-facing H2 connection. We have multiple strategies of connection reuse so this remains configurable, but this was the best compromise we could find.
Probably that we could propose something like this: Also, please note that the same situation happens the other way around, with multiple uploads over a single H2 connection to multiple servers behind working at different speeds. It's generally less of an issue though, but the large windows necessary to permit fast POST uploads may very well block the retrieval of small objects by the same client within the same connection.
Is this just an over-commitment of flow control then? That is, you advertise a window of N when you only really have space for M < N? What is interesting is the implication here that you might be forced to advertise the larger window because if you don't you can't get decent throughput, but to do so you are betting that most of that capacity will be used by in-network state (data that is on the wire or in routers and whatnot, along with all the in-system buffers you don't see). Of course, if that stalls, all that stuff in those buffers will all be forced into your buffers. That can affect all connections that are somehow entangled with the affected stream. That means all requests on the same connection, but it could affect other connections. All that said, that's much more involved than your text here. You appear to be looking for something much simpler. We should have advice about the connection-level flow control credit being backed by real resources. You say stream, but the purpose of connection-level flow control is to limit resource allocation. While the stream-level values more directly determine stream-level throughput, the general idea is that they can be set with minimal regard to resource management if there is a good connection-level limit (the main consequence of not jointly managing these is a range of complicated priority issues). The current document is not very good about saying much about any of this. So any addition along these lines, including a version of that text, would be worth adding.
When flow-control is set to values that the endpoint will respect, it minimizes retransmits due to buffer overruns. Minimizing retransmits does not always guarantee highest bandwidth; The more conservative one is with respect to preventing retransmits or buffer overruns, the more probable it is that some resources will be unused. This is similar to the tradeoff space of congestion control, however the buffers here are visible to the endpoints.
The thing is that due to the window-in-window you don't even benefit from the outer buffering (including the wire) if you have too small stream windows. With a single stream (or even H1) this problem does not exist at all as TCP acks at the edge. Here it's different, the data need to be delivered to userland and wait for its turn before the window can open again. I've seen POST requests slow down to 1 Mbps where the equivalent in H1 were 20 times higher using the same buffer size and TCP stack. So users bump the advertised window to compensate for this horrible performance and face random trouble. And on the server side the issue is the same but less visible, it mostly appears when the coalescing gateway and the servers are located on different datacenters a few milliseconds apart. But even at 4 ms with 64k windows it's only 128 Mbps per stream, so you can be sure that users are quite tempted to increase this for certain use cases where they see their devices twiddling thumbs on their 10/40/100G NICs. This is where it is possible to use the whole pipe's length when it's reserved or all streams of a given connection. What I'm seeking is not to fix the protocol, it is how it is and we all knew about this limitation during its design, but rather to warn users against the risks and effects of not accurately calculating the advertised stream window size in case of combined streams.
By the way, one easy trap to fall into is to advertise a window corresponding to the buffer used at the connection level, while what matters is how much you can extract from this connection to be able to immediately parse subsequent frames. And often you have no precise ideas of how much space there is outside.
Let's try to pull something in, maybe copy from QUIC: URL

Related discussion:
text: timely fashion.  Failure to do so could lead to a deadlock when critical frames, such as WINDOW_UPDATE, are not read and acted upon. <ins> 5.2.3. If an endpoint cannot ensure that its peer always has available flow control window space that is greater than the peer's bandwidth-delay product on this connection, its receive throughput will be limited by HTTP/2 flow control.  This will result in degraded performance. Sending timely WINDOW_UPDATE frames can improve performance. Endpoints will want to balance the need to improve receive throughput with the need to manage resource exhaustion risks, and should take careful note of dos in defining their strategy to manage window sizes. </ins> 5.3. In a multiplexed protocol like HTTP/2, prioritizing allocation of