Change:
Proposed update to to get better H2 parity. Note: This is a PR on Alan's PR, rather than a replacement PR.
The current spec only allows GOAWAY to be sent from the server to the client and contains only the last processed bidirectional stream ID. This makes sense for the vanilla HTTP use case, but may not play well with extensions. Unidirectional streams are a flexible extension point, and it may be useful for each endpoint to communicate to the peer the last processed unidirectional stream id. An HTTP extension may also define a use for server initiated bidirectional streams, and may want the GOAWAY signal for these also. True if it's an extension then the extension author can also extend GOAWAY to cover these cases or introduce another mechanism, but it would be a shame if many extensions had to cover the same ground. I suggest we allow GOAWAY to be sent by either endpoint, and that it should contain both the last processed peer initiated bidirectional and unidirectional stream.
I was compelled by Alan's discussion about this. It sounds like you are suggesting having two fields in the GOAWAY frame, one for each type. My thought was to mirror the way MAX_STREAMS works and have independent GOAWAY frames.
I think this argument makes sense, but this feels increasingly like a transport feature and not an HTTP feature. If you add a unidirectional limit, I'd prefer a single frame, unless NAME has a reason he thinks separate frames are likely to be useful in practice.
I don't have a strong opinion here. For reference, the original issue about moving GOAWAY out of transport is URL For the separate frames topic: HTTP/3 currently lacks explanation about how GOAWAY works with MAXSTREAMS (see ). So my suggestion pre-empts that without much thought put into the final fix. Therefore it might be wrong. Some weak arguments: Before sending a GOAWAY, there is no explicit limit set by the server. The stream ID limit is implicit and real but more importantly, you don't invoke the requirement . Imagination required on how we fix this language up for MAXSTREAMS. From Alan's problem statement description, I imagined GOAWAY to operate more like a drain state trigger. So having separate frames would allow the draining of unidirectional traffic and bidirectional traffic independently. E.g. a client declares "I'm intending to leave soon, and I don't want to accept new things from you (pushes) but I want the ability to create new requests, so please continue to give me bidi stream credits". On a more practical level, a frame that includes both fields when only one type wants to be drained will likely end up with a set to max_int. So we'd waste bytes to express "don't do anything". Modelling this by making fields optional adds complexity for little gain. Finally, we a separate GOAWAY for unidirectional streams opens the door for it to be an extension frame. However, I think this would be the wrong thing to do as it would make managing extensions very difficult.
The salient comment from for me is "I think that the GOAWAY frame needs to identify requests, not streams" Making GOAWAY bidirectional and cover unidirectional and bidirectional streams seems to run counter to that statement, which is why it was starting to feel more transport-y. But, I don't feel that strongly about this, unless it becomes an implementation hassle of some sort.
I think the use case I'm not sure is compelling is the one where we only want to drain one type of stream, but no the other. My model of GOAWAY is that it's used for graceful shutdown, and I agree that could be useful on either side(moreso for non-HTTP applications, but maybe with WebTransport?). I have a hard time thinking of an application where graceful shutdown requires continued creation of one of the stream types, but they may exist.
I think this is the pragmatic argument for making a single frame. You don't want to make it easy for implementations to get into semi-draining states unintentionally. Sending GOAWAY with a limit of both forces that decision, which is good.
I had thought the H2 requirement was that after sending you will not accept or generate new streams, but I can't actually find that text in 7540. If we're concerned about being semi-draining, this seems like a restriction I'd want to have as well.
I don't think that we need to make special accommodations for possible future extensions, where the requirements for graceful shutdown are unpredictable. Strange. There's an example in h3. In order to answer a request, a server might create multiple push streams after the point that shutdown is initiated. The model for GOAWAY in h3 is based on the realization that the primary purpose of HTTP is to provide answers to requests. Stop the requests and you eventually run out of work. For the client, no signaling is required. For the server, it just has to have a way to block incoming requests. To Ian's point about identifying requests, I believe that it does (if the text doesn't support that, that's just a bug). Extensions can change anything. But we can't really say what the graceful shutdown model would be without understanding the extension. Assuming that once shutdown is initiated there will no longer be any need for new streams is not safe. I can easily think of cases where new streams are needed to fulfill existing commitments. Any protocol that treats streams as messages with an 1:N or N:M relationship between the message that initiates a transaction and the messages required to complete that transaction is open to this. Take a SQL or similar exchange where a transaction is opened, some queries and updates performed, and the transaction is committed or rolled back. A graceful shutdown might trigger rollback or it needs to allow for existing transactions to complete. If you want to allow transactions to complete, then you might have multiple streams created after the shutdown order is issued. And the SQL case is still simple, because it is still fundamentally transactional. If the protocol is truly bidirectional, with either peer initiating activity that might take time, then you need a more sophisticated shutdown process. The quanta used to measure what is accepted or rejected during shutdown might not be a discrete linear sequence as it is for simple transactional protocols; it could be multi-dimensional. Indeed, the reason for this issue is the possibility that h3 with extensions could end up being multi-dimensional. The stuff people are doing with bidirectional transport use could end up like that, especially where it is intended to be multiplexed with h3. Expecting things to line up around streams is probably reasonable, but when streams are used for multiple purposes, things get messy. You need to be identifying the unit. We get to use streams for GOAWAY because client-initiated bidirectional streams are exclusively used for requests, but we use stream identifiers only as means of identifying requests there (again, if the text implies otherwise, we need to fix that). So my view is that GOAWAY is good enough for shutdown of requests. But that is all we can realistically provide for at this layer. If an extension needs to be shutdown gracefully, then the extension that adds that thing will need to provide signaling in support of that shutdown. Practically, that means multiple frames involved in shutdown, but that's not terrible, just wait for all shutdown-y things to complete shutdown before tearing down the connection.
I think I'm fine with GOAWAY as is, but PUSH is never a required feature, it's a server-side optimization/pessimization.
This wasn't making a value-judgment of server push, just a note that there are cases that don't fit the model proposed.
To clarify, I was trying to note push is optional, and not required. In practice, I don't think PUSH would work in this case, at least in browsers. If the client receives a GOAWAY, then I wouldn't expect any new requests to go to that connection, including those which may be matched with results from the push cache.
That's true, but URLs may have already been promised (not yet pushed) and there may already be requests waiting to rendezvous with the pushed stream. WDYT?
That's also a detail of one particular push implementation, not an inherent way that push works. I think the Push implementation in WinInet would still work, because the push cache is at the level of a request group rather than an individual connection. (Somewhat moot for deployment use, but existence proof that there's more than one possible implementation of that.)
I currently see 4 options: 1) Do nothing and assume there are some implementation specific edge cases that are going to go fairly sideways, but they're unlikely to cause user visible issues. 2) Move GOAWAY to transport and have it include uni and bidi. 3) Add a max push id to the existing GOAWAY 4) Add a new frame for push. I'm ok with 1 and 2, and would be hesitant to pursue 3 or 4.
I prefer option 1. I dont see what 3 provides. I think we could achieve the same by adding text that says something like "the server commits to resolving all existing promises sent on streams up to and including the stream ID indicated in GOAWAY. Pushes may be cancelled or reset as normal". My 2p. GOAWAY is starting to sound like more of a STOP_CREATING (you have credit but it's a waste to use it in the current state) and PROCESSED (endpoint processed all entities up to ID, it is safe to retry higher IDs on a different connection).
NAME besides the theoretical interest of how GOAWAY works with extensions, do you have any HTTP/3 extension in mind where the current GOAWAY breaks badly? And if so, would it be feasible to add another GOAWAY_EXTENSION frame for that use case? ie: Can we just not fix this, or make some editorial tweaks?
The extension we are developing allows complete bidirectionality to HTTP, so a server can initiate a request/response to the client. The H2 mapping of this extension uses a client-sent GOAWAY to clue the server in to stop doing this. I don't know that missing this will break-badly; it probably depends on what application developers do with the bidirectionality. It's asymmetric with H2, seemingly because we think we got it wrong in H2 and are trying to correct it? I'm trying to say there is some value in keeping it symmetric. I think Lucas correctly identified the semantics of what GOAWAY is - STOP_CREATING and PROCESSED. As mentioned in the original issue, anyone writing an extension can also extend the GOAWAY frame under that extension that does what I proposed, so doing nothing is of course viable.
Discussed in London; we should mimic whatever H2 does, and there's some uncertainty in the room about what a client-sent GOAWAY is intended to mean. H2 GOAWAY says: So HTTP/2 does define it in terms of streams, and a server upon receipt of a GOAWAY frame MUST NOT open more streams, which would include push. The other side probably can't police you, however, because they can't reliably know that the opening of the stream wasn't already in flight; they'll just ignore it.
It was mentioned in Montreal that the HTTP WG was asked about this. Was the response reflected back in this ticket? apologies, this one completely popped off my stack
To summarize the discussion on the HTTP list: H2 GOAWAY is symmetric A GOAWAY sent by the client will prevent any pushes higher than the last-stream-id in the GOAWAY Martin feels like this behavior is wrong, and it shouldn't be carried over into H3. Do we need to build consensus from the HTTP working group that it is OK to deviate from H2 semantics in H3, like we are now planning to do for priorities?
While this is true, we have - as a working group - previously discussed the principles underlying this and agreed that the unit we should care about is the request. However, I agree that this is something we should discuss at the HTTP working group. Where we can each put on a different set of hats.
What's ill defined is whether the pushes that come on a request are part of the request or not. I think a reasonable argument can be made for either interpretation.
A different use case came up: Consider streaming a (potentially infinite) live video as a sequence of pushes. How does a client signal its intent to close this connection, and stop the server from pushing more video segments? The client can send a STOPSENDING frame on the request stream which is receiving the PUSHPROMISEs, but this may result in less than graceful termination of that stream.
In this case what do you advertise MAXPUSHID as? By limiting the value and not incrementing it, you can cause the server to be unable to drive the connection, and allow the client to hit its idle timeout.
Yeah, limiting credit via MAXPUSHID is one way, but I still prefer the explicit signal from GOAWAY - it's like immediately zeroing out the credit.
Agree. This is like writing a cheque but asking the other side not to cash it (out of band) to avoid it bouncing and causing pain all round.
Even with a client sent GOAWAY, you'd receive a few more pushes within the next RTT. If you managed MAXPUSHID like flow control and only gave out enough extra for the time window you wanted to shut down within(ie: 1-2RTTs), I believe this achieves the desired behavior? If that time window is <1RTT, then your only real option is CONNECTION_CLOSE?
I think it's more about challenging the assumption that stopping sending requests (or push credits) is the same as explicitly signaling your intent to close the connection. If the endpoint knows it will shutdown, it signals the peer, and the peer can take actions in response to ensure the close is graceful.
NAME Can you clarify what your current proposal is? Is it to move GOAWAY to transport or keep it in H3 and make it bidi?
Discussed in Cupertino: Clients can send GOAWAY. Add transition considerations to talk about relaying GOAWAY from an H2 to H3 connection / vice versa. We would like the client-sent GOAWAY to carry a Push ID. There is still disagreement over what the precise semantics of that are. NAME to write a PR to provoke argument.
The semantics of an identifier in the server-to-client GOAWAY are twofold. The identifier clearly identifies requests that were not "processed", which allows the client to retry those requests. The identifier also identifies which of those requests can be cleaned up expeditiously. My view is that the former is significantly more important. Enabling clean retry is one of the best features of h2. For pushes, the need to signal whether the data was "processed" is moot, because only safe requests can be pushed. So most of this comes down to what optimizations are enabled by the explicit Push ID. In h2, requests are automatically "cancelled", though the bits are still retransmitted by TCP if the connection is kept alive. A client isn't required to reset or finalize any associated streams if the requests were not complete, but the transport will automatically repair lost segments. In h3, the same applies, though there is an optimization that might be enabled. The transport layer can be told not to retransmit data for "cancelled" streams if packets are lost. This doesn't require explicit RESETSTREAM or STOPSENDING frames, reprioritization might be enough. Depending on the transport implementations, endpoints might need to use RESET_STREAM to ensure that transport stacks are aware of the need to suppress retransmissions. From that perspective, an explicit Push ID would enable new capabilities in terms of suppression of unnecessary retransmission. For example, say that there is an open request that is receiving associated pushes for data that arrives on a fixed cadence. Say that a client wants to move to a different connection (because it was told with Alt-Svc, for instance). The client might initiate a request starting at a particular time on a new connection. When the client cancels this connection, there might be a number of pushes in flight. If the client can precisely identify the push that immediately precedes the request it initiates on the new connection, the server can identify which of these pushes need to be fully delivered, and which can be abandoned. However, this optimization really only works if the client can make this identification precisely. If there are multiple requests in flight on the connection, the client might not be able to identify or predict the Push ID that will be used for the next push in the sequence. The server might avoid this by providing promises well ahead (>1RTT) of the actual pushes so that the client doesn't need to guess, but that might not be possible in the general case. This also requires that Push IDs are used strictly sequentially with respect to the global clock. This doesn't guarantee clean separation even if Push IDs monotonically increase for each resource. For instance, if two resources were providing the same sort of pushes, response 0 might promise Push ID 10 for time=44, but response 4 might promise Push ID 9 that corresponds to time=45. That makes it difficult for a client to make a clean cut with a single value. This also doesn't interact well with a model where the answering of a request depends on receiving a set of pushes. If you have a resource that produces a continuous set of pushes on the same connection as a resource that (ideally) produces a fixed number of pushes (see also Prefer-Push), then the cut for the continuous resource might be clean, but it might split the pushes for the resource with a fixed set of associated pushes. Of course, all of these arguments could easily be countered by saying that this is just an optimization, but given that it's an optimization we don't have for h2 (because TCP), I think that I'd prefer not to provide the capability. Finer-grained signals might be valuable: such as the idea that individual requests might have separate budgets for push that can be progressively expanded by clients as they are ready for more. That seems like something worth considering for future versions of this protocol.
makes a subtle but major change to the semantics, and I think we need to carefully consider whether that's something we want. We were conflating these two models during interim discussion, and I don't think we're clear on our direction yet. Instead of "Clients MUST NOT send new requests on the connection after receiving GOAWAY", we now have "Endpoints MUST NOT initiate new requests or pushes on the connection with an identifier greater than or equal to the smallest identifier received in a GOAWAY frame." In other words, GOAWAY() has changed from meaning "I'll service your in-flight requests, but don't make any new ones" to a no-op, since the client couldn't have initiated such a request anyway. I understand that this might be a desirable pattern in the server-to-client direction, where certain use patterns consider the set of pushes part of the fulfillment of the client's request, and the client actively wants the server to continue initiating pushes until the request is complete. This doesn't seem a wise semantic to bring to the client-to-server direction. In HTTP/2, the recipient of the GOAWAY was prohibited from initiating any more activity (requests or pushes) regardless of the value in the frame. My first preference is to preserve that, and actually have the same semantics as HTTP/2.
I agree with NAME that's a useful semantic we should preserve in HTTP/3.
I think Mike articulated a point I was trying to make on Slack but in a much better way. +1.
Ah, I was trying to get the behavior NAME was after for pushes (eg: let the server keep pushing if it wants to), and didn't intend to allow the client to send more requests after receiving a GOAWAY -- that doesn't work. But it sounds like we'd prefer to keep the push behavior symmetric with h2 (no more pushes after GOAWAY receipt) than introduce a new push semantic where the client can allow pushes to continue even after signaling its intent to close. This is more along the lines that NAME was advocating for - where the intent to close and how pushes are wound down are completely decoupled frames. The discussion on slack I thought was different - it was about what the sender of a GOAWAY should do with new incoming streams. I intended the PR to reflect that the sender of a GOAWAY should continue allowing requests or pushes with ID's smaller than the sent GOAWAY value. Do we agree this is the right behavior?
The sender of GOAWAY needs to continue accepting incoming streams up to the indicated ID, yes. If we decide the other semantic is useful, it's easy enough to define another frame (SOFTGOAWAY, INCONCLUSION, etc.) in-spec or as an extension that simply hints to the other side that they should start cleaning up, but doesn't fully prohibit new streams.
It sounds like LAST_ORDERS would be the classic optional-to-understand extension -- "I'd like you to wrap up gracefully now, so it doesn't hurt when I forcibly close in a few seconds." Can we agree on (and update the PR to) keeping H2 semantics of "no new requests/pushes" after receipt of GOAWAY in either direction, and leave that for another day? I've created for this direction.

Discussion:
A client has more flexibility in the value it chooses for the Push ID in a GOAWAY that it sends.  A value of 2^62 - 1 indicates that the <del> server can continue fulfilling pushes promised while processing outstanding requests, and the client can continue granting push credit as needed (see frame-max-push-id).  A smaller value indicates the client will reject pushes with Push IDs greater than or equal to this value.  Like the server, the client MAY send subsequent GOAWAY frames so long as the specified Push ID is strictly smaller than all previously sent values. </del> <ins> server can continue fulfilling pushes which have already been promised, and the client can continue granting push credit as needed (see frame-max-push-id).  A smaller value indicates the client will reject pushes with Push IDs greater than or equal to this value. Like the server, the client MAY send subsequent GOAWAY frames so long as the specified Push ID is strictly smaller than all previously sent values. </ins> Once all accepted requests and pushes have been processed, the endpoint can permit the connection to become idle, or MAY initiate an