Code change:
3.4. <del> When using the "connect-tcp" Upgrade Token, templated TCP proxies do not use the Capsule Protocol RFC9297.  Clients MAY request use of the Capsule Protocol by offering the Upgrade Token "connect-tcp-capsule" instead.  When offering or accepting the "connect-tcp-capsule" Upgrade Token, clients and servers SHOULD include a "Capsule- Protocol: ?1" header to facilitate processing by intermediaries. Clients of this specification MAY implement "connect-tcp", "connect- tcp-capsule", or both.  Accordingly, a templated TCP proxy server MUST implement both Upgrade Tokens unless its use is restricted to a subset of compatible clients. When using "connect-tcp-capsule", TCP payload data is sent in the payload of a new Capsule Type named DATA (data-capsule).  The ordered concatenation of DATA capsule payloads represents the main payload data stream in any protocol where this is well-defined. Intermediaries MAY split or merge DATA capsules.  Endpoints MAY indicate a TCP connection error by sending an incomplete DATA capsule, as an alternative to using TCP, TLS, or HTTP stream errors. </del> <ins> In each HTTP version, any requirements related to closing connections in Classic HTTP CONNECT also apply to "connect-tcp", with the following modifications: In HTTP/1.1, endpoints SHOULD close the connection in an error state to indicate receipt of a TCP connection error (e.g., a TCP RST or timeout).  Acceptable error states include sending an incomplete DATA capsule (as defined in RFC9297), a TLS Error Alert (RFC8446), or a TCP RST (if TLS is not in use).  When a connection is terminated in an error state, the receiving endpoint SHOULD send a TCP RST if the underlying TCP implementation permits it. In HTTP/2 and HTTP/3, senders MAY use an incomplete DATA capsule to indicate a TCP connection error, instead of (or in addition to) the signals defined for TCP connection errors in Classic HTTP CONNECT.  Recipients MUST recognize any incomplete capsule as a TCP connection error. Intermediaries MUST propagate connection shutdown errors, including when translating between different HTTP versions. </ins> 4.

Related discussion:
text: This leaves only the Capsule Protocol mode
I do think we should add FINAL_DATA, but I'm fine with merging this first and then discussing that in
As of draft-ietf-httpbis-connect-tcp-05, connect-tcp now optionally supports the capsule protocol. However, this optionality adds non-trivial complexity to implementations. As currently written, if a client wishes to use capsules, the server can reject the use of capsules, and now the client needs to cache that this server doesn't support them. However it's undefined for how long the client should cache this information, and within what scope. We could potentially figure out recommendations for all these questions, but fundamentally it might be best to instead make capsules mandatory for connect-tcp, like they are for connect-udp and connect-ip. That would simplify implementations and improve consistency.
Individually, I think it would be interesting to try out this direction — probably based on implementation experience for how easy it is to add. On one hand, this makes connect-tcp less of a trivial drop-in. On the other, it opens up more possible use cases.
I heard a good deal of support for this direction in the HTTPBIS meeting.
Thanks!