Code change:
The reason for the SHOULD is that the current text says: do a key update as described in {{key-update}} prior to reaching these limits. Note that it is not possible to perform a KeyUpdate for early data and therefore implementations SHOULD not exceed the limits when sending early data. We could of course change both.
DTLS 1.3 introduces integrity limits for the AEAD algorithms, which is good. TLS 1.3 does not have any AEAD integrity limits, probably because the assumption that only a single AEAD forgery can be attempted. My understanding is that this is not true for 0-RTT data which works more like DTLS and where an attacker can attempt any number of forgery attempts on a single AEAD key. RFC 8446 has relatively strong anti-replay protection: "The server MUST ensure that any instance of it (be it a machine, a thread, or any other entity within the relevant serving infrastructure) would accept 0-RTT for the same 0-RTT handshake at most once." but this does not stop an attacker from attempting an unlimited number of forgeries. I think consideration / mitigation of this is missing from RFC8446. Some alternative suggestions: Should the 0-RTT requirement above be strengthen to: "The server MUST ensure that any instance of it (be it a machine, a thread, or any other entity within the relevant serving infrastructure) would process a 0-RTT handshake with the same PSK, URL pair at most once." should RFC8446bis continue to ignore AEAD limits for 0-RTT? 0-RTT data already has lower security (weaker replay protection)? In this case I think some considerations should be added describing that the integrity protection of is weaker than for normal application data. should RFC8446bis introduce DTLS 1.3 type integrity limits for clientearlytraffic_secret? This aligns the integrity limits of 0-RTT data in TLS with the normal application data in DTLS.
Could you say a bit more about how this would work in practice? I did not consult the spec just now, but I had the impression that 0-RTT still had only a single attempt at forgery (at least, per replay cache instance/backend server).
Ah, the linked DTLS issue mentions long-lived external PSKs specifically. I agree that our story for those is less well specified and probably not great. Off the top of my head, I would suggest "don't use 0-RTT with long-lived PSKs".
I think that the original is fine. I don't think that there is any meaningful distinction between 0-RTT handshake and the (PSK, URL) tuple, at least as far as the derived keys go. Change the PSK, PSK identity, or random and you get a new key derivation. I had expected that early data was subject to the same constraints as later epochs. The only caveat being that it is impossible to update early data keys using KeyUpdate. The answer is to finish the handshake of course, so a hard error is fine. We might advise servers to cap maxearlydata_size to help avoid any issues of overrun, though with certain perverse padding arrangements, that might not provide any real guarantee. As above integrity limits should also apply.
URL fixes (1) The case for (3) is actually more subtle, as the problem is not clientearlytraffic_secret but rather the trial decryption skipping phase. We might need to apply DTLS 1.3 like limits there.

Related discussion:
text: an analysis of these limits under the assumption that the underlying primitive (AES or ChaCha20) has no weaknesses.  Implementations SHOULD do a key update as described in key-update prior to reaching <del> these limits. </del> <ins> these limits.  Note that it is not possible to perform a KeyUpdate for early data and therefore implementations SHOULD not exceed the limits when sending early data. </ins> For AES-GCM, up to 2^24.5 full-size records (about 24 million) may be encrypted on a given connection while keeping a safety margin of