jiebi/CodeConvo huggingface dataset:
There's still a difference when the response mixes strong and weak validators, right? If both types are present, the weak ones would get ignored. Is that intentional?
Yes, that is intentional.
is silent about what to do when there's more than one validator in the message. Freshening responses with HEAD (just below) specifies that any validators must match. (use of 'match' there is a bit fuzzy, should probably link to where it's defined)
I think this section would be better written in terms of entity tags and dates, rather than the abstract notion of strong validators. But I don't know if we have time for that. The basic idea is that a cache might have sent several strong validators in the request and is now being told that at least some of those validators indicate valid responses, so each validator in the 304 indicates a valid response. I don't think the origin server has a preference on which one, but in general the strong etag validators should be considered first.
... it's just deployments of very very old caches that would be of concern here. Given how unlikely this is in 2020, how broken they would be for other reasons, and the prevalence of HTTPS, it seems safe to modify these statements.
Is this warning still relevant? If we want to keep it, I think we need to qualify it; e.g., "A few old HTTP/1.0 caches that are unlikely to be used on the open Internet..." Similar text in cc: no-cache.
I think the warnings can be removed.
The practice of combining list-form fields into a single value is widespread, and the document addresses this pretty clearly. However it never really says that this combination needs to be limited to a single field section. It's pretty clear that trailers need to be kept discrete, but there is nothing so direct. It's probably worth adding "within a field section" somewhere.
Agree, but I can't find the definition of "field section" neither in MESSAGING nor in SEMANTICS :)
I was pointed to URL which is good, but it doesn't really say that this combining only applies to a single field section (and yes, that is not a term that is defined in the document as far as I can see).
Thanks, this works nicely. Shame that you had to add a new term, but we already got that in QUIC.
by explicitly specifying the namespace. Also consistently changes to the (b) notation to indicate a variable length sequence of bytes.
This PR got lost when merging 289.
Assume a setup with two publishers, a single relay and one subscriber. P1 ANNOUNCE's track namespace fooba P2 ANNOUNCE's track namespace foo S1 sends a SUBSCRIBE REQUEST for Full Track Name How does the relay know whether is Track Namespace: and Track Name: , or Track Namespace: and Track Name , and hence which publisher to route the subscribe to? I see a few possibilities: 1) The relay should not allow this scenario (via namespace construction and/or authorization). If so the draft should include text explaining that this is disallowed (normatively?) or "bad things(tm)" happen. 2) Change SUBSCRIBE REQUEST to be able to disambiguate the Track Namespace from the Track Name. This could be done by sending name and namespace in separate message fields, or having stricter rules for the structure of Full Track Name such that a relay can extract the constituent parts. 3) Explicitly state that Full Track Names are matched to namespaces via "longest prefix match".
Option 3 looks interesting. P1 can use a longer track namespace to steer the sub request from P2. Like a traffic engeering for content.
I like option 2. I don't agree with treating track namespace as merely a string prefix. Even if you go with option 3, there's still a race condition where a late ANNOUNCE will cause a SUBSCRIBE to an unintended track. I want to give the track namespace more properties so it's treated more like a broadcast identifier. A few folks are headed down this path without realizing it, such as proposing that there's a single catalog per track namespace. I don't see how that would even work if ANNOUNCE is used purely for routing like a BGP announcement.
NAME : are you saying you believe that the relay resolves the described ambiguity by using 'longest prefix' matched (eg: option 3)? Another option I hadn't considered based on this comment is that the relay would send OBJECTs from BOTH publishers since they both match the Full Track Name. This will break in the current design, since each publisher has their own Object Sequence and Group Sequence and when merged the consumer will be left with garbage.
It also violates the logic described that says a Full Track Name can be used as a cache key.
Yeah that seems quite problematic NAME Let's say a relay have two connected publishers who send IN ORDER: A subscriber connects and issues: What happens next? The relay forwards the SUBSCRIBE to publisher 1. The relay returns SUBSCRIBE ERROR notfound The relay forwards the SUBSCRIBE to publisher 2. The relay forwards the SUBSCRIBE to publisher 1&2. These are in the order of Alan's options. I added option 4 based on Suhas' comment. Would occur if the relay rejects publisher2's ANNOUNCE because of thr shared prefix. Would occur because the track namespace is an ID, not a prefix, and must fully match an ANNOUNCE. Would occur because publisher2 has the longer path prefix. Would occur because both publishers could_ produce the requested track. And to throw a wrench in things, a third publisher joins.
There's also the case where the relay replies to the subscribe requests on behalf of the publisher(s), so it doesn't have to make the routing decision at subscription time. But it does need to decide where to forward OBJECTs when they arrive.
Implementing this now so I have some more thoughts. We should explicitly support UNANNOUNCE and UNSUBSCRIBE, which implicitly occur when the connection is dropped. On ANNOUNCE, the relay MUST ensure that the namespace not a subset/superset of an existing ANNOUNCE. On SUBSCRIBE, the relay looks up the track based on the prefix and appends itself to a list of subscribers. If not found, it looks up the namespace and creates a track entry if valid. On UNSUBSCRIBE, the relay looks up the track based on the full name and removes itself from a list of subscribers. On UNANNOUNCE, the relay deletes any namespaces and tracks matching the prefix. A radix tree is used for both so the relay can iterate through any values that match the given prefix. This is a global radix tree, similar to how QUIC connections are identified when variable length connection IDs are supported. On ANNOUNCE, the relay inserts the upstream into the upstreams hash map and errors on a duplicate. On SUBSCRIBE, the relay looks up the namespace and then the track, creating an entry if not found.
On UNSUBSCRIBE, the relay looks up the namespace and then the track, deleting an entry if found. On UNANNOUNCE, the relay deletes from the namespace and track hash map. This is a strictly better than option 1. It provides the same functionality but faster, and without the restriction on namespace prefixes. Runs into issues with ANNOUNCE. example: and are subscribed to different publishers, despite using THE SAME NAME. This invalidates the claim that the track name is the cache key. We can't migrate to the new longer prefix from unless there's a guarantee they are producing the same content. A similar issue occurs when UNANNOUNCE is fired, when happens implicitly when the connection is dropped: Each subscription needs to keep reference to the track when the SUBSCRIBE was received, instead of doing a lookup based on full track name when each OBJECT is received. This will cause havok when resubscriptions occur, for example with ABR, as switching away from and back to a track MAY result in different content. It's not viable. The subscription would intermingle objects from separate publishers that happen share a prefix, all while using the same . I really don't want to support subscribing based on wildcards or prefixes anyway. This is also the slowest option, as there needs to be a global RadixTree that maps tracks to subscriptions on receipt of each object. All other options listed above would keep an array of subscribers per track, resolved using a (shardable) HashMap.
I think this seems like the right approach. Having 2 producers with conflicting namespaces is a bad idea, unless the authz policy and application is somehow OK with the consequences. I have added to clarify the thinking further
I'm of the view of two things 1) apps should not do this if it is a problem for that app usage of relay 2) relay should use the latest "announce" to override any earlier announces that are more specific ( or the same )
Discussed at IETF 117: Support in the room for changing "Full Track Name" from a single string to a tuple of Namespace and Track Name, and matching on Namespace exactly.
IIRC, my recollection was not the same, also I don't think we concluded that the namespace match exactly. The discussion went over what the authz model defines and how that impacts the flow.
I wonder how much the transport draft can mandate how a given relay application should implement the business rules. The transport draft must just indicate relay behavior and the course of actions to take for what happens when " if match succeeds vs match fails ", which is what is in the draft today already
Uh, my understanding of what we talking about what not at all removing Full Track Name - it was about we did not need to define a separator for concatenating things in a tuple. I don't agree there was support for chaning Full Track Name to Namesapce and Track Name tuple and only doing full matches. That is not going to work for many of the use cases and not what we are discussing.
I thought what came out of this discussion is 1) we do not know how duplicate ANNOUNCE work even when the names are identical ( do we keep first, last , error etc ) 2) once we figure that out, we can start to deal with the more complicated issue of partial match or overlapping Perhaps I have this recollection wrong but if so point me at right place in recording for that.
My motivation for ANNOUNCE was for the contribution case. A client like OBS will connect to the server, ANNOUNCE a new catalog track, and then the server will issue a corresponding SUBSCRIBE to get the data flowing. This is modeled after the QUICR PUBLISHINTENT message, except that data doesn't flow until the other side issues a SUBSCRIBE. However, announcing a namespace prefix subverts this behavior: The relay doesn't know if an ANNOUNCE is a prefix for routing, or a single broadcast being published. The relay effectively needs a business agreement with each client to determine. I think we should split this ANNOUNCE functionality into separate messages if just for the sake of discussion:
In the case of , a tuple makes perfect sense as it mirrors the arguments to SUBSCRIBE. This message would be primarily used for contribution to "push" a new broadcast. It also makes sense as a reconnect signal where the last message wins. In the case of , matching the longest prefix might make sense instead. I know very little about BGP but maybe there's some inspiration there. This message would primarily be used for inter-relay routing.
I assumed that full track name was a tuple already, and "full track name as a concatenation" was a notational convenience.
Now that I read the draft more carefully, it does send full track name in messages, rather than a tuple of track name and track NS. As far as I'm concerned, this is a mistake -- I don't think this actually works, and I have no idea how one is supposed to implement that. I believe "error" is actually equivalent to "keep first" here. I think it's a safety vs practicality trade-off. On one hand, if two different programs announce the same track, it will not work, since they will publish conflicting objects under the same name; if we disallow re-announcing, we will prevent that. On the other hand, re-announcing makes reconnects more practical.
NAME Sorry if I misread the room and/or the author's meeting. Let's continue discussing. That is being discussed in / , let's keep this issue focused on the SUBSCRIBE REQUEST ambiguity?
Individual comment: Just to illustrate the problem further here, I wrote a short sketch of a generic relay (python syntax): So the concrete proposal I have is to change to have two fields, tracknamespace and trackname. Then the first bit would change to: This is not to say the every relay has to do matches in this way. By squishing the namespace and name together it's not possible to implement a relay that does exact matching. If we put the tuple in , a relay that wants non-exact matching rules based on full track name can always construct it by concatenating namespace and track_name together. NAME Can you say more about the use cases that would break with a change like this, or suggest an alternate way to implement an exact matching relay?
In a recent editor's call there was some discussion, which split into three separate issues - capturing it here: 1) SUBSCRIBE REQUEST should have a way to denote which part of the Full Track Name is the namespace. Either a Track Namespace Length field or just transmitting Track Namespace and Track Name in two fields. There was consensus to add this. 2) Does relay matching behavior (eg: exact match vs longest prefix vs ??) need to be negotiated in band? () 3) Is there a way to subscribe to more than one track at a time (eg: wildcard subscribe, ). I'd like to keep this issue focused on 1.
Based on discussions with NAME NAME and others, there are potential key confusion attacks when the server is checking the signature using the wrong public key. I think the simplest solution is to add the public key to the TLS key exporter. That should solve the issue without increasing the amount of bytes sent on the wire
Including the public key in the exporter context makes it significantly harder to implement the server side of this. Including the hash of the public key in the signature appears to work also. I have a sketch model in Tamarin that doesn't find any attacks when the hash of the public key is in the signature, which gives me some confidence that it's doable, although NAME wasn't convinced. We should discuss this in the room at 117.
I agree with NAME The implementation complexity that comes from putting the public key in the exporter context is real, at least for some edge networks. I think we should pursue alternatives that don't weaken the security properties, even if they make the registration step a bit more explicit.
If this design makes implementation hard then we should totally change it. Can I ask you to elaborate on what causes this to be hard on the server side?
The basic problem in our particular case is that the component that terminates TLS and computes exported values is upstream of the component that has the public key. So when a request comes in with the header to the TLS terminator, the public key needs to get relayed back for computation. If, instead, the public key was signed, then the TLS terminator could simply forward the computed exporter without any sort of round trip. (I punched this in on my phone, but can draw pictures on a laptop if the above is not clear.) We don't necessarily need to cater to one implementation, but I would be surprised if this were not a common arrangement in practice.
Thanks, that makes sense. To explore the problem space: if we were to include the public key in both the context and a header parameter, would that work for your implementation? (That way the TLS terminator can call the exporter without a round trip.)
Yep, that would address the problem too.
SIGMA seems like it might be worth considering as an alternative. That is, sign the nonce, but then include a MAC that covers all of the relevant parameters.
Something like SIGMA should also work, but I'd like to minimize the number of primitives required to implement this.
Currently, modifying an HTTPS stack to support this draft only requires a signature implementation and a TLS key exporter API. Using a MAC would require a MAC primitive and opens the question "which MAC should we use?"
That's an easy question: you are using HKDF to get the values, so HMAC is a logical choice. The only reason I suggested that is that I'm increasingly nervous about these bespoke constructions. We understand what sign and mac provides.
I think we need to first identify what property it is we actually want, and then see what is the simplest way to get there. NAME what is the property we want?
NAME had a long chat about this today. We didn't come to any solid conclusions - the authentication property here is really subtle and the attacker model not yet clear. This is going to need some really careful work and detailed formal analysis to get right. In terms of this discussion, my personal take would be: I don't think a mac & sign approach is going to give you anything strictly better or different than an exporter + signature approach. In both cases its just a question of what you put in the exporter / mac. Having the public key conveyed in the header and bound explicitly in the exporter (rather than via the keyid) seems like a conservative and reasonable approach.
Thanks for thinking through this! I've written up to add the public key to both the context and the header
: "A server MUST ignore a Range header field received with a request method other than GET." : "Likewise, if the new method might have some use for partial response semantics (Section 14.2), it ought to document this, too."
I can see why someone might want to use it with SEARCH as well, though they would more likely use a page parameter. I can't think of any other use case.
FWIW, that's exactly how it was used by MS in Exchange SEARCH: We need to decide which of the statements in our spec is correct.
Huh, that's news for me. We could have used that example for Range extensions.
Change requirement to MUST ignore Range when not defined for that method or the method is unrecognized.
Updates some out of date portions of the IANA considerations section. See for more info
NAME Ping
NAME Ping
Some aspects of the IANA considerations section are outdated and should be updated: The “Cookie” and “Set-Cookie” registrations from RFC 6265 have been moved from the “Permanent Message Header Field Names” registry to the “HTTP Field Name Registry” Update Sections 9.1 (Cookie) and 9.2 (Set-Cookie)The word "Registry" should be omitted from registry names. Update Section 9.3 (Cookie Attribute Registry)
LGTM. Apologies I missed this.
HTTPSEM has been published as RFC9110, update all reference to reflect this.
NAME Can you PTAL? I spot checked that existing section #s still matched up, but I would appreciate another look.
The sections should be stable, see . I'd also change "HTTPSEM" to just "HTTP2".
Sorry, not sure I understand.
Oops. I meant to say: I'd also change "HTTPSEM" to just "HTTP". See URL
Thanks, the latest commit should fix that.
Still LGTM. Thanks!
This leaves only the Capsule Protocol mode
I do think we should add FINAL_DATA, but I'm fine with merging this first and then discussing that in
As of draft-ietf-httpbis-connect-tcp-05, connect-tcp now optionally supports the capsule protocol. However, this optionality adds non-trivial complexity to implementations. As currently written, if a client wishes to use capsules, the server can reject the use of capsules, and now the client needs to cache that this server doesn't support them. However it's undefined for how long the client should cache this information, and within what scope. We could potentially figure out recommendations for all these questions, but fundamentally it might be best to instead make capsules mandatory for connect-tcp, like they are for connect-udp and connect-ip. That would simplify implementations and improve consistency.
Individually, I think it would be interesting to try out this direction — probably based on implementation experience for how easy it is to add. On one hand, this makes connect-tcp less of a trivial drop-in. On the other, it opens up more possible use cases.
I heard a good deal of support for this direction in the HTTPBIS meeting.
Thanks!
This adds support for a server-provided sf-string "id" of up to 1024 characters that is echoed back in a "Dictionary-ID" request header when the dictionary is advertised as being available. This allows for a server to use something other than the hash to retrieve the dictionary (i.e. a cache key). This should be more flexible than the client sending the original request URL and will allow for servers to store arbitrary context information with the dictionary. It doesn't open up any additional privacy or security concerns because the dictionary hash was already being treated as if it was user-identifiable information. For
NAME could you please take a look and see if this meets your needs and looks reasonable?
Yes, looks great! Thanks!
WIP.
NAME NAME quick once over before I merge?
NAME thanks, updated, squashed.. merging.
:
: The and rules are not used in the document. The rule is not used in the document, unless you accept . The rule is not used in the document, but I think it can be useful to clarify the definition of , i.e.:
I agree that using "field-name" over there makes this a bit clearer.
Addressed in: URL
My understanding is that MoQ is always hop-by-hop encrypted by virtue of working over QUIC or WebTransport, and therefore a "relay" as described in the draft can only inspect Track/Group/Object metadata if it terminates QUIC. The document should clarify the requirements.
I agree with Alan. As a QUIC enthusiast, I guess the current draft implicitly suggests hop-by-hop QUIC termination at relay, e.g., which is necessary to enable cache at relay while having MOQT Message Type and OBJECT metadata as QUIC payload. Yet it is definitely more clarified for readers to explicitly see that information in an early section (e.g., sec 1.1.4).
Given MoQ is over either QUIC or WebTransport, I believe it's impractical to restrict it to only QUIC based transports.
The context of the question is whether one can build an MoQ relay that can view object properties without terminating the TLS session underpinning MoQ or WebTransport (eg an untrusted middlebox).
LG, but given Webtransport is also a substrate MoQ runs over, I think we want to avoid using QUIC connection?LGTM
I added three sections I feel are important: ) Credential Stuffing/Abuse callout in security considerations. Browser-based front ends have lots of security checks inherently built in to an IdP's flow. If we move the authNZ to be backend API based then it's important to call out to implementors that they should treat that API with the same security concerns as whatever API recieves request when a login form is submitted. ) New Grant Type needed. ) DCR Registration consideration and 4.) corrected minor typo Happy to discuss. I have other updates I'll add and submit PRs/issues for later.
[x] 5. I think you should come out and say that digest-algorithm values MUST be compared in a case-insensitive fashion. [x] 5. 'The IANA acts a registry' reads oddly; it maintains the registry. Also, saying 'the registry contains the tokens listed below' is misleading; these are initial registrations, which can be modified and added to after the RFC is published. [x] 5. There should not be RFC2219-MUSTs in IANA registry text. [x] This can be addressed by having a requirement similar to "Deprecated digest algorithms MUST NOT be used."
[ ] 5. Also, is the entire checksum value case-insensitive, or just the token identifying the value? NAME the checksum value is defined by the digest-algorithm, which defines its case. [ ] 5- Also, these might be better placed in IANA Considerations. NAME can we just drop any mention of IANA in this section? I like it. [x] * 5. The ability to have a digest algorithm that ignores content-codings is something of a surprise after reading section 2; it needs to be at least mentioned up there. NAME it is stated in the introduction URL . how would you specify it?
lgtm but keen to see mnot's approval too