jiebi/CodeConvo huggingface dataset:
I made few comments on the PR, but one thing to note here too: I think recommending claim per comment is inconsistent with a comment that says that "this draft intentionally doesn't prescribe how keys are distributed, identified, selected, trusted, etc. any more than JWT/JWS do. so I think recommending is redundant and using in examples is sufficient.
That latter comment was about (and in reply to a suggestion about) validating the signature on the Issuer-signed JWT and checking that the signing key belongs to this Issuer. While the former comment is about the holder's public key inside the Issuer-signed JWT itself. Different things so I don't believe there's really an inconsistency in my statements. Nor do I think that being a little more a bit more prescriptive and more clear about using RFC7800 cnf/jwk to convey the Key Binding key is redundant. Rather this review of a review seemed like a good opportunity to provide a little more clarity and guidance in an area the draft that could benefit from such.
one nit, otherwise looks good - thank you, Brian!
Clarifies . 1- + HEAD is used to get a resource checksum 2- no Digest in request for clarity 3- only use sha-256 to focus on the actual example.
Digest can be sent in requests and response Nope, in the example Digest is both in the request and in the response. See the hidden 607 line URL When used in conjuction with signatures, the client that want to certify that no-payload was attached will: compute the Digest of an empty string sign that Digest header with some signature mechanism attach the signature and digest to the request send the request with no payload data The server will then: verify the signature according to the chosen mechanism if no payload is provided, the Digest is the one of an empty string eventually store the signature to prove that the request had no payload Clearly the signature mechanism should provide enough information to define the semantic of the request (eg. the method and all relevant headers). HTH, R.
Well. So the Digest is for the empty request payload. Maybe it would be good to state this in the spec? (That said, this is IMHO a very misleading example, as the HTTP spec is very clear that GET/HEAD request bodies ("A payload within a HEAD request message has no defined semantics; sending a payload body on a HEAD request might cause some existing implementations to reject the request."). It would probably better to have this example on a request that actually can take meaningful payload...
+1
NAME PTAL :)
I'll repeat what I said earlier: "That said, this is IMHO a very misleading example, as the HTTP spec is very clear that GET/HEAD request bodies ("A payload within a HEAD request message has no defined semantics; sending a payload body on a HEAD request might cause some existing implementations to reject the request."). It would probably better to have this example on a request that actually can take meaningful payload..."
In order to make progress here, I suggest we simplify the example to remove the Request digest. That keeps the example focused on it's name "Server Returns No Representation Data".
NAME removed Digest from the HEAD request.
NAME PTAL ;)
NAME PTAL :)
YAY :rocket:
In 10.2 Server Returns No Representation Data: what does the Digest request header field on the HEAD request refer to, and what is the recipient supposed to do with it?
Fixes URL
nit: can you wrap the lines at 80 chars?
In the call last Wednesday, we agreed this design was the correct approach, so merging even though there are some reviews outstanding requesting changes.
Consider an edge relay, receiving and forwarding relative SUBSCRIBE requests for -5, -3, and -1 for the same track. The first object it receives back is group 37. It cannot tell where to place this object in the sequence and must therefore hold it until it knows more information. If the next object is Group 39, it still cannot forward it, because the head sate is undefined. We can solve this problem simply by having the SUBSCRIBEOK response indicate the latest group and object number: SUBSCRIBEOK { Track Namespace (b), Track Name (b), Track Alias (i), Expires (i), LastestGroup(i), LastestObject(i) } One consequence of adding these fields is that an edge, which has no active subscription for the track, cannot send the SUBSCRIBEOK response until it itself receives a SUBSCRIBEOK after forwarding the SUBSCRIBE. This makes subscription an async process in which the acknowledgment of the subscription may take in some cases a RTT back to the origin. I think this is necessary for stable state management.
+1 this is required to build a relay and a non-realtime player. This was required anyway, otherwise the relay would swallow any application SUBSCRIBEERROR codes. My mental model is that SUBSCRIBEOK == HTTP response headers.
Moqt is a pub/sub protocol and is hop-by-hop protocol. A subscribe ok means the susbcribe request was validated at the relay. A OK isn't tied to getting a OK from the original publisher. Its sub/sub-ok is a async operation ( like any typical pub/sub systems out there). An ok for subscribe means a promise that if there is a matching data ( either in cache or coming in from a live publisher at some point), it will be forwarded .
Clarifying question , if the use-case is something like VOD or non real-time flows, I think the same can be done by
I absolutely don't want this behavior. It means that a generic relay will always return SUBSCRIBEOK, even if it cannot route (or authenticate) a SUBSCRIBE. A typo or finished broadcast would be indistinguishable from a valid request unless OBJECTs start to flow.
SUBSCRIBEOK would be equivalent to HTTP 100 CONTINUE, which is truly useless, and there would be no equivalent to 200 OK or 404 Not Found. Pub/sub doesn't mean black hole simulator either. I've only used RabbitMQ a long time ago, but if you tried to bind a queue (ie. SUBSCRIBE) to an exchange (ie. ANNOUNCE) it will return an error if the exchange doesn't exist. Is the justification that pub/sub protocols are supposed to black hole, or is this behavior that you actually want? Instead of hanging indefinitely if a relay can't route a SUBSCRIBE, wouldn't it just be better to receive a 404 Not Found? You're allowed to try again, but now there's no timers or guess work as to if the SUBSCRIBE could be routed to an origin.
It's for higher latency targets. HLS/DASH clients don't start at the latest segment, but rather start further back based on the target latency. If I understand correctly, you're saying the relay: has an empty cache. receives a and . issues issues a live subscription upstream: waits for the first OBJECT to arrive for that subscribe, using it as the live playhead: issues a backfill subscribe upstream: forwards any objects based on the mapping from relative to absolute. It works for relays, but at the cost of an extra round-trip to origin. However, it won't work for clients that want to start playback N groups back (jitter buffer size), because the player doesn't know when/what to render without first receiving N distinct groups. Here's what we would like: has an empty cache. receives a and . (re)issues a subscription upstream: waits for forwards any objects based on the mapping from relative to absolute.
Individual Comment: I think SUBSCRIBEOK needs to resolve all relative Subscribe Hints that were passed in the Subscribe. Otherwise, I'm really not sure how my library will know when an Object arrives if it belongs to a particular subscription or not -- the OBJECTs that arrive have only track name and absolute group/object numbers. I see the use case for some relays wanting to optimistically SUBSCRIBEOK, but then there would need to be a follow up message that resolves the relative hints when the relay finds out where they map.
Couple of observations what use-cases would involve same client sending multiple subscription hints at the same time ? Hints are read access points on the same track and if a client ask for multiple such points, it will get objects on the same track regardless .. how relays puts them together in such a case for a given client cannot be normatively specified NAME can you give me a concrete use-case example of "resolve all relative Subscribe Hints that were passed in the Subscribe" from a single client so that it helps understand the issue better. I personally don't think the issue being discussed is directly related to this ...
The use case of an edge relay forwarding simultaneous requests from multiple connected clients. This is a common use-case for any edge or fan-out relay.
Individual Comment: This is definitely one use case. Another is a client making two simultaneous subscriptions: eg live edge and range request starting some time in the past.
One thing that will not scale is every subscription needing to go back to the original publisher. Consider the cases of a million users are joining a session that starts at the top of the hour and that is published from a mobile device. I think that in any real system there is also going to be no single idea of the live edge. The live edge is relative to which relay you subscribed to. If the relay you are on is over a geo sync satellite, it is going to be behind ones that are not. So here is what I think we should do. We should not allow relative subscription to things in the past. I think the uses cases I have heard for this can be deal with as subscribe to live edge forward and as once the client find out what object that is, do an absolute subscribe for any data in the past to fill in buffers etc. So to be clear, no relative group or object sequences that are negative but still allows absolute. This seems to meet the use cases and remove a ton of edge case design problems. To be clear, I think we should be able to subscribe to the current group, but give me object 0 forward in that group. If it is a group per GOP, this allows one to get everything needed to decode the current frame.
But not every subscription has to go back to publisher. It only has to go back to a node that is already subscribed and understands the live edge. Furthermore, concurrent requests would be coalesced at each node.
Imagine in the worst case that 1 million requests are made within 100ms to a (really powerful) edge. That edge would only forward one request to the origin. All the rest would be held pending that response. This is exactly what happens when a few million people ask for a live HAS stream. Due to fan out and request coalescing, the CDN only sends a few requests back to the actual origin.
Yeah exactly. Only the first SUBSCRIBE will propagate upstream and the SUBSCRIBEOK is used to bootstrap the live edge. The relay can infer the live edge while the single upstream subscribe remains active. Because of the SUBSCRIBEOK, the relay knows that the received OBJECT only gets forwarded to the first downstream subscribe. Otherwise it would have to guess the live edge. The relay keeps track of while the upstream subscription remains active. The initial value is set by but all future updates are based on received messages. This way is can map from relative->absolute and reply with without issuing a new upstream subscription.
For the original use-case reported on this issue, if there are multiple subscribes at a relay with varying relative offsets, the relay has no choice but to send each combination of subscribe and a given -ve relative offsite upstream since that identifies each unique data point in the track. This has couple of issues it doesn't scale across relays and also when there are very high rate of subscriptions across relays and clients it makes the lookup and state management complicated latencies across relays and client locations ends up producing varying results for the same query since it depends on when the query was made and how long would it take to for conferencing cases when there are multiple publishers who can be active (have live edge), unjoined yet ( so no live edge) , the expectation for subscribes to get a OK back from original publishers will not scale in terms of latency of joins The more i think of it, subscribing back into paste using a relative offsets and expecting the answer to be vetted by the original publisher "unnecessarily complicates the protocol".
The group sequence numbers are consistent between subscriptions . If a relay receives relative subscribes of -5,-3 and -1, all it has to is send a single subscribe upstream of -5. The response would give it the live edge group as well as all the content for the various subscriptions.
If the subscriptions were -345, -2, -1, the relay could decide to make two subscriptions upstream - one for -345, the other for -2, because it didn't want to force the smaller relative subscriptions to wait for the larger one. The point is, this is optimization logic for the relay. It does have a choice in what subscriptions it sends upstream based on the timing and diversity of the input subscriptions. Why is the latency of the join affected? Consider a subscriber talking to an edge where the subscriber is 100ms away from the origin. We consider two scenarios: in the first case the subscriber receives a SUBSCRIBEOK immediately after subscribing but then waits 100ms for the audio and video to arrive from the origin. In the second case the the subscriber waits 100ms to receive SUBSCRIBEOK but it is immediately followed by the audio and video. So in both cases the subscriber gets its audio and video after 100ms. The media latency is consistent under either scenario.
If the relay logc is to send only a single subscribe upstream, then you don't need anything to be said in Subscribe OK either. Since when the first object arrives it can compute the live edge by adding 5 to the object being reported and the compute the absolute values for others (in this case for -3 and -1). That would definitely make things simpler for sure.
For cases where a client wants to go back 345 groups in past, i will be very much inclined to do an absolute request after learning the latest known state of the group as an end application OR something catalog can describe more information to make the right absolute request on the first go
It can't do this because the order of items it receives as Streams is not guaranteed. The origin may send the groups in order -5,-4,-3,-3,-1, but they could be received at the relay (or any hop) as -3, -5,-4,-1,-2. That's why you need an explicit notification from the origin as to where the live edge is. Only that plus the group sequence numbers allow the correct sequencing, not the order of arrival.
In this case there was just a single subscribe request with -5 was sent upstream, based on your explanation above. This avoids all the confusion which would happen when there are multiple inflight -ve offsets.
This seems like it will have many race conditions.
I've opened issues for the things that were previously TODO comments in the draft; this removes them from the draft itself.
Issues and for posterity
NAME wrote: Public Keys is maybe worth considering. This too is this is an area for further discussion and consideration should this draft progress.
My take: keep the scope of this narrow. If raw keys are needed - or other certificate types for that matter - then we can define another field.
I concur with Martin's take. The TODO made its way into the individual draft (and subsequently became this issue) based largely on this brief suggestion in one part of an off-list email on the topic, "While I do not know of a use case yet, it might be worth keeping raw public keys in mind as a use case." Which is not unreasonable but I'm going to favor narrow scope here over speculative need.
NAME wrote: might be needed/wanted by the backend application (to independently evaluate the cert chain, for example, although that seems like it would be terribly inefficient) and that any intermediates as well as the root should also be somehow conveyed, which is an area for further discussion should this draft progress. One potential approach suggested by a few folks is to allow some configurability in what is sent along with maybe a prefix token to indicate what's being sent - something like Client-Cert: FULL \ \ \Client-Cert: EE \ as the strawman. Or a perhaps a parameter or other construct of {{?RFC8941}} to indicate what's being sent. It's also been suggested that the end-entity certificate by itself might sometimes be too big (esp. e.g., with some post-quantum signature schemes). Hard to account for it both being too much data and not enough data at the same time. But potentially opening up configuration options to send only specific attribute(s) from the client certificate is a possibility for that. In the author's humble opinion the end-entity certificate by itself strikes a good balance for the vast majority of needs and avoids optionality. But, again, this is an area for further discussion should this draft progress.
I would propose a separate header for the chain that would use the structure List value format.
I'd also argue strongly for enabling the certificate chain (even if as a list of sha256 checksums of entities in the chain).
End-entity certificates are effectively namespaced by the rest of the chain, so not including the full chain and/or not tying the end-entity cert to the chain are some of the top foot-guns I've found with people using this.
I like NAME suggestion: separating the EE certificate out is sensible. The assumption here is that the terminating proxy validates the signature from the EE cert when establishing the connection. That makes it different. That said, the origin server might need to assemble and validate the chain, so including all the data presented in TLS is likely necessary. (I don't think hashes are as generically useful; space savings can be achieved with header compression, assuming that you use VERY large tables.)
From a header compression standpoint, I suspect we should note that multiple instances of the header will compress more efficiently than a single instance with a long list, assuming there are multiple possible intermediate cert paths.
NAME wrote: http-core. This should be pretty simple to add, but for the purposes of having a clean PR pulling out the TODOs, I'm moving it to an issue for now anyway.
URL points to draft-ietf-httpbis-semantics for this
NAME wrote: possible with HTTP1.1 and maybe needs to be discussed explicitly here or somewhere in this document? Naively I'd say that the header will be sent with the data of the most recent client cert anytime after renegotiation or post-handshake auth. And only for requests that are fully covered by the cert but that in practice making the determination of where exactly in the application data the cert messages arrived is hard to impossible so it'll be a best effort kind of thing.
This change passes 'null' to the content-negotiation algorithm if the request header is absent. For list headers, the algorithms can always turn that into the empty list, but for non-list headers they need to come up with something appropriate to do. This also tries to do better about , but may not completely fix it.
Thanks.
URL says "... For each variant in variants-header ... Let request-value be the field-value(s) associated with field-name in incoming-request." I don't see any handling of the case where there is no header in . The algorithms in URL then also don't handle the case where request-value is empty. Sorry if I've just missed where you handle the empty case.
add a note about cert retention on TLS session resumption (to
Hi, given that I had the question today relayed from a customer by our support team, I had a second look at the draft to see what was planned for resumed connections, and I'm seeing this: The problem is, on resumed connections, the client does not present the certificate again, it will use a ticket or a session ID to match a cached entry or any other mechanism I'm not familiar with (please bear with me, I'm not SSL-fluent). In our case when this happens, the related fields are empty (since the info is missing from the TLS stack), and we encourage customers to log something else as a complement to match new connections against a previously verified session (e.g. session ID). I think it's important to explicitly state that such a certificate info is not necessarily available in such a case, and that either the proxy+application have to find other ways to persist the information between connections, or that the cases where the certificate is matched need to be narrowed down to the strict minimum. For example, I remember a decade ago when working for a bank that the security team absolutely wanted to distinguish really authenticated connections and resumed ones and that in particular they refused to consider as 2FA something that resulted from a resumed connection because they wouldn't be able to prove that the authentication happened at this precise instant if they had to defend this in court. After the different versions of TLS evolved, I'm not sure whether there is a stable session identifier that's available across multiple versions, but I do think that if we document a common method to pass this client-cert info to the server, we also need to provide "something" for all the resumed connections that do not convey that info anymore, otherwise users might fall back to poor solutions due to the difficulty of understanding all the details of the various versions (and it's very likely that I've been one of those giving such wrong advices, not knowing how to do better). It's particularly important in load-balanced environments because you cannot count on the front LB to keep a copy of the client-cert, as the resumed connection might very well end up on another node, and you cannot count on the server either since different requests will often reach different servers.
So the only thing that remains is that "thing" that the client uses to be recognized on resumption (sorry for the imprecise language, it seems for me that up to TLS1.2 it was a ticket but I'm not sure for 1.3).
Hi Willy, This is likely a limitation of your TLS implementation. A perfectly reasonable limitation, but not something that is consistent across all implementations. What happens with resumption is that any state from the initial connection can be carried across over into a resumed session. Obviously, things that happened on the first connection after a ticket was issued can't be assumed to be available, but generally anything that happened in the original handshake could be used. What the specification assumes is that the details of the handshake are remembered and can be used again. There's a bunch of other stuff like which TLS version and whatnot, some of which the specification requires endpoints to remember (see for example , which insists that the KDF remain the same). When it comes to authentication state, however, TLS is not very consistent and so implementations are not. Some implementations (like NSS and Boring SSL) remember the client certificate. The way they do this is they put the whole certificate chain in the session ticket. That makes the tickets pretty large, which can badly affect the resumption handshake: whatever you save on sending certificates is eaten up with a huge ticket in the first message. Worse, this first message is bad DoS mojo as you have to accept a whole lot of state before you are really sure that the client is OK (there are ways around this, but they are all pretty bad). Others might save the certificates server side, but that adds a whole different type of fragility. Others just "forget" that the client offered a certificate. That last one is probably what you are seeing here. This is a whole new sort of problem to solve. What you might need in this case is a way to recover an old certificate from a log based on some identifier. The problem here is that the identifier isn't stable either. TLS doesn't have a single consistent value you can use across all versions and all types of resumption.
Hi Martin, thanks for the detailed explanation.