jiebi/CodeConvo huggingface dataset:
In case there is no rough consensus to keep federation section, I raised URL to remove it.
I'd suggest we've had a few different conversations about federation of everything in a transparency service, vs. selective promotion of a subset of information across Transparency Services. The later being the ability for a consumer of one or more Transparency Services to promote the content for a subset of their software. Microsoft ships lots of software, However a consumer may only want to promote the Office suite of products into their Transparency Service for internal validation. I'd suggest the wording, as current, isn't well defined and I'd vote to cut this section until more well defined.
If this PR is dropped then we should create an issue to address the underspecification of unpotected headers at registration, especially the idea to preserve unprotected headers. Federation is a simple use case that highlights this problem
Its not simple, and we have seen no implementation support for it to date.
If we are at the point where we need to start from scratch on configuration/registration policy and scrap the initialization section, I personally do not see how we can keep federation and not just remove it (not to drive you and others mad, we can always add back later).I think the new text is too specific. To me (and as in the prior text) Federation refers to having multiple TS, and one TS accepting receipts from another TS at registration time is just an advanced case, for which we don't have clarity yet. As discussed earlier, letting the TS decide what unprotected headers to authenticate in its receipts is a major complication. My preference would be (1) keep a general placeholder on Federation --- better than nothing; or else (2) cut the section. But this leaves many questions unanswered, starting with the two TS in the arch diagram.
Strengthened requirement for content identifiers for header fields to be lower-case (changed from SHOULD to MUST). Relaxed guidance on processing Creation Time and Expiration Time to only require verifiers to examine them and test against the verifier’s requirements. Minor editorial corrections and readability improvements.
The current text allows mixed-case header field names when they are being used as content identifiers. This is unnecessary – header field names are case-insensitive – and creates opportunity for incompatibility. Instead, content identifiers should always be lowercase.
The processing instructions for Creation Time and Expiration Time imply that verifiers are not permitted to account for clock skew during signature verification.
This can safely be fixed adding a -like claim, so that the signer remains in control of the signature validity. This is especially important if the signature is created with the signer's private key. imho this kind of document should not define a clock-skew tolerance threshold, and implementors shoud be suggested to fix their clocks. There may be corner cases (eg. IoT/disconnected devices ) that we should address properly
I'd split the clock-skew part in a separate PR. The rest LGTM.
Based on the discussion in URL, this PR would allow servers to repeatedly generate 104 responses to inform the client about the upload progress. The formulation allows the client to free associated resources (e.g. file buffers) because the server guarantees that the upload offset will never be less than the last reported value. Closes URL
If this is not a mandatory part of the protocol, I wonder how the client would know that the server supports it. It's possible that the client has already forgot about the initial bytes beyond the first ACK when the ACK is received.
I don't think the client must know in advance whether the server will send these additional informational responses. This mechanism is an additional enhancement, but not much should be lost if the server does not implement it. Yes, that can happen, but this can also occur in the current draft version. If the client forgot bytes that come before the reported by the server, it cannot continue the upload. The only save way for the client to regularly forget data is by chunking the upload into multiple requests using and thereby getting a confirmation from the server about the new upload offset. The addition of 104 for progress information does not change much about this approach. If you want to be sure about the offset, you should chunk the upload. But the 104s provide a slight improvement, where the client can already forget data before the request is completed. In a sense, the client should never forget data before the request is completed or a 104 is received if the client wants to recover from upload failures. This PR should just be an enhancement for the client, but not pose any additional issues for it. Does that make sense?
Based on the feedback from and IETF 118, I adjust the text in this PR to clarify what header fields can be present in informational responses: Upload creation: Server may send one or more 104 responses The first 104 response must contain the Location header, subsequent responses must not. The Upload-Offset header is optional Upload appending: Server may send one or more 104 responses The 104 response must not contain the Location header. The Upload-Offset header is optional In theory, this allows a 104 to be generated with no Location or Upload-Offset header, but I don't see a reason against disallowing this.
The text now also ensures that the client knows the upload resource URL before it receives a 104 with Upload-Offset. This avoids implementation issues where the client might free upload data before it knows the upload resource URL to resume the upload in cases of failure. Please let me know if these changes improve the draft's readability.
If we are adding strict incrementing requirement to 104 responses, should we also do so for plain offset retrieving?
I would prefer so for consistency.
In a message to the mailing list (URL), Austin Wright (NAME proposed to use informational responses to send information about the upload progress from the server to the client. During the Upload Creation Procedure and Upload Appending Procedure, the server could regularly send an informational response with the latest offset in the Upload-Offset header. The value should reflect the amount of data that has been received and safely stored by the server. This way, the client knows what data has been successfully transmitted and is not needed for resumption anymore. Any resources attached for this data (e.g. memory buffers) can be released then. In contrast, If the client only monitors which,data it has sent, it cannot be sure if the server properly stored it yet. There is a short example of this how communication could look like (note: it uses server-generated upload URLs (URL), but it could also be done using client-generated tokens, of course):
I think doing this is fine, and I agree that it has benefits as pointed out above. General guideline of HTTP is that servers are allowed to send any number of informational responses. There's no reason to not use that. Regarding the status code, I think we can reuse 104.
+1 Kazuho
I think we need some communication between client and server to enable this. The protocol is designed to avoid depending on 1xx status codes. Does the client need to explicitly opt in to this? Can the server opt out? What if the server doesn't support 1xx? If we want to keep this optional without adding a new header field, a possible design is: If the server wants to send periodic updates, it can start by doing so in the initial 104. The client can enable additional buffering behavior based on this information (but it might have already dropped some bytes by the initial 104). This is a server driven solution and the client won't be able to opt out.
Or maybe URL
I'm not fussed on the specific response code value but the 102 seems deprecated, for whatever that means, and the linked wording seems at odds with the system design we want
It's not deprecated, but wasn't carried over to WebDAV-bis due to lack of implementations. It's in the registry, so it's a real HTTP status code. That said, I now agree it doesn't fit well, as 102 is for situations where the client is actually waiting for a response (as opposed to still sending).
I guess that's accurate to the historical record, after a very cursory delve into RFC 2518 and 4918. But I wonder how clear that interpretation is to the broader HTTP world. MDN has a perspective you might want to follow up on. Yeah this is where I'm hitting a hard edge. A registered status code, where the behaviour, the important stuff, is described in an obsoleted RFC doesn't really seem great.
FWIW, I would push back hard on that line of argument. The code is registered, so it's a valid status code. And yes, a spec that wants to use might want to either include an updated definition, or make sure the definition is moved to a separate new document.
I also think that 102 is OK to use here (but maybe a new status code is still warranted, see below). The 102 isn't depreciated as a bad idea or because of another better solution, it was merely omitted from recent updates to the WebDAV specification for lack of implementations. I imagine another specification could take over ownership of the definition/registration. I think the only reason I didn't specifically write 102 in my email is, I've to indicate "upload complete, the server is now performing additional processing" and it may be useful to distinguish this signal from "upload has been partially received and committed to storage". The former is probably closer to the intended usage, so it may still be good to mint a new status code for the latter usage. Alternatively, both cases might be distinguished by the HTTP fields that are returned, and maybe servers want to indicate both at the same time ("50% committed to disk, 20% processed").
I share the view that 102 does not fit very well here.
It would be awkward to send an URI to which the missing part of the request should be sent, while indicating that "the server has accepted the complete request" (). If we are to choose from an existing status code, 100-continue might be a better choice, though I would prefer using a new status code considering that we did so in introducing 103. All 1xx codes are interim and therefore kind of implies continuation. We can burn a new status code for a particular case of contiuation, which Resumable Upload is.
I opened a PR for this in URL
I think this would be a good addition to the spec as long as it's not mandatory for the server to send this data. As previously stated in this thread It would help the client to discard in-memory data that is no longer required to resume. Would it make sense to also include the location header in the additional 104 responses? This would make the solution more robust in case one (or several) of the 104 responses get lost and the extra header field would not add any significant amount of extra bytes in each request.
Yes, this would be optional. I am unsure how big the benefit would actually be. This would only help if a 104 is lost without the connection being destroyed, so the HTTP response is still open and a subsequent 104 could be received. Maybe this is possible with HTTP/2 or HTTP/3, but I don't know that. Other than that, repeating Location would be redundant.
1xx response are allowed to be dropped by intermediaries. But I'm not aware of any that would drop some in a request. Typically they just swallow all or pass all.
NAME Is that so? I could be wrong but states that a proxy MUST forward 1xx responses unless the proxy itself requested the generation of the 1xx response.
I suspect what NAME meant to say is that some proxies do drop 1xx responses, whether they're allowed to or not. Also, some HTTP libraries consume all the 1xx responses without exposing them to the caller, even if the library itself doesn't do anything with that particular 1xx code; such a library is indistinguishable from a poorly-behaved intermediary.
My concern isn't so much proxies dropping one or more response as much as e.g.
switching network and losing one or more of the 104 response (think spotty Wi-Fi moving over to 4G). Another input here would be consistency, especially if we add the header to the first 104. I agree that the benefit might not be huge but it's not zero either. It does however seem like the consensus in Prague was that this is not needed and I can get behind that.
One or more of the 104 responses could be dropped, along with the final response, due to the connection being interrupted, but selective 104s can't be dropped due to network issues. Clients need to deal with failed connections no matter what, so I don't see how this is an issue.
NAME Right, then this is a non-issue that can be skipped. I just wanted to bring it up so that we do not miss aspects. :)
Thanks for the feedback! I adjusted URL to clarify what header fields can be present in 104 responses.
Micha proposed to change REVOKED to INVALID, as it gives less intention whether this is a permanent state and the issuers could reflect a suspension also with a 1bit size status list as well
I'm in agreement with Kristina, the more I look at this the more I think we should have one possible enumeration of statuses rather than allowing every issuer to define their own, the question then becomes how big that set needs to be as it controls how many bits are allocated to each credential in the status list.
I'm fine with defining the first bit as valid/invalid. I think providing the option for additional bits offers benefits with acceptable tradeoffs
I've made some changes, please recheck NAME NAME
umm I thought we agreed to remove the concept of the "typ" attribute.. so no and no .
Yeah that was captured here NAME URL
But I assume the Status List Token will have a JOSE header to properly distinguish it from other JWTs, right?
I approve, but moved to separate issue
NAME typo on my end. yes, JWT should be typed ( claim present), the current spec also introduces a body claim as far as I can see, which is what I am not in agreement with.
NAME I am confused. so the idea is to merge this PR and then remove the "type" JWT claim in PR ?
comment edited, I got confused here as well. There is an issue open already as a task and I would like to not blow up the PRs too much
NAME ready for merge
I am a little confused with calling "VALID", "INVALID" Status List , but agreed this should be mergedOne minor typo
Inspired by some discussion on : while just the word "security" is indeed used in marketing literature for proxies, it's meaning to different parties is so varied so as to not really convey much useful information. In the IETF we try hard to be clear about what security provides we need and/or provide, avoiding the vague catch-all term. However (as I am reminded out of band), the audience of this document is not exactly security experts, and it is easy to go overboard trying to be precise, at least in this instance. [actual commit message body retained below] The mere act of inserting a proxy into the chain does not, in and of itself, do much of anything for the security of the system (and a badly implemented proxy can make the security of the system much worse). A proxy can, however, provide security services such as auditing access, annotating content from untrustworthy sources, exfiltration avoidance, etc. It is these services that are the security-related motivation for using a proxy, so say that "security services", rather than just "security", are being provided by the proxy.
Some diffs are due to me reflowing text to fit the line limits, so please ignore those. Once we have this and a limit on announce IDs, it seems likely that we'll be able to get rid of 'Role', but for now it can stay.
Thanks NAME . this looks like a good improvement. I have few clarification questions and bit of extra work for you to add some informative text, if my comment seems ok to you.
Unfortunately, that makes it much more difficult for the server to tell abusive clients from valid ones, as we discovered with the recent HTTP/2 rapid reset attacks. In the end we did develop heuristics, but they're imperfect and in the process we broke a bunch of applications in an effort to defend against DDoS attacks. In contrast, QUIC had a system like this and experienced no issues and we didn't need to develop any new heuristics.
Currently, there's no way to limit the number of simultaneous subscriptions. QUIC stream limits cannot effectively be used to limit the number of subscriptions, so I'd suggest MoQ have a limit. The simplest idea would be to have a limit specified as a parameter in the server's SETUP message. As a related question, even with limits, what happens if a client subscribes to a number of streams that it doesn't have the bandwidth to support? ie: 10 4k video streams and they're on a slow link? Possibly the subscriptions could fail with an error code indicating that the server didn't have enough bandwidth to request them and/or serve them to the client?
rofl I just came here to post the same issue. See: URL I think we follow QUIC and have a MAXSUBSCRIBES frame. The initial value can be set in the SETUP message. The subscribe is considered active until the publisher sends or (TODO). I think we're okay with the subscriber/publisher using arbitrary subscription IDs if there's a single control channel. It would be easier if we required incremental subscribe IDs and you couldn't reuse them though.
Also do we need ? That seems like a potential attack vector too, since it involves inserting into a potentially CDN-wide routing table.
A few months ago, I would have said we should go with the HTTP/2 style design of just having a limit on number of streams, but given the recent issues, I'd be inclined towards the QUIC style of specifying the max number. But I think that works better if we also have a subscription ID that's monotonically increasing for subscriptions, ie: Announces also likely need some limits, but I'm less clear on how that might work.
I propose a monotonically increasing in the message. You can't reuse Subscribe IDs. We add a message containing the maximum allowed ID and possibly a SETUP parameter to set the initial value. The corresponding messages use that ID instead of echoing the full track name. This removes ambiguity and overhead. We would still have Track ID for compressing the full track name (maybe renamed?). You could have multiple SUBSCRIBE reference the same Track ID but contain different Subscribe IDs. For example: Same for .
Adding a Subscribe ID used as you're describing SGTM and as you said remove any ambiguity about what subscription is being referenced. Unless we are absolutely never going to have multiple subscriptions, I believe we'll need a Subscribe ID.
This is related to
may be better to name it as transaction_id . since it is id tied to a given subscribe transaction ..
As a follow-up to adding a Subscribe ID, we'll add a SETUP field or param to indicate the initial value and a MAXSUBSCRIBEID frame to increase the limit, similar to QUIC flow control.
The consensus at the session today was to go with a QUIC style approach, so I wrote There was interest in a similar approach for announces, but announces don't yet have an ID, so that will have to wait.
Will assign to Mike English
This seems like a really complicated way to limit the subscribes. I understand how it mirror streams but that had pretty different constrains. Why don't we just have an subscribe error of "too many subscriptions"? Discussed open issues in the 7 August 2024 virtual, not recorded here. Remaining issue (just do a SUBSCRIBE_ERROR?) went to the list and chairs declared rough consensus, although at least 2 participants would prefer not to have this.
This strengthens the language around Structured Fields so that it's clear they're preferred, and targeted fields are now primarily defined as structured fields. It still leaves using a Cache-Control parser as an option. If we want to go further, we could remove the paragraph starting with 'However,...' and the two bullet points below that.
Reverted in URL for the record
Right now, parsing of targeted Cache-Control is specified like this: The idea here is that while we'd like the interop benefits of SF, we recognise that some -- perhaps many -- implementers will have a strong motivation to reuse their Cache-Control parsers. This seems like it might be the worst of both worlds, in that we now have two possible implementation strategies. Given that there are several now, I wonder whether should reconsider this. E.g., 1) Strengthen the first MAY to a SHOULD, making the field sf-first, or even 2) Require SF, don't allow reuse of CC parsers
Reopening to double-check that we still want to allow CC parsers.
To be clear: the remaining issue (as discussed at the interim) is whether we want to remove the text allowing parsers to be reused for this field.
This seems like the right balance: this is a profile of SF that a CC parser will accept, assuming that a CC parser won't choke on parameterization of "no-cache" or similar.
Spotted another mention of accidental while I was at it, so changed that too.
From NAME WGLC review: Corruption could also be non-accidental. Let's remove the accidental qualifier or add words about explicit manipulation (for good or bad purposes).
Agree, but I think that accidental/bad implementation has to be mentioned. See URL iirc there were some comments regarding the fact that Digest alone (eg. without other mechanisms) is not enough to protect against malicious agents... NAME do you remember anything on that? The current text imho is correct, since it adds:
NAME NAME I'd close with no-action :)
NAME Removing "accidental" from the sentence in question would fix the issue.
I completely agree, I'll do that!
LGTM
reorder section on Referenced Token and Status List Token URL
I did a first review of this draft. As a newbie to the topic, I found the following issues: I miss a description of the fundamental concept, i.e. the status list itself is a JWT (issued by the same issuer issuing the JWTs), the status list JWT is obtained from a URL hosted by that issuer or another party, the link between the JWT and its status list is provided through data in the JWT. I suggest to add this in the Introduction (or an Overview) section. When reading Section 3.1. for the first time I was confused whether it describes the status list JWT or the JWT referring to the status list. I think it is the latter and it should be clear to the reader on first sight. I think the order of definition should be changed. Right now, I'm supposed to understand how a status list JWT is used in a JWT without even knowing what that status list JWT is. I suggest to change order: 1) Overview of the concepts, 2) specify status list JWT (structure and where/how to obtain), 3) embedding in a JWT (syntax, processing rules, ...).
Agree on 1+2. As mentioned in we need a better word describing the "Token" that is being described by the status List. I will make a PR using "Subject Token" as this comprises both JSON and CBOR formats. Agree on 3 as well.
For reordering it would make sense to get forward with the existing PRs first and do it afterwards..
Addressing some of these issues in
What's left to do: reorder Status List Token first and Referenced Token afterwards