jiebi/CodeConvo huggingface dataset:
Chrome and Safari treat it as LF in those places, but they do not allow two CRs to end the header block (network error in Safari, empty body in Chrome). It does seem that a stricter stance is possible here, given the intersection of the results. cc NAME NAME
If we can be stricter here and just fail on CRs in headers, that sounds good to me. That having been said, I think changing behavior to anything but just rejecting the response would be worse than the status quo (Treating CRs as part of header values, for instance).
Yeah, it seems like CR CR could be treated as a network error at least. CR "after" reason phrase or individual header is less clear. But I agree that making CR part of the value isn't good.
Regarding at the end of the line, we currently say?
NAME Fetch Metadata will add a bunch of headers that are supposedly structured headers, but given their simplicity can be compared using equality on servers. If servers did that it would likely prevent adding parameters in the future however or making other supposedly-compatible extensions. Should we accept this for headers clients transmit to servers or should we do something akin to TLS's GREASE?
In general, greasing HTTP headers is interesting -- although I'm a little wary, given the extent of the grease-related discussions we've had in QUIC. As a baseline -- I think that the goal of grease should be to prevent casually incomplete implementation (e.g., someone using regex or splitting on commas and extracting the raw values). It can never assure complete and correct implementation overall. I think the interesting thing to grease in SH is parameters, since that's a primary extensibility mechanism, and is (now) uniform across values. Ideally, grease parameters would contain values that frustrate the casual approaches above; e.g., strings with commas in them. A couple of options I can see; We could advise / require sending implementations to generate extra parameters sporadically (not super-often, so as to limit bandwidth consumption, but not so rarely that it creates debugging nightmares). To be recognisable as grease to the receiving application, their names would have to have some convention. We could advise / require headers using SH to define their own greasing mechanisms when appropriate. I suspect that truly automatic greasing might increase friction against adoption of SH, since some people's reaction will be "what is this thing doing to my header?!?". Invoking in on a header-by-header basis (perhaps by adding an appendix with guidance) might help.
I think the main benefit is derived from clients doing this so the main party to convince would be the user agents (who are already generally on board with SH).
So, should anything in the spec change? At this point the most I can see is adding something like:
I guess I'm mainly interested if implementations are willing to do this uniformly. NAME NAME NAME
I think NAME suggestion is reasonable to recommend (though it risks a quagmire when we attempt to ship a new header defining a quotient of quietly quixotic quilting), but I'll defer to NAME or NAME to weigh in from Chromium's networking team's broader perspective for headers generally.
I am, unsurprisingly, in favor of GREASEing things on general principle. :-) Although the network stack only produces a few core headers while others, like Fetch metadata, mostly come from outside anyway. GREASE would want to be applied where the header is produced, most likely. (Well, we could implement some generic post-processing scheme, but then all the headers would need compatible mechanisms and we'd need to know which they are.)
I also think this sounds reasonable. One problem is reproducibility - if we modify a header 1 in 100 times, then a developer would have trouble reproducing the issue. If we apply the same modification per browser session, it would become a new cross-site tracking vector, unless we also took NIK into account when deciding whether/how to modify the header.
For TLS, we've never fed state in as input to GREASE, which is the safest option privacy-wise. But you're right that you then need to worry about reproducibility. To that end, if we GREASE some TLS codepoint, we always GREASE it, with the randomness coming in only in the particular values, as a low-pass discouragement against hardcoding values. But if the implementation is intolerant to only values that start with 5, it is indeed unlikely to be reproducible enough.
If we always grease something, couldn't that result in problems around servers always expecting an extra value?
It could, but that'd be a really weird bug for servers to have, whereas not properly skipping over unknown keys is much easier to do on accident. (Maybe I just parse the whole thing with a giant regex and make assumptions about the order.)
I don't think it's that weird - if I want to test parsing something with a variable number of fields, I'd test on 0, 1, 2, and possible something much greater than 2 (like 3!). I think it's very easy to mess up on any one of those cases, and get the others right. Could assume an initiate search for a delimiter will always succeed, or use a regexp with "+" instead of "*", etc.
Beyond the text above, we could add a character to those allowed to start keys, and reserve it for greasing. E.g., . I think I'd do that purely at the parse/serialise layer, so a grease value never appears in the actual data model.
Is that attractive?
NAME I'm not sure if that is beneficial. I think our goal is to let every HTTP extension that uses a SH list or a SH dictionary to be extensible. To achieve that, greasing has to carry a value that can be recognized by the extensions. Assuming that that would happen for a fair number of HTTP extensions, the SH codec shared among those HTTP extensions would encode / decode arbitrary number of undefined values. I think that would be sufficient.
Maybe you can use "X-". One of the reasons we avoided reserving a range of values was to make grease values a little harder to separate from genuine values. The point being that they need to be seen by the code that processes them without filtering. From that, I take two conclusions: A prefix won't work as it is too easy to distinguish. Having a generic processor strip grease values would mean that the code we want inoculated would never get the opportunity to benefit from our attempts at inoculation. Therefore, I suggest that we find a subset of valid strings that we will reserve. Maybe that is all strings of 8 characters or more that fit a certain pattern. We don't have any established pattern for strings, but anything should do as long as new values can be generated cheaply and it doesn't take too much of the available space.
I'm going to disagree on pretty much all points, NAME The target here is assuring that a consuming implementation isn't grepping, string-mangling, etc. If they're aware of it being a SH and actively NOT using a SH parser, there's not much we can do, reasonably. To that end, is just as effective as, and hiding it from the application is a benefit, because we don't pollute their data model, we don't make them think about it, and once it's parsed into the data model, they CAN'T interfere in the ways we're trying to prevent.
NAME what does "recognised by the extensions" mean? Are you saying that a recipient has to be aware of grease's presence AFTER parsing to serve the purpose?
NAME Let me use an example. Consider the case of the Priority header field. The header would be generated, encoded, decoded, and used in the following manner: We would definitely want to grease (4), so that it would be possible to define new keys.
Therefore, there is no benefit in letting (3) absorb the keys used for greasing. We should prohibit (3) doing such thing. That is my argument. Who adds the greasing keys is debatable. That could happen purely in (1) or in cooperation between (1) and (2). Or we could define a common vocabulary, so that it could be done purely in (2). I do not have a strong opinion here.
OK, so you want to protect against an additional case -- someone using an SH parser (rather than grep, etc.) but raising an error if any parameters are present. Personally, I'm not as concerned about this case; someone is going to have to go out of their way to write that code. What I'm concerned about is lazy programmers doing some quick string hacks.
While I might agree that we might not need to be concerned about this case, I wonder if there would be a benefit in requiring an SH parser (3) to absorb certain keys, assuming that most, if not all of the extensions that use SH dictionary or SH array would allow future extensions. How about saying that "a SH encoder MAY add a greasing entry (i.e. a key that starts from ""), unless the extension using SH forbids that", and that "a SH decoder MUST NOT absorb the greasing entries"? That would give us end-to-end greasing by default with least hassle.
I'm not against that. The only thing bothering me is that "unless the extension using SH forbids that" means that APIs will need another bump on them...
Yeah we can remove "unless", if the key names used for greasing is not going to be a valid key name in terms of the spec. E.g., grease keys added by SH encoder starts with "", while applications are forbidden from using a key starting from underscores.
That's what I was suggesting up-thread :)
Yeah you stated "do that purely at the parse/serialise layer," and my complaint was absorbing the values in the parsing logic. Sorry I wasn't clear about what I am complaining about.
SGTM - anyone else have thoughts?
Looking at how to specify this, having grease parameters appear in the data model don't make me very happy; we either have to make them valid but tell people not to use them (which is likely to be abused), or we have to make them invalid but say they might appear anyway, just to be ignored, which is weird. Either way, it's confusing to write/read the spec, and I suspect some people will use the grease prefix for some purpose because it works (most of the time). Stepping back, I think the scenario we're talking about is the possibility that a consumer of a specific SH header (i.e., post-parse) is likely to erroneously reject headers that has unrecognised parameters, even when the header allows them (keeping in mind that some header definitions might disallow them, even though that's not a great extensibility strategy). To me, that's not the purpose of greasing, because there are lots of other possible erroneous constraints that a consumer (rather than an SH implementation) might put on a header, and we don't try to prevent that. If someone wants to purposefully reject something in a way that's contrary to the header's spec, they're going to do it. Rather, greasing (to me) is about making sure that someone who's ignorant of the spec / too lazy to read it doesn't just do the "natural" thing and e.g., feed an integer into a parser, or split a string on commas. So, to me adding parameters / dict members and "disappearing them" at parse is adequate. I'm not against addressing the case NAME points out, but I don't want to complicate things to achieve it. Do other folks on this thread have any leaning in one direction or the other?
I wouldn't say that that is mutually exclusive with greasing each user (e.g., priority). Obviously it is true that the SH spec would be simpler if we grease only SH parser. Though, the flip side of that would be that each user (e.g., priority) will have its own greasing scheme. That's the reason why I tend to think that the total cost might become smaller if we recommend: SH serializers add members using keys that users would never generate require them to be passed through by the SH parser so that each user would be required to ignore members using unknown keys PS.
Besides, assuming that we want grease the SH parser, I think it might be a good idea to allow ordinary character to use the escaped encoding (e.g., \x4A), so that parsers incapable of unescaping those characters can be detected at an earlier point.
I'm not at all sure that header authors are going to find greasing desirable across the board; it's very intrusive into the application (unlike other forms of greasing). And, as I said above, it's not at all clear that all header authors are going to be happy to ignore unrecognised keys. I'd rather be conservative here -- we haven't done this kind of greasing before (into the application space, rather than just within a protocol), and we're not sure how it will work out (especially considering the potential performance overhead; for small header values, it might seriously inflate their size, even if just in the dynamic table). But, again, folks other than us have an opinion, it'd be good to hear it.
I've said my part, you disagreed, but I haven't really changed my mind. If we're building something with must-ignore-unknown semantics, and if we do grease (and we don't have to), then it needs to reach as far as possible.
SH is only a data model and a serialisation, it can't say anything about semantics of values (including must-ignore or must-not). I mean, you could introduce weird sentinel values, but that seems like a bad kind of kluge. (And adding to my lazy regexp isn't the intended outcome.) It is up to individual header fields' spec authors to define the semantics of values in the context of that header field, including which SH types are allowed, what parameter names are ok, what to do about unrecognised ones, etc. That includes greasing their header field_ for extensibility. Maybe if we had a way of discussing a schema – for example a formal grammar an application could pass to the parser to allow it to fail on valid-but-disallowed data – we could talk about greasing there. But in the data model itself, it doesn't make much sense.
NAME that's the thing; we're not enforcing must-ignore-unknown, merely suggesting it as a default. Whether that's the semantic or not is up to individual header definitions. I'll do a PR shortly based upon the text above.
Late to the party... I've been doing some thinking on GREASE and request headers in the last few days as part of URL As part of the feedback that we received on that, there are . If the GREASEd request headers are used for content adaptation, each new GREASE variant may create another copy in the cache, at least if is involved. Not saying that this means we should avoid GREASEing - just saying that it's a consideration that will apply to UA Client Hints, and is probably applicable in the general case as well.
I'm going to merge the PR and (since this is our last open issue) cut a new I-D.
Good catch.
...currently defined in RFC 7538.
This would obsolete 7538?
Yes. Now the good question is how much of RFC 7538 include. My proposal would be just to include parts of URL, but to point back to RFC 7538 for additional information (non-normatively).
Seems reasonable to me.
Maybe we should also add the table from URL somewhere.
Possibly at the top of the 3xx section.
A response is incomplete if the TLS connection close was incomplete. Remove text about TLS reuse/resumption, which was very inconsistent. On the first, I removed a case of "partial closure" rather than "incomplete closure". I also linked the incomplete message handling section with the TLS connection closure section.
Apparently I'm not allow to link to after the fact. Sorry for not having done so in the opening comment.
This is okay, but the paragraph before your first change should be rewritten instead. I can fix that later.
This solves Issue by moving away from an example that wasn't ideal anyway (it encoded a different bytestring for the value Möbius). The Disclosure JSON should be:
"it shows as is there any way to change that..? (this is not the change introduced in this PR)" on the text in the doc at URL "A different way to encode the umlaut (two dots &; placed over the letter):" Originally posted by NAME in URL
Not sure why this broke now. I created a PR because the example wasn't ideal anyway: URL
I just noticed that the PR does not fully solve the problem. We have the same problem somewhere else as well: !
it's seems to have been an issue at least back to -01 URL and/or -02 URL (so basically since inception). I'm sure we could fight with the tool chain to fix it. But at what cost? I'm inclined to just reword that "The two representations... " part to make the issue go away. Which I did with this 48efde35a81ca3520eb30ceea6c43d62e8cd96f8 update to URL
PR merged
reads: This seems to have the effect that 412 cannot happen as part of a 100-continue response, but instead, the entire upload body must be parsed for errors (such as ), and only if it is free of errors can the server return 412 (Precondition Failed). This seems unnecessary and wasteful of bandwidth, and perhaps a result of the fact that server-side validation of documents, including the 422 Unprocessable Entity status code, is a comparatively newer feature of HTTP servers. [1] If this is the case, consider relaxing this requirement so that 412 (Precondition Failed) is the last error tested before the request body is parsed. [1] See mailing list thread:
Can I suggest we use 'converted' instead of 'normalized' ?
WFM, will update.
It's annoying to test for otherwise. Still allow both cases in parse.
Postel's law ("Be liberal in what you accept, and conservative in what you send.") isn't as popular these days as it once was. Not sure we need to accept upper case % anymore.
I am sure that we don’t. (Though I have thoughts on the use of the Robustness Principle to guide this sort of decision. Generally, it is better to look at what the case needs. Here, the values are created by code, so permissiveness is not a valuable feature.)
Fine. See URL
This is an alternate proposal to to This PR: Keeps Client as : an application making protected Transparency Service resource requests on behalf of the resource owner and with its authorization. Replaces Verifier with Relying Party: a relying party depends on Signed or Transparent Statements to verify an Artifact.,
Thanks, NAME Definitely looking for more input, and I’m also ok to close the PRs and issue and use the existing terminology. My main concern is a Relying Party is a consumer of information. They rely on the receipts and signed statements from the Transparency Service. Which makes it a good candidate to replace Verifier If we replace Client with Relying Party, this confuses what we call the entity that registers signed statements. They are not the issuer, as the issuer signs statements, but is not the entity with access rights to the Transparency Service. Both the producer and consumer need access rights. That was what we had as client.
There is still additional work that needs to be done. Client applications submit signed statements, and obtain receipts, so they can include transparent statements in downstream products. A consumer of a downstream product is a relying party, they either verify the transparent statements themselves, or they delegate the verification role to the "verifier" role.
As discussed on today's call. Folks expressed concern that "Verifier" was not friendly enough to "Consumers or Relying Parties", that are supposed to process receipts. Editors Update: We can choose to: [ ] Accept [ ] Accept [ ] Close both PRs and this issue, and keep the doc with Client and Verifier
NAME wanted to take a look for a proposal
Just an update, I plan to send a PR with proposed changes this week. NAME shall we sync up in the comments or out of band elsewhere?
A.J. Yes, let’s get together and work out a proposal. I’m booked today but tomorrow is open except for 07:00-08:00 and 14:00-15:00 PST (10:00-11:00 and 17:00-18:00 EST) Monty
Sorry, NAME I had to dig a bit to find your email in my inbox from the mailing lists but I found it. Invite sent. I think we can make quick progress and draft a PR tomorrow.
Met with NAME and we discussed perspectives and work on this in the near-term, summarized with URL More to follow.
Adding some more notes to plot out the work, Monty and I discussed aligning the diagram with the separate/distinct RATS architecture, but the separate call out of a distinct Relying Party role and how it pertains to agent was discussed when we met, per discussion I missed in today's author/editor meeting. For my future reference, I need to re-review the doc, as well as compare/contrast ,
So I did not make as much tangible progress today and have spent a lot of time thinking and reading as I re-read this SCITT Architecture draft and related WG drafts (RATS, OAuth) where Relying Party is defined. So, before I spend a significant effort in drawing this with aasvg, I would like to quickly highlight the gaps in my understanding via Mermaid so I can quickly iterate. I have tried reading the mailing list but it not 100% clear where the dichotomy between Auditor and Verifier occurred, but they seem overlapping and redundant to me unless Verifier is, in fact, some non-person entity, like the software doing the verification itself (so this goes to what should we rename client to in , separate of this), but does that not in fact mean Auditor is the Relying Party of sorts? It seems Auditor is just giving a specific name to a general kind of consumer and that is not what a Verifier is. So on that note, this is the current version of the SVG diagram in Mermaid ( I hope I got this right, I cannot just have labels like "Issuer" on the left-hand side so I use subgraphs to box those and not do something clever with dotted lines and cause more problems). (Source: URL) It would seem we really need to change the diagram and the terminology to get rid of Auditor altogether, so first, we would move to this. (Source: URL) I was going to do a subsequent edit, but at this point I will have to ask the group: what is a verifier doing that is different from a relying party owner with a relying party? Could you separate one from the other? (Don't you need to replay the log to verify the sequence of statements?)
For reference, there is so much overlap with I think we need to either add compare statements to terminology and touch up the current state diagram to disambiguate the use of these terms. Screenshot below. ! So would that mean, if they are in fact separate, Edit 2 incrementing after Edit 1 to reflect my understanding would be ... this? (Source: URL) If we start to think this way, do we consider the software (client/agent/what have you) used by the Issuer to be distinct? Similar but with one of two functionalities (an agent/client/what have you can be for Issuers to get receipts of signed statements or a verifier that reads all of them)? /cc NAME I am not sure this update at the end of the day reflects our discussion, but I feel like I lost the thread of what we discussed and seemed concrete in discussion. Translating it into diagram and wording updates has left me scratching my head.
And just for the sake of dissing RATs... ! IMO this is reversed... to rely on something you must trust a verifier or be a verifier. I think what they mean is for the relying party to depend on a verifier, but what it looks like is that a verifier sits in front of an RP.
A verifier can be a person or automation that was configured by a person. This is likely a great conversation for the NAME would you be able to lead that discussion with NAME
hm... if we are talking RATS an entity can take on both the Roles of Verifier and Relying Party at the same time (rendering the creation and conveyance of Attestation Results moot)
It seems now obsoletes the diagrams in URL and review in a previous meeting reflecting improvements and viable changes based upon answers to the questions posed in strawman diagrams in that comment.
Also of not my updates to the potential (and beneficial) overlap and possible alignment between relying party and client per URL
Ping NAME NAME
I wrote some of my thoughts on URL and aj-stein-nist/ietf-wg-scitt, but for me Client was going to replace Relying Party (Client==RP), so I am not sure if making Verifier==RP is what my intent was when I revived the issue, so I will leave to others to decide.