jiebi/CodeConvo huggingface dataset:
Code change:
once in the signature input.  Parameterized keys MAY appear in any order. <ins> If a dictionary key is named as a covered component but it does not occur in the dictionary, this MUST cause an error in the signature input string generation. </ins> Following are non-normative examples of canonicalized values for Dictionary Structured Field Members given the following example header field, whose value is known to be a Dictionary:

Related discussion:
text: We didn't want to make it seem like the parameter was the "right" way to signal algorithms here, and most certainly is not required. You'll notice that none of the other sections have "MUST use the alg parameter value of 'foo'" either. Older Cavage drafts relied on the parameter, albeit in a much less secure fashion, and implementations were susceptible to algorithm substitution in the same way that naive JOSE implementations tend to be -- the "alg: none" attack all over again, more or less. The value of the parameter already MUST be something from the registry, and the registry does not (and will not) contain JWA values. So you already aren't allowed to do it, normatively, and the note is our attempt at explaining the why. Ultimately, an implementation is allowed to use whatever algorithm they want and makes sense. What this whole section does is define a set of common usable algorithms as well as provide a pattern of applying crypto primitives that applications can follow for their own algorithms. If someone defines a new algorithm that's generally useful, like we recently did with the ed25519 here, it can be registered for use with the parameter. But if a system can base the algorithm selection off of the key material or another part of the protocol (like with GNAP), that's highly preferable and less vulnerable to substitution at runtime. I'd be happy to know how we can make this clearer in the text, but I'm not comfortable with that kind of normative construct in that section.
NAME -- would you like to chime in? I recall from our discussions you having another, more specific reason for this decision, but I can't remember the details beyond us agreeing to do it the way we have it in the spec right now.
Neither Java nor JS subtleCrypto come out of the box with PKCS1 PEM support. (for java ). It is not immediately obvious to work out that the PEM formats of the private key in in PKCS1 format (the spec could point that out). When building an implementation it helps to have the test suite clearly map to the data in the spec (e.g. ) so as to make error reporting clearer. That is even though using the following openssl command to convert the keys work, it makes it difficult to see if an error was introduced at some point.
Code change:
once in the signature input.  Parameterized keys MAY appear in any order. <ins> If a dictionary key is named as a covered component but it does not occur in the dictionary, this MUST cause an error in the signature input string generation. </ins> Following are non-normative examples of canonicalized values for Dictionary Structured Field Members given the following example header field, whose value is known to be a Dictionary:

Related discussion:
text: To support PKCS1 the developer has to depend on further less well tested libraries and add additional code (as I ) leading possibly to new bugs. Furthermore, I will need to go through the same process for JavaScript now, doubling the work. Since PKCS8 is as expressive as PKCS1 ( and see ), it would make life easier for devs if the keys were all in PKCS8 format. Unless there really is something that PKCS1 offers that PKCS8 does not? I will try doing this using the Web Crypto API next. But there we find that supports the PKCS format.
While I am at it. I am having trouble with verifying the signature for 3 out of 4 examples from the spec that I tried out in . Currently the only one that works is the . I am able to produce and verify a signature for , and examples, but the signatures don't correspond to what is in the spec. That may very well be a mistake in my code, which I am working on as a to bobcats scala and scala-js crypto library. But I just wanted to check how confident you were in your examples. I am going to re-implement next in JS, so that will give me another reference point.
I am working my way through this with JS. I was able to parse the and keys after translating them to pkcs8 format with openssl, but am having trouble importing in the console of Chrome, Chrome-beta, Chrome-Dev and Safari. I guess that is in pkcs8 format already. I don't have trouble parsing that key with BouncyCastle... I guess this is a little config problem, but it's difficult to work out what that could be... Perhaps someone has an idea?
NAME Since this all has to do with the examples in the text and not the spec itself, I think we'd be fine with having the keys in multiple formats if it's useful for developers. However I'll note that in many code libraries you need to add the PEM headers of and for their compliant parsers to parse an EC key in PEM format, for example. I've been able to parse these keys in both Java and Python, and in fact all the examples are generated using a python script: URL Also, note that RSA-PSS and ECDSA are non-deterministic signatures that incorporate a source of randomness into the signature calculation.
Code change:
once in the signature input.  Parameterized keys MAY appear in any order. <ins> If a dictionary key is named as a covered component but it does not occur in the dictionary, this MUST cause an error in the signature input string generation. </ins> Following are non-normative examples of canonicalized values for Dictionary Structured Field Members given the following example header field, whose value is known to be a Dictionary:

Related discussion:
text: This might be why you're getting valid signatures that look different from the examples -- you shouldn't expect to replicate the exact signature values. RSA 1.5 is deterministic, and will always give this same signature for the same input (as will HMAC). This is just one of the reasons why it's not recommended in many environments. I think we can add some text in the examples sections that points this out, if that would be helpful.
NAME You can also test keys and examples at URL for another implementation. I am pretty confident in the values in the examples, but if there are in fact bugs in these implementations (which is always possible!) I'd love to know and fix them.
NAME you were right. I had been verifying the keys by string comparison, so I fixed that. I also worked out what the Java names for the various algorithms were and got nearly all the tests to run on JS and Java platforms. But there is one error left that is consistent across both, so I wrote it up here. URL
Nevertheless, I have not yet been able to import the private key PEM file in JS. So I looked for a way to convert it to JWK to see how that would fare. I looked online, but none of the services worked, I looked at npm but that did not work, finally I found in Java that did the conversion in 2 lines of code. With this I could get signing to work: So there is something problematic with that PEM, at least insofar as it makes it very difficult to get the example to work in JS which should be one of the main platforms. (I have not yet verified the signature in JS as I am having trouble getting verification in JS to work with a valid PSS key, even though I can create the public key with the PEM, and am able to do that Scala-JS)
I think I found the problem to be with the private key. After going through this process: transforming the spec key to JWK as described above transforming the JWK to PEM PKCS1 using node's I get the following transforming the above using Java's Bouncy Castle to PEM I get this: And with that all the JS crypto API tests pass. See the .
Code change:
once in the signature input.  Parameterized keys MAY appear in any order. <ins> If a dictionary key is named as a covered component but it does not occur in the dictionary, this MUST cause an error in the signature input string generation. </ins> Following are non-normative examples of canonicalized values for Dictionary Structured Field Members given the following example header field, whose value is known to be a Dictionary:

Related discussion:
text: I think to understand why that final private key works with the Java Web Crypto API but not the original one from which it was derived (which worked with Bouncy-Castle) we'd need the input of someone like NAME . With that public key replaced (in commit ) and a few others commits to get Continuous Integration to work, we have all the tests pass for Scala compiled to JavaScript Those running in the browser are a bit interleaved due to asynchronous execution, something that needs still to be fixed.
Looks good!
Code change:
3.3. <del> Parameterized Lists are arrays of parameterized items, with one or </del> <ins> Parameterized Lists are arrays of parameterized items, with zero or </ins> more members. A parameterized item is an item (item) with associated parameters, an

Related discussion:
text: This modifies parsing to consider empty or not-present header fields as empty containers; serialisation omits them from output.
The SH syntax does not directly allow empty field values, such as defined for Accept, although this might be allowed by field definitions like MyField = [ sh-list ] Likewise, the list types do not allow for empty lists. sh-list = list-member *( OWS "," OWS list-member ) list-member = sh-item I think this is a very odd choice for a generic syntax because the empty set is often given semantic meaning in generic communication. It's hard to do math without zero. If this is intended, the rationale should be explained in the spec, and there should be some discussion with examples about defining fields as an optional value.
FWIW, even if empty lists may be problematic top-level, the situation is different when they are nested...
ietf104 - interest in the room in supporting empty (perhaps null?) syntax
I agree this is something important. It may be used to mean "I know I must pass the info but I don't know the info", which is different from "I don't care about the protocol" or passing a dummy value just to fill the hole.
In the original base material of header definitions, empty values were generally just not sent at all. Having a specific Null value may or not make sense, not sure.
I personally like the distinction between "known to be empty" and "not presented either because I don't know or because it's empty". Some semantics might explicitly require the presence of some fields which we would enforce by the protocol. This doesn't mean they can't be empty, they must be present, for example, to make sure they weren't dropped in the middle. For example, "Connection: foo" gets rid of the field, not just the value. Just my two cents, Willy
Do we have any RFC'ed headers where there is a difference between a non-present and an empty header ?
There's an accept-something if I remember well. Also I've been using "Host:" in many occasions with HTTP/1.1 requests when I didn't know the host name and this tends to work where no Host field fails ;-) Willy
I'm not against a '?N' null token (along the lines of true/false), but I have a hard time thinking of a use for it.
Code change:
3.3. <del> Parameterized Lists are arrays of parameterized items, with one or </del> <ins> Parameterized Lists are arrays of parameterized items, with zero or </ins> more members. A parameterized item is an item (item) with associated parameters, an

Related discussion:
text: URL
Wouldn't an empty list be equivalent to an empty value, thus we don't need a new syntax?
Putting an empty list where for instance an integer is expected, just to say that there will be none, sounds counter-intuitive to me.
On Fri, Apr 05, 2019 at 01:37:11AM -0700, Julian Reschke wrote: Indeed, probably.
On Fri, Apr 05, 2019 at 03:11:00AM -0700, Poul-Henning Kamp wrote: Agreed in general. My point was that sometimes specs use "MUST send" when the sender doesn't know, resulting in stupid values appearing. That's exactly what led a number of agents to place a content-length with stupid values (0 or huge ones) with the CONNECT method a long time ago. An empty value where an integer is needed by the application is more of an application-level error than a protocol-level one in my opinion : it's up to the application to decide how to recover from this missing piece of information (is it really needed to process the request or not). For me it can mean "don't care", "don't know" or "not representable". After all floats support "NaN" for the same reason :-) Willy
If there is a "MUST send" requirement, there must also be something available in the same spec that can be sent ?
There is not, and never has been, an interop problem with sending empty field values. They are a list value of zero items or a string item of zero length. Accept and Host, respectively.
NAME how do you know that?
Because if a recipient drops an empty Host field the request will be rejected with 400. There have been no reports of such breakage in 25 years. It isn’t even a likely scenario given how message fields are parsed and reproduced.
If a recipient doesn't drop an empty host header, the request will still be rejected with 400, surely? Regardless, I'd note that many headers address this situation with a reserved value like . Is there value in standardising on one such value for future headers?
Code change:
3.3. <del> Parameterized Lists are arrays of parameterized items, with one or </del> <ins> Parameterized Lists are arrays of parameterized items, with zero or </ins> more members. A parameterized item is an item (item) with associated parameters, an

Related discussion:
text: I'm not convinced, but I'd note that we also have many headers that use to denote "all"; it seems to me that if we're going to do null, we should do that too (but again, I'm not convinced this is adding value).
On Mon, Apr 08, 2019 at 05:16:41PM -0700, Mark Nottingham wrote: It depends on the header field that was empty ;-) I've used the empty Host header field quite a bit (even in health check requests). My observation has been that while HTTP/1.1 without Host naturally causes a 400, HTTP/1.1 with empty Host is often processed similar to HTTP/1.0 without Host or as 1.1 with an unknown Host. I'd say "why not", but it's useful to consider lists and the iterative processing that can happen, removing one item at a time. Having to place a specific value to denote emptiness would require specific processing (i.e. check for emptiness after removing any item just for the sake of replacing it with the special value). By the way, speaking about lists makes me think we may complicate interop when a header field appears multiple times. For example if we have this in a request : List: a,b,c,d Foo: bar List: e,f,g,h It's not unreasonable to think that some simplified implementations may replace the first occurrence of List with "none" after processing it, leading to this : List: none Foo: bar List: e,f,g,h Or possibly this : List: none Foo: bar List: none Which may be processed as "List: none,e,f,g,h" or "List: none,none" respectively. Just my two cents, Willy
Willy, I think that would be clearly broken, and we should be able to catch it with tests. I think we're talking about all of the container types here (dict, list, list of lists, parameterised lists), correct? Julian, you gave as an example. Given that the behaviour recommended is exactly the same as if the header was missing, that's not very compelling. Besides the potential stripping issue (I have heard of implementations that strip empty headers, but don't have details on hand), some header libraries may not make a distinction between missing and empty headers.
Code change:
3.3. <del> Parameterized Lists are arrays of parameterized items, with one or </del> <ins> Parameterized Lists are arrays of parameterized items, with zero or </ins> more members. A parameterized item is an item (item) with associated parameters, an

Related discussion:
text: Overall, I don't see a lot of value in accommodating empty headers directly, and I don't think we should be promoting it as a practice.
Agree, I'm not convinced either. I see the potential, but Gettys rule applies: "The only thing worse than generalizing from one example is generalizing from no examples at all." (May actually be from Phil Karlton originally, sources differ.)
Hm, no. "A request without an Accept-Encoding header field implies that the user agent has no preferences regarding content-codings." vs. "An Accept-Encoding header field with a combined field-value that is empty implies that the user agent does not want any content-coding in response."
Yes, but in the real world, the outcome is always the same.
No. HTTP/1.1 introduced an IESG requirement that Host must always be sent and recipients must always reject with 400 if Host is not received. When I pointed out that not all URIs have a host, the response was that clients must send an empty Host field in that situation. It is a MUST requirement in Semantics section 5.4: This is ingrained in HTTP semantics and commonly used by non-browser HTTP implementations. There is only one example of an HTTP header field using "none" and that is Accept-Ranges. It isn't a good example for anything. To be clear, it would be a bug for an HTTP message processor to drop a received header field just because the value is empty. Nothing about SH will change that. What we are discussing is whether SH is capable of representing an empty list or an empty header field, both of which it is currently incapable of doing.
If that spec specifically says "empty field-value", a SH "empty" token will not be usable for Host: anyway, and we're back in the "generalizing from no example at all" again.
On Thu, Apr 11, 2019 at 03:53:15PM +0000, Poul-Henning Kamp wrote: After having thought a bit more about this, I now think that we don't need empty list elements in the end because lists are not strongly ordered, they're not arrays with a position so an empty element in a list brings no value.
Code change:
3.3. <del> Parameterized Lists are arrays of parameterized items, with one or </del> <ins> Parameterized Lists are arrays of parameterized items, with zero or </ins> more members. A parameterized item is an item (item) with associated parameters, an

Related discussion:
text: I'd further say that removing an element from a list must result in removing the list once the last one was removed as it is a set. So I think I'm fine with not dealing with empty lists. This then leaves us only with the empty header field alone, likely where a string or a token are expected. I don't see a reason not to support an empty string just like any other string. A header field is not just a value, it's a name+value combination. With a void value the name still serves as a boolean (present or not) and this is exactly how it's used when the field is made mandatory to distinguish between agents generations or as a signal to detect support for something. We've all filled administrative forms where the requested information was irrelevant but mandatory in the form and just left it void so that the person in charge deals with the particular case (on paper) or filled with whatever was accepted (on web forms) hoping it would just be ignored since irrelevant. For me the Accept and Host fields are exactly in this category. My concern is to be certain that a valid HTTP request can be transmitted over SH. And ironically, the only mandatory HTTP header ("Host") is also the one requiring the support for an empty value due to its mandatory nature. Willy
I have lost count of the number of packages I have received from USA where part of the address is "Denmark does not have states" :-)
Again, the purpose of SH is not to allow all possible HTTP header to be represented in the data model -- it's to solidify the 80% case and promote best (and interoperable) practice. Representing a boolean as an empty header field is not good practice.
See PR above.
I've put in the changes for the list-based header fields. It's less clear to me that it's desirable to default a missing or empty item to an empty string (strings and tokens) 0 (integers and floats), False (boolean) or a 0-length binary array (binary data); to me, those are all going to be application-specific defaults that are better specified explicitly. I'm happy to adjust the spec to make that more clear, if folks feel it already isn't.
Code change:
The terms Byte Sequence, Item, String, Token, Integer, and Boolean are imported from STRUCTURED-FIELDS. <del> The terms client and server are from HTTP. </del> <ins> The terms "representation", "representation data", "representation metadata", "content", "client" and "server" are from HTTP. </ins> 3.

Related discussion:
text: adds more used definitions from HTTP.
representation metadata, content, ... in Conventions and Definitions.
Code change:
The terms Byte Sequence, Item, String, Token, Integer, and Boolean are imported from STRUCTURED-FIELDS. <del> The terms client and server are from HTTP. </del> <ins> The terms "representation", "representation data", "representation metadata", "content", "client" and "server" are from HTTP. </ins> 3.

Related discussion:
text: The definition of upload creation requests allows the client to include the entire file, some file data or no data at all. While we provide this flexibility, we don't give guidance on when which approach is best suited. This PR adds this missing piece and explains how an optimistic client can attempt to upload the file in one request and how a careful client uses a separate request for creating the upload. In addition, it also mentions how to transparently upgrade to resumable uploads, which wasn't explained in detail before. Let me know if you have any feedback on this.
Are "Optimistic Upload Creation" and "Upgrading To Resumable Uploads" the exact same way? Maybe the 3rd way could be checking first before uploading?
The way they are described right now they are similar, yes. In theory, an optimistic upload creation can also be used where the client doesn't upload the entire file, but just a large part of it (think 500 MB of a 10 GB file). When upgrading a non-resumable request to a resumable upload, this act of division is not allowed. But I am not sure if we should go into that much detail here. If not, then we can merge these two sections. Upload limits are a good point! With the careful upload, the client also knows the limits before sending the first PATCH request. The optimistic client must be prepared to receive a 104 response with limits that it has to adapt to. We can mention that a careful client may also send an OPTIONS request before hand if it deems that necessary.
I think upload-limit can just be enforced on the server side. If it sends a 104 and a 5xx response, the client would automatically retry. We have deployed a client-side implementation that is not aware of the upload-limit header field.
Thanks for all of your comments! I updated the PR to put the section about upgrading to resumable uploads under the optimistic upload creation since they are similar. I also added a brief mention of Upload-Limit and shortened the text a bit. Let me know if you think the text needs more adjustments.
Just a minor nit. Otherwise I think this is a very good addition and agree with NAME that we should add some text with regards to the .
Code change:
The terms Byte Sequence, Item, String, Token, Integer, and Boolean are imported from STRUCTURED-FIELDS. <del> The terms client and server are from HTTP. </del> <ins> The terms "representation", "representation data", "representation metadata", "content", "client" and "server" are from HTTP. </ins> 3.

Related discussion:
text: Proposal for optimistic: The client needs to adapt to the header provided in the intermediate response and fail the request if the file is larger than the maximum allowed size. Proposal for Careful: The client can use the header provided in the to verify that the file is allowed to be uploaded before sending any data.Some editorial suggestions, but one big question: What is the material difference between the first and third strategy? Could these be the same strategy?