jiebi/CodeConvo huggingface dataset:
The definition of upload creation requests allows the client to include the entire file, some file data or no data at all. While we provide this flexibility, we don't give guidance on when which approach is best suited. This PR adds this missing piece and explains how an optimistic client can attempt to upload the file in one request and how a careful client uses a separate request for creating the upload. In addition, it also mentions how to transparently upgrade to resumable uploads, which wasn't explained in detail before. Let me know if you have any feedback on this.
Are "Optimistic Upload Creation" and "Upgrading To Resumable Uploads" the exact same way? Maybe the 3rd way could be checking first before uploading?
The way they are described right now they are similar, yes. In theory, an optimistic upload creation can also be used where the client doesn't upload the entire file, but just a large part of it (think 500 MB of a 10 GB file). When upgrading a non-resumable request to a resumable upload, this act of division is not allowed. But I am not sure if we should go into that much detail here. If not, then we can merge these two sections. Upload limits are a good point! With the careful upload, the client also knows the limits before sending the first PATCH request. The optimistic client must be prepared to receive a 104 response with limits that it has to adapt to. We can mention that a careful client may also send an OPTIONS request before hand if it deems that necessary.
I think upload-limit can just be enforced on the server side. If it sends a 104 and a 5xx response, the client would automatically retry. We have deployed a client-side implementation that is not aware of the upload-limit header field.
Thanks for all of your comments! I updated the PR to put the section about upgrading to resumable uploads under the optimistic upload creation since they are similar. I also added a brief mention of Upload-Limit and shortened the text a bit. Let me know if you think the text needs more adjustments.
Just a minor nit. Otherwise I think this is a very good addition and agree with NAME that we should add some text with regards to the .
This change allows client to discover support for resumable uploads and potential limits via an OPTIONS request. Closes URL A brief comment on this sentence: Once the server has created an upload resource it might have more information available (user details, file metadata etc) and apply different limits than it would have known upfront for an OPTIONS request where this information was not available. We allow this difference but recommend that the upfront announced limits are not looser than the limits after the upload resource has been created. The client can then assume that it can create an upload resource if it obeys to the upfront announced limits even if looser limits will then apply later. Does this make sense? Or is this addition rather unnecessary?
Is OPTIONS a required feature on the server side? What about requests?
Yes, I think a server should be required to support this. If it only wants to provide resumable uploads for specific users, it does not have to announce this for unauthorized users. If a server supports resumable uploads for all target URIs, then it should respond with to an request. But I am unsure how often this actually happens.
introduced for the server to indicate limits on the upload resource (e.g. maximum size). These limits are announced to the client during upload creation in the informational or final responses. However, the client cannot discover these limits without attempting to create an upload resource. A client may want to avoid creating an upload resource if it knows the the server limits cannot be satisfied. A possible solution is to allow clients to send an OPTIONS request upfront to the URL from upload creation. The server might then include the Upload-Limit header in its response, so that the client knows them before trying to create an upload resource. However, a client is not required to send this OPTIONS request upfront. This would also be handy in a possible future where browsers natively implement resumable uploads. The response for a CORS preflight request might then already include information whether the endpoints supports resumable uploads and, if so, under which constraints. The browser can use this data to optimize the uploads.
Early thought: Would it be beneficial to include some kind of marker that this endpoints supports resumable upload in the OPTIONS response? For client's that does not support upgrading requests (e.g.
due to lack of support for 1xx responses) who whishes to know upfront if the upload can be split (Upload-Complete: ?0) or if a regular upload (without resumability) should be performed.
Interesting idea! The presence of the Upload-Limit header could be used as an indicator for support of resumable uploads. In theory, its value can be an empty dictionary as well if the server does not impose any limits. If the response does not include the Upload-Limit header, the client can either attempt an upgradable upload or directly fall back to regular uploads.
URL
Too bad we can't use the empty dictionary header as an indicator. It's probably an edge case anyway that a client, not knowing the support of the server, would issue an OPTIONS request to find out? Or maybe not, given CORS? What do others think?
Thanks for the pointer, Lucas! If we cannot use an empty value, a server could also fill in a dummy value, e.g. . This is only necessary if the server does not apply any limit at all (not even ), which I suppose is very rare in reality. In general, the situation where a client wants to know upfront whether the server supports resumable uploads appears like an edge case to me. I assume the client will often know that the server supports it or otherwise will attempt to transparently upgrade a traditional upload into a resumable one. So I don't think we have to spend too much energy on this. How do you feel about this, Stefan?
NAME Yeah I think that is fair enough. The use cases where a generic client (without prior knowledge of server support nor support for informational responses) would send an extra OPTIONS request seems slim to none. The CORS example is interesting but mostly (always?) happens in a browser which does support informational responses and thus would use the 104.
Opened URL for this.
While already discouraged by the grammar, this warning further discourages servers from sending nameless cookies which have a history of causing compatibility problems. It also reorders the notes in the syntax section to group similar notes together.
Ideally we'd define server-side parsing of cookie-string (or at least recommend behaviour here for things that don't match cookie-string!), given this does cause problems for some servers (e.g. URL). Until then we should at least make it clear that the "the algorithm to compute the cookie-string from a cookie store and a request-uri" can create things that don't match cookie-string (e.g., nameless cookies after ). There's a number of WPT expected results that don't match cookie-string.
Can you clarify what a fix would look like? Bonus points for a PR. (I mean, I can also attempt a PR, just not clear on what the recommended fix would be)
You attempting would probably be as good as me attempting!
I'm skeptical, however willing to try! And now I think I actually understand the bug report (and didn't when I commented). NAME is saying that the ABNF in URL has a bug, since is defined as: But nameless and valueless cookies exist. cf. URL URL
I think maybe this can be solved by making the following changes: This is pretty simple to grok: This is supposed to convey "a nameless cookie, OR a valueless cookie (but not both)" That should work for the following valid cookie-pairs: (nameless) (nameless2) (named cookie with empty , i.e., valueless) (name + value) If that's the route we go, then in URL cookie-string = cookie-pair *( ";" SP cookie-pair ) That should work. However... unless we tweak , I think you can end up with the following invalid cookie: . Will need to formalize "if no name, there must be a value". Maybe we can just add an and then do something like:
Actually that won't fix the bug I described.
Maybe this super elegant solution (but the above ABNF would have to be tweaked):
FWIW nameless and valueless cookies are no longer RFC compliant after URL Up until then only Set-Cookie strings with empty names and empty values weren't accepted, but there were ways around this to get empty-name-empty-value cookies into user agents' cookie jar using the CookieStore API.
There are nameless cookies and there are valueless cookies (and, as you said, there are no nameless-and-valueless cookies). FWIW, the syntax in section 4.1.1 where is defined is a SHOULD, not a MUST, on the server's part. I think one issue is that reusing it to define makes that also implicitly a SHOULD, if I'm reading this correctly: Edit: To clarify, I'm thinking there's no accuracy issue with the spec as written, since if the server does things as recommended (and sends a cookie with name=value), then the user agent will respond as given by . But that is still totally confusing, at least to me, so I think we should still define this better.
(love the ambiguities of english -- thanks for the explicit callout and pointer to NAME
In the nameless cookie case, it seems that an unhandled edge case arises because of the following, from the section: To illustrate the problem with some examples: if is empty and is , the algorithm above works - the resulting cookie line would be if is empty and is , though, the resulting cookie line would be , which would be interpreted as a cookie with name and value . One solution for this, which is currently specified by the , is to forbid cookies with an empty name and a value containing '=' from being set. Limiting from containing a '=' is not mentioned in the section or the section of RFC6265bis, though. It seems like using the following in the section would be a better approach: This would fix the issue in the example above (the resulting cookie lines would be and respectively, both of which would be interpreted correctly). If we want to allow to be produced in the first case, we could replace step 2 above with something like: I'll submit a PR for this change.
The empty-name-but-nonempty-value case is serialized as and not for historical/compat reasons. There's some earlier discussion in .
The case where the value contains an '=' is interesting though, and not tested by WPTs I believe. The Set-Cookie header can't produce a nameless cookie with value so this is only a concern for non-HTTP APIs that don't parse a set-cookie-string. Could we instead just add to the Storage Model section the restriction that the cookie-name and cookie-value cannot contain '=' or ';' characters? (Side note, more generally, this is getting out of hand and we should try to factor out common logic in the Storage Model and set-cookie-string parsing sections.)
hmm, looks like there is a WPT for this case, but the expected output for it would be interpreted incorrectly :( URL;l=22-26 Wouldn't be allowed in a Set-Cookie header? Also, '=' is a valid base64 character, so not allowing it in might cause a lot of headaches
Oh, my mistake, you're right that creates a nameless cookie with value . Chrome/Edge/Firefox agree on how to serialize these, but Safari ignores nameless cookies altogether. I would be a bit concerned about breaking stuff if this serialization changed. :-/ Theoretically there's a use case for this behavior... since and are the same cookie (the nameless cookie) but with different values, and they overwrite each other, a site could be relying on getting the output or (but never both) as a way of (sort of) setting two mutually exclusive cookies named and .
Maybe we just fix this with a note cautioning against using nameless cookies, and point to this issue as a warning.
is deprecated due to the ubiquity of UTF-8, but the section makes no mention of UTF-8. This might be worth a mention.
It's only used for though and that section already mentions UTF-8. As does the Media Type section. Since is deprecated, perhaps the production should simply be removed and takes directly?
Accept-Charset is deprecated primarily for privacy. It also happens to be less useful now because of utf-8. In contrast, the charset parameter is still used most of the time, mainly to reduce sniffing. We can add something about utf-8, but promoting it too hard will cause other (social) problems.
NAME observation is correct. But then, charset is used in Content-Type as well, just not referenced directly as it is used over there as a regular parameter.
It's not clear to me at least that the production section also applies to .
Syntactically, it does not (because that relies on a more generic syntax for arbitrary parameters). But I agree, this needs cleanup.
Proposal is to remove the ABNF production and to clarify the text (so that it applies to Content-Type as well).
Hi! I post here my combined comments for draft-ietf-httpbis-semantics-16, as I don't think they deserve one issue each :) I hope that's fine. My comments range from minor to very minor to nit, and are mostly editorials and clarifications. As this can possibly be me, I'll leave it up to you to decide if they can be useful to improve readability, or not. Thanks for your work! Francesca [x] 1. ----- Additional (social) requirements are placed on implementations, FP: I am not sure I understand the meaning behind the term "social requirement". Editors: URL (URL) [x] 2. ----- When a major version of HTTP does not define any minor versions, the minor version "0" is implied and is used when referring to that protocol within a protocol element that requires sending a minor version. FP: I had a hard time parsing this sentence (starting at "and is used..."), maybe breaking it up with commas would be enough. Especially the two "that" are unclear to me. Editors: URL (URL) [x] 3. ----- RFC 3986, which allows for an empty path to be used in references, FP: very minor, I'd prefer "in URI references". Editors: URL, change applied in URL [x] 4. ----- The process for determining that access is defined by the URI scheme and often uses data within the URI components, such as the authority component when the generic syntax is used. However, authoritative FP: I had to read the sentence above 3 times before realizing that the subject of "is defined" is not "access", but "The process", and understanding it. Editors: URL, change applied in URL [x] 5. ----- A client MAY discard or truncate received field lines that are larger FP: I assume and It might have been good to clarify that "truncate" a field lines is equivalent to dropping a member of the list of values. Editors: URL [x] 6. ----- Note that double-quote delimiters are almost always used with the quoted-string production; using a different syntax inside double- FP: I am not sure I understand "quoted-string production" Editors: added missing fwd reference - URL [x] 7.
----- content, it is often desirable for the sender to supply, or the recipient to determine, an identifier for a resource corresponding to that representation. FP: It might have been good here to give a hint or example on why such identifiers could be desirable. Editors: URL (URL) [x] 8. ----- Fields (Section 5) that are located within a trailer section are are referred to as "trailer fields" (or just "trailers", FP: s/are are/are Editors: URL [x] 9. ----- All responses, regardless of the status code (including interim responses) can be sent at any time after a request is received, even if the request is not yet complete. A response can complete before FP: I think at this point in the text there is no definition of what qualifies a request or response being "complete". Editors: actually, the definition is earlier in the text; added a reference anyway: URL [x] 10. ----- An intermediary MAY combine an ordered subsequence of Via header field list members into a single member if the entries have identical received-protocol values. For example, FP: Is there any requirement on the intermediary to be able to collapse the ordered sequence into a single member (such as storing and being able to expand this information)? Might want to expand a little. Editors: Declined because there is no requirement to do so. We already explain that one might want to do so for information hiding of intermediaries within a private network (URL and URL) [x] 11. ----- This definition of safe methods does not prevent an implementation from including behavior that is potentially harmful, that is not entirely read-only, or that causes side effects while invoking a safe method. What is important, however, is that the client did not request that additional behavior and cannot be held accountable for it. For example, most servers append request information to access FP: I might be missing the motivation for it, but this paragraph seems strange to me. Why is it important that the client cannot be held accountable for it? The concern I have is that the text seems to be saying "safe is essentially read only/not harmful/..., but not really". (And same for idempotent) Editors: URL [x] 12. ----- referring to making a GET request.
A successful response reflects the quality of "sameness" identified by the target URI. In turn, FP: I am confused by this terminology: quality of "sameness". Editors: URL [x] FP: The text could add something for when it is acceptable or expected to make sense (so why this is a SHOULD). Editors: Because the protocol has historically preferred to separate semantics from parsing requirements so that the semantics can be extended over time without breaking parsers. It is a SHOULD NOT for GET because the original protocol did not define a meaning for such message content and some implementations took advantage of that leniency to implement and deploy features that work among consenting implementations but are non-interoperable with intermediaries and other implementations. Hence, it's a bad idea, but the protocol does not prevent it from happening. See for why this is impossible to change. [x] 14. ----- A server SHOULD NOT use the From header field for access control or authentication. FP: Same as the previous comment, when is it acceptable to use the From header field for access control or authentication? Editors: URL [If it is acceptable to the server that does it, then HTTP doesn't care in terms of interop, but it is still a bad idea because the protocol has other mechanisms for doing access control and authentication which are less likely to be leaked by accident (URL)] [x] 15. ----- FP: I don't know if I missed it, and please correct me if I did, but I was hoping to find a sentence stating something of the sort for the header fields: "If not otherwise stated, these fields can be used with any method or response code". Editors: URL [x] 16. ----- FP: Most of the header fields contain examples, which I find helpful. I think it would be good to have examples for all of them (including TE, Trailer). Editors: URL [x] 17. ----- | Accept. Future media types are discouraged from registering | any parameter named "q". FP: I wonder if it would be worth making a not out of it in the media type registry (so that it's easily remembered by experts) Editors: URL [x] 18. ----- Note that an If-None-Match header field with a list value containing "\" and other values (including other instances of "\") is unlikely to be interoperable.
FP: what does it mean for a header field specific value to be not-interoperable? I am also not sure how "*" between the list of values would be processed. 2. above does not specify it, only stating: If the field value is a list of entity-tags, the condition is false if one of the listed tags matches the entity-tag of the selected representation. FP: I would possibly interpret this as "search for tags matching the entity tag - ignore all others". Editors: URL and URL [x] 19. ----- senders might send erroneously send multiple values, and both FP: remove one "send". Editors: URL [x] 20. ----- the String data type of [RFC8941]), or a field-specific encoding). FP: remove the first parenthesis. Editors: URL [x] 21. ----- FP: This is more of a check, but I see that all references only point to sections. I hope this will be translated into "x.x.x of RFCthis", is that correct? Editors: URL
You mean in the tables for IANA registrations? That's for brevity; IANA knows how to deal with that.
The media type registry is not owned by "us". I don't think we have the authority to change it.
I will look into this with IANA and the DEs and get back. Just to be clear, the document states "future media types are discouraged..." but what we really mean is "experts should be conservative with assigning ..." , is that accurate? Yes, in theory we'd like to discourage media types to be registering "q", but the practical way to make sure that doesn't happen is to make the DEs react to it if it does.
maybe "...whenreferring to that protocol version..."? NAME ?
Yes, thanks!
I don't think it's restricted to that.
For instance, to provide a bookmark/make something accessable for a subsequent GET... NAME - do you want to add that?
I don't get that question. The spec allows collapsing. It doesn't require it. Could you please clarify your question?
The motivation is that if something is defined as "safe", I can URL do that operation without being held accountable.
For instance, I can always GET a URI (such as for prefetch/preload/indexing) and if the server does do something stupid with that request (such as deleting a resource, or adding something to a shopping basked), it's their problem. Re idempotenty: this makes it possible to repeat a request.
The text states that intermediaries are allowed to collapse "an ordered subsequence of Via header field list members into a single member", and they can only do so "if the entries have identical received-protocol values." So I read 2 rules: the order matter, i.e. if (A,B) collapses to C, (B,A) does not map to C (A,B) must have identical recived-protocol values I was wondering about any additional rules that the intermediaries have to follow in order to collapse the list into one: for example, is it ever expected for the intermediary to remember that this single member was this collapsed ordered sequence? Are they ever requested to un-collapse?
Order matters for the purpose of collapsing. If you have "D, E, F", and E has a different protcol than D or F, you can't collapse. If "A, B" can be collapsed, so can "B, A" (because they have the same protocol) We are not saying that, so there is no expectation that the collapsing can be undone.
Ok, so yes I understand - the idea of this text is to say: "this is the intent, BUT be aware that implementations might do something stupid and not comply with this intent." I guess what I got stuck with was "the client can't be held accountable", as it wasn't clear to me what would it mean for a client to be held accountable, as clients in general are not aware of what's the action on the server their request will result in.
This might require a separate issue. A field value containing more than one list element and including "*" is invalid, and the spec really doesn't say what that means. The intent of saying "interoperable" is (IMHO) to prevent people from assuming the different servers will do the same thing with it. I believe we should clarify that a value like that is invalid, and thust MUST NOT be sent. NAME , NAME ?
That refers to URL - I guess we should cite that? NAME ?