jiebi/CodeConvo huggingface dataset:
We wanted to ensure that the output was consistent (primarily for testing/comparison purposes), so we devised the "normalized path" to ensure that paths were output in a known format. After the refactor, it basically just means that the path doesn't use the shorthand notations; all of the segments are bracketed.
The following sentence in the Normalized Paths section provides some motivation: but maybe we need to expand it along the lines of NAME comment above.
Yes, that's my recollection. Yes, Normalized Paths use only bracket notation, but bracket notation can also produce paths which are not Normalized, such as and .
I repeat my basic question: What are these things for and what benefit do they offer, and to whom? I'm not sure what "used in the output" even means. It seems to me that more motivation is needed. I'd also be OK with just removing the whole section.
Here goes... A Normalized Path plays the same role as a JSON Pointer in situations where JSONPath syntax is required instead of JSON Pointer. A Normalized Path identifies a node in a given JSON value. Given a JSONPath expression which identifies a particular node in a given JSON value, there is a unique Normalized Path which identifies the same node in the given JSON value. Software (applications, libraries, JSONPath implementations, etc.) which outputs Singular Paths identifying nodes in a JSON value can output the equivalent Normalized Paths instead. A benefit of doing this is that the behaviour of the software is then more predictable and more easily tested (because the tests need to accommodate fewer possibilities). If there are multiple implementations of the software, sharing a single test suite is made more tractable. Who does this benefit? Human or programmatic users of the software observe more predictable behaviour. People writing tests have less work to do. Sharing a test suite avoids people duplicating effort.
Normalized paths are introduced in Stefan Goessners's , and are implemented in the two most important implementations, Goessner JavaScript JSONPath, and . I assume Stefan Goessner can speak to the original motivation, and if you want to know how they are used in practice, I suggest surveying users, perhaps starting on the Jayway site. Among other uses, they are helpful for diagnostics, for discovering what a perhaps non-obvious query resolves to in terms of the individual nodes selected.
They have other uses for selecting specific nodes given a much larger query.
I think the spec needs a statement with a MUST/SHOULD/MAY about supporting these things and a clear suggestion as to what useful purpose they're designed to serve. Otherwise I seriously suggest removing them because to be honest I can't think of a single scenario where I'd use them.
I think they're intended not to be written but consumed. I agree in that I doubt anyone would write out except maybe in the context of a stackoverflow question or something. They'd just use . The point of this, though, is to be able to declare a preferred syntax that implementations should use to identify a known location in a document. It's just saying that we prefer over . (One advantage of using the bracketed syntax for normalization is that any location in a document can be written in this syntax, whereas there are restrictions on the dot syntax.) Again, I think this is most apparent in output when the implementation is returning paths to locations instead of / along with the values.
Another consideration is the spec's definition of nodelist as the output of applying a query: The following statement in the Normalized Paths section: steps into implementation detail. In the rest of the spec we studiously avoid prescribing implementation details. One way to reconcile these would be to put some non-normative text near the start of the spec which talks about implementation options for nodelist. This could then include language such as "If a nodelist is represented as a list JSONPath expressions, these SHOULD be Normalized Paths.". (But would it make sense to use "SHOULD" in non-normative text??)
I don't think that's implementation detail. It's defining what the output may be. We can talk about this in another issue if it concerns you, though.
I would say it's a statement about the representation of the abstract nodelist. A list of Normalized Paths is a string representation. Other possible representations are lists of JSON values in string form, a programming language list of nodes, etc. The choice of representation is an implementation decision. Feel free to raise a separate issue if you'd like to change the current text.
Normalized Paths ... are a concrete string presentation of nodelists (JSON text). are already used in the spec within every example table (Result Paths).
I believe we can merge this now as it is. If we want to add more guidance later we should open a separate issue and another PR.
This issue came up during the discussion of issue . We say that ACK or ACK_MP frames can be send over any path, however, maybe we should discuss some guidance and trade-offs. E.g. sending on the same path as the packets received also for RTT measurements; using the shortest delay path might have some performance benefits...?
I agree. We should add some guidance here. We did some experiment regarding this issue. Our finding is returning ACK on the shortest path is helpful. But the improvement is only more significant when the path rtt ratio is larger than 4:1.
As discussed on Slack, currently the text makes no mention of the case where the PUSHPROMISE arrives behind (some of) the data on the push stream itself. However, it does mention this situation for the DUPLICATEPUSH frame. This proposed change recommends buffering pushed data for an unknown PUSHID and using flow control to prevent this from being a problem. I am unsure if more text is needed (e.g., what happens if the PUSHID is never resolved (which would mean a misbehaving server)), but I think the proposed change should suffice. This is also related to issue . Thanks to NAME for his help.
LGTM. Non-normative language seems to be the best form of guidance here
the push id that links a push stream to a push promise is now an unframed varint that comes right after the stream preface. this is right now the only unframed data (except made for the unidirectional stream prefaces) in the specs and I am proposing to make that a frame as well for consistency and to simplify implementation and allow code reuse. motivation: the specs already define frames that only carry a single varint (mostly push related) max push id duplicate push cancel push and goaway! here instead we decided to leave it unframed just because we can since it is at the beginning of the stream cost: two extra bytes per push stream the text will need to specify that on a push stream this new frame (PUSH_ID ?) MUST be the first one, otherwise it is a protocol error. I am happy to put up a PR, in case we reach consensus
The varint type and length of a frame is also unframed. So you are asking to trade one unframed varint for two. Is your assertion that this is unique and difficult code to write, or that the unframed content is somehow difficult to manage?
This seems to be going back to some of the early discussion we had on a thread I can't find. I'm not against discussing things again because we have more experience. However, I don't see a compelling reason for the change. My implementation needs to parse unframed data because of the stream header and for QPACK streams. I think this change might make my implementation more complex.
Reading multiple varints requires more work: more checks to see if the stream has enough data, more checks to ensure only 1 frame sent at a specific time (this is the same as SETTINGS)
NAME those are in the frame header -- so they are part of the frame. I am not sure what your point is here then. Here I am assuming that an implementation has generic code to parse frames, and that parsing anything that is not part of a frame requires special handling. While this is implementation specific and there can definitely be counter examples, from an abstraction and consistency point of view I see no reason for doing it the way we are doing it now. (If we discount the unidirectional stream preface) I believe there should be two type of streams: framed (control, request, and push streams) and unframed (QPACK, and potentially extension streams). The push stream right now is a hybrid, that only becomes a framed stream after receiving that first PUSH_ID. If I am recalling correctly when we discussed potentially having DATA frames with zero-length to transition to unframed data, it was also considered a bit unusual. Here I am arguing that we are sort of doing the reverse here where we have an unframed stream that then transitions to framed. I think this generates complexity. And really the benefit we get is saving two bytes and one frame type. NAME I do vaguely remember that discussion and not 'wasting' frame types may have been one of the arguments when we made that decision, but that is not a problem now that we have a 62bit space for frame types.
I don't care about the cost, I care about things like having to deal with PUSH_ID (or whatever) frames in places other than the start of a push stream. And I don't believe that it is difficult to take the special-case code for pulling a varint off a partially-available stream and use that for push ID. We've not had too much trouble with that so far in our implementation.
it is definitely not difficult, but it is a special case and as any special case forces an increased complexity in the possible codepaths, even if not that big in this case. I am making my argument for the sake of consistency mostly, and for the fact that I believe it would also help to better define error handling.
With the current text, if I receive something bogus on a push stream (after the preface) I'll have to keep the stream around (because the push stream may come out of order) waiting for a PUSH_PROMISE with that ID that may never come. So I believe that framing could help error handling as well. I do not have super strong feelings about this proposal but at the same time I'd like us to try and recall what the motivations are for the current text and for making this special since it was a while back and personally I can't recall what they were.
This sounds like the case NAME is looking at. I'd argue this is a good reason for the current design, frame processing should only take place once the PUSH_PROMISE is received. But I'd like to understand how framing could help avoid errors.
I was not trying to solve the same problem as here, but I was mentioning the fact that I believe framing would slightly help in a subset of cases. Take a server with a buggy push implementation, rather than a malicious one. Something that doesn't send the PUSHID at all. On the client you'll start seeing a flow of data and you can't do anything better than parsing the first few bytes of that as a PUSHID, and basically see random IDs. With Framing, if a server forgets to send the PUSHID, it is clear at the receiver that it is a protocol error. That being said there are many other classes of bugs that may trigger the same behavior even with frames, so this is minor. NAME now, that seems like an implementation dependent claim :) as says (and I like it) a client should buffer any data after receiving the PUSHID and before it has a PUSH PROMISE for it. and that is sufficient. no mention of frames there.
Right now in draft 19 we have: Parse varint - stream type Parse varint - push ID Parse varint - frame type (HEADERS). The assertion then is, if step 2 gets skipped by the sender, the receiver will parse the HEADERS frame type as a varint expecting it to be the push ID (value 0x1, with no restriction on min or max encoding). It will then parse the HEADERS frame length as a varint expecting it to be the HEADERS frame type (length is unlikely to be 1).
Doesn't the client just blow up at that point?
It was just an example. And as I said I don't feel strongly about the error handling nit. But if you must, yes if you assume headers is the first frame after the pushid then yes you would read 0x1 which is conveniently a server initiated bidirectional stream which is not allowed anyway, so you could reject it immediately. But I find this just fortuitous. For example, one could make a counter argument: why are we framing SETTINGS at all given that it is the first message on the control stream ? We also defined it quite clearly in this table what to do with it [Certain frames can only occur as the first frame of a particular stream type; these are indicated in Table 1 with a (1).] URL Do you think there is zero value in doing that for the PUSHID as well? I personally think there is some value. Anyway, I think we are diverting this the wrong route. All I am looking for here is a good explanation for why it is defined like it is now, and I am failing to find it. As far as the implementation goes it makes little difference, and I guess it is more of a consistency problem for me. That naked varint on the wire triggers my OCD :) So instead of going back and forth like this, perhaps NAME can shed some light on the good reasons why it is this way.
+1 to levelling up, I'll take acknowledge my guilt here. But before I move on: If your at the stage of parsing QUIC STREAM frame payload, you should already know exactly what the stream ID is. Some historic points I remember: In Paris MT presented some ideas for unidirecitonal streams - URL This relates to issue URL and PR URL Push and control streams were identified then. And we were happy with reserving specific stream IDs for certain things. All non-reserved streams were server push streams. Subsequent data on the stream only makes logical sense to handle if you know what push it relates to, so those bytes go first. Roll forward some, QPACK design requires additional streams so those get reserved to. Around Kista, the topic of unidirectional stream types comes up. Without types, we make uncoordinated unidirecitonal stream extensions quite a difficult thing.
So add more bytes at the start of stream and remove reservations. The SETTINGS frame can contain a set of parameters, indeed it SHOULD contain at least one GREASE setting. A single SETTINGS frame that describes the length of all parameters is quite useful, you know when the peer has finished sending all of them. I don't see as much value for PUSH_PROMISE because you're just framing a single integer that describes its own length already.
oh gosh yeah, I confused PUSH_ID with a stream id. scratch that completely
One can imagine a scenario where we'd want to place something before the SETTINGS frame on the control stream. On the other hand, it's less likely we'd ever want anything other than the Push ID to begin the push stream.
Discussed in London; no.
Seems like a good editorial improvement.
You don't want to age out before you get epoch 3 because otherwise you might be dropping when you have handshake packets.
For my understanding: what is the harm of keeping the 0-RTT keys around till the end of the handshake?
"Yes, the goal is to encourage people not to keep 0-RTT keys too long, because they might not have PCS. The replay risk is essentially a non-issue, except to the extent that it might already have been replayed."
I think that the answer here is that DDR resolution is SVCB-reliant as defined in the SVCB spec. Can you make that explicit please?
says It sounds like you're saying that DDR should adopt this without adjustments. That seems reasonable to me.
Right. I sort of assumed that it was SVCB-reliant, but it doesn't say that. Certainly, this is not a case where any deviation from spec is necessary. (The more of those we have, the better.)
DDR predates when the reliant and optional terms came about, which is why it wasn’t covered. I’m not clear on why anything would need to be said here. Reliant and optional seem to be modes for the client to block on getting SVCB records or falling back and ignoring them if they don’t arrive. Discovery based on IP addresses is done with an SVCB query for URL, and no other query types are allowed for URL By definition, this only works with SVCB. That doesn’t mean that a client can’t be SVCB optional for its general operation — it just means that if SVCB is blocked or fails, you don’t get to discover the encrypted resolver. What do we need to say here that would actually benefit things?
So SVCB-optional doesn't even make sense for this specification. I was just looking for a nudge in the form of: "The resolver/client uses SVCB in a SVCB-reliant mode." Just so that there is no confusion.
Yes, optional doesn’t make sense. But I still think that the entire reliant/optional choice is orthogonal to how DDR works. These are the two lines that mention reliant mode in the SVCB spec: Both of these seem to be about client behavior for connection establishment to a hostname. It’s specifically about connecting. It doesn’t make sense to connect to URL; this is a special purpose SVCB query used for discovery, which can lead to a connection. I don’t see any requirement from the SVCB spec that protocols that use SVCB need to define themselves as “reliant” or “optional”. Is there something I’m missing there?
How is this not about connecting from the point at which you feed the name into a SVCB query? A DDR client is trying to connect to a (secure) resolver. To that end, it is making a SVCB query.
That it uses "URL" doesn't change the overall goal.
Until a client uses the SVCB query for discovery, it doesn’t know if an encrypted resolver exists that is designated. I’m not sure if saying a DDR-aware client is “SVCB-reliant” means that it will not use the unencrypted resolver for application queries until it discovers an encrypted resolver (which is not a requirement, obviously, since clients can be allowed to continue using unencrypted DNS if their local policy allows it), or it means that the connection to the discovered encrypted DNS server is SVCB-reliant, which is tautologically true since the existence of said server was only discovered via SVCB. If it’s the former, we shouldn’t say it because it’s really about client policy for allowing unencrypted DNS. If it’s the latter, I don’t see the value of adding anything, and I’m not sure it really fits the meaning of “reliant”.
I believe DDR clients' recommended behavior is exactly the behavior recommended in Section 8.2 of SVCB-DNS, quoted above. It is initially SVCB-optional, and becomes SVCB-reliant once SVCB resolution succeeds. Specifically: if you do discover an encrypted resolver, and you attempt connection, you SHOULD NOT fall back to unencrypted if that connection fails. Otherwise, you've added a downgrade vulnerability to attackers on the "transport path" (which may not be the same as the bootstrap-query path). As this is the default recommendation for SVCB-DNS, I don't think any additional normative language is strictly required. However, I think a reminder might be appropriate. This does not address the question of whether to wait for SVCB resolution before issuing any initial queries. As a practical matter, I don't think we can require that.
Okay, I agree with the behavior you describe there of being initially optional and then becoming reliant. I can try to find a place where it makes sense to reference that section.
Some definitional work for terms that were poorly used. Note that "host" in the remaining uses is strictly interpreted in the RFC 1122 sense, so I left those alone. I considered sorting the terms section differently, and might still alphabetize that, but this isn't problematic just yet.
The word "host" is undefined. If it is equal to "endpoint", it should be replaced. Otherwise, "host" should" be defined in Sec 1.2. The word "address" sometime means "IP address" and "a pair of IP address and port" in other cases. Sec 1.2 should define "address" as "a pair of IP address and port". And "IP" should be added in front of "address" which means "IP address".
fixed this.
Thank you!
Originally raised by Magnus: Section 5.2.1: o Session Information ("i="), URI ("u="), Email Address ("e="), Phone Number ("p="), Bandwidth ("b="), Repeat Times ("r="), and Time Zones ("z=") lines are not useful in this context and SHOULD NOT be included. Considering that b=CT is not useless, I am a bit surprised to see b= on this list. Yes, b=CT: is only useful is one like to provide a upper total bandwidth limitation. So, maybe some discussion of the possible choice?
This is just a mistake in 5.2.1 - we cover how to deal with it later in section 5. Prob need to remove the mention of b= here.
Adding a description to signaling options to improve readability. I have pending to update L2TP after the other issue is solved URL
Thanks, Victor. Does the text reflect the latest version of signaling/discovery split?
leaf l2tp-type { type identityref { base t-ldp-pwe-type; } description "T-LDP PWE type."; }
The type should be taken from this registry: URL
A new IANA-maintained module is created to fix this issue.
From NAME Please consider using the same wording for the title as for 6.2 and 6.1 (nit) Who is "we" in " We do have experience" ? While I share some concerns about deploying a new transport in user space as opposed to more stable and controlled kernel space, I wonder whether this paragraph is relevant in this document. Does it bring any value ? If no, then please remove it.
We'll look at the date for CUBIC being enabled by default in the Linux kernel as "experience". We'll delete the paragraph as suggested.
was fixed elsewhere.
LGTM. I'm almost sad to lose the ref to "Congestion Defense in Depth", but I guess it makes good sense, that's not a consensus position.
This distinguishes between the direction which was last set via setDirection (which simply affects offer/answer generation) from the direction that was negotiated, which affects whether a transceiver can send/recv media. Addresses issue .
According to WebRTC 1.0 Section 5.4 setDirection() does not take effect immediately: "The setDirection method sets the direction of the RTCRtpTransceiver. Calls to setDirection() do not take effect immediately. Instead, future calls to createOffer and createAnswer mark the corresponding media description as sendrecv, sendonly, recvonly or inactive as defined in [JSEP] (section 5.2.2. and section 5.3.2.). Calling setDirection() sets the negotiation-needed flag." My interpretation of the above is that setDirection immediately sets the transceiver.direction attribute, but that the "current direction" (distinct from the direction attribute) is only changed by calls to setLocal/setRemoteDescription. I would like to suggest that JSEP indicate precisely when the "current direction" changes. My reading of JSEP is that this is determined from the currentLocalDescription and currentRemoteDescription. For example: The direction provided in the currentLocalDescription (e.g. the "current direction") is "sendrecv" and setDirection("recvonly") is called. Since setDirection does not change the direction in currentLocalDescription, the RtpSender does not stop sending. createOffer() is called. A "recvonly" offer is generated. setLocalDescription(RecvOnlyOffer) is called. Now the direction in pendingLocalDescription changes to "recvonly", but the direction in currentLocalDescription is still "sendrecv", so the RtpSender does not stop sending. setRemoteDescription(SendOnlyAnswer) is called. Now the direction in currentLocalDescription is "recvonly", so the RtpSender will stop sending.
Bernard's description makes sense to me; currentLocal/Remote indicate the current direction state.
Fixed by