jiebi/CodeConvo huggingface dataset:
Update the aggregate request text to reflect recent protocol changes. Relax the requirement that all sub-requests use the same key config. Do URL, which was previously merged into the wrong branch.
NAME NAME please have a look when you can.
Per , the first step in the upload protocol is a POST to the leader server which returns information about helpers. In some yet to be specified cases, the response could include a challenge to be incorporated into reports. However I suspect that in most cases, the response will be static and so an idempotent, cacheable GET would be a better fit, especially since it would save clients a roundtrip to the helper on the hot path of submitting reports, and would make it possible for helpers to implement uploadstart by putting some static content in a CDN. HTTP GET requests don't have bodies, so we would have to figure out how to encode all the data in the current into the URI, which could be done as either path fragments in the URI (i.e., ) or as query parameters (i.e., ). We would of course need to make sure that don't preclude proof systems like the one alleged to exist in section 5.2 of "BBCp19".
Also, because the current protocol uses a POST for uploadstart, the implication is that for every report they submit, clients need to hit and then . If we make idempotent and cacheable, then that request can be skipped in the majority of cases. I suppose the significance of this depends on how we expect clients to use PDA: are they going to be continuously streaming hundreds or thousands of reports into the leader as events of interest occur (in which case I think eliminating requests is interesting) or are they expected to accumulate several reports locally before submitting them all in one or a few requests?
I agree this would be a good change, if we can swing it. It's notable that Prio v2 and HH don't require a random challenge from the leader. But as discussed, this is useful for other proof systems one might want to implement for Prio. For example, see the data type implemented here: URL See below for an explanation. There might be other ways to implement the challenge. One possibility is to derive the challenge from the TLS key schedule (or from whatever state the client and leader share for the secure channel).
NAME and NAME have both pointed out that adding this dependency on the underlying transport is problematic. Another possibility might be to use some external source of randomness that the leader trusts. comes to mind, though it is somewhat problematic because clients would effectively use the same randomness for some period of time. The security model of BBCp19 would need to be extended to account for this. (I believe this could be done.) Some other solution I'm missing? _ This data type is used to compute the mean and variance of each integer in a sequence of integers. To do so, the client encodes each integer of its input vector as a pair , where is the representation of as a bit vector and is equal to . To prove validity of its input, the proof needs to do two things: verify that (a) is indeed a bit vector, and (b) the integer encoded by is the square root of . In order to check these things efficiently, the proof system uses a random challenge generated by the leader. At a very high level, the client wants to prove that the following boolean conjunction holds: is 1 or 0 AND is 1 or 0 AND ... AND encodes the square root of . In Prio, we need to represent this conjunction as an arithmetic circuit, which the random challenge helps us do. In a bit more detail now: Let's rewrite this expression as w[0] AND w[1] AND ... AND w[s]. As an arithmetic circuit, this expression is actually , where if and only if w[i] is true. If the input is valid, then the expression should evaluate to . But because we're talking about field elements, a malicious client may try to play games like setting and so that the sum is still , but the statement is false. To detect this, we actually compute the following expression: , where is a randomly chosen field element. If the statement is true, then the expression evaluates to . If the statement is false, then with high probability, the expression will evaluate to something other than . (This idea comes from Theorem 5.3)
Thank you for the detailed explanalation NAME Having thought about it more, I suspect an HTTP GET wouldn't be the right way to support proof systems that need a challenge.
Assuming that a unique challenge is needed for every report[^1], then intuitively the leader would have to maintain a mapping of reports to issued challenges so that the leader and helper(s) can later use the challenge value when evaluating the client's proof. But now that the leader has to maintain state per-report, I think the upload protocol is missing a notion of a report identifier. The contains a , but IIUC the will be the same across many reports[^2] (e.g., if a browser is sending daily reports on how often users click a button, they use the same every time they submit). So needs to include a report ID, which must be echoed back to the leader in so that the leader can look up the challenge for the report. [^1] Is this true? Or would it suffice for each to use the same challenge across multiple reports? [^2] I notice that the design doc lacks a strong definition of a report and how it relates to tasks, inputs, measurements: because GET should be "safe" in the sense of not changing any server state, and it seems like the server would have to maintain a mapping of reports to issued challenges so that the leader and helper can later use the challenge value when evaluating the client's proof.
Continuing on the assumption that causes a state change on the leader, POST is then the appropriate HTTP method to use. , though it's a little unusual. There may be some big drawback to cacheable POST responses, but I think that might let us support proof systems that require live challenges while also eliminating the majority of requests when the proof system is such that the will change very rarely.
The requirement for the leader to keep state across upload start and upload finish may be avoidable. Suppose the leader has a long-lived PRF key K. It would could choose a random nonce N, compute R = PRF(K, N) as the challenge, and hand (N, R) to the client in its upload start response. The client would then use R to generate the proof and send N to the leader in its upload finish request. The leader could then re-derive R. NAME and I kicked around the idea of having the client generate a report ID. I think this may end up being useful, but we weren't sure if it was better to have the leader choose it or the client.
To conform to the security model of BBCp19, each report would need its own challenge. It may be possible to relax this, but this would be risky.
I like this a lot! I'm a big fan of deterministic derivation schemes like this -- besides reintroducing the possibility of an idempotent GET, eliminating storage requirements for challenges makes it much easier to scale up leaders. If the report IDs are big enough (UUIDs?) we could even use them as . I would say the leader, because if you let the client do it, then the leader would have to check for things like collisions with existing report IDs or other malicious ID choices. If the leader chooses it, then all it has to do is generate a few bytes of randomness. I agree.
NAME we'd appreciate your feedback on this issue because it involves security considerations for ZKP systems on secret-shared data [1]. I'll quickly sum up the problem so that you don't need to worry about the conversation above. The problem at hand regards systems like [1, Theorem 5.3] that require interaction between the prover and verifier. Specifically, we're considering a special case of Theorem 5.3 in which the verifier sends the prover a random "challenge" that is used to generate and verify a (non-interactive) FLPCP (call it ): We refer to the prover's first and second messages as "upload start" and "upload finish" respectively. Each message corresponds to an HTTP request made by a web client to a web server. Ideally, the verifier would be able to handle these requests statelessly, i.e., without having to store . Consider the following change. Suppose the verifier has a long-term PRF key . In response to an upload start request, it chooses a unique "upload id" --- maybe this is a 16-byte integer chosen at random --- and computes and sends to the prover. In the upload finish request, the prover sends so that the verifier can re-derive instead of needing to remember it: The (potential) problem with this variant is that a malicious client is able to replay a challenge as many times as it wants.
This behavior isn't accounted for in Definition 3.9, so when you try to "compile" this system into a ZKP on secret-shared data as described in Section 6.2 and consider its security with respect to Definition 6.4 (Setting I), you can no longer appeal to Theorem 6.6 to reason about its security. Our question is therefore: What impact do you expect this change to have on the soundness of the system? Can you think of an attack that exploits the fact that challenges can be replayed? My expectation is that the problem is merely theoretical, i.e., we ought to be able to close the gap with an appropriate refinement of Definition of 3.9 and reproving Theorem 5.3. [1] URL
Thanks for looping me in. In the discussion below, I am assuming that the client sends its data vector (called "x" in the paper [1]) at the same time as it sends its proof to the verifier. Even if the client sends its data vector in the first message, it seems that the client can reuse an (R,N) pair from upload request 1 to later submit upload request 2, in which case the client sees (R, N) before it chooses its data vector and proof. So I think we can assume that the client's data vector and proof can depend on the verifier's random value R. (I'll call this value "r" to match the paper.) For soundness: the client/prover must commit to its data vector before it learns the verifier's random value r. If the prover can choose its data value and proof in a way that depends on the verifier's randomness r, the soundness guarantee no longer holds. There is also an efficient attack. So I suspect that the stateless-verifier optimization is unsound. The attack works as follows: given r, the verifier finds a non-satisfying vector x such that sumi ri C(Ai(x)) = 0, again using the notation from Theorem 5.3 in the paper [1]. For the circuits C used in most applications, it will be easy to find such a vector x. Then, the prover constructs a proof that x that sumi ri C(Ai(x)) = 0.
The proof is valid, so the verifier will accept always. To remove interaction, the Fiat-Shamir-like technique in Section 6.2.3 is probably the best way to go. We only sketched the idea in the paper (without formal analysis or proof) but if it's going to be important for your application, I'm happy to sanity-check the protocol you come up with. Another idea—the details of which I have not checked carefully—would be to modify your protocol as follows: The prover sends its data vector x to verifiers, split into k shares x1, ..., xk—one for each of the k verifiers. Each verifier holds a PRF key ki computes ri = PRF(ki, xi) and they return r = r1 + ... + rk to the prover. The verifiers store no state. The prover computes the proof pi and sends (x1, pi1), ..., (xk, pik) to the verifiers—again, one for each of the k verifiers. Each verifier computes ri = PRF(ki, xi), they jointly compute r = r1 + ... + r_k, and check the proof using randomness r. The Fiat-Shamir-like approach seems slightly cleaner to me, since it only requires a single prover-to-verifier message. Either way, please let me know if I misunderstood your question or if anything else here is unclear. [1] URL
Thanks for the nice explanation, Henry. It's perfectly clear and answers my question. It also points to something that I, for one, missed the importance of: the need for the prover to "commit" to the input shares before generating the r-dependent proof. I appreciate you pointing this out. The only requirement for making upload start not idempotent is so that the randomness r can be sent to the client. It would be nice to do away with this requirement. I think the way forward is using the Fiat-Shamir technique. NAME I'll get cracking on this and let you know when I have something for you to look at.
NAME I believe the change is something like this. Let be a hash function with range . Suppose there are verifiers. The prover proceeds as follows: Splits its inputs into shares .
For each , do: Choose blinding factor from at random. Let . Set . Derive field element from (e.g., seed a PRNG with and map the output to a field element by rejection sampling on chunks of the output). Generate the proof using and . Split into shares . For each send to the -th verifier.
Yes, this looks right to me!
As of URL, the upload start request no longer exists. When we're ready to specify Prio, we'll want to include the Fiat-Shamir approach for protocols that use joint randomness. I've noted this and am closing the issue.
You'll also need to update the last paragraph of the MTI Extensions section (currently section 8.2). Now that we're going to be down to only one client-only extension defined in this document, that whole paragraph can probably be cut in favor of a one-off note in the remaining extension's section (signature_algorithms will be the last).
So that the clients can learn about what groups a server supports. The idea here is to allow a client to offer P256 but then learn that a server supports 25519 for next time without the server having to reject.
What data can and cannot be retried safely?
That's not something that QUIC knows. HTTP does though. As advice, I don't think that this needs much more expansion, but I'd be happy to take a PR.
I'm not so sure this is a HTTP-only issue. Unlike TLS 1.3, there is QUIC data being sent under encryption, and that might have side effects.
LGTM, one nit. ("using applications" is fine, it's just a bit awkward.)
Follow-up from
Consider the following sequence of packets being sent In our old loss recovery code (before unifying TLP and RTO, ), we used the following logic for determining if an RTO was not spurious, which would then lead to collapse of the congestion window: In the example above would be packet 14. This means that to count as a verified RTO, an ACK must acknowledge 15 or 16, but not any packets smaller or equal than 14. With , we now collapse the congestion window when is larger than 2 and we receive an ACK which leads us to declare any packet lost. In the example above this means any ACK received after sending packet 15 which declares a packet lost would suffice. Even worse, this logic already triggers if we receive the ACK immediately after sending 15, i.e. when the peer didn't even have a chance to receive it. This seems a lot more aggressive than what we had before before. Is this intentional?
This is unintentional. I belIeve Jana is intending to fix this in PR , so please take a look at that.
If I understand correctly, is a fix for and only makes the description of the constant and the pseudo code consistent.
I believe the text is now correct, but the pseudocode is still incorrect in the way you describe. I think we should introduce a largestsentbefore_congestion variable as a replacement, but I'm open to other solutions.
I'm removing design. I consider this a bug fix, but still very important.
See which I believe fixes the text.
I've been using to mark things that will result in changes that affect implementations. This would seem to fit that description. Whether you consider it worth discussing is different. If it isn't worth discussing, maybe it will just be fixed by and everything will be fine without discussion in Tokyo. WIthout the label, this won't make the change log, and that might be a problem.
Fair point, I think fixes the text, but I'll re-label this as design to ensure it's included in the changelogs.
The spurious RTO detection calls an RTO spurious if any unacked packets sent prior to the RTO are acked after the RTO.
The implication here is that any packet that was received but not acked can cause the RTO to be labeled spurious, even if there were more packets that were in fact lost. This is not always right. Consider the following example: Packets 5, 6, 7 are sent Packet 5 is received, but 6 and 7 are dropped Ack of 5 is dropped TLPs (packets 8 and 9) are sent and dropped RTO (packet 10) is sent and received ACK acks 10 and 5 Since 5 was sent prior to RTO, RTO is marked spurious. A crisp definition of spurious RTO may be useful to continue. Here's what I think is sensible: A spurious RTO is an unnecessary RTO. If the RTO timer had not gone off, things would have been fine. Based on this definition, clearly the RTO was necessary in the example, and therefore not spurious. We need to change the mechanism for detecting spurious RTOs, and I'll propose the following. An RTO is spurious if an ack is received after the RTO fires that acks only packets sent prior to the RTO. In other words, RTO is spurious if ack is received after the RTO, and (instead of , as the draft currently states).
This was recently changed because Lars ran into an issue() where the RTO'd packet was acknowledged for the first time in the same ACK as the pre-RTO packets. In that case, I think it was due to an overly aggressive MinRTO and some pretty coarse grained timers, but I think we want to declare that RTO spurious, since everything would have been fine if it hadn't been sent?
NAME : does not change the semantics substantively of what was in the draft. The intent was always to declare spurious RTO if (packet prior to RTO was newly acked). I'm proposing that we change that to (no packet sent after RTO was newly acked).
In light of , I think this is OBE?
Yes, this is OBE. Closing.
Clean-up after URL As the content-from-edhoc branch seems to be the most up-to-date one (in which the parts changed are introduced in the first place), this builds on , and only the latest commit is actually relevant.
Compared to the original bstridentifier mapping, this is a different int-to-bstr mapping, but one that's most likely efficient to implement. I have not made any statement on the efficiencies, but some guidance we could give is that from a coding density point of view, numeric Cx should only be used with -24..23; for all other values, doing it in a bstr is just as good and saves a byte in OSCORE. (In fact we may want to make that normative, so that only the single-byte case needs to be implemented where this needs explicit implementation).
I realize that the specific review feedback was for the Introduction - but might we also want to "minimize" the number of WS configurations discussed in Section 4 to focus on the browser motivation?
While the third configuration is not mentioned in the introduction, all three are motivated by in-browser app use cases. So I don’t see a need to cut down the discussion in Section 4. Grüße, Carsten
: The introduction motivates the layering of COAP over WebSockets to allow it to be used in a network that only allows external access through a HTTP proxy. In fact, it is not necessary to use WebSockets to do so; HTTP CONNECT provides a tunnel in this scenario. WebSockets is a specialised protocol that allows bidirectional, stream-based transport from a Web browser -- it is effectively TCP wrapped into the Web browser's security model, to make doing so safe. Using it for other purposes (i.e., non-browser use cases) isn't really appropriate. I'd suggest either removing the WebSockets binding, or changing its motivation in the Introduction. For reference, the original URL motivation:
PR had a mistake in it, and some very confusing language. The rules for generating stateless resets was all mixed up in some complementary rules about generating packets that might trigger stateless resets. This attempts to split out the rules for generating stateless resets (minimum size is 5 + 16 = 21), and for generating packets that might trigger a stateless reset (min_cid + 22).
Do we need discussion on the fixed QUIC header bit in the triggering packet and outgoing reset packet?
NAME if you can lay out what you think the security concerns are in another issue, that would helpful. I realize that there are plenty of compromises in this design, which make this less than ideal. But first I'd like to try to understand where you think this is falling short.
Do we need discussion on the fixed QUIC header bit in the triggering packet and outgoing reset packet? Originally posted by NAME in URL
Maybe I'm misunderstanding the question, but isn't the spec is completely clear on that? A stateless reset has the first bit unset and the second bit set.
Agreed, I think this is already clear and I don't see any need to change it.
Thanks both. I just wanted to check and didn't feel like I had the brain cells to answer that. I don't like relying on the image only, but the text says "two fixed bits", and stateless reset - unlike Version Negotiation - is version-specific. So I'm closing with no action.
I believe this PR addresses the stated issue, which is that the text is unclear. However, I am still concerned about the security properties of this design.
URL in this example, it would be very stupid of the client to select version C and start another negotiation after the sever has indicated D is what it prefers. The server can again select D as preferred and there exits a potential loop. However, it is possible. What are the rule we have in the specification that stops this kind of loop?
I'm not sure that I understand this question. A client makes a choice about what version to attempt (and offer) when it creates a connection. It might learn that its versions are incompatible with a server (with a Version Negotiation packet) and try again, but that retry would be a separate action. But if the connection is successfully established, the negotiation is done.
There is at most one round of incompatible VN followed by one connection that may or may not use compatible VN to switch versions again. There is no potential for a loop here. NAME can you clarify what you meant?
The scenario i was thinking is a follows - Chosen = A, Other Versions = (A, B) -----------------> <----------------- Chosen = G, Other Versions = (G,H) ........ the draft only says the client's first flight contain This contains the list of versions that the client knows its first flight is compatible with (A,B). It does not force the client to list ALL the compatible versions, so it might not list all the compatible versions. I think, there is also no requirement on the server to list ALL the compatible versions. Now server sends back another list of compatible versions (D,C). This new list triggers the client to realise it knows about versions that are compatible (E,F), and sends it back to the client as it has not interest in D or C. That trigger the server to reply with new list of compatible version (G,H).. and this goes on. It is also possible that client made a mistake by not listing all the compatible version (well it does not have to) and as it is not interested in D or C, it wants to list (E,F) and keep in negotiating. I would like to understand the with current specification this kind of interaction is not possible and the negotiation should not go on like that.
The scenarios you describe aren't allowed: if the server performs compatible version negotiation it MUST select a that was included in the client's :
yes, but servers are allowed to do incompatible version negotiation.