Change:
Specifically: Specify use of the "one-shot" HPKE APIs defined in RFC9180, replacing equivalent use of the "multi-shot" APIs. (This change is intended to be a clarification, not a functional change.) Move the task_id parameter from the "info" parameter to the "aad" parameter. This is intended to protect against key-commitment-related attacks[1].) This is a functional change. Clarify that the URL field is an encapsulated key, rather than an encryption context. (This is a clarification, not a functional change.) This is a second copy of a previous PR () which was merged too hastily, and then reverted. There are no changes from that PR. [1] See discussion in .
Per URL section 8.1: PPM effectively uses the single-shot APIs; we should consider refactoring PPM's usage of HPKE to use only one of applicationinfo & AAD. Following the advice of the HPKE RFC, we would want to use applicationinfo. This would permit slightly simpler implementations. I am not sure how much it would simplify the HPKE computations.
Should we then formally use the single shot HPKE APIs from ?
That would be my recommendation (though, somewhat confusingly given the advice quoted above, those APIs still allow specifying both application info & AAD). Implementations can & do already use the single-shot APIs, so I strongly suspect specifying the single-shot APIs will be feasible. & .]
+1 to using single-shot APIs, though I'm not sure I agree that using either info or add (but not both) is simpler. What's the reasoning?
The reasoning is: a) it follows the advice of the HPKE spec (quoted above) b) AFAICT, in a single-shot setting, application info & AAD serve exactly the same purpose, so using both is arbitrary/confusing/a potential source of mistakes. (in a multi-shot setting, application info is bound to all encryptions/decryptions done by the same context; AAD is bound to a single encryption/decryption)
TBH I'm not sure I agree with the quoted guidance here. There is a concept in protocol design known as "domain separation", the goal of which is to provide some kind of binding of long-term secret key operations to the context in which they were used. Imagine, for example, that someone used an HPKE secret key for PPM and some other protocol. Suppose further that both are using the empty string for . Then the security of our PPM deployment depends on how the other protocol uses the derived AEAD key, since in both protocols may end up deriving the same AEAD key. One way to avoid creating this attack surface is to pick an string that no other application is likely to choose so that derived keys are guaranteed to not collide (except with some negligible probability). That said, I'd go for changing how we use and . Something like this might be simpler: Let be a fixed string that identifies the protocol. It would be good if this string were relatively long, say the SHA-256 hash of "ppm-00". Let encode the task ID, sender/receiver roles, etc.
I definitely agree that application info should include information allowing domain separation, for the reasons you mention. (Indeed, the current PPM spec includes the strings "ppm input share" or "ppm aggregate share" in its application infos, presumably for this purpose.) But I think that's orthogonal to the question of whether we should also use AAD, since the application info could also include information other than the "domain separation parameter". Picking one example to make things concrete, a client upload request sets its application info to and its AAD to . But it could just as easily set the application info to (i.e. the concatenation of the two previous strings) and not use AAD. (Or alternatively, it could place that string in the AAD and not use application info.) My argument is that it's simpler to use only one of the parameters, since in the single-shot setting that PPM is using HPKE, the two parameters serve exactly the same purpose (i.e. including additionally-authenticated data).
As I see it, our goal is to make sure that decryption only succeeds if the sender and receiver agree on the "context", i.e., the task ID, sender/receiver role, nonce, extensions, and so on. This way a MITM cannot force the aggregator to interpret an input share incorrectly. Does this sound reasonable to you? From the point of view of this threat model, I don't think it's immediately clear that sticking the context in or is equivalent. For example, if we stuff the context in , then it influences the derivation of the AEAD key; but depending on the AEAD, it might be feasible for an attacker to find a find two keys that decrypt the same ciphertext. (See URL) More analysis will be needed to say for sure whether this weakness in the AEAD amounts to an attack against our protocol. In the meantime, sticking the "context" in seems like a more conservative choice to me.
Given your concern, your approach SGTM. It sounds like this may be a concern worth raising with the folks working on the HPKE spec, given the advice to "use the Setup info parameter for specifying auxiliary authenticated information", but I'm not going to chase this down currently.
So I think there's two tasks for this issue: Move task ID from to Use single-shot APIs for HPKE Anything else?
Yep, I think that's it.
(reopening for now as the fix PR has been reverted temporarily)
See .
Please note that (regular, not necessarily incremental) DPFs can also be used for aggregation, and are described in a series of papers: Proof of concept implementations of DPFs can be found or .
Strong +1 to this idea. I had a longer write-up motiving a use case and outlining a high level solution which leverages this technology for a web API use-case. Then I found this existing issue. Here's what I had: The existing Prio protocol enables distributed computation of histograms which have buckets determined by a set of aggregation ids. This construction requires communication overhead per contribution per client that scales linearly with the size of the domain of aggregation (i.e. the size of the histogram vector). This solution works for modest domains (e.g. ~1000s of entries), but starts to incur significant communication overhead as domains get larger (~1M entries or more). With the effort, we are looking at aggregation use-cases where domain sizes for aggregation are extremely large and sparse e.g. on the order of ~1B possible aggregation buckets, with ~1k - 1M non-zero buckets. This is analogous to the “heavy hitters” problem. Additionally, we are looking at protecting the output with central differential privacy noise. In trying to solve for these use-cases, we are exploring a few optimizations: Improving communication overhead via compression techniques that grow only logarithmically in the size of the output domain Improving server-side performance by allowing for dynamic / hierarchical query models and partial evaluations of the output domain These techniques are inspired by . Note: we are also exploring MPC protocols which don’t grow proportionally to the output domain at all, but they are at an early stage. A new cryptographic primitive can help us achieve these goals. Given a function f(x): X -> Y, a function secret sharing is a primitive defined by two algorithms GenKey(f) outputs two keys K0 and K1 Eval(Kb, x) outputs value zb for b=0, 1 and all x∈X such that z0 and z1 are two random shares of f(x) in Zp, i.e z0 + z1 = f(x) mod p. A point function fa,b(x) : X -> Y is a function such that f(a) = b and f(a) = 0 for all x ≠ a. A distributed point function is a function secret sharing for a point function fa,b(x). There exist such that the size of the DPF keys K0 and K1 is logarithmic in the function input domain . By themselves, DPFs can be drop-in replacements for vector secret shares (via calling Eval on the full domain), and there are secure ways of verifying input consistency, like in Prio, with interaction between the servers. This trades off a cpu increase for a large communication overhead decrease. Additionally, Boneh et. al. have shown that these techniques can be extended to allow for evaluation on prefixes of the input point (“incremental” DPFs or "IDPFs"). In other words: Eval(Kb, xt) outputs value zb for b=0, 1 and all x∈X such that z0 and z1 are two random shares of f(x) in Zp, i.e z0 + z1 = f(x) mod p, for xt a t-bit prefix of x. Prefix evaluation critically allows for evaluating input in a hierarchical manner (i.e. counts at a fixed prefix). By leveraging IDPFs, we can allow for dynamic queries on large, sparse domain data that traverses a hierarchy. For example, imagine a domain of size 2^25 where only a small fraction of the entries are expected to be non-zero. Both vector secret sharing and naive DPF expansion would struggle with this problem. With IDPFs though, a client could: Evaluate the data at a 12-bit prefix Find that only 2^5 prefixes hold meaningful amounts of data Partially evaluate the data again (unprefixed), but only for entries which match the 2^5 prefixes that have data. This will end up evaluating 2^(5 + 25 - 12) = 2^18 entries, which is more manageable (~250 kb expansion per record) than expanding the whole domain from the beginning Timings from preliminary C++ benchmarks: 526ms for a full domain evaluation (size 2^25) 12ms for incremental / sparse evaluation. This single pruning step is a simple example, but this technique can be leveraged in much more complex ways to solve real world private metrics problems. Of course, if we are applying DP noise in the computation we may need to split a privacy budget across each of these dynamic queries. Internally at Google we are working on a production C++ implementation of IDPFs (for fast performance), to be open sourced very soon. We are seeing ~10ms to fully expand a ~4 mb vector from a ~350 byte key, and ~1ms to expand a 500kb vector from a ~300 byte key. There is a (non-production) of IDPFs by Henry Corrigan-Gibbs NAME referenced a golang and C++ implementation of DPFs above
NAME we'd welcome your thoughts in this thread :-)
Thanks for looping me in here! It would definitely be nice if Prio v3 could support heavy-hitters queries. There are a number of applications that need this, so I like the idea of building it into the design. At the same time, the machinery needed to support these heavy-hitter applications involves a quite a few moving parts that are not needed for Prio v1/v2 functionality. In particular, Prio v1/v2 uses: Two or more servers (where privacy holds if at least one is honest) Simple additive secret sharing with finite-field arithmetic ZK proofs on secret-shared data (called "SNIPs" in the Prio paper) The simplest version of the heavy-hitters scheme from the paper needs: Two servers (where privacy holds if at least one is honest) Simple additive secret sharing with finite-field arithmetic Distributed point functions Sketch-verification schemes (Section 4 in the BBCGI21 paper) Do we want to expand the scope of this document to include the more recent and more advanced techniques? Or, is it better to stick with the Prio v1/v2 technology, which is quite stable and which we understand pretty well at this point—even if it does not support all of the functionality that we want?
I think it's worth spinning up a new doc, though presumably there are pieces shared by both (i.e., ZK proofs on secret shared data). These could live in a third doc.
The C++ implementation of (I)DPFs mentioned by NAME above is now public: URL
FYI I published an example query model for how the flexible, non-interactive heavy hitters could work via some configuration here: URL It basically requires either the collector or leader to give some configuration outlining which bit-prefixes to expand.
I think we're well passed the relevant point on the decision tree here :) Any reason to keep this issue open? I suppose the links are useful, but it's easy enough to rummage around in the closed issues to find them.
Let's close it out.

Discussion:
To generate the report, the client begins by sharding its measurement into a sequence of input shares as specified by the VDAF in use.  To <del> encrypt an input share, the client first generates an HPKE RFC9180 context for the aggregator by running </del> <ins> encrypt an input share, the client generates an HPKE RFC9180 ciphertext and encapsulated key for the aggregator by running </ins> <del> where "pk" is the aggregator's public key and "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for the helper).  In general, the info string for computing the HPKE context is suffixed by two bytes, the first of which identifies the role of the sender and the second of which identifies the role of the intended recipient. "enc" is the HPKE encapsulated key and "context" is the HPKE context used by the client for encryption.  The payload is encrypted as where "input_share" is the aggregator's input share and "nonce" and "extensions" are the corresponding fields of "Report".  Clients MUST NOT use the same "enc" for multiple reports. </del> <ins> where "pk" is the aggregator's public key; "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for the helper); "task_id", "nonce", and "extensions" are the corresponding fields of "Report"; and "input_share" is the aggregator's input share. </ins> The leader responds to well-formed requests to "[leader]/upload" with HTTP status code 200 OK and an empty body.  Malformed requests are

report share as invalid with the error "hpke-unknown-config-id". Otherwise, it decrypts the payload with the following procedure: <del> where "sk" is the HPKE secret key, "task_id" is the task ID, "nonce" and "extensions" are the nonce and extensions of the report share respectively, and "server_role" is 0x02 if the aggregator is the leader and 0x03 otherwise.  If decryption fails, the aggregator marks the report share as invalid with the error "hpke-decrypt-error". Otherwise, it outputs the resulting "input_share". </del> <ins> where "sk" is the HPKE secret key, and "server_role" is the role of the aggregator ("0x02" for the leader and "0x03" for the helper).  If decryption fails, the aggregator marks the report share as invalid with the error "hpke-decrypt-error".  Otherwise, it outputs the resulting "input_share". </ins> 4.3.1.4.

"AggregateShareReq" is done as follows: where "pk" is the HPKE public key encoded by the collector's HPKE <del> key, and server_role is is "0x02" for the leader and "0x03" for a helper. </del> <ins> key, "server_role" is "0x02" for the leader and "0x03" for a helper. </ins> The collector decrypts these aggregate shares using the opposite process.  Specifically, given an encrypted input share, denoted "enc_share", for a given batch interval, denoted "batch_interval", decryption works as follows: <del> where "sk" is the HPKE secret key, "task_id" is the task ID for a given collect request, and "server_role" is the role of the server that sent the aggregate share ("0x02" for the leader and "0x03" for the helper). </del> <ins> where "sk" is the HPKE secret key, "task_id" is the task ID for the collect request, and "server_role" is the role of the server that sent the aggregate share ("0x02" for the leader and "0x03" for the helper). </ins> 4.4.5.