Draft: draft-ppm-protocol

Commentary: is never transmitted over the wire , nor do we otherwise depend on is structure , so this commit removes that struct definition and instead enumerates the pre - negotiated task parameters each participant must hold , leaving implementations free to store , represent and transmit those parameters any way they wish . is still around , but instead of being a SHA-256 over the serialiation of Param , it 's now just 32 random bytes . 
 Mind rebasing , 
 Sounds reasonable to me . I 'll rebase after is merged . 
 Rebased on main to incorporate . 
 Currently , the derivation of HPKE context entangles a constant for , which is 0x01 for the leader and 0x00 for the helper . Some iterations of PR , which aimed to generalize message definitions to support more than one helper , introduced a notion of an aggregator ID or role , and then used that role in the key derivation . That notion wound up being a little more complicated than what needed to be addressed in that PR , but we should still consider whether and how we want to formalize the notion of an aggregator 's role and whether it should be involved in encryption key derivation . 
 I 'm not sure if this should go in label parking - lot . The server role constants we have now are sufficient for the deployments that will realistically materialize in the near or medium term , and I think we could define server roles in the future in a manner that is compatible with the existing constants . 
 I 'm in favor of the generalization , but am fine with putting this in the parking lot . 
 One important question we must address is the maximum number of aggregators in a protocol instantiation . We are already encountering places where . 
 We 've long since introduced a formal enumeration whose layout was chosen to allow adding more helpers in the future . Closing . 
 The current specification describes a PPMParam struct and defines a TaskID as SHA-256(PPMParam ) . However , because the PPMParam is never rendered on the wire . Rather it is just serialized in order to be fed into SHA-256 . The parameters are provided to the endpoints out of band . Consider a client implementor who behaves as follows : Create a JSON structure J that defines the parameters . Translate J into    the client the pair [ J , TaskID = SHA-256(P ) ] You 'll note that the client never sees the PPAParam at all . As far as I can tell , this would be a completely conformant implementation . Similar comments apply to the aggregators as we also do not define how they are configured . So , similarly they could just be sent  and have a TaskID -JSON lookup table . This is n't just a possible implementation , but it 's likely to be a common one , because people have whatever central configuration system they have and do n't necessarily want to carry around some binary blob . In fact , the only reason people need to see the PPAParam structure at all as opposed to the information in it is to hash it to create TaskID ( yes , i appreciate that I suggested the hashing thing ) . So what I suggest is we merely make a list of the things that the client and the aggregators need to know respectively and not define any structure that contains that list . Then we can just pick the TaskID randomly . When we originally discussed hashing , the idea was to have parameters be transitively incorporated into the protocol to avoid situations where each side is using different parameters . It 's not clear to me that this is that important . A malicious client can just lie and use its own parameters , and if there is a bug , then it 's just as likely it involves misinterpreting the parameters . 
 I think our goal was to force clients to commit to a set of PPMParams , but you 're right , nothing really ties the actual behavior of clients or aggregators to the PPMParams . We put a 16 byte nonce into PPMParam with the intention of guaranteeing that  would be unique , but I think we can just instead have  be that nonce . As for the sizes , we picked len(PPMTaskID ) = 32 to match the length of a SHA-256 , and len ( ) = = 16 to match a v4 UUID . If  is going to be randomly generated instead of derived , then 16 bytes should suffice . 
 The problem we 're trying to solve by binding the PPM params to encryption has less to do with malicious clients than honest ones . Namely , it 's meant to ensure that honest aggregators reject reports that were generated with incorrect parameters . As to whether this binding is strictly necessary , I agree that it 's not clear that there 's an attack it prevents right now . This binding is conservative in the sense that it aims to mitigate attacks that may come up in the future , as the set of parameters evolve . Another benefit of this binding is that it ensures the client knows what privacy - sensitive parameters are enforced by honest aggreagators . Namely , the batch size , minimum batch window , and , eventually , parameters for DP . To conclude , there may not be a need to define a serialization format for PPM params , and it would be fine to just randomly the task ID . However , there should be some sort of binding between the params and HPKE encryption . 
 You seem to be assuming a very specific implementation , but nothing in the spec requires that , as shown by my example above . 
 Based on our conversation in the call on 8/18 , I think the consensus is we should not define but instead informally list the parameters that all participants must agree on ( which does appear in several on - the - wire messages ) will become a randomly generated sequence of bytes instead of being a SHA-256 of the Unless someone objects ( tagging to make sure he sees this issue as he has n't participated yet ) I 'm going to make this doc change this week , after landing . 
 seems like it complicates this , since we 'd have to spell out some way to sort aggregators . Do we want to land this change before that one ? 
 I want to land first , since it 's just about done . I do n't think it 'll be too hard to change references to into references to an ordered list of aggregators . 
 Sounds good ! 
 A few nits , but otherwise I 'm good with landing this .
Paragraph (version A):
"uuid": A unique sequence of bytes used to ensure that two
otherwise identical "Param" instances will have distinct
"TaskID"s.  It is RECOMMENDED that this be set to a random 16-byte
string derived from a cryptographically secure pseudorandom number
generator.
Paragraph (version A):
"aggregator_endpoints": Vector of URLs relative to which an
aggregator's API endpoints can be found.  The leader's endpoint
MUST be the first in the vector.  The order of the
"encrypted_input_shares" in a "Report" (see uploading-reports)
MUST be the same as the order in which aggregators appear in this
vector.
Paragraph (version A):
"collector_config": The HPKE configuration of the collector
(described in key-config).  Putting the collector's HPKE
configuration directly in "struct PPMParam" absolves collectors of
the burden of operating an HTTP server.  See #102 [1] for
discussion.
Paragraph (version A):
"max_batch_lifetime": The maximum number of times a batch of
reports may be used in a collect request.
Paragraph (version A):
"min_batch_size": The minimum batch size, i.e., the minimum number
of reports that appear in a batch.
Paragraph (version A):
"proto": The PPM protocol, e.g., Prio or Hits.  The rest of the
structure contains the protocol specific parameters.
Paragraph (version A):
Where param is the serialization of Param.
Paragraph (version B):
We start with some basic type definitions used in other messages.
Paragraph (version B):
"aggregator_endpoints": A list of URLs relative to which an
aggregator's API endpoints can be found.  Each endpoint's list
MUST be in the same order.  The leader's endpoint MUST be the
first in the list.  The order of the "encrypted_input_shares" in a
"Report" (see uploading-reports) MUST be the same as the order in
which aggregators appear in this list.
Paragraph (version B):
"collector_config": The HPKE configuration of the collector
(described in key-config).  Having participants agree on this
absolves collectors of the burden of operating an HTTP server.
See #102 [1] for discussion.
Paragraph (version B):
"max_batch_lifetime": The maximum number of times a batch of
reports may be used in collect requests.
Paragraph (version B):
"min_batch_size": The minimum number of reports that appear in a
batch.
Paragraph (version B):
"protocol": named parameter identifying the core PPM protocol,
e.g., Prio or Hits.
Paragraph (version A):
"encrypted_input_shares" contains the encrypted input shares of
each of the aggregators.  The order in which the encrypted input
shares appear MUST match the order of the "aggregator_endpoints"
in the "Param" corresponding to "task_id" (i.e., the first share
should be the leader's, the second share should be for the first
helper, and so on).
Paragraph (version A):
"config_id" is equal to "HpkeConfig.id", where "HpkeConfig" is the
key config of the aggregator receiving the input share.
Paragraph (version B):
"encrypted_input_shares" contains the encrypted input shares of
each of the aggregators.  The order in which the encrypted input
shares appear MUST match the order of the task's
"aggregator_endpoints" (i.e., the first share should be the
leader's, the second share should be for the first helper, and so
on).
Paragraph (version B):
"aggregator_config_id" is equal to "HpkeConfig.id", where
"HpkeConfig" is the key config of the aggregator receiving the
input share.
Paragraph (version A):
The "time", "nonce", and "extensions" fields have the same value as
those in the report uploaded by the client.  Similarly, the
"helper_share" field is the "EncryptedInputShare" from the "Report"
whose index in "Report.encrypted_input_shares" is equal to the index
of "[aggregator]" in "Param.aggregator_endpoints".  [OPEN ISSUE: We
usually only need to send this in the first aggregate request.  Shall
we exclude it in subsequent requests somehow?]  The remainder of the
structure is dedicated to the protocol-specific request parameters.
Paragraph (version B):
The "time", "nonce", and "extensions" fields have the same value as
those in the report uploaded by the client.  Similarly, the
"helper_share" field is the "EncryptedInputShare" from the "Report"
whose index in "Report.encrypted_input_shares" is equal to the index
of "[aggregator]" in the task's aggregator endpoints.  [OPEN ISSUE:
We usually only need to send this in the first aggregate request.
Shall we exclude it in subsequent requests somehow?]  The remainder
of the structure is dedicated to the protocol-specific request
parameters.
Paragraph (version A):
where "pk" is the HPKE public key encoded by the collector's HPKE key
configuration, "task_id" is "OutputShareReq.task_id" and
"aggregator_id" is the index of the helper's endpoint in
"Param.aggregator_endpoints". "output_share" is the serialized
"OutputShare", and "batch_interval" is obtained from the
"OutputShareReq".
Paragraph (version B):
where "pk" is the HPKE public key encoded by the collector's HPKE key
configuration, "task_id" is "OutputShareReq.task_id" and
"server_role" is the role of the server ("0x01" for the leader and
"0x00" for the helper).  "output_share" is the serialized
"OutputShare", and "batch_interval" is obtained from the
"OutputShareReq".
Paragraph (version A):
"collector_hpke_config_id" is "collector_config.id" from the
"Param" corresponding to "CollectReq.task_id".
Paragraph (version A):
"encrypted_output_share" is an encrypted "OutputShare", whose
structure is given below.
Paragraph (version B):
"collector_hpke_config_id" is "collector_config.id" from the task
parameters corresponding to "CollectReq.task_id".
Paragraph (version B):
"payload" is an encrypted "OutputShare".
Paragraph (version A):
"task_id", the task ID associated with the PPM parameters "Param".
Paragraph (version B):
"task_id", the PPM task ID.
Paragraph (version A):
First the aggregator checks that the request's batch interval
respects the boundaries defined by the PPM parameters "Param".
Namely, it checks that both "batch_interval.start" and
"batch_interval.end" are divisible by "Param.min_batch_duration" and
that "batch_interval.end - batch_interval.start >=
min_batch_duration".  Unless both these conditions are true, it
aborts and alerts the peer with "invalid batch interval".
Paragraph (version A):
If "len(X) < Param.min_batch_size", then the aggregator aborts and
alerts the peer with "insufficient batch size".
Paragraph (version A):
The aggregator keeps track of the number of times each input share
was added to the batch of an output share request.  If any input
share in "X" was added to at least "Param.max_batch_lifetime"
previous batches, then the helper aborts and alerts the peer with
"request exceeds the batch's privacy budget".
Paragraph (version B):
First the aggregator checks that the request's batch interval
respects the boundaries defined by the PPM task's parameters.
Namely, it checks that both "batch_interval.start" and
"batch_interval.end" are divisible by "min_batch_duration" and that
"batch_interval.end - batch_interval.start >= min_batch_duration".
Unless both these conditions are true, it aborts and alerts the peer
with "invalid batch interval".
Paragraph (version B):
If "len(X) < min_batch_size", then the aggregator aborts and
alerts the peer with "insufficient batch size".
Paragraph (version B):
The aggregator keeps track of the number of times each input share
was added to the batch of an output share request.  If any input
share in "X" was added to at least "max_batch_lifetime" previous
batches, then the helper aborts and alerts the peer with "request
exceeds the batch's privacy budget".
Paragraph (version A):
Clients have limited capabilities and requirements.  Their only
inputs to the protocol are (1) the Param structure configured out of
band and (2) a measurement.  Clients are not expected to store any
state across any upload flows, nor are they required to implement any
sort of report upload retry mechanism.  By design, the protocol in
this document is robust against individual client upload failures
since the protocol output is an aggregate over all inputs.
Paragraph (version B):
Clients have limited capabilities and requirements.  Their only
inputs to the protocol are (1) the parameters configured out of band
and (2) a measurement.  Clients are not expected to store any state
across any upload flows, nor are they required to implement any sort
of report upload retry mechanism.  By design, the protocol in this
document is robust against individual client upload failures since
the protocol output is an aggregate over all inputs.
Paragraph (version A):
Collectors statefully interact with aggregators to produce an
aggregate output.  Their input to the protocol is the Param
structure, configured out of band, which contains the corresponding
batch window and size.  For each collect invocation, collectors are
required to keep state from the start of the protocol to the end as
needed to produce the final aggregate output.
Paragraph (version A):
Collectors must also maintain state for the lifetime of each Param
value, which includes key material associated with the HPKE key
configuration.
Paragraph (version B):
Collectors statefully interact with aggregators to produce an
aggregate output.  Their input to the protocol is the task
parameters, configured out of band, which include the corresponding
batch window and size.  For each collect invocation, collectors are
required to keep state from the start of the protocol to the end as
needed to produce the final aggregate output.
Paragraph (version B):
Collectors must also maintain state for the lifetime of each task,
which includes key material associated with the HPKE key
configuration.
Paragraph (version A):
An important parameter of a PPM deployment is the minimum batch size.
If an aggregation includes too few inputs, then the outputs can
reveal information about individual participants.  Aggregators use
the batch size field of the "Params" structure to enforce minimum
batch size during the collect protocol, but server implementations
may also opt out of participating in a PPM task if the minimum batch
size is too small.  This document does not specify how to choose
minimum batch sizes.
Paragraph (version B):
An important parameter of a PPM deployment is the minimum batch size.
If an aggregation includes too few inputs, then the outputs can
reveal information about individual participants.  Aggregators use
the batch size field of the shared task parameters to enforce minimum
batch size during the collect protocol, but server implementations
may also opt out of participating in a PPM task if the minimum batch
size is too small.  This document does not specify how to choose
minimum batch sizes.