Draft: draft-pda-protocol

Commentary: This PR restructures the document to have a more traditional IETF flow , including an overview of operation and then serially going through each operation . It also moves the crypto detail to an appendix in preparation for moving it to a separate CFRG document . 
 As noted above , I think this is an unnecessary limitation in the current text . There is no a priori reason why it needs to be so for either protocol . Even for  , one might imagine having the leader preconfigured with a specific heavy hitters measurement strategy . I actually think there are three things going on : sending in reports . leader issuing aggregation requests collector asking for results . That 's how this text is structure . Sure , but this is just one instance of where the collector may have to wait . For instance , the computation may just take some time . 
 I some clarifying text . Maybe we 're now at the point where this is probably close enough and we should merge and then fix other things later .  ? 
 This seems to be a dupe fo -- can we close ? 
 Per discussion in the call today , I think this issue is about updating the various PDU definitions to allow for multiple helpers . For instance , currently has fields and , but it should instead have . is the more complex problem of extending the protocol to handle 1 helper , which probably involves more rounds of communication . I 'm going to grab this issue because I ran into which I think is related , so I 'll fix both . 
 Protocol messages are defined in the context of the PPM document , so the " PPM " seems redundant . I think we should just call things as they are , i.e. , instead of . 
 A batch of reports can be processed incrementally in a set of sub - batches . Each sub - batch may mean multiple RTs between leader and helper . Right now , this is hidden in the Prio or HH - specific pieces , but it probably should be manifest above . 
 Yes , it seems to me that needs a boolean in it so that helper can indicate to leader whether more rounds are needed . 
 I believe this has been resolved with PrepareStepResult ( indicating " continued " or " finished " ) .
Paragraph (version A):
This document describes a framework for specifying protocols for
privacy-preserving data-aggregation.  Each protocol is executed by a
large set of clients and a small set of servers.  The servers' goal
is to compute some aggregate statistic over the clients' inputs
without learning the inputs themselves.  This is made possible by
distributing the computation among the servers in such a way that, as
long as at least one of them executes the protocol honestly, no input
is ever seen in the clear by any server.
Paragraph (version B):
This document describes a protocol for privacy preserving
measurement.  The protocol is executed by a large set of clients and
a small set of servers.  The servers' goal is to compute some
aggregate statistic over the clients' inputs without learning the
inputs themselves.  This is made possible by distributing the
computation among the servers in such a way that, as long as at least
one of them executes the protocol honestly, no input is ever seen in
the clear by any server.
Paragraph (version A):
This section defines some terminology we will use in the remainder of
this document.
Paragraph (version B):
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in
BCP 14 RFC2119 RFC8174 when, and only when, they appear in all
capitals, as shown here.
Paragraph (version A):
The protocol is executed by a large set of clients and a small set of
servers.  We call the servers the _aggregators_. Each client's input
to the protocol is a set of measurements (e.g., counts of some user
behavior).  Given the input set of measurements "x_1, ..., x_n" held
by "n" users, the goal of a _private data aggregation (PPM) protocol_
is to compute "y = F(x_1, ..., x_n)" for some aggregation function
"F" while revealing nothing else about the measurements.
Paragraph (version A):
It's impossible to deduce the measurement without knowing _all_ of
the shares.
Paragraph (version A):
A common PPM task that can't be solved efficiently with Prio is the
"t"-_heavy-hitters_ problem BBCp21.  In this setting, each user is in
possession of a single "n"-bit string, and the goal is to compute the
compute the set of strings that occur at least "t" times.  One reason
that Prio doesn't apply to this problem is that the proof generated
by the client would be huge.
Paragraph (version A):
An essential task of any data collection pipeline is ensuring that
the input data is "valid".  Going back to the example above, it's
often useful to assert that each measurement is in a certain range,
e.g., "[0, 2^k)" for some "k".  This straight-forward task is
complicated in our setting by the fact that the inputs are secret
shared.  In particular, a malicious client can corrupt the
computation by submitting random integers instead of a proper secret
sharing of a valid input.
Paragraph (version A):
To solve this problem, in each PPM protocol, the client generates a
zero-knowledge proof of its input's validity that the aggregators use
to verify that their shares correspond to as valid input.  The
verification procedure is designed to ensure that the aggregators
learn nothing about the input beyond its validity.
Paragraph (version A):
After encoding its measurements as an input to the PPM protocol, the
client generates a _proof_ of the input's validity.  It then splits
the proof into shares and sends a share of both the proof and input
to each aggregator.  The aggregators use their shares of the proof to
decide if their input shares correspond to a valid input.
Paragraph (version A):
As noted above, each client has a collection of measurements that it
wants to send.  Each measurement is characterized by a set of
parameters that are centrally configured and provided to each client
(see pa-config).
Paragraph (version A):
Once the client has collected the measurements to send, it needs to
turn them into a set of reports.  Naively, each measurement would be
sent in its own report, but it is also possible to have multiple
measurements in a single report; clients need to be configured with
the mapping from measurements to reports.  The set of measurements
that go into a report is referred to as the "input" to the report.
Because each report is independent, for the remainder of this
document we focus on a single report and its inputs.
Paragraph (version A):
Each PPM task consists of two sub-protocols, _upload_ and _collect_,
which are executed concurrently.  Each sub-protocol consists of a
sequence of HTTP requests made from one entity to another.
Paragraph (version A):
The operational capabilities of each entity are described further in
entity-capabilities.
Paragraph (version B):
This document uses the protocol definition language of RFC8446.
Paragraph (version B):
is to compute "y = F(x_1, ..., x_n)" for some aggregation function
"F" while revealing nothing else about the measurements.
Paragraph (version B):
Prio CB17, which allows for aggregate statistics such as sum,
mean, histograms, etc. over a single value.
Paragraph (version B):
Heavy Hitters BBCp21, which allows for finding the most common
strings among a collection of clients (e.g., the URL of their home
page) as well as counting the number of clients that hold a given
string.
Paragraph (version B):
It allows the aggregators to compute the final output by first
aggregating up their measurements shares locally, then combining
the results to obtain the final output.
Paragraph (version B):
The overall system architecture is shown in pa-topology.
Paragraph (version B):
The PPM scheme to use.  This is to some extent dictated by the
previous choices.
Paragraph (version B):
The minimum "batch size" of reports which can be aggregated.
Paragraph (version B):
The rate at which measurements can be taken, i.e., the "minimum
batch window".
Paragraph (version B):
These parameters are distributed out of band to the clients and to
the aggregators.  Each task is identified by a unique 32-byte ID
which is used to refer to it in protocol messages.
Paragraph (version B):
During the duration of the measurement, each client records its own
value(s), packages them up into a report, and sends them to the
leader.  Each share is separately encrypted for each aggregator so
that even though they pass through the leader, the leader is unable
to see or modify them.  Depending on the measurement, the client may
only send one report or may send many reports over time.
Paragraph (version B):
The leader distributes the shares to the helpers and orchestrates the
process of verifying them (see validating-inputs) and assembling them
into a final measurement for the collector.  Depending on the PPM
scheme, it may be possible to incrementally process each report as it
comes in, or may be necesary to wait until the entire batch of
reports is received.
Paragraph (version B):
An essential task of any data collection pipeline is ensuring that
the input data is "valid".  In PPM, input validation is complicated
by the fact that none of the entities other than the client ever sees
the values for individual clients.
Paragraph (version B):
In order to address this problem, each PPM client generates a zero-
knowledge proof that its report is valid and attaches it to the
report.  The aggregators can then jointly verify this proof prior to
incorporating the report in the aggregation and reject the report if
it cannot be verified.  However, they do not learn anything about the
individual report other than that it is valid.
Paragraph (version B):
The specific properties attested to in the proof vary depending on
the measurement being taken.  For instance, if we want to measure the
time the user took performing a given task the proof might
demonstrate that the value reported was within a certain range (e.g.,
0-60 seconds).  By contrast, if we wanted to report which of a set of
N options the user select, the report might contain N integers and
the proof would demonstrate that N-1 were 0 and the other was 1.
Paragraph (version B):
It is important to recogize that "validity" is distinct from
"correctness".  For instance, the user might have spent 30s on a task
but the client might report 60s.  This is a problem with any
measurement system and PPM does not attempt to address it; it merely
ensures that the data is within acceptable limits, so the client
could not report 10^6s or -20s.
Paragraph (version B):
Communications between PPM entities are carried over HTTPS RFC2818.
HTTPS provides server authentication and confidentiality.  In
addition, report shares are encrypted directly to the aggregators
using HPKE I-D.irtf-cfrg-hpke.
Paragraph (version B):
where "task_id" is the associated PPM task (this value is always
known, see task-configuration) and "payload" contains a string
indicating the error that occurred.  When sent by an aggregator in
response to an HTTP request, the response status is 400.  When sent
in a request to an aggregator, the URL is always
"[aggregator]/error", where "[aggregator]" is the URL of the
aggregator endpoint.
Paragraph (version B):
The following list defines a set of common errors.
Paragraph (version B):
This document uses the verbs "abort" and "alert with "[some error
message]"" to describe how protocol participants react to various
error conditions.
Paragraph (version A):
Where "SHA-256" is as specified in FIPS180-4.
Paragraph (version A):
We call this a _key configuration_. The key configuration is used to
set up a base-mode HPKE context to use to derive symmetric keys for
protecting: (1) input shares sent from the client to an aggregator;
or (2) output shares sent from the helper to the collector.  The
_config id_, "HpkeConfig.id", is forwarded by the sender to the
receiver to help the receiver decide if it knows the decryption key.
Paragraph (version A):
Before the client can upload its report to the leader, it must first
discover the key configs of each of the aggregators.  To do so, the
client sends a GET request to "[aggregator]/key_config", where
"[aggregator]" is the aggregator's endpoint URL.  The aggregator
responds to well-formed requests with status 200 and an "HpkeConfig".
Paragraph (version B):
Where param is just the serialization of PPMParam.
Paragraph (version B):
Clients periodically upload reports to the leader, which then
distributes the individual shares to each helper.
Paragraph (version B):
Clients SHOULD follow the usual HTTP caching RFC7234 semantics for
key configurations.  Aggregators SHOULD favor long cache lifetimes to
avoid frequent cache revalidation.
Paragraph (version A):
The leader responds to well-formed requests to "[leader]/upload" with
status 200 and an empty body.  Malformed requests are handled as
described in pa-error-common-aborts.  Clients SHOULD NOT upload the
same measurement value in more than one report if the leader responds
with status 200 and an empty body.
Paragraph (version B):
The leader responds to well-formed requests to "[leader]/upload" with
status 200 and an empty body.  Malformed requests are handled as
described in errors.  Clients SHOULD NOT upload the same measurement
value in more than one report if the leader responds with status 200
and an empty body.
Paragraph (version A):
The collector interacts with the leader to produce the final
aggregate output.  This process consists of a sequence of _collect
requests_ issued to the leader.  Before a request can succeed, the
aggregators must have verified and aggregated enough reports and the
leader must have obtained the helper's encrypted output share (see
pa-aggregate).  In general, the procedure by which the aggregators
verify and aggregate reports depends on parameters carried by the
collect request.
Paragraph (version A):
The batch window of the request aligns with the size of the batch
window for the task, i.e., "batch_start" and "batch_end" are
multiples of "PPMParam.batch_window".
Paragraph (version A):
The batch window of the request is at least the minimum batch
window for the task, i.e., "batch_end - batch_start >=
PPMParam.batch_window".
Paragraph (version A):
After the client uploads a report to the leader, the leader and
helper verify in zero knowledge that the input is valid.  The exact
procedure for doing so is protocol specific, but all protocols have
the same basic structure.  In particular, the protocol is comprised
of a sequence of _aggregate requests_ from the leader to the helper.
At the end of this procedure, the leader and helper will have
aggregated a set of valid client inputs (though not necessarily a
complete batch).
Paragraph (version B):
Once a set of clients have uploaded their reports to the leader, the
leader can send them to the aggregators to be verified and
aggregated.  In order to enable the system to handle very large
batches of reports, this process can be performed incrementally.  To
aggregate a set of reports, the leader sends an AggregateReq to the
helper containing those report shares.  The helper then processes
them (verifying the proofs and incorporating their values into the
ongoing aggregate) and replies to the leader.
Paragraph (version B):
Some PPM schemes (e.g., Prio) allow the leader to start
aggregating reports proactively before all the reports in a batch
are received.  Others (e.g., Hits) require all the reports to be
present and must be initiated by the collector.
Paragraph (version B):
Processing the reports - especially verifying the proofs - may
require multiple round trips.
Paragraph (version B):
Note that it is possible to aggregate reports from one batch while
reports from the next batch are coming in.
Paragraph (version B):
This process is illustrated below in pa-aggregate-flow.  In this
example, the batch size is 20, but the leader opts to process the
reports in sub-batches of 10.  Each sub-batch takes two round-trips
to process.  Once both sub-batches have been processed, the leader
can issue an OutputShareReq in order to retrieve the helper's
aggregated result.
Paragraph (version B):
In order to allow the helpers to retain minimal state, the helper can
attach a state parameter to its response, with the leader returning
the state value in the next request, thus offloading the state to the
leader.  This state value MUST be cryptographically protected as
described in helper-state.
Paragraph (version A):
Once the aggregators have verified at least as many reports as
required for the PPM task, the leader issues an _output share
request_ to the helper.  The helper responds to this request by
extracting its output share from its state and encrypting it under
the collector's HPKE public key.
Paragraph (version B):
to the helper.  The helper responds to this request by extracting its
output share from its state and encrypting it under the collector's
HPKE public key.
Paragraph (version A):
where "pk" is the HPKE public key encoded by the collector's HPKE key
configuration and "output_share" is its serialized output share.
Paragraph (version A):
where "task" is the associated PPM task (this value is always known)
and "payload" is the message.  When sent by an aggregator in response
to an HTTP request, the response status is 400.  When sent in a
request to an aggregator, the URL is always "[aggregator]/error",
where "[aggregator]" is the URL of the aggregator endpoint.
Paragraph (version A):
The following specify the "boiler-plate" behavior for various error
conditions.
Paragraph (version B):
where "pk" is the HPKE public key encoded by the collector's HPKE key
configuration and "output_share" is its serialized output share.
This encryption prevents the leader from learning the actual result,
as it only has its own share and not the helper's share, which is
encrypted for the collector.
Paragraph (version A):
Prio is _robust_ against malicious clients, and _private_ against
malicious servers, but cannot provide robustness against malicious
servers.  Any aggregator can simply emit bogus output shares and
undetectably spoil aggregates.  If enough aggregators were available,
this could be mitigated by running the protocol multiple times with
distinct subsets of aggregators chosen so that no aggregator appears
in all subsets and checking all the outputs against each other.  If
all the protocol runs do not agree, then participants know that at
least one aggregator is defective, and it may be possible to identify
the defector (i.e., if a majority of runs agree, and a single
aggregator appears in every run that disagrees).  See #22 [2] for
discussion.
Paragraph (version B):
against malicious servers, but cannot provide robustness against
malicious servers.  Any aggregator can simply emit bogus output
shares and undetectably spoil aggregates.  If enough aggregators were
available, this could be mitigated by running the protocol multiple
times with distinct subsets of aggregators chosen so that no
aggregator appears in all subsets and checking all the outputs
against each other.  If all the protocol runs do not agree, then
participants know that at least one aggregator is defective, and it
may be possible to identify the defector (i.e., if a majority of runs
agree, and a single aggregator appears in every run that disagrees).
See #22 [2] for discussion.
Paragraph (version A):
Aggregate requests are ordered as follows: We say that a report "R2"
_follows_ report "R1" if either "R2.time > R1.time" or "R2.time ==
R1.time" and "R2.jitter > R1.jitter".  If "R2.time < R1.time", or
"R2.time == R1.time" but "R2.jitter <= R1.jitter", then we say that
"R2" _does not follow_ "R1".
Paragraph (version B):
"R1".
Paragraph (version B):
Each PPM task consists of two sub-protocols,
Paragraph (version B):
problem BBCp21.  In this setting, each user is in possession of a
single "n"-bit string, and the goal is to compute the compute the set
of strings that occur at least "t" times.  One reason that Prio
doesn't apply to this problem is that the proof generated by the
client would be huge.
Paragraph (version B):
How big the field needs to be depends on the type of data being
aggregated and how many users there are.  The field size also
impacts the security level: the longer the validity circuit, the
larger the field needs to be in order to effectively detect
malicious clients.  Typically the soundness error (i.e., the
probability of an invalid input being deemed valid by the
aggregators) will be 2n/(p-n), where n is the size of the input
and p is the prime modulus.
Paragraph (version B):
In order to make Prio practical, it's important that
implementations employ FFT to speed up polynomial operations.  In
particular, the prime modulus p should be chosen so that "(p-1) =
2^b * s" for large "b" and odd "s".  Then "g^s" is a principle,
"2^b"-th root of unity (i.e., "g^(s\*2^b) = 1"), where "g" is the
generator of the multiplicative subgroup.  This fact allows us to
quickly evaluate and interpolate polynomials at "2^a"-th roots of
unity for any "1 <= a <= b".  Note that "b" imposes an upper bound
on the size of proofs, so it should be large enough to accommodate
all foreseeable use cases.  Something like "b >= 20" is probably
good enough.
Paragraph (version B):
We use rejection sampling to map a PRNG seed to a pseudorandom
sequence of field elements (see prio-prng).  In order to minimize
the probability of a simple being rejected, the modulus should be
as close to a power of 2 as possible.