Draft: draft-ietf-ppm-dap

Commentary: Having to track all report IDs is a burden for sharding anti - replay storage across time . This change clarifies that , while tracking report IDs is sufficient , it is also sufficient to track any report metadata that contains the reprot ID . 
 cc/ 
 LGTM 
 Squashed 
 Anti - replay is enforced in . Both DAP-01 and DAP-02 read : What 's not fully specified , at least in this section , is what it means for a report to " have been aggregated ( with this aggregation parameter ) " . For that we have been referring to the anti - replay section . reads : The report ID is defined as On the other hand , reads : The nonce is defined as Thus in DAP-01 we are required to check for collisions of the timestamp+random value , whereas in DAP-02 we are required to check for collisions of just a random value ( report ID ) . The latter is stricter , since it disallows repeated random values . After discussing some implementation stuff with I realized that this may have been an unintended regression . From a security standpoint , I think the DAP-02 requirement is better , since it forces the client implementation to generate a fresh report ID each time it generates a report . Otherwise it is easier to fingerprint reports from the same client . Does this violate assumptions in anyone 's implementation ? Where I can imagine this being problematic is if your implementation currently shards nonce sets into time windows . does this , but we have to amend this behavior to implement fixed - size queries anyway . Reply to this thread if this causes an issue for you and you think we should reconsider this requirement . 
 From Janus ' perspective , I think we are OK with this change . Janus also breaks report sets into time windows -- I do n't think this change will affect Janus ' implementation of that functionality , though as you note this sharding behavior will change with tasks . 
 I 'm nearly done with the implemenation ... once the code is reviewed and landed I 'll try to remember to write up how I ended up organizing data . 
 Seems fine to me . I ca n't think of a reason why the client would want to reuse a reportID with a different time for the same task . 
 I am in favor of recommending that clients generate a good random i d for every report AND for allowing the timestamp to be considered as part of the anti - replay strategy . While an unbounded memory of seen report ids would indeed be excellent protection , in practice this requires unbounded storage of report ids , which is a practical problem for implementations . Replay of a client report can leak information , so we want to assure that a report is aggregated at most once . A report has a report i d and a timestamp , both generated by the reporting client . Both are included in the authenticated additional data , and can not be altered without the tampering being evident . For the rest of this note , we will assume that the client is doing its part for its own privacy by choosing a new random report i d for every report , and by having a clock is not hugely inaccurate . Even if the clients are doing their part , we must still be concerned with the actions of other entities , such as a compromised leader , helper , or collector . We also need to prevent third party manipulation , e.g. by replaying captured reports . The use of the timestamp can help us here , because the timestamp can not be altered by any of these parties without detection . If at least one aggregator is honest and imposes the policy below , then we can use the timestamp to ease our storage burdens . Let ’s consider the timestamp in more detail . For reporting accuracy we do n’t want to permit just any timestamp to be aggregated . At any given time T by the server ’s clock , there are bounds we want to apply based on the timestamp . Reports earlier than some time MinTime = T - MinDelta should be rejected as too old , and reports farther in the future than MaxTime = T + MaxDelta should be rejected for being so confused about the time . We allow reports slightly in the future as we want to allow for clock - skew . Call the closed interval [ MinTime , MaxTime ] for time T Allowed(T ) . We will discard reports outside of this interval when we receive them , and also when we process them . We impose the “ on reception ” policy to avoid queuing work that we ’d never do , and we impose on processing to prevent backlog and to allow the definition of LeastRetainedTime ( below ) . Assume a monotonically non - decreasing system clock . Whenever an aggregation begins , we durably record the current MinTime and associate it with the active state for the aggregation job , using appropriate locking in concurrent implementations . Define the LeastRetainedTime as the minimum of the current MinTime and the recorded MinTime over all active aggregation jobs ( if any ) . It is impossible to aggregate any report before this time , as it will be rejected in any new job and is n’t present in any existing job . Garbage collection may safely remove any recorded report metadata with a timestamp before the LeastRetainedTime . We now only have to remember report ids that are greater than or equal to the LeastRetainedTime . If clients choose random ids with every report submitted , and there is at least one honest aggregator , then any given report can not be replayed . Note that in cases where a server is so flooded with reports that it can not process or remember them , then the DAP spec still has the " report_dropped " error available to ensure that reports are not aggregated when anti - replay can not be assured . 
 Thanks Full disclosure :  is helping out with Daphne development . When he showed me this analysis , I was pretty convinced that relaxing this requirement would be safe and worth it to reduce burden on implementations . Although I 'm not strongly opposed to keeping the stronger requirement if folks think it is useful . I think the only change that would need to be made is to : 
 I also agree with  's analysis . I think we should make the change described by cjpatton in DAP-03 . 
 This argument is good . One point : while it certainly justifies including the timestamp in the overall anti - replay strategy , it does not necessarily justify checking rather than only for the ID - based check . That is , the overall strategy could be : the timestamp ; if it 's out of the acceptable window , fail due to too old / too new . if [ or ] is in the set of received reports ; if it is , fail due to replayed - report . I would have a preference for doing the ID - based check on only , since the report ID is the primary identifier of the report , but plausibly implementations may want to check timestamps as well ( for example , if they bucket report IDs by timestamp ) . Perhaps we could phrase things such that either implementation is allowed ? In practice , as long as the report IDs are properly generated as random values , the timestamp will be irrelevant to the ID - based check . If the leader / helper disagree on whether to check or , the more restrictive check will be the one that is effectively in place . edit : another way of looking at this argument -- for the ID - based replay check , what justification is there for checking over , say , or ? 
 I think if we assume that honest clients always choose their report ID at random , then any anti - replay check that includes the report ID ought to be sufficient . I agree we could write this in a way that makes this implementation - specific perhaps suggesting that (  , timestamp ) is one particularly useful relaxation .
Paragraph (version A):
A report ID used by the Aggregators to ensure the report appears
in at most one batch.  (See anti-replay.)  The Client MUST
generate this by generating 16 random bytes using a
cryptographically secure random number generator.
Paragraph (version B):
A report ID used by the Aggregators to ensure the report appears
in at most one batch.  (See input-share-validation.)  The Client
MUST generate this by generating 16 random bytes using a
cryptographically secure random number generator.
Paragraph (version A):
The Leader MUST ignore any report pertaining to a batch that has
already been collected.  (See early-input-share-validation for
details.)  Otherwise, comparing the aggregate result to the previous
aggregate result may result in a privacy violation.  (Note that the
Helpers enforce this as well.)  The Leader MAY ignore any reports
whose timestamp is past the task's "task_expiration".  When it does
so, the leader SHOULD abort the upload protocol and alert the client
with error "reportTooLate".  Client MAY choose to opt out of the task
if its own clock has passed "task_expiration".
Paragraph (version B):
The Leader MUST ignore any report pertaining to a batch that has
already been collected.  (See input-share-validation for details.)
Otherwise, comparing the aggregate result to the previous aggregate
result may result in a privacy violation.  (Note that the Helpers
enforce this as well.)  The Leader MAY ignore any reports whose
timestamp is past the task's "task_expiration".  When it does so, the
leader SHOULD abort the upload protocol and alert the client with
error "reportTooLate".  Client MAY choose to opt out of the task if
its own clock has passed "task_expiration".
Paragraph (version B):
Implementation note: To detect replay attacks, each Aggregator
is required to keep track of the set of reports it has
processed for a given task.  Because honest Clients choose the
report ID at random, it is sufficient to store the set of IDs
of processed reports.  However, implementations may find it
helpful to track additional information, like the timestamp, so
that the storage used for anti-replay can be sharded
efficiently.
Paragraph (version A):
Using a client-provided report multiple times within a single batch,
or using the same report in multiple batches, may allow a server to
learn information about the client's measurement, violating the
privacy goal of DAP.  To prevent such replay attacks, this
specification requires the aggregators to detect and filter out
replayed reports.
Paragraph (version A):
To detect replay attacks, each aggregator keeps track of the set of
report IDs pertaining to reports that were previously aggregated for
a given task.  If the leader receives a report from a client whose
report ID is in this set, it either ignores it or aborts the upload
sub-protocol as described in upload-flow.  A Helper who receives an
encrypted input share whose report ID is in this set rejects the
report as described in early-input-share-validation.
Paragraph (version A):
Aggregators prevent the same report from being used in multiple
batches (except as required by the protocol) by only responding to
valid collect requests, as described in batch-validation.
Paragraph (version A):
Implement some form of batch-to-report index, as well as inter-
and intra-batch replay mitigation storage, which includes some way
of tracking batch report size.  Some of this state may be used for
replay attack mitigation.  The replay mitigation strategy is
described in anti-replay.
Paragraph (version B):
Implement some form of batch-to-report index, as well as inter-
and intra-batch replay mitigation storage, which includes some way
of tracking batch report size.  Some of this state may be used for
replay attack mitigation.  The replay mitigation strategy is
described in input-share-validation.
Paragraph (version A):
Implement and store state for the form of inter- and intra-batch
replay mitigation in anti-replay.
Paragraph (version B):
Implement and store state for the form of inter- and intra-batch
replay mitigation in input-share-validation.
Paragraph (version A):
However, Aggregators are also required to implement several per-
report checks that require retaining a number of data artifacts.  For
example, to detect replay attacks, it is necessary for each
Aggregator to retain the set of report IDs of reports that have been
aggregated for the task so far.  Depending on the task lifetime and
report upload rate, this can result in high storage costs.  To
alleviate this burden, DAP allows Aggregators to drop this state as
needed, so long as reports are dropped properly as described in
early-input-share-validation.  Aggregators SHOULD take steps to
mitigate the risk of dropping reports (e.g., by evicting the oldest
data first).
Paragraph (version B):
However, Aggregators are also required to implement several per-
report checks that require retaining a number of data artifacts.  For
example, to detect replay attacks, it is necessary for each
Aggregator to retain the set of report IDs of reports that have been
aggregated for the task so far.  Depending on the task lifetime and
report upload rate, this can result in high storage costs.  To
alleviate this burden, DAP allows Aggregators to drop this state as
needed, so long as reports are dropped properly as described in
input-share-validation.  Aggregators SHOULD take steps to mitigate
the risk of dropping reports (e.g., by evicting the oldest data
first).