Change:
Make the handling of collect requests on the leader asynchronous, since servicing a collect request could entail running the preparation protocol over all the reports that fall into the batch interval, especially in VDAFs like poplar1 where the aggregation parameter is not available until the collector provides it. I've only made the collect request handling asynchronous. My reasoning is that handling other requests synchronously should be feasible: upload requests should only contain shares of a single report and thus never be all that big. Additionally, the client needs an explicit acknowledgement from the aggregator that the uploaded report was accepted and has been persisted. the requests in the aggregation/preparation sub-protocol should also never get that big: even when preparing a large number of reports, the leader can choose to carve that work up into any number of aggregation jobs, each of which covers a small enough number of reports that the message sizes and processing times should be reasonable to handle in synchronous request/response exchanges.
With this change, we will have two differing definitions of structs named . My understanding is that clients will know which one to expect based on the URL of the request, and the status code in the response. (they will receive a URL from /collect on success, and the URL will return a URL with status code 202, or shares with status code 200) I think we should change one of these for clarity to the reader, i.e. or .
It's easy to imagine that a request will take a long time, especially if you are using Collect and that has to trigger a bunch of work from the leader. We need some way of handling this.
This intersects with something we were discussing last week, which is whether or not should have in it, or a URL from which collector config can be fetched dynamically. We decided on the former to spare collectors of running an online HTTP server. However, if we adopt an asynchronous model for collect requests, then we might need collectors to be an HTTP server anyway so that the leader can notify them that a collect request is complete, in which case we should make collectors have a endpoint like the other servers.
NAME and I were talking about this today. We agree that / definitely needs to be made asynchronous since in the case, servicing a will definitely incur lots of expensive work between the aggregators (this is less likely in the case because the aggregators aren't blocked on obtaining the aggregation parameter from the collector in that VDAF). What I'm not sure if is whether we should make all the interactions between the aggregators during preparation and verification asynchronous, and I'm not sure how to decide. It seems like it should be possible for leaders to scope each aggregate requests to a small enough number of shares that each one can reasonably be serviced synchronously.
For collect requests, I'd also note here that asynchronicity is probably going to be important for , as it's possible that the aggregation flow will fall behind the upload flow. (Imagine the helper can only handle 1000 reports/sec, but the clients are uploading 1million reports/sec.) As for aggregate requests, I would expect these to be able to be completed fairly quickly (in a matter of seconds, in the worst case). Based on our experience so far, the most expensive part appears to be interacting with long-term storage, which is required for anti-replay and, if we take , looking up report shares. I think we can hold off on making these requests asynchronous until we've gotten enough deployment experience to determine if this is going to be a problem.
NAME provided some interesting prior art that we should rely on when addressing this: URL
Nicely done. Hi Tim, Maybe this might help you: URL (URIs, not UUIDs) You might also find this informative: URL You are probably familiar with how ACME does asynchronous processing, which is largely OK. This probably comes down to preference at some level. However, asynchronous interactions are a fair bit more complex than synchronous ones, so avoiding that complexity is good. Keep in mind that modern HTTP versions aren't stalled if there are a handful of long-running requests, so unless you are talking a fairly significant time, it might be cleaner and easier to just have the client wait on hold. Asynchronous requests do come with some benefits to go with the complexity. For instance, robustness can be improved if the initial request creates resources that can be referred to later on a different connection without having to rely on the connection remaining viable. RFC 7240 does suggest that you might offer both, but I would only go that far if you thought that implementations would consistently use both options.

Discussion:
"[leader]/collect", where "[leader]" is the leader's endpoint URL. The body of the request is structured as follows: <ins> [OPEN ISSUE: Decide if and how the collector's request is authenticated.  If not, then we need to ensure that collect job URIs are resistant to enumeration attacks.] ~~~ struct { TaskID task_id; Interval batch_interval; opaque agg_param<0..2^16-1>; // VDAF aggregation parameter } CollectReq; ~~~ </ins> The named parameters are: "task_id", the PPM task ID.

"agg_param", an aggregation parameter for the VDAF being executed. Depending on the VDAF scheme and how the leader is configured, the <del> collect request may cause the leader to send a series of aggregate requests to the helpers in order to compute their share of the aggregate.  Alternately, the leader may already have made these requests and can respond immediately.  In either case it responds to the collector's request as follows. It begins by checking that the request meets the requirements of the batch parameters using the procedure in batch-parameter-validation. If so, it obtains the helper's encrypted aggregate share for the batch interval by sending an AggregateShareReq to the helper as described in aggregate-share-request.  (This request may too have been made in advance.) Next, the leader computes its own aggregate share by aggregating all of the verified output shares that fall within the batch interval. Finally, it responds with HTTP status 200 and a body consisting of a CollectResp message: [OPEN ISSUE: What happens if this all takes a really long time.] [TODO: Decide if and how the collector's request is authenticated.] </del> <ins> leader and helper may already have prepared all the reports falling within "batch_interval" and be ready to return the aggregate shares right away, but this cannot be guaranteed.  In fact, for some VDAFs, it is not be possible to begin preparing inputs until the collector provides the aggregation parameter in the "CollectReq".  For these reasons, collect requests are handled asynchronously. Upon receipt of a "CollectReq", the leader begins by checking that the request meets the requirements of the batch parameters using the procedure in batch-parameter-validation.  If so, it immediately sends the collector a response with HTTP status 303 See Other and a Location header containing a URI identifying the collect job that can be polled by the collector, called the "collect job URI". The leader then begins working with the helper to prepare the shares falling into "CollectReq.batch_interval" (or continues this process, depending on the VDAF) as described in pa-aggregate. After receiving the response to its CollectReq, the collector makes an HTTP GET request to the collect job URI to check on the status of the collect job and eventually obtain the result.  If the collect job is not finished yet, the leader responds with HTTP status 202 Accepted.  The response MAY include a Retry-After header field to suggest a pulling interval to the collector. Once all the necessary reports have been prepared, the leader obtains the helper's encrypted aggregate share for the batch interval by sending an "AggregateShareReq" to the helper as described in aggregate-share-request.  The leader then computes its own aggregate share by aggregating all of the prepared output shares that fall within the batch interval. When both aggregators' shares are successfully obtained, the leader responds to subsequent HTTP GET requests to the collect job's URI with HTTP status 200 OK and a body consisting of a "CollectResult": </ins> "shares" is a vector of "EncryptedAggregateShare"s, as described in aggregate-share-request, except that for the leader's share, the "task_id" and "batch_interval" used to encrypt the "AggregateShare" are obtained from the "CollectReq". <ins> If obtaining aggregate shares fails, then the leader responds to subsequent HTTP GET requests to the collect job URI with an HTTP error status and a problem document as described in errors. The leader MUST retain a collect job's results until the collector sends an HTTP DELETE request to the collect job URI, in which case the leader responds with HTTP status 204 No Content. [OPEN ISSUE: Allow the leader to drop aggregate shares after some reasonable amount of time has passed, but it's not clear how to specify that.  ACME doesn't bother to say anything at all about this when describing how subscribers should fetch certificates: https://datatracker.ietf.org/doc/html/rfc8555#section-7.4.2] </ins> [OPEN ISSUE: Describe how intra-protocol errors yield collect errors (see issue#57).  For example, how does a leader respond to a collect request if the helper drops out?]

CollectReq pa-collect: "message/ppm-collect-req" <del> CollectResp pa-collect: "message/ppm-collect-req" </del> <ins> CollectResult pa-collect: "message/ppm-collect-result" </ins> The definition for each media type is in the following subsections.