Change:
Currently parsing AggreateInitializeReq, AggregateshareReq, CollectReq, and CollectResp requires the query type associated with the task. The query type is determined by the task, but it is necessary to parse the task ID from the message before the remainder can be parsed. This change ensures that parsing of these messages does not require the query type.
The query type is determined by the task config, which in turn is determined by the task ID. The task ID is encoded by the message, so it is necessary to parse the task ID before the Aggregator knows how to parse it. PR URL introduced the following problem: If an Aggregator implements multiple query types ("time-interval" and "fixed-size", say), then a message's format depends on the query type for the task. This means it is necessary to parse the task ID before the rest of the message. This is perfectly implementable, but it has two downsides: This likely breaks abstraction boundaries in current implementations and will require refactoring. (For example, . Notice that the message is parsed in one step.) Going forward, we'll need to make sure that the task ID always appears first in any DAP message so that it can be easily parsed. This is already true today, but we might want to add some text to Operational Considerations that spells this requirement out so that we minimize the chance of regression. Here's my question for folks: Do we want to try and do something about this? I can think of a couple of solutions: Add the query type byte to messages whose format depends on this. One downside is that it creates a new error case for the Aggregator to check, namely, that the query type indicated in the message matches the query type of the task. (In fact, we tried to avoid this in .) Move the task ID out of the message payload and into the URL query string, or perhaps the HTTP header. The only downside to this is that it complicates request authentication for deployments that want to sign the payload and bind it to the task config. (This may be desirable from a security perspective.) One possible upshot is that it might put us in a better position to address . (NAME please weigh in here!)
I noticed this issue while working on the implementation of "fixed-size". I think my preference would be to do nothing here, i.e., shift the burden to implementations and maybe make a note in Operational Considerations.
I thought about this too, but I was waiting to finish Janus' DAP-02 implementation to see how impactful it was. (implementation is still ongoing, I'll update this bug w/ my thoughts once we're closer to done) If we do decide to do something, I think I'd prefer solution . We've had a few issues now of the form "correctly parsing this message requires knowing the task ID, which requires parsing part of the message." This is technically feasible (as you note), but is fairly awkward from an implementation standpoint (IMO). Moving the task ID out from the message sidesteps this problem, and IMO fits well with . I think solution would allow implementations to have simpler parsing logic, but I'm not sure this upside is worth the cost of an additional byte on the wire & an additional error case. Though I suppose this new error case exists either way -- but without a discriminator byte, the error case would most likely present itself as a message parsing failure. (e.g. "message has too many/too few bytes")
After conferring with NAME NAME et al offline, we discovered another consideration here which makes me lean towards making protocol messages self-describing (solution (1.)). We're working on an . At a high-level, an Aggregator that supports this extension uses information provided by the client (in a Report extension) to derive the task ID. This information includes, among other things, the query type to use for the task. The problem is that decoding of the depends on the query type, which, when this extension is in-use, is not known until parsing the report extensions: If we take solution (1.), then is encoded by , thus resolving this bug. There are other ways to resolve this. I can think of a few possibilities if there were an extension field in . However there are lots of things to consider for this sort of change, and I would prefer we wait until later. Does anyone object to taking solution (1.)? I will have a PR ready for this by the end of the week. I would like to land this in DAP-02 if at all possible.
PR is up, would appreciate review.
I think that the right solution in the long run is Chris' , namely putting the task ID, and anything else needed to parse the request body, into the HTTP request path. But I'm OK with taking for now in the interest of making implementable. Sure, we'll have to take the query type byte out of these messages later, but there are quite a few messages that currently have a or member that will get moved up into the request path in draft-03 () I think there's plenty of prior art on including things like HTTP headers, path or query params in request signatures. See for instance , which explains how to canonicalize an HTTP request into a string that can be "signed" (well, it's an HMAC, but AWS insists that it's a signature).
Option (2.) doesn't quite solve my extension problem (URL), but I have no objection to moving the task ID (and maybe agg job ID) out of the message in any case. Seems to me like a way cleaner design that would likely solve other problems.
LGTM, one typo needs fixingThis accomplishes what it sets out to, modulo some typos.

Discussion:
Information used to bind the aggregate result to the query.  For fixed-size tasks, this includes the batch ID assigned to the batch <del> by the Leader.  The indicatd query type MUST match the task's </del> <ins> by the Leader.  The indicated query type MUST match the task's </ins> query type. [OPEN ISSUE: What should the Collector do if the query type