jiebi/CodeConvo huggingface dataset:
Code change:
Had offline conversation with Lucas, conclusion was that this is good enough to merge to allow interop - we'll have plenty of time to figure out details in subsequent issues and PRs.

Related discussion:
text: arbitrary values.  These values MUST NOT be assigned by IANA and MUST NOT appear in the listing of assigned values. <del> 10.4. </del> <ins> 9.3. </ins> This document establishes a registry for HTTP datagram context extension type codes.  The "HTTP Context Extension Types" registry
Code change:
Had offline conversation with Lucas, conclusion was that this is good enough to merge to allow interop - we'll have plenty of time to figure out details in subsequent issues and PRs.

Related discussion:
text: assigned by IANA and MUST NOT appear in the listing of assigned values. <del> 10.5. </del> <ins> 9.4. </ins> This document establishes a registry for HTTP context extension type codes.  The "HTTP Context Close Codes" registry governs a 62-bit
Code change:
LGTM
From NAME The very first sentence is mostly unparseable due to its length (7 lines). Please consider splitting it.
I added this, from NAME review: This section and the next one should be a list with 2 bullets rather than 2 sections to make the text more readable.

Related discussion:
text: 5.6. <del> In addition to measurements media players use to guide their segment- by-segment adaptive streaming requests, streaming media providers may also rely on measurements collected from media players to provide analytics that can be used for decisions such as whether the adaptive encoding bitrates in use are the best ones to provide to media players, or whether current media content caching is providing the best experience for viewers.  To that effect, the Consumer Technology Association (CTA) who owns the Web Application Video Ecosystem (WAVE) project has published two important specifications. 5.6.1. </del> <ins> Media players use measurements to guide their segment-by-segment adaptive streaming requests, but may also provide measurements to streaming media providers. In turn, providers may base analytics on these measurements, to guide decisions such as whether adaptive encoding bitrates in use are the best ones to provide to media players, or whether current media content caching is providing the best experience for viewers. To that effect, the Consumer Technology Association (CTA) who owns the Web Application Video Ecosystem (WAVE) project has published two important specifications. CTA-2066: Streaming Quality of Experience Events, Properties and Metrics </ins> CTA-2066 specifies a set of media player events, properties, quality of experience (QoE) metrics and associated terminology for
Code change:
LGTM
From NAME The very first sentence is mostly unparseable due to its length (7 lines). Please consider splitting it.
I added this, from NAME review: This section and the next one should be a list with 2 bullets rather than 2 sections to make the text more readable.

Related discussion:
text: defining a common terminology as well as how each metric should be computed for consistent reporting. <del> 5.6.2. </del> <ins> CTA-5004: Common Media Client Data (CMCD) </ins> Many assume that the CDNs have a holistic view into the health and performance of the streaming clients.  However, this is not the case.
Code change:
I actually used the URL in the draft to clone a copy, so I think this is good to go.

Related discussion:
text: 4.1. <del> (To the editor: check this repository URL after the draft is adopted. The working group may create its own repository) </del> <ins> This document is in the Github repository at: </ins> <del> This document is in the Github repository at https://github.com/GrumpyOldTroll/ietf-mops-drafts.  Readers are welcome to open issues and send pull requests for this document. </del> <ins> https://github.com/ietf-wg-mops/draft-ietf-mops-streaming-opcons Readers are welcome to open issues and send pull requests for this document. </ins> Substantial discussion of this document should take place on the MOPS working group mailing list (mops@ietf.org). <ins> Join: https://www.ietf.org/mailman/listinfo/mops Search: https://mailarchive.ietf.org/arch/browse/mops/ </ins> 4.2. Presentations: IETF 105 BOF: https://www.youtube.com/watch?v=4G3YBVmn9Eo&t=47m21s <del> IETF 106 Mops meeting: https://www.youtube.com/ </del> <ins> IETF 106 meeting: https://www.youtube.com/ </ins> watch?v=4_k340xT2jM&t=7m23s <del> List archive search for doc mentions: https://mailarchive.ietf.org/arch/browse/mops/?q=draft-ietf-mops- streaming-opcons </del> 5. This document requires no actions from IANA.
Code change:
looks good.
From NAME
NAME and NAME - I'm not sure who wrote this text, except I don't think it was me. >And as with other parts of the ecosystem, new technology brings new challenges. For example, with the emergence of ultra-low-latency streaming, responses have to start streaming to the end user while still being transmitted to the cache, and while the cache does not yet know the size of the object. Some of the popular caching systems were designed around cache footprint and had deeply ingrained assumptions about knowing the size of objects that are being stored, so the change in design requirements in long-established systems caused some errors in production. Incidents occurred where a transmission error in the connection from the upstream source to the cache could result in the cache holding a truncated segment and transmitting it to the end user's device. In this case, players rendering the stream often had the video freeze until the player was reset. In some cases the truncated object was even cached that way and served later to other players as well, causing continued stalls at the same spot in the video for all players playing the segment delivered from that cache node. NAME was curious if there's a reasonable reference that we could provide for this.
Yeah, that was me. There was not a public reference for this, alas, just a word of mouth anecdote.
NAME - I agree with you about your comment on Section 3.6. It happens that I got comments that Section 3.6 and 3.7 didn't need to be nearly as detailed as they were, so they were heavily rewritten. The text where "throttled" was used is now in Section 3.6.1, and the emphasis is now on the cause of unexpected traffic profile changes, rather than the ISP response to the unexpected traffic profile changes in this specific incident, so the text that included "throttled" has been removed.
NAME - I agree with your comment on Section 4.4, that some description or reference should be provided for the word "manifest". The next occurrence of "manifest" is in Section 5.2, which does provide a description. I don't think that's quite right - I'd say So I'm changing this definition, and moving it to the first occurrence of "manifest" in Section 4.4. Thanks for helping us fix problems you didn't even tell us about. :upsidedownface:
lgtm

Related discussion:
text: Media servers can provide media streams at various bitrates because the media has been encoded at various bitrates.  This is a so-called "ladder" of bitrates that can be offered to media players as part of <del> the manifest that describes the media being requested by the media player, so that the media player can select among the available </del> <ins> the manifest, so that the media player can select among the available </ins> bitrate choices. The media server may also choose to alter which bitrates are made
Code change:
All protocol request messages contain a field, which allows aggregators to service multiple tasks from a single set of endpoints. The exception is : since it is an HTTP GET request, there is no body, and thus no place for the client to indicate the desired to the aggregator. We don't want to introduce a request body to , because then it would have to be a POST instead of a GET, and that would make it harder for implementations to cache HPKE config values. So instead we have clients encode the task ID in RFC 4648 Base16 (a.k.a. hexadecimal) and provide it as a query parameter. See issue
My goal here is to enable our aggregator implementations in the interop target to use one HPKE config per task. The alternative to this would be for us to use aggregator endpoints that somehow incorporate the task ID, as I described , which is already permitted by the spec as written.
The Base64 alphabet includes , and , which means you have to URL encode (a.k.a. percent encode) Base64 strings before you can use them in a URL, which is a chore and yields less readable URLs (not that 64 character hex strings is especially user-friendly). An alternative would be to use RFC 4648's , omitting the padding characters as suggested in since task IDs are of a known length.
Gotcha. I think the main thing is to be consistent. I would either: Change this PR to use base64url; or Tack on a commit that makes all encodings of the taskID (except those that appear in the body, of course) the same (i.e., hexadecimal). I think (2.) would be my preference.
I agree with the consistency argument. I'm leaning towards using everywhere because it's more compact than Base16/hex, and also I think implementations are less likely to be surprised by case requirements in base64url (i.e., it's obvious that base64 is case sensitive, but you have to read RFC 4648 attentively to notice that it mandates uppercase hex strings).
Base64Url sounds good to me. I took a quick look and it appears that most standard base64 libraries also support the URL encoding.
I will clean this up and merge it once I have implemented it in Janus and convinced myself it's sound.
SGTM, but why not base64 encode the taskID for consistency with URL

Related discussion:
text: ppm-urn-space).  Clients SHOULD display the "detail" field of all errors.  The "instance" value MUST be the endpoint to which the request was targeted.  The problem document MUST also include a <del> "taskid" member which contains the associated PPM task ID, encoded with base64 using the standard alphabet RFC4648 (this value is always known, see task-configuration). </del> <ins> "taskid" member which contains the associated PPM task ID, encoded in Base 64 using the URL and filename safe alphabet with no padding as defined in sections 5 and 3.2 of RFC4648 (this value is always known, see task-configuration). </ins> In the remainder of this document, we use the tokens in the table above to refer to error types, rather than the full URNs.  For
Code change:
All protocol request messages contain a field, which allows aggregators to service multiple tasks from a single set of endpoints. The exception is : since it is an HTTP GET request, there is no body, and thus no place for the client to indicate the desired to the aggregator. We don't want to introduce a request body to , because then it would have to be a POST instead of a GET, and that would make it harder for implementations to cache HPKE config values. So instead we have clients encode the task ID in RFC 4648 Base16 (a.k.a. hexadecimal) and provide it as a query parameter. See issue
My goal here is to enable our aggregator implementations in the interop target to use one HPKE config per task. The alternative to this would be for us to use aggregator endpoints that somehow incorporate the task ID, as I described , which is already permitted by the spec as written.
The Base64 alphabet includes , and , which means you have to URL encode (a.k.a. percent encode) Base64 strings before you can use them in a URL, which is a chore and yields less readable URLs (not that 64 character hex strings is especially user-friendly). An alternative would be to use RFC 4648's , omitting the padding characters as suggested in since task IDs are of a known length.
Gotcha. I think the main thing is to be consistent. I would either: Change this PR to use base64url; or Tack on a commit that makes all encodings of the taskID (except those that appear in the body, of course) the same (i.e., hexadecimal). I think (2.) would be my preference.
I agree with the consistency argument. I'm leaning towards using everywhere because it's more compact than Base16/hex, and also I think implementations are less likely to be surprised by case requirements in base64url (i.e., it's obvious that base64 is case sensitive, but you have to read RFC 4648 attentively to notice that it mandates uppercase hex strings).
Base64Url sounds good to me. I took a quick look and it appears that most standard base64 libraries also support the URL encoding.
I will clean this up and merge it once I have implemented it in Janus and convinced myself it's sound.
SGTM, but why not base64 encode the taskID for consistency with URL

Related discussion:
text: MUST be in the same order.  The leader's endpoint MUST be the first in the list.  The order of the "encrypted_input_shares" in a "Report" (see uploading-reports) MUST be the same as the order in <del> which aggregators appear in this list. </del> <ins> which aggregators appear in this list.  Aggregators MAY use the same endpoint for multiple tasks. </ins> "max_batch_lifetime": The maximum number of times a batch of reports may be used in collect requests.
Code change:
Justify text after bullets consistenly Make sure all lines break at 80 characters
LGTM, modulo conflicts. It'd be nice if we had a linter that blocked PRs until their formatting was right, since changes like this taint git history.

Related discussion:
text: Initialize VDAF preparation and initial outputs as described in input-share-prep. <del> [[OPEN ISSUE: consider moving the helper nonce check into #early- </del> <ins> [[OPEN ISSUE: consider moving the helper nonce check into early- </ins> input-share-validation]] Once the helper has processed each valid report share in
Code change:
As of draft-irtf-cfrg-vdaf-05 (not yet published), the random coins used for sharding are an explicit parameter of . Add this parameter and explasin how it is generated.
NAME I believe this is the only API change we need to account for, correct?

Related discussion:
text: The last input comprises the randomness consumed by the sharding algorithm.  The sharding randomness is a random byte string of length specified by the VDAF.  The Client MUST generate this using a <del> cryptographicallly secure random number generator. </del> <ins> cryptographically secure random number generator. </ins> The Client then wraps each input share in the following structure:
Code change:
All references to "metadata" are now to "report_metadata". References to "CollectReq" are now "CollectionReq". References to "CollectResp" are now "Collection".
The current version of the draft renames field "metadata" to "report_metadata" in the Report structure, but does not do the same renaming in the ReportShare and InputShareAad structures. I think this naming should be consistent, though I don't have a preference between the alternatives.
Thanks for pointing this out! Can you send prepare a PR?
Fixed in
Thanks a bunch NAME

Related discussion:
text: Each report share has a corresponding task ID, report metadata (report ID and, timestamp), the public share sent to each Aggregator, and the recipient's encrypted input share.  Let "task_id", <del> "metadata", "public_share", and "encrypted_input_share" denote these values, respectively.  Given these values, an aggregator decrypts the input share as follows.  First, it constructs an "InputShareAad" message from "task_id", "metadata", and "public_share".  Let this be denoted by "input_share_aad".  Then, the aggregator looks up the HPKE config and corresponding secret key indicated by "encrypted_input_share.config_id" and attempts decryption of the payload with the following procedure: </del> <ins> "report_metadata", "public_share", and "encrypted_input_share" denote these values, respectively.  Given these values, an aggregator decrypts the input share as follows.  First, it constructs an "InputShareAad" message from "task_id", "report_metadata", and "public_share".  Let this be denoted by "input_share_aad".  Then, the aggregator looks up the HPKE config and corresponding secret key indicated by "encrypted_input_share.config_id" and attempts decryption of the payload with the following procedure: </ins> where "sk" is the HPKE secret key, and "server_role" is the role of the aggregator ("0x02" for the leader and "0x03" for the helper).
Code change:
All references to "metadata" are now to "report_metadata". References to "CollectReq" are now "CollectionReq". References to "CollectResp" are now "Collection".
The current version of the draft renames field "metadata" to "report_metadata" in the Report structure, but does not do the same renaming in the ReportShare and InputShareAad structures. I think this naming should be consistent, though I don't have a preference between the alternatives.
Thanks for pointing this out! Can you send prepare a PR?
Fixed in
Thanks a bunch NAME

Related discussion:
text: Implementation note: The Leader considers a batch to be collected once it has completed a collection job for a <del> CollectReq message from the Collector; the Helper considers a batch to be collected once it has responded to an </del> <ins> CollectionReq message from the Collector; the Helper considers a batch to be collected once it has responded to an </ins> AggregateShareReq message from the Leader.  A batches is determined by query (query) conveyed in these messages. Queries must satisfy the criteria covered in batch-validation.
Code change:
All references to "metadata" are now to "report_metadata". References to "CollectReq" are now "CollectionReq". References to "CollectResp" are now "Collection".
The current version of the draft renames field "metadata" to "report_metadata" in the Report structure, but does not do the same renaming in the ReportShare and InputShareAad structures. I think this naming should be consistent, though I don't have a preference between the alternatives.
Thanks for pointing this out! Can you send prepare a PR?
Fixed in
Thanks a bunch NAME

Related discussion:
text: the hash values with a bitwise-XOR operation. Then, for each Aggregator endpoint "{aggregator}" in the parameters <del> associated with "CollectReq.task_id" (see collect-flow) except its </del> <ins> associated with "CollectionReq.task_id" (see collect-flow) except its </ins> own, the Leader sends a POST request to "{aggregator}/tasks/{task- id}/aggregate_shares" with the following message:
Code change:
All references to "metadata" are now to "report_metadata". References to "CollectReq" are now "CollectionReq". References to "CollectResp" are now "Collection".
The current version of the draft renames field "metadata" to "report_metadata" in the Report structure, but does not do the same renaming in the ReportShare and InputShareAad structures. I think this naming should be consistent, though I don't have a preference between the alternatives.
Thanks for pointing this out! Can you send prepare a PR?
Fixed in
Thanks a bunch NAME

Related discussion:
text: 4.5.6. <del> Before an Aggregator responds to a CollectReq or AggregateShareReq, it must first check that the request does not violate the parameters associated with the DAP task.  It does so as described here. </del> <ins> Before an Aggregator responds to a CollectionReq or AggregateShareReq, it must first check that the request does not violate the parameters associated with the DAP task.  It does so as described here. </ins> First the Aggregator checks that the batch respects any "boundaries" determined by the query type.  These are described in the subsections
Code change:
All references to "metadata" are now to "report_metadata". References to "CollectReq" are now "CollectionReq". References to "CollectResp" are now "Collection".
The current version of the draft renames field "metadata" to "report_metadata" in the Report structure, but does not do the same renaming in the ReportShare and InputShareAad structures. I think this naming should be consistent, though I don't have a preference between the alternatives.
Thanks for pointing this out! Can you send prepare a PR?
Fixed in
Thanks a bunch NAME

Related discussion:
text: batch IDs.  Thus the Aggregator needs to check that the query is associated with a known batch ID: <del> For a CollectReq containing a query of type "by_batch_id", the </del> <ins> For a CollectionReq containing a query of type "by_batch_id", the </ins> Leader checks that the provided batch ID corresponds to a batch ID <del> it returned in a previous CollectResp for the task. </del> <ins> it returned in a previous Collection for the task. </ins> For an AggregateShareReq, the Helper checks that the batch ID provided by the Leader corresponds to a batch ID used in a
Code change:
All references to "metadata" are now to "report_metadata". References to "CollectReq" are now "CollectionReq". References to "CollectResp" are now "Collection".
The current version of the draft renames field "metadata" to "report_metadata" in the Report structure, but does not do the same renaming in the ReportShare and InputShareAad structures. I think this naming should be consistent, though I don't have a preference between the alternatives.
Thanks for pointing this out! Can you send prepare a PR?
Fixed in
Thanks a bunch NAME

Related discussion:
text: AggregateShare collect-flow: "application/dap-aggregate-share" <del> CollectReq collect-flow: "application/dap-collect-req" </del> <ins> CollectionReq collect-flow: "application/dap-collect-req" </ins> Collection collect-flow: "application/dap-collection"
Code change:
I added a clarification that the nonce is random. IMO the question of who randomly generates the nonce goes with the question of how participants agree upon PDAParams, which is to say it's outside the protocol.
On the 2021/7/7 design call we decided that the ID should be derived deterministically from the serialized PAParam by, for example, hashing it. This solves a few problems simultaneously. This settles the question of whether there should be 1:1 correspondence between a task ID and the parameters used for that task. NAME brought up an interesting question, which is how to make the task ID unique when two different data collection tasks use the same parameters. Thoughts?
The consensus here is to do SHA-256 over the PDAParams structure and use that as the 32 byte PDATaskID. However, hashing a protocol message requires us to decide on a canonical representation of a protocol message, which I think forces us to decide how to encode messages on the wire. I filed for that question. I'll proceed with a PR for this issue on the assumption that we'll pick JSON.
As I mentioned there, the assumption would be that we're adopting TLS' binary encoding. If so, in TLS notation, you would write as the hash of , which itself is a serialized data structure.
All we require is that each PAParam have some unique value, right? A random nonce would probably suffice. r.e. the encoding, yes, let's stick with TLS's wire format as NAME suggests.
Yes, my thinking was that we would just stick 16 bytes of random data in (a UUID's worth). I agree that we should stick with TLS binary encoding.
Approved, with the following suggestion: Let's leave a [TODO: ...] comment under the definition of for clarifying how it should be chosen.

Related discussion:
text: structure, which specify the protocol used to verify and aggregate the clients' measurements: <ins> "nonce": A unique sequence of bytes used to ensure that two otherwise identical "PDAParam" instances will have distinct "PDATaskID"s.  It is RECOMMENDED that this be set to a random 16-byte string derived from a cryptographically secure pseurandom number generator. </ins> "leader_url": The leader's endpoint URL. "helper_url": The helper's endpoint URL.