jiebi/CodeConvo huggingface dataset:
watch for increasing numbers of end users uploading significant amounts of content. <ins> 2.6. The causes of unpredictable usage described in sec-unpredictable were more or less the result of human choices, but we were reminded during a post-IETF 107 meeting that humans are not always in control, and forces of nature can cause enormous fluctuations in traffic patterns. In his talk, Sanjay Mishra Mishra reported that after the CoViD-19 pandemic broke out in early 2020, Comcast's streaming and web video consumption rose by 38%, with their reported peak traffic up 32% overall between March 1 to March 30 Comcast, AT&T reported a 28% jump in core network traffic (single day in April, as compared to pre stay-at-home daily average traffic), with video accounting for nearly half of all mobile network traffic, while social networking and web browsing remained the highest percentage (almost a quarter each) of overall mobility traffic ATT, and Verizon reported similar trends with video traffic up 36% over an average day (pre COVID-19) Verizon. We note that other operators saw similar spikes during this time period.  Craig Labowitz Labovitz reported Weekday peak traffic increases over 45%-50% from pre-lockdown levels, A 30% increase in upstream traffic over their pre-pandemic levels, and A steady increase in the overall volume of DDoS traffic, with amounts exceeding the pre-pandemic levels by 40%. (He attributed this increase to the significant rise in gaming-related DDoS attacks (LabovitzDDoS), as gaming usage also increased.) </ins> 3. 3.1.
compound annual growth rate continuing at 34% (from Appendix D of CVNI). <ins> A substantial part of this growth is due to increased use of streaming video, although the amount of video traffic in real-time communications (for example, online videoconferencing) has also grown significantly.  While both streaming video and videoconferencing have real-time delivery and latency requirements, these requirements vary from one application to another.  For example, videoconferencing demands an end-to-end (one-way) latency of a few hundreds of milliseconds whereas live streaming can tolerate latencies of several seconds. This document specifically focuses on the streaming applications and defines streaming as follows: Streaming is transmission of a continuous media from a server to a client and its simultaneous consumption by the client.  Here, continous media refers to media and associated streams such as video, audio, metadata, etc.  In this definition, the critical term is "simultaneous", as it is not considered streaming if one downloads a video file and plays it after the download is completed, which would be called download-and-play. This has two implications.  First, server's transmission rate must (loosely or tightly) match to client's consumption rate for an uninterrupted playback.  That is, the client must not run out of data (buffer underrun) or take more than it can keep (buffer overrun) as any excess media is simply discarded.  Second, client's consumption rate is limited by not only bandwidth availability but also the real- time constraints.  That is, the client cannot fetch media that is not available yet. </ins> In many contexts, video traffic can be handled transparently as generic application-level traffic.  However, as the volume of video traffic continues to grow, it's becoming increasingly important to
further.  In this case, the immersive content is typically referred to as volumetric media.  One way to represent the volumetric media is to use point clouds, where streaming a single object may easily <del> require a bitrate of 30 Mbps or higher.  Refer to PCC for more details. </del> <ins> require a bitrate of 30 Mbps or higher.  Refer to MPEGI and PCC for more details. </ins> 2.2.
3.1. Adaptive BitRate (ABR) is a sort of application-level response <del> strategy in which the receiving media player attempts to detect the available bandwidth of the network path by experiment or by observing the successful application-layer download speed, then chooses a video bitrate (among the limited number of available options) that fits within that bandwidth, typically adjusting as changes in available bandwidth occur in the network or changes in capabilities occur in the player (such as available memory, CPU, display size, etc.). </del> <ins> strategy in which the streaming client attempts to detect the available bandwidth of the network path by observing the successful application-layer download speed, then chooses a bitrate for each of the video, audio, subtitles and metadata (among the limited number of available options) that fits within that bandwidth, typically adjusting as changes in available bandwidth occur in the network or changes in capabilities occur during the playback (such as available memory, CPU, display size, etc.). </ins> The choice of bitrate occurs within the context of optimizing for <del> some metric monitored by the video player, such as highest achievable video quality, or lowest rate of expected rebuffering events. </del> <ins> some metric monitored by the client, such as highest achievable video quality or lowest chances for a rebuffering (playback stall). </ins> 3.2. <del> ABR playback is commonly implemented by video players using HLS </del> <ins> ABR playback is commonly implemented by streaming clients using HLS </ins> RFC8216 or DASH DASH to perform a reliable segmented delivery of <del> video data over HTTP.  Different player implementations and receiving devices use different strategies, often proprietary algorithms (called rate adaptation or bitrate selection algorithms), to perform available bandwidth estimation/prediction and the bitrate selection. Most players only use passive observations, i.e., they do not generate probe traffic to measure the available bandwidth. </del> <ins> media over HTTP.  Different implementations use different strategies ABRSurvey, often proprietary algorithms (called rate adaptation or bitrate selection algorithms) to perform available bandwidth estimation/prediction and the bitrate selection.  Most clients only use passive observations, i.e., they do not generate probe traffic to measure the available bandwidth. </ins> This kind of bandwidth-measurement systems can experience trouble in several ways that can be affected by networking design choices.
9. <del> Thanks to Mark Nottingham, Glenn Deen, Dave Oran, Aaron Falk, Kyle Rose, Leslie Daigle, Lucas Pardue, Matt Stock, Alexandre Gouaillard, and Mike English for their very helpful reviews and comments. </del> <ins> Thanks to Alexandre Gouaillard, Aaron Falk, Dave Oran, Glenn Deen, Kyle Rose, Leslie Daigle, Lucas Pardue, Mark Nottingham, Matt Stock, Mike English, Roni Even, and Will Law for very helpful suggestions, reviews and comments. (If we missed your name, please let us know!) </ins> 10.  References
errors.  The "instance" value MUST be the endpoint to which the request was targeted.  The problem document MUST also include a "taskid" member which contains the associated PPM task ID, encoded <del> with base64 RFC4648 (this value is always known, see task- configuration). </del> <ins> with base64 using the standard alphabet RFC4648 (this value is always known, see task-configuration). </ins> In the remainder of this document, we use the tokens in the table above to refer to error types, rather than the full URNs.  For
This protocol is extensible and allows for the addition of new cryptographic schemes that implement the VDAF interface specified in <del> I-D.draft-cfrg-patton-vdaf.  Candidates include: </del> <ins> VDAF.  Candidates include: </ins> "prio3", which allows for aggregate statistics such as sum, mean, histograms, etc.  This class of VDAFs is based on Prio CGB17 and
client ever sees the values for individual clients. In order to address this problem, the aggregators engage in a secure, <del> multi-party computation specified by the chosen VDAF I-D.draft-cfrg- patton-vdaf in order to prepare a report for aggregation.  At the beginning of this computation, each aggregator is in possession of an input share uploaded by the client.  At the end of the computation, each aggregator is in posession of either an "output share" that is ready to be aggregated or an indication that a valid output share could not be computed. </del> <ins> multi-party computation specified by the chosen VDAF VDAF in order to prepare a report for aggregation.  At the beginning of this computation, each aggregator is in possession of an input share uploaded by the client.  At the end of the computation, each aggregator is in posession of either an "output share" that is ready to be aggregated or an indication that a valid output share could not be computed. </ins> To facilitiate this computation, the input shares generated by the client include information used by the aggregators during aggregation
"vdaf_verify_param": The aggregator's VDAF verification parameter output by the setup algorithm computed jointly by the aggregators <del> before the start of the PPM protocol I-D.draft-cfrg-patton-vdaf). [OPEN ISSUE: This is yet to be specified.  See issue#161.] </del> <ins> before the start of the PPM protocol VDAF).  [OPEN ISSUE: This is yet to be specified.  See issue#161.] </ins> Finally, the collector is configured with the HPKE secret key corresponding to "collector_hpke_config".
To generate the report, the client begins by sharding its measurement into a sequence of input shares as specified by the VDAF in use.  To <del> encrypt an input share, the client first generates an HPKE RFC9180 context for the aggregator by running </del> <ins> encrypt an input share, the client generates an HPKE RFC9180 ciphertext and encapsulated key for the aggregator by running </ins> <del> where "pk" is the aggregator's public key and "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for the helper).  In general, the info string for computing the HPKE context is suffixed by two bytes, the first of which identifies the role of the sender and the second of which identifies the role of the intended recipient. "enc" is the HPKE encapsulated key and "context" is the HPKE context used by the client for encryption.  The payload is encrypted as where "input_share" is the aggregator's input share and "nonce" and "extensions" are the corresponding fields of "Report".  Clients MUST NOT use the same "enc" for multiple reports. </del> <ins> where "pk" is the aggregator's public key; "server_role" is the Role of the intended recipient ("0x02" for the leader and "0x03" for the helper); "task_id", "nonce", and "extensions" are the corresponding fields of "Report"; and "input_share" is the aggregator's input share. </ins> The leader responds to well-formed requests to "[leader]/upload" with HTTP status code 200 OK and an empty body.  Malformed requests are
report share as invalid with the error "hpke-unknown-config-id". Otherwise, it decrypts the payload with the following procedure: <del> where "sk" is the HPKE secret key, "task_id" is the task ID, "nonce" and "extensions" are the nonce and extensions of the report share respectively, and "server_role" is 0x02 if the aggregator is the leader and 0x03 otherwise.  If decryption fails, the aggregator marks the report share as invalid with the error "hpke-decrypt-error". Otherwise, it outputs the resulting "input_share". </del> <ins> where "sk" is the HPKE secret key, and "server_role" is the role of the aggregator ("0x02" for the leader and "0x03" for the helper).  If decryption fails, the aggregator marks the report share as invalid with the error "hpke-decrypt-error".  Otherwise, it outputs the resulting "input_share". </ins> 4.3.1.4.
"AggregateShareReq" is done as follows: where "pk" is the HPKE public key encoded by the collector's HPKE <del> key, and server_role is is "0x02" for the leader and "0x03" for a helper. </del> <ins> key, "server_role" is "0x02" for the leader and "0x03" for a helper. </ins> The collector decrypts these aggregate shares using the opposite process.  Specifically, given an encrypted input share, denoted "enc_share", for a given batch interval, denoted "batch_interval", decryption works as follows: <del> where "sk" is the HPKE secret key, "task_id" is the task ID for a given collect request, and "server_role" is the role of the server that sent the aggregate share ("0x02" for the leader and "0x03" for the helper). </del> <ins> where "sk" is the HPKE secret key, "task_id" is the task ID for the collect request, and "server_role" is the role of the server that sent the aggregate share ("0x02" for the leader and "0x03" for the helper). </ins> 4.4.5.
"batch_interval" is the batch interval of the request. <ins> "agg_param", an aggregation parameter for the VDAF being executed. </ins> "report_count" is the number of reports included in the aggregation. "checksum" is the checksum computed over the set of client reports. <del> "job_id" is the ID of the aggregation job. </del> This request MUST be authenticated as described in https-sender-auth. To handle the leader's request, the helper first ensures that the request meets the requirements for batch parameters following the
where "[leader]" is the first entry in the task's aggregator endpoints.  The payload is structured as follows: <del> This message is called the Client's report.  It consists of a "header" and the encrypted input share of each of the Aggregators. The header consists of the task ID and report "metadata".  The metadata consists of the following fields: </del> <ins> This message is called the Client's report.  It consists of the report metaata, the "public share" output by the VDAF's input- distribution algorithm, and the encrypted input share of each of the Aggregators.  The header consists of the task ID and report "metadata".  The metadata consists of the following fields: </ins> A timestamp representing the time at which the report was generated.  Specifically, the "time" field is set to the number of
4.4.1.3. Each report share has a corresponding task ID, report metadata <del> (timestamp, nonce, and extensions), and encrypted input share.  Let "task_id", "metadata", and "encrypted_input_share" denote these values, respectively.  Given these values, an aggregator decrypts the input share as follows.  First, the aggregator looks up the HPKE config and corresponding secret key indicated by </del> <ins> (timestamp, nonce, and extensions), the public share sent to each Aggregator, and the recipient's encrypted input share.  Let "task_id", "metadata", "public_share", and "encrypted_input_share" denote these values, respectively.  Given these values, an aggregator decrypts the input share as follows.  First, the aggregator looks up the HPKE config and corresponding secret key indicated by </ins> "encrypted_input_share.config_id".  If not found, then it marks the report share as invalid with the error "hpke-unknown-config-id". Otherwise, it decrypts the payload with the following procedure:
"vdaf_verify_key" is the VDAF verification key shared by the aggregators; "agg_id" is the aggregator ID ("0x00" for the Leader and <del> "0x01" for the helper); and "agg_param" is the opaque aggregation parameter distributed to the aggregtors by the collector. </del> <ins> "0x01" for the helper); "agg_param" is the opaque aggregation parameter distributed to the aggregtors by the collector; "public_share" is the public share generated by the client and distributed to each aggregator; and "input_share" is the aggregator's input share. </ins> If either step fails, the aggregator marks the report as invalid with error "vdaf-prep-error".  Otherwise, the value "out" is interpreted
aggregate shares into an aggregate result using the VDAF's "agg_shares_to_result" algorithm.  In particular, let "agg_shares" denote the ordered sequence of aggregator shares, ordered by <del> aggregator index, and let "agg_param" be the opaque aggregation parameter.  The final aggregate result is computed as follows: </del> <ins> aggregator index, let "report_count" denote the report count sent by the Leader, and let "agg_param" be the opaque aggregation parameter. The final aggregate result is computed as follows: </ins> 4.5.4.
participating clients or permit some attacks on robustness.  This auxiliary information could be removed by having clients submit reports to an anonymizing proxy server which would then use Oblivious <del> HTTP I-D.thomson-http-oblivious to forward inputs to the DAP leader, without requiring any server participating in DAP to be aware of whatever client authentication or attestation scheme is in use. </del> <ins> HTTP I-D.draft-ietf-ohai-ohttp-07 to forward inputs to the DAP leader, without requiring any server participating in DAP to be aware of whatever client authentication or attestation scheme is in use. </ins> 7.4.
*Collect:* The collector makes one or more requests to the leader in order to obtain the final output of the protocol.  Before the output can be computed, the aggregators (i.e, the leader and <del> helpers) need to have verified and aggregated a sufficient number </del> <ins> helper) need to have verified and aggregated a sufficient number </ins> of inputs.  Depending on the PA protocol, it may be possible for the aggregators to do so immediately when reports are uploaded. (See prio.)  However, in general it is necessary for them to wait until (a) enough reports have been uploaded and (b) the collector has made a request.  (See hits.) <del> [TODO: Say that the protocol involves multiple helpers.  The downside of using secret sharing is that the protocol requires at least two servers to be online during the entire data aggregation process.  To ameliorate this problem, we run the protocol in parallel with multiple helpers.] </del> 3. This section specifies a protocol for executing generic PA tasks.
3.3.2. <del> For each URL "[helper]" in "PAUploadStartResp.helper_urls", the client sends a GET request to "[helper]/key_config".  The helper responds with status 200 and an "HpkeConfig" message.  Next, the client collects the set of helpers it will upload shares to.  It ignores a helper if: </del> <ins> Let "[helper]" denote the helper URL encoded by "PAUploadStartResp.param.helper_url".  When a client sends a GET request to "[helper]/key_config", the helper responds with status 200 and an "HpkeConfig" message.  The client aborts if any of the following happen: </ins> the client and helper failed to establish a secure, helper- authenticated channel;
the key config specifies a KEM, KDF, or AEAD algorithm the client doesn't recognize. <del> If the set of supported helpers is empty, then the client aborts and alerts the leader with "no supported helpers".  Otherwise, for each supported helper the client issues a POST request to "[leader]/upload_finish" with a payload constructed as described below. </del> [OPEN ISSUE: Should the request URL encode the PA task?  This would be necessary if we make "upload_start" an idempotent GET per issue#48.]
that about a client?  I'm not sure, so I'd be inclined to remove this unless we have a concrete use case.] <del> The client begins by setting up an HPKE I-D.irtf-cfrg-hpke context for the helper by running </del> <ins> Next, the client issues a POST request to "[leader]/upload_finish". It begins by setting up an HPKE I-D.irtf-cfrg-hpke context for the helper by running </ins> where "pk" is the KEM public key encoded by "HpkeConfig.public_key". The outputs are the helper's encapsulated context "enc" and the
SetupBaseS() will be, as well as the aad for context.Seal().  The aad might be the entire "transcript" between the client and helper.] <del> [OPEN ISSUE: Is it safe to generate the proof once, then secret-share between each (leader, helper) pair?  Probably not in general, but maybe for Prio?] </del> [OPEN ISSUE: allow server to send joint randomness in UploadStartResp, and then enforce uniqueness via double-spend state or something else (see issue#48).]
We sometimes refer to this message as the _report_. The message contains the "task" fields of the previous request.  In addition, it includes the time (in seconds since the beginning of UNIX time) at <del> which the report was generated, the helper's HPKE config id, endpoint URL, and the helper and leader shares.  The helper share has the following structure: </del> <ins> which the report was generated, the helper's HPKE config id, and the helper and leader shares.  The helper share has the following structure: </ins> Field "enc" encodes the helper's encapsulated HPKE context.  The remainder of the structure contains the share itself, the structure
3.4.2.1. The process begins with a PACollectReq.  The leader collects a <del> sequence of reports that are all associated with the same PA task, helper URL, and helper HPKE config id.  Let "[helper]" denote "PACollectReq.helper_url".  The leader sends a POST request to "[helper]/aggregate" with the following message: </del> <ins> sequence of reports that are all associated with the same PA task. Let "[helper]" denote "PAParam.helper_url", where "PAParam" is the PA parameters structure associated "PAAggregateReq.task.id".  The leader sends a POST request to "[helper]/aggregate" with the following message: </ins> The structure contains the PA task, the helper's HPKE config id, an opaque _helper state_ string, and a sequence of _sub-requests_, each
The term "Remote Attestation" is a common expression and often associated or connoted with certain properties.  The term "Remote" in this context does not necessarily refer to a remote entity in the <del> scope of network topologies or the Internet.  It rather refers to a decoupled system or entities that exchange the payload of the </del> <ins> scope of network topologies or the Internet.  It rather refers to decoupled systems or entities that exchange the payload of the </ins> Conceptual Message type called Evidence I-D.ietf-rats-architecture. This conveyance can also be "Local", if the Verifier role is part of the same entity as the Attester role, e.g., separate system <del> components of the same Composite Device (a single RATS entity). Examples of these types of co-located environments include: a Trusted Execution Environment (TEE), Baseboard Management Controllers (BMCs), as well as other physical or logical protected/isolated/shielded Computing Environments (e.g. embedded Secure Elements (eSE) or Trusted Platform Modules (TPM)).  Readers of this document should be familiar with the concept of Layered Attestation as described in Section 4.3 Two Types of Environments of an Attester in I-D.ietf- rats-architecture and the definition of Attestation as described in I-D.ietf-rats-tpm-based-network-device-attest. </del> <ins> components of the same Composite Device (a single RATS entity).  Even if an entity takes on two or more different roles, the functions they provide typically reside in isolated environments that are components of the same entity.  Examples of such isolated environments include: a Trusted Execution Environment (TEE), Baseboard Management Controllers (BMCs), as well as other physical or logical protected/isolated/shielded Computing Environments (e.g. embedded Secure Elements (eSE) or Trusted Platform Modules (TPM)).  Readers of this document should be familiar with the concept of Layered Attestation as described in Section 4.3 Two Types of Environments of an Attester in I-D.ietf-rats-architecture and the definition of Attestation as described in I-D.ietf-rats-tpm-based-network-device- attest. </ins> 3.