Original: R OBUST A LGORITHMS ON A DAPTIVE I NPUTS FROM B OUNDED A DVERSARIES
Revised: R OBUST A LGORITHMS ON A DAPTIVE I NPUTS FROM B OUNDED A DVERSARIES

Original: A BSTRACT
Revised: A BSTRACT

Original: We study dynamic algorithms robust to adaptive input generated from sources with bounded capabilities, such as sparsity or limited interaction.
Revised: We study dynamic algorithms robust to adaptive input generated from sources with bounded capabilities, such as sparsity or limited interaction.

Original: For example, we consider robust linear algebraic algorithms when the updates to the input are sparse but given by an adversary with access to a query oracle.
Revised: For example, we consider robust linear algebraic algorithms when the updates to the input are sparse but given by an adversary with access to a query oracle.

Original: We also study robust algorithms in the standard centralized setting where an adversary queries an algorithm in an adaptive manner, but the number of interactions between the adversary and the algorithm is bounded.
Revised: We also study robust algorithms in the standard centralized setting where an adversary queries an algorithm in an adaptive manner, but the number of interactions between the adversary and the algorithm is bounded.

Original: We first recall a unified framework of (Hassidim et al., 2020;
Revised: We first recall a unified framework of (Hassidim et al., 2020;

Original: Beimel et al., 2022; Attias et al., 2023) for answering Q adaptive queries that incurs (cid:101) O ( √ Q ) overhead in space, which is roughly a quadratic improvement over the na ¨ ıve implementation and only incurs a logarithmic overhead in query time.
Revised: Beimel et al., 2022; Attias et al., 2023) for answering Q adaptive queries that incurs (cid:101) O ( √ Q ) overhead in space, which is roughly a quadratic improvement over the na ¨ ıve implementation and only incurs a logarithmic overhead in query time.

Original: Although the general framework has diverse applications in machine learning and data science, such as adaptive distance estimation, kernel density estimation, linear regression range queries, point queries and serves as a preliminary benchmark, we demonstrate even better algorithmic improvements for reducing the preprocessing time for adaptive distance estimation and
Revised: Although the general framework has diverse applications in machine learning and data science, such as adaptive distance estimation, kernel density estimation, linear regression range queries, point queries and serves as a preliminary benchmark, we demonstrate even better algorithmic improvements for reducing the preprocessing time for adaptive distance estimation and

Original: ) permitting an unlimited number of adaptive queries for kernel density estimation.
Revised: ) permitting an unlimited number of adaptive queries for kernel density estimation.

Original: Finally, we complement our theoretical results with additional empirical evaluations.
Revised: Finally, we complement our theoretical results with additional empirical evaluations.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Robustness to adaptive inputs or adversarial attacks has recently emerged as an important desirable characteristic for algorithm design.
Revised: Robustness to adaptive inputs or adversarial attacks has recently emerged as an important desirable characteristic for algorithm design.

Original: An adversarial input can be created using knowledge of the model to induce incorrect outputs on widely used models, such as neural networks (Biggio et al., 2013; Szegedy et al., 2014; Goodfellow et al., 2015; Carlini & Wagner, 2017; Madry et al., 2018).
Revised: An adversarial input can be created using knowledge of the model to induce incorrect outputs on widely used models, such as neural networks (Biggio et al., 2013; Szegedy et al., 2014; Goodfellow et al., 2015; Carlini & Wagner, 2017; Madry et al., 2018).

Original: Adversarial attacks against machine learning algorithms in practice have also been documented in applications such as network monitoring (Chandola et al., 2009), strategic classification (Hardt et al., 2016), and autonomous navigation (Papernot et al., 2016;
Revised: Adversarial attacks against machine learning algorithms in practice have also been documented in applications such as network monitoring (Chandola et al., 2009), strategic classification (Hardt et al., 2016), and autonomous navigation (Papernot et al., 2016;

Original: Liu et al., 2017; Papernot et al., 2017).
Revised: Liu et al., 2017; Papernot et al., 2017).

Original: The need for sound theoretical understanding of adversarial robustness is also salient in situations where successive inputs to an algorithm can be possibly correlated; even if the input is not adversarially generated, a user may need to repeatedly interact with a mechanism in a way such that future updates may depend on the outcomes of previous interactions (Mironov et al., 2011; Gilbert et al., 2012;
Revised: The need for sound theoretical understanding of adversarial robustness is also salient in situations where successive inputs to an algorithm can be possibly correlated; even if the input is not adversarially generated, a user may need to repeatedly interact with a mechanism in a way such that future updates may depend on the outcomes of previous interactions (Mironov et al., 2011; Gilbert et al., 2012;

Original: Bogunovic et al., 2017; Naor & Yogev, 2019; Avdiukhin et al., 2019).
Revised: Bogunovic et al., 2017; Naor & Yogev, 2019; Avdiukhin et al., 2019).

Original: Motivated by both practical needs and a lack of theoretical understanding, there has been a recent flurry of theoretical studies of adversarial robustness.
Revised: Motivated by both practical needs and a lack of theoretical understanding, there has been a recent flurry of theoretical studies of adversarial robustness.

Original: The streaming model of computation has especially received significant attention Ben-Eliezer et al.
Revised: The streaming model of computation has especially received significant attention Ben-Eliezer et al.

Original: ; Hassidim et al.
Revised: ; Hassidim et al.

Original: (2020); Woodruff & Zhou (2021);
Revised: (2020); Woodruff & Zhou (2021);

Original: Kaplan et al.
Revised: Kaplan et al.

Original: (2021); Braverman et al.
Revised: (2021); Braverman et al.

Original: ; Chakrabarti et al.
Revised: ; Chakrabarti et al.

Original: ; Ajtai et al.
Revised: ; Ajtai et al.

Original: ; Chakrabarti et al.
Revised: ; Chakrabarti et al.

Original: Ben-Eliezer et al.
Revised: Ben-Eliezer et al.

Original: ; Assadi et al.
Revised: ; Assadi et al.

Original: ; Attias et al.
Revised: ; Attias et al.

Original: ; Dinur et al.
Revised: ; Dinur et al.

Original: ; Woodruff et al.
Revised: ; Woodruff et al.

Original: More recently, there have also been a few initial results for dynamic algorithms on adaptive inputs for graph algorithms (Wajc, 2020;
Revised: More recently, there have also been a few initial results for dynamic algorithms on adaptive inputs for graph algorithms (Wajc, 2020;

Original: Beimel et al., 2021; Bernstein et al., 2022).
Revised: Beimel et al., 2021; Bernstein et al., 2022).

Original: These works explored the capabilities and limits of algorithms for adversaries that were freely able to choose the input based on previous outputs by the algorithm.
Revised: These works explored the capabilities and limits of algorithms for adversaries that were freely able to choose the input based on previous outputs by the algorithm.

Original: However, in many realistic settings, an adversary is limited in its abilities.
Revised: However, in many realistic settings, an adversary is limited in its abilities.

Original: For example, adversarial attacks in machine learning are often permitted to only alter the “true” input by a small amount bounded in norm.
Revised: For example, adversarial attacks in machine learning are often permitted to only alter the “true” input by a small amount bounded in norm.

Original: For the L 0 norm, this restriction means that the adversary can only add a sparse noise to the true input.
Revised: For the L 0 norm, this restriction means that the adversary can only add a sparse noise to the true input.

Original: More generally, it seems reasonable to assume that adversarial input is generated from a source that has bounded computation time or bounded interactions with an honest algorithm.
Revised: More generally, it seems reasonable to assume that adversarial input is generated from a source that has bounded computation time or bounded interactions with an honest algorithm.

Original: 1. O UR C ONTRIBUTIONS
Revised: 1. O UR C ONTRIBUTIONS

Original: In this paper, we study algorithms robust to adaptive/adversarial input generated from sources with bounded capabilities.
Revised: In this paper, we study algorithms robust to adaptive/adversarial input generated from sources with bounded capabilities.

Original: We first study dynamic algorithms for adaptive inputs from a source that is restricted in sparsity.
Revised: We first study dynamic algorithms for adaptive inputs from a source that is restricted in sparsity.

Original: Namely, we consider robust linear algebraic algorithms when the updates to the label can be adversarial but are restricted in sparsity.
Revised: Namely, we consider robust linear algebraic algorithms when the updates to the label can be adversarial but are restricted in sparsity.

Original: We then study robust algorithms in the standard centralized setting, where an adversary queries an algorithm in an adaptive manner, but the number of interactions between the adversary and the algorithm is bounded.
Revised: We then study robust algorithms in the standard centralized setting, where an adversary queries an algorithm in an adaptive manner, but the number of interactions between the adversary and the algorithm is bounded.

Original: We first show that combining novel subroutines for each of these problems in conjunction with a simple but elegant idea of using differential privacy to hide the internal randomness of various subroutines previously used by Hassidim et al.
Revised: We first show that combining novel subroutines for each of these problems in conjunction with a simple but elegant idea of using differential privacy to hide the internal randomness of various subroutines previously used by Hassidim et al.

Original: Beimel et al.
Revised: Beimel et al.

Original: ; Attias et al.
Revised: ; Attias et al.

Original: (2023) suffices to achieve robust algorithms across these different settings.
Revised: (2023) suffices to achieve robust algorithms across these different settings.

Original: Dynamic algorithms on adaptive input for regression.
Revised: Dynamic algorithms on adaptive input for regression.

Original: Motivated by the problem of label shift in machine learning, we consider a dynamic version of least-squares regression, where the labels get updated.
Revised: Motivated by the problem of label shift in machine learning, we consider a dynamic version of least-squares regression, where the labels get updated.

Original: In this model, we are given a fixed design matrix and a target label that receives a sequence of updates.
Revised: In this model, we are given a fixed design matrix and a target label that receives a sequence of updates.

Original: After each one, the algorithm is asked to output an estimate of the optimal least-squares objective.
Revised: After each one, the algorithm is asked to output an estimate of the optimal least-squares objective.

Original: The goal of the algorithm is to maintain the objective value within a multiplicative factor (1 + ε ) to the optimal.
Revised: The goal of the algorithm is to maintain the objective value within a multiplicative factor (1 + ε ) to the optimal.

Original: More specifically, the algorithm is given a fixed design matrix A ∈ R n × d with n ≥ d and an initial response vector (i.e., label) b (1) , which receives updates over time.
Revised: More specifically, the algorithm is given a fixed design matrix A ∈ R n × d with n ≥ d and an initial response vector (i.e., label) b (1) , which receives updates over time.

Original: We are interested in estimating the least-squares objective value F ( A , b ) =
Revised: We are interested in estimating the least-squares objective value F ( A , b ) =

Original: min x ∈ R d ∥
Revised: min x ∈ R d ∥

Original: Ax − b ∥ 22 as the target label b undergoes updates.
Revised: Ax − b ∥ 22 as the target label b undergoes updates.

Original: The updates to b are adaptively chosen by an adversary but can only affect at most K entries of b per step.
Revised: The updates to b are adaptively chosen by an adversary but can only affect at most K entries of b per step.

Original: Formally, on the i -th round:
Revised: Formally, on the i -th round:

Original: (1) The adversary provides an update to K entries of b ( i − 1) , possibly depending on all previous outputs of the algorithm.
Revised: (1) The adversary provides an update to K entries of b ( i − 1) , possibly depending on all previous outputs of the algorithm.

Original: The algorithm updates its data structure and outputs an estimate (cid:98) F i of F i = F (cid:0) A , b ( i ) (cid:1) .
Revised: The algorithm updates its data structure and outputs an estimate (cid:98) F i of F i = F (cid:0) A , b ( i ) (cid:1) .

Original: The adversary observes and records the output (cid:98) F i .
Revised: The adversary observes and records the output (cid:98) F i .

Original: The goal of the adversary is to create a sequence of labels (cid:0) b ( i ) (cid:1) T i =1 that induces the algorithm to output an inaccurate estimate.
Revised: The goal of the adversary is to create a sequence of labels (cid:0) b ( i ) (cid:1) T i =1 that induces the algorithm to output an inaccurate estimate.

Original: To deal with adaptivity, a na ¨ ıve idea is to treat each step as an independent least-squares regression problem.
Revised: To deal with adaptivity, a na ¨ ıve idea is to treat each step as an independent least-squares regression problem.

Original: However, this approach uses a completely new approximation of the objective value for each update, which seems potentially wasteful.
Revised: However, this approach uses a completely new approximation of the objective value for each update, which seems potentially wasteful.

Original: On the other hand, any randomness that is shared by computations over multiple updates can potentially be leveraged by the adversary to induce an incorrect output.
Revised: On the other hand, any randomness that is shared by computations over multiple updates can potentially be leveraged by the adversary to induce an incorrect output.

Original: Our main result is an algorithm that beats the na¨ıve algorithm in this challenging, adaptively adversarial setting.
Revised: Our main result is an algorithm that beats the na¨ıve algorithm in this challenging, adaptively adversarial setting.

Original: We provide a general result with run-time dependence on n, d, K , and the number of nonzero entries in A , nnz( A ) .
Revised: We provide a general result with run-time dependence on n, d, K , and the number of nonzero entries in A , nnz( A ) .

Original: Theorem 1.1 (Informal; see Theorem 2.1) .
Revised: Theorem 1.1 (Informal; see Theorem 2.1) .

Original: Let κ ( A ) =
Revised: Let κ ( A ) =

Original: O (1) and ε ∈ (0 , 1) .
Revised: O (1) and ε ∈ (0 , 1) .

Original: There exists a dynamic algorithm that given adaptively chosen K -sparse updates to b and a fixed design matrix A ∈ R n × d , outputs a (1 + ε ) approximation to the least-squares objective F ( A , b ( i ) )
Revised: There exists a dynamic algorithm that given adaptively chosen K -sparse updates to b and a fixed design matrix A ∈ R n × d , outputs a (1 + ε ) approximation to the least-squares objective F ( A , b ( i ) )

Original: every round with high probability.
Revised: every round with high probability.

Original: The algorithm uses (cid:101) O
Revised: The algorithm uses (cid:101) O

Original: (cid:16)(cid:112) K nnz( A ) /ε 3 (cid:17) amortized time per step of update.
Revised: (cid:16)(cid:112) K nnz( A ) /ε 3 (cid:17) amortized time per step of update.

Original: Specifically, the update time is d 1 . 5 when K ≤ d and n = O ( d ) and square root of the input sparsity when K = O (1) .
Revised: Specifically, the update time is d 1 . 5 when K ≤ d and n = O ( d ) and square root of the input sparsity when K = O (1) .

Original: Notice that this significantly betters the na¨ıve approach of treating each step independently and solving for the least-square objective, which requires nnz( A ) + poly( d ) time by sketching (Woodruff (2014)).
Revised: Notice that this significantly betters the na¨ıve approach of treating each step independently and solving for the least-square objective, which requires nnz( A ) + poly( d ) time by sketching (Woodruff (2014)).

Original: We mention that a recent work by Jiang et al.
Revised: We mention that a recent work by Jiang et al.

Original: (2022) considers a row-arrival model for dynamic linear regression.
Revised: (2022) considers a row-arrival model for dynamic linear regression.

Original: Our setting is different since we allow arbitrary updates to the target label, whereas in their setting the design matrix undertakes incremental change.
Revised: Our setting is different since we allow arbitrary updates to the target label, whereas in their setting the design matrix undertakes incremental change.

Original: We note that their algorithm maintains a solution vector, while we focus on the cost only.
Revised: We note that their algorithm maintains a solution vector, while we focus on the cost only.

Original: Adaptive query framework.
Revised: Adaptive query framework.

Original: We then consider robust algorithms in the standard centralized setting, where an adversary queries an algorithm in an adaptive manner.
Revised: We then consider robust algorithms in the standard centralized setting, where an adversary queries an algorithm in an adaptive manner.

Original: In many key algorithmic applications, randomization is necessary to achieve fast query time and efficient storage.
Revised: In many key algorithmic applications, randomization is necessary to achieve fast query time and efficient storage.

Original: This necessitates the need for robust versions of these algorithm which can efficiently employ the power of randomness while also being accurate across multiple possibly correlated inputs.
Revised: This necessitates the need for robust versions of these algorithm which can efficiently employ the power of randomness while also being accurate across multiple possibly correlated inputs.

Original: Our main parameters of interest are query time and the space used by a robust algorithm compared to their na¨ıve, non-robust, counterparts.
Revised: Our main parameters of interest are query time and the space used by a robust algorithm compared to their na¨ıve, non-robust, counterparts.

Original: Formally, we define the model as a two-player game between an algorithm HonestAlg over a data set
Revised: Formally, we define the model as a two-player game between an algorithm HonestAlg over a data set

Original: X and an adversary A that makes adversarial queries about X to HonestAlg .
Revised: X and an adversary A that makes adversarial queries about X to HonestAlg .

Original: At the beginning of the game, HonestAlg uses pre-processing time to compute a data structure D from X to answer future queries from A .
Revised: At the beginning of the game, HonestAlg uses pre-processing time to compute a data structure D from X to answer future queries from A .

Original: The game then proceeds in at most Q rounds for some predetermined Q , so
Revised: The game then proceeds in at most Q rounds for some predetermined Q , so

Original: that in the t -th round, where t ∈ [ Q ] :
Revised: that in the t -th round, where t ∈ [ Q ] :

Original: (1) A computes a query q t on X , which depends on all previous responses from HonestAlg .
Revised: (1) A computes a query q t on X , which depends on all previous responses from HonestAlg .

Original: (2) HonestAlg uses D to output a response d t to query q t .
Revised: (2) HonestAlg uses D to output a response d t to query q t .

Original: (3) A observes and records the response d t .
Revised: (3) A observes and records the response d t .

Original: The goal of A is to formulate a query q t for which the algorithm HonestAlg produces an incorrect response d t .
Revised: The goal of A is to formulate a query q t for which the algorithm HonestAlg produces an incorrect response d t .

Original: We remark that the algorithm may not have access to X , after constructing D , to respond to the query q t .
Revised: We remark that the algorithm may not have access to X , after constructing D , to respond to the query q t .

Original: On the other hand, A can use previous outputs to possibly determine the internal randomness of the data structure D and make future queries accordingly.
Revised: On the other hand, A can use previous outputs to possibly determine the internal randomness of the data structure D and make future queries accordingly.

Original: In this case, the analysis of many randomized algorithms fails because it assumes that the randomness of the algorithm is independent of the input.
Revised: In this case, the analysis of many randomized algorithms fails because it assumes that the randomness of the algorithm is independent of the input.

Original: Consequently, it does not seem evident how to handle Q adaptive queries without implementing Q instances of a non-adaptive data structure, i.e., each instance handles a separate query.
Revised: Consequently, it does not seem evident how to handle Q adaptive queries without implementing Q instances of a non-adaptive data structure, i.e., each instance handles a separate query.

Original: Thus, a natural question to ask is whether a space overhead of Ω( Q ) is necessary.
Revised: Thus, a natural question to ask is whether a space overhead of Ω( Q ) is necessary.

Original: As a preliminary benchmark, we show that a space overhead of Ω( Q ) is unnecessary by giving a unified framework with only an (cid:101) O (cid:0) √ Q (cid:1) space overhead.
Revised: As a preliminary benchmark, we show that a space overhead of Ω( Q ) is unnecessary by giving a unified framework with only an (cid:101) O (cid:0) √ Q (cid:1) space overhead.

Original: Theorem 1.2.
Revised: Theorem 1.2.

Original: Given a data structure D that answers a query q with probability at least 34 using space
Revised: Given a data structure D that answers a query q with probability at least 34 using space

Original: S and query time T , there exists a data structure that answers Q adaptive queries, with high probability, i.e., 1 − 1 poly( n,Q ) , using space O (cid:0) S √ Q log( nQ ) (cid:1) and query time (cid:101) O (cid:0) T log( nQ ) + log 3 ( nQ ) (cid:1) .
Revised: S and query time T , there exists a data structure that answers Q adaptive queries, with high probability, i.e., 1 − 1 poly( n,Q ) , using space O (cid:0) S √ Q log( nQ ) (cid:1) and query time (cid:101) O (cid:0) T log( nQ ) + log 3 ( nQ ) (cid:1) .

Original: Theorem 1.2 invokes the framework of Hassidim et al.
Revised: Theorem 1.2 invokes the framework of Hassidim et al.

Original: Beimel et al.
Revised: Beimel et al.

Original: ; Attias et al.
Revised: ; Attias et al.

Original: (2023) to the centralized setting, where a number of queries are made only after the data structure is created.
Revised: (2023) to the centralized setting, where a number of queries are made only after the data structure is created.

Original: To concretely instantiate our framework and state an example, we consider the adaptive distance estimation problem defined as follows.
Revised: To concretely instantiate our framework and state an example, we consider the adaptive distance estimation problem defined as follows.

Original: In the adaptive distance estimation problem, there exists a set
Revised: In the adaptive distance estimation problem, there exists a set

Original: X = { x (1) , . . .
Revised: X = { x (1) , . . .

Original: , x ( n ) } of n points in R d .
Revised: , x ( n ) } of n points in R d .

Original: We are also given an accuracy parameter ε > 0 .
Revised: We are also given an accuracy parameter ε > 0 .

Original: A query is of the form q , and the algorithm must output a (1 + ε ) -approximation to ∥ x ( i ) − q ∥ p for all i .
Revised: A query is of the form q , and the algorithm must output a (1 + ε ) -approximation to ∥ x ( i ) − q ∥ p for all i .

Original: The trivial solution of storing all n points and computing all n distances to a query point uses space and query time O ( nd ) .
Revised: The trivial solution of storing all n points and computing all n distances to a query point uses space and query time O ( nd ) .

Original: Cherapanamjeri & Nelson (2020) improved the query time to (cid:101) O (cid:0) n + d ε 2 (cid:1) at the cost of using (cid:101) O (cid:16) ( n + d ) d ε(cid:17) space and (cid:101) O (cid:16) nd 

Revised: Cherapanamjeri & Nelson (2020) improved the query time to (cid:101) O (cid:0) n + d ε 2 (cid:1) at the cost of using (cid:101) O (cid:16) ( n + d ) d ε(cid:17) space and (cid:101) O (cid:16) nd

Original: ε 2 (cid:17) pre-processing time, while permitting an arbitrary number of queries.
Revised: ε 2 (cid:17) pre-processing time, while permitting an arbitrary number of queries.

Original: By comparison, our data structure handles Q queries of approximate distances from a specified point in X , using query time (cid:101) O (cid:0) n + d ε 2 (cid:1) , pre-processing time (cid:101) O (cid:16) nd √ Q ε 2 (cid:17) , and space (cid:101) O (cid:16) ( n + d ) √ Q ε(cid:17) .
Revised: By comparison, our data structure handles Q queries of approximate distances from a specified point in X , using query time (cid:101) O (cid:0) n + d ε 2 (cid:1) , pre-processing time (cid:101) O (cid:16) nd √ Q ε 2 (cid:17) , and space (cid:101) O (cid:16) ( n + d ) √ Q ε(cid:17) .

Original: Thus, in the regime where d ≫ n √ Q , our data structure already improves on the work of Cherapanamjeri & Nelson (2020).
Revised: Thus, in the regime where d ≫ n √ Q , our data structure already improves on the work of Cherapanamjeri & Nelson (2020).

Original: A noticeable weakness of our construction is that the Q queries return only the approximate distance between a query point and a single point in X , whereas Cherapanamjeri & Nelson (2020) outputs approximate distances to all points in X .
Revised: A noticeable weakness of our construction is that the Q queries return only the approximate distance between a query point and a single point in X , whereas Cherapanamjeri & Nelson (2020) outputs approximate distances to all points in X .

Original: Moreover, Cherapanamjeri & Nelson (2022) subsequently improve the pre-processing time to (cid:101) O (cid:0) nd ε 2 (cid:1) .
Revised: Moreover, Cherapanamjeri & Nelson (2022) subsequently improve the pre-processing time to (cid:101) O (cid:0) nd ε 2 (cid:1) .

Original: Thus we open up our framework to (1) show that it can be further improved to handle the case where we return the approximate distances of all points in X from Q adaptive query points and (2) achieve pre-processing time (cid:101) O (cid:0) ndε 2 (cid:1) .
Revised: Thus we open up our framework to (1) show that it can be further improved to handle the case where we return the approximate distances of all points in X from Q adaptive query points and (2) achieve pre-processing time (cid:101) O (cid:0) ndε 2 (cid:1) .

Original: Theorem 1.3.
Revised: Theorem 1.3.

Original: There is a data structure which, when instantiated with dataset X = { x i } i ∈ [ n ] ⊂ R d and query bound Q ≤ d , answers any sequence of Q adaptively chosen distance estimation queries correctly with probability at least 0 .
Revised: There is a data structure which, when instantiated with dataset X = { x i } i ∈ [ n ] ⊂ R d and query bound Q ≤ d , answers any sequence of Q adaptively chosen distance estimation queries correctly with probability at least 0 .

Original: Furthermore, the space complexity of the data structure is (cid:101) O ( ε − 2 · n √ Q ) and the setup and query times are (cid:101)
Revised: Furthermore, the space complexity of the data structure is (cid:101) O ( ε − 2 · n √ Q ) and the setup and query times are (cid:101)

Original: O ( ε − 2 · nd ) and (cid:101) O
Revised: O ( ε − 2 · nd ) and (cid:101) O

Original: ( ε − 2 · ( n + d )) , respectively.
Revised: ( ε − 2 · ( n + d )) , respectively.

Original: Another application of our framework is the adaptive kernel density estimation problem, where there exists a set X
Revised: Another application of our framework is the adaptive kernel density estimation problem, where there exists a set X

Original: = { x (1) , . . .
Revised: = { x (1) , . . .

Original: , x ( n ) } of n points in R d and the goal is to output a (1 + ε ) -approximation to the quantity 1 n (cid:80) i ∈ [ n ] k
Revised: , x ( n ) } of n points in R d and the goal is to output a (1 + ε ) -approximation to the quantity 1 n (cid:80) i ∈ [ n ] k

Original: ( x ( i ) , q ) , for an accuracy parameter ε > 0 , a query q , and a kernel function k , under the promise that the output is at least some threshold τ > 0 .
Revised: ( x ( i ) , q ) , for an accuracy parameter ε > 0 , a query q , and a kernel function k , under the promise that the output is at least some threshold τ > 0 .

Original: Backurs et al.
Revised: Backurs et al.

Original: (2019) give an algorithm for kernel density estimation that uses O (cid:0) 1 τε 2 (cid:1) space and O (cid:16) d √ τε 2 (cid:17) query time, improving over the standard algorithm that samples O (cid:0) 1 τε 2 (cid:1) points and then uses O (cid:0) dτε 2 (cid:1) query time to output the empirical kernel density.
Revised: (2019) give an algorithm for kernel density estimation that uses O (cid:0) 1 τε 2 (cid:1) space and O (cid:16) d √ τε 2 (cid:17) query time, improving over the standard algorithm that samples O (cid:0) 1 τε 2 (cid:1) points and then uses O (cid:0) dτε 2 (cid:1) query time to output the empirical kernel density.

Original: However, the analysis for both of these algorithms fails for the adaptive setting, where there can be dependencies between the query and the data structure.
Revised: However, the analysis for both of these algorithms fails for the adaptive setting, where there can be dependencies between the query and the data structure.

Original: By using the data structure of Backurs et al.
Revised: By using the data structure of Backurs et al.

Original: (2019) as a subroutine, our framework immediately implies an algorithm for adaptive kernel density estimation that uses (cid:101) O (cid:16) √ Q τε 2 (cid:17) space and O (cid:16) d log Q √ τε 2 (cid:17) query time to answer each of Q adaptive queries.
Revised: (2019) as a subroutine, our framework immediately implies an algorithm for adaptive kernel density estimation that uses (cid:101) O (cid:16) √ Q τε 2 (cid:17) space and O (cid:16) d log Q √ τε 2 (cid:17) query time to answer each of Q adaptive queries.

Original: In this case, we are again able to go beyond our framework and give a data structure that handles an unlimited number of adaptive kernel density queries:
Revised: In this case, we are again able to go beyond our framework and give a data structure that handles an unlimited number of adaptive kernel density queries:

Original: Theorem 1.4.
Revised: Theorem 1.4.

Original: Suppose the kernel function k is L -Lipschitz in the second variable for some L > 0 , i.e., | k
Revised: Suppose the kernel function k is L -Lipschitz in the second variable for some L > 0 , i.e., | k

Original: ( x , y ) − k ( x , z ) | ≤ L ∥ y − z ∥ 2 for all x , y , z ∈ R d .
Revised: ( x , y ) − k ( x , z ) | ≤ L ∥ y − z ∥ 2 for all x , y , z ∈ R d .

Original: Moreover, suppose that for all ∥ x − y ∥ 2 ≤ ρ , we have k ( x , y ) ≤ τ 3 .
Revised: Moreover, suppose that for all ∥ x − y ∥ 2 ≤ ρ , we have k ( x , y ) ≤ τ 3 .

Original: Then an algorithm that produces a kernel density estimation data structure D that is L -Lipschitz over a set X of points with diameter at most ∆ and outputs a (1+ ε ) -approximation to KDE queries with value at least τ with probability at least 1 − δ using space S ( n, ε, τ, log δ ) and query time T ( n, ε, τ, log δ ) , then there exists a KDE data structure that with probability at least 0 .
Revised: Then an algorithm that produces a kernel density estimation data structure D that is L -Lipschitz over a set X of points with diameter at most ∆ and outputs a (1+ ε ) -approximation to KDE queries with value at least τ with probability at least 1 − δ using space S ( n, ε, τ, log δ ) and query time T ( n, ε, τ, log δ ) , then there exists a KDE data structure that with probability at least 0 .

Original: 99 , outputs a (1 + ε ) -approximation to any number of KDE queries with value at least τ using space
Revised: 99 , outputs a (1 + ε ) -approximation to any number of KDE queries with value at least τ using space

Original: S (cid:16) n, O ( ε ) , O ( τ ) , O (cid:16) d log (∆+ ρ ) L ετ (cid:17)(cid:17) and query time T (cid:16) n, O ( ε ) , O ( τ ) , O (cid:16) d log (∆+ ρ ) L ετ (cid:17)(cid:17) .
Revised: S (cid:16) n, O ( ε ) , O ( τ ) , O (cid:16) d log (∆+ ρ ) L ετ (cid:17)(cid:17) and query time T (cid:16) n, O ( ε ) , O ( τ ) , O (cid:16) d log (∆+ ρ ) L ετ (cid:17)(cid:17) .

Original: Additionally, we show that our framework guarantees adversarial robustness for a number of other important problems such as nearest neighbor search, range queries, point queries, matrix-vector norm queries, and linear regression.
Revised: Additionally, we show that our framework guarantees adversarial robustness for a number of other important problems such as nearest neighbor search, range queries, point queries, matrix-vector norm queries, and linear regression.

Original: Finally, we supplement our theoretical results with a number of empirical evaluations, which are in the appendix.
Revised: Finally, we supplement our theoretical results with a number of empirical evaluations, which are in the appendix.

Original: 1. O UR T ECHNIQUES
Revised: 1. O UR T ECHNIQUES

Original: Dynamic regression on adaptive inputs.
Revised: Dynamic regression on adaptive inputs.

Original: Our dynamic algorithm for dynamic maintenance of least-squares objective exploits two main ideas.
Revised: Our dynamic algorithm for dynamic maintenance of least-squares objective exploits two main ideas.

Original: First, standard results in sketching and sampling show that it suffices to solve for the sketched objective of min x ∈ R d ∥ SAx − Sb ∥ 22 , where S is an ℓsubspace embedding for A .
Revised: First, standard results in sketching and sampling show that it suffices to solve for the sketched objective of min x ∈ R d ∥ SAx − Sb ∥ 22 , where S is an ℓsubspace embedding for A .

Original: Here, we exploit several techniques from numerical linear algebra and in particular use leverage score sampling to obtain a subspace embedding S of A .
Revised: Here, we exploit several techniques from numerical linear algebra and in particular use leverage score sampling to obtain a subspace embedding S of A .

Original: By standard results in sketching, a (1 + ε ) optimal solution is given by x ∗ =
Revised: By standard results in sketching, a (1 + ε ) optimal solution is given by x ∗ =

Original: ( SA ) † Sb .
Revised: ( SA ) † Sb .

Original: Moreover, since the goal is to output the objective value instead of the solution vector, we may take a Johnson-Lindenstrauss (JL) sketch to further reduce dimensionality and run-time.
Revised: Moreover, since the goal is to output the objective value instead of the solution vector, we may take a Johnson-Lindenstrauss (JL) sketch to further reduce dimensionality and run-time.

Original: This allows us to focus on ∥ GAx ∗ − Gb ∥ 22 , where G ∈ R O (log d ) × n is a JL sketch.
Revised: This allows us to focus on ∥ GAx ∗ − Gb ∥ 22 , where G ∈ R O (log d ) × n is a JL sketch.

Original: As a result, our algorithm dynamically maintains a solution GA ( SA ) † b in this sketched space.
Revised: As a result, our algorithm dynamically maintains a solution GA ( SA ) † b in this sketched space.

Original: To achieve that, we first explicitly solve GA ( SA ) † in pre-processing.
Revised: To achieve that, we first explicitly solve GA ( SA ) † in pre-processing.

Original: Since GA has few rows, this reduces to a small number of linear solves and can be computed fast via conjugate gradient-type methods.
Revised: Since GA has few rows, this reduces to a small number of linear solves and can be computed fast via conjugate gradient-type methods.

Original: To handle the updates, we leverage their sparsity to efficiently maintain the solution and show that each round takes roughly O ( K ) time.
Revised: To handle the updates, we leverage their sparsity to efficiently maintain the solution and show that each round takes roughly O ( K ) time.

Original: Amortizing the pre-processing with the update costs over all iterations yields our desired run-time.
Revised: Amortizing the pre-processing with the update costs over all iterations yields our desired run-time.

Original: Finally, we apply techniques from differential privacy to ensure adversarial robustness, by aggregating independent copies of the algorithm via a private median mechanism.
Revised: Finally, we apply techniques from differential privacy to ensure adversarial robustness, by aggregating independent copies of the algorithm via a private median mechanism.

Original: Intuitively, the private mechanism hides the internal randomness of the algorithm and therefore prevents the adversary from otherwise choosing a “bad” input based on knowledge of internal parameters.
Revised: Intuitively, the private mechanism hides the internal randomness of the algorithm and therefore prevents the adversary from otherwise choosing a “bad” input based on knowledge of internal parameters.

Original: Adaptive query framework.
Revised: Adaptive query framework.

Original: Our framework maintains (cid:101) O (cid:0) √ Q (cid:1) instances of the non-adaptive data structure and crucially uses differential privacy (DP) to protect the internal randomness of the data structures.
Revised: Our framework maintains (cid:101) O (cid:0) √ Q (cid:1) instances of the non-adaptive data structure and crucially uses differential privacy (DP) to protect the internal randomness of the data structures.

Original: In addition to our previous results for dynamic regression, the technique of using DP to hide randomness has recently been used in the streaming model (Hassidim et al., 2020; Kaplan et al., 2021) and the dynamic model (Beimel et al., 2021).
Revised: In addition to our previous results for dynamic regression, the technique of using DP to hide randomness has recently been used in the streaming model (Hassidim et al., 2020; Kaplan et al., 2021) and the dynamic model (Beimel et al., 2021).

Original: These works elegantly use the advanced composition property of DP to bound the number of simultaneous algorithms that must be used in terms of the number of times the output changes “significantly” over the course of the stream.
Revised: These works elegantly use the advanced composition property of DP to bound the number of simultaneous algorithms that must be used in terms of the number of times the output changes “significantly” over the course of the stream.

Original: In the streaming model, the robust algorithms proceed by instantiating many “hidden” copies of a standard randomized algorithm.
Revised: In the streaming model, the robust algorithms proceed by instantiating many “hidden” copies of a standard randomized algorithm.

Original: As the stream arrives, the algorithms are updated and an answer, aggregated using DP, is reported.
Revised: As the stream arrives, the algorithms are updated and an answer, aggregated using DP, is reported.

Original: Crucially, many of these results exploit the fact that the output answer is monotonic in the stream so that there is a known upper bound on the final output.
Revised: Crucially, many of these results exploit the fact that the output answer is monotonic in the stream so that there is a known upper bound on the final output.

Original: Thus, the reported answers can only increase by a multiplicative factor at most a logarithmic number of times, which is used to bound the initial number of algorithms which are initialized.
Revised: Thus, the reported answers can only increase by a multiplicative factor at most a logarithmic number of times, which is used to bound the initial number of algorithms which are initialized.

Original: In our centralized setting, this can be imagined as setting the parameter Q .
Revised: In our centralized setting, this can be imagined as setting the parameter Q .

Original: The main parameter of interest in the streaming literature is the space used by the streaming algorithms, whereas we are concerned with both space usage and query times.
Revised: The main parameter of interest in the streaming literature is the space used by the streaming algorithms, whereas we are concerned with both space usage and query times.

Original: Furthermore, stream elements are only accessed one at a time and cannot be processed together unless memory is used.
Revised: Furthermore, stream elements are only accessed one at a time and cannot be processed together unless memory is used.

Original: In our case, the dataset is given to us upfront
Revised: In our case, the dataset is given to us upfront

Original: and we can pre-process it to construct a data structure towards solving a centralized problem.
Revised: and we can pre-process it to construct a data structure towards solving a centralized problem.

Original: The work by Beimel et al.
Revised: The work by Beimel et al.

Original: (2021) shares many of these ideas: the authors are concerned with dynamic graph algorithms where an adversary can update the graph in an adaptive fashion.
Revised: (2021) shares many of these ideas: the authors are concerned with dynamic graph algorithms where an adversary can update the graph in an adaptive fashion.

Original: Similar tools such as multiple randomized initialization and aggregated responses using DP are utilized.
Revised: Similar tools such as multiple randomized initialization and aggregated responses using DP are utilized.

Original: The main difference is their parameters of interest: the goal of Beimel et al.
Revised: The main difference is their parameters of interest: the goal of Beimel et al.

Original: (2021) is to have a fast amortized update time across many queries.
Revised: (2021) is to have a fast amortized update time across many queries.

Original: This necessitates the need to “throw away” existing algorithms and start with fresh randomness at intermittent points.
Revised: This necessitates the need to “throw away” existing algorithms and start with fresh randomness at intermittent points.

Original: In contrast, we study a centralized setting where the underlying dataset is not updated but we wish to answer Q adaptive queries on the dataset.
Revised: In contrast, we study a centralized setting where the underlying dataset is not updated but we wish to answer Q adaptive queries on the dataset.

Original: Inspired by these works, our main framework also uses advanced composition to show the sufficiency of maintaining (cid:101) O (cid:0) √ Q (cid:1) data structures to answer Q adaptive queries in the centralized setting, which gives a rich set of applications.
Revised: Inspired by these works, our main framework also uses advanced composition to show the sufficiency of maintaining (cid:101) O (cid:0) √ Q (cid:1) data structures to answer Q adaptive queries in the centralized setting, which gives a rich set of applications.

Original: Moreover, to improve the query time of our algorithms, we further invoke the privacy amplification of sampling to show that it suffices to output the private median of a small subset, i.e., a subset of size O (log Q ) , of these (cid:101) O (cid:0) √ Q (cid:1) data structures.
Revised: Moreover, to improve the query time of our algorithms, we further invoke the privacy amplification of sampling to show that it suffices to output the private median of a small subset, i.e., a subset of size O (log Q ) , of these (cid:101) O (cid:0) √ Q (cid:1) data structures.

Original: Thus our framework only incurs a logarithmic overhead in query time and an (cid:101) O (cid:0) √ Q (cid:1) overhead in space.
Revised: Thus our framework only incurs a logarithmic overhead in query time and an (cid:101) O (cid:0) √ Q (cid:1) overhead in space.

Original: Surprisingly, our simple framework gives diverse applications for adaptive algorithms on a number of important problems, including estimating matrix-vector norms, adaptive range query search, adaptive nearest neighbor search, and adaptive kernel density estimation, to name a few.
Revised: Surprisingly, our simple framework gives diverse applications for adaptive algorithms on a number of important problems, including estimating matrix-vector norms, adaptive range query search, adaptive nearest neighbor search, and adaptive kernel density estimation, to name a few.

Original: These applications are discussed in depth in Section C.
Revised: These applications are discussed in depth in Section C.

Original: Adaptive distance estimation.
Revised: Adaptive distance estimation.

Original: To achieve better pre-processing time for adaptive distance estimation, our main technique is to sample groups of rows from a Hadamard transform and argue that an interaction with a separate group should be considered in separate privacy budgets, effectively arguing that outputting n approximate distances to a single adaptive query only uses one unit of privacy budget.
Revised: To achieve better pre-processing time for adaptive distance estimation, our main technique is to sample groups of rows from a Hadamard transform and argue that an interaction with a separate group should be considered in separate privacy budgets, effectively arguing that outputting n approximate distances to a single adaptive query only uses one unit of privacy budget.

Original: By contrast, our black-box framework charges one unit of privacy budget per approximate distance, so that outputting n approximate distances would use n units of privacy budget.
Revised: By contrast, our black-box framework charges one unit of privacy budget per approximate distance, so that outputting n approximate distances would use n units of privacy budget.

Original: Adaptive kernel density estimation.
Revised: Adaptive kernel density estimation.

Original: Theorem 1.4 is based on showing that with constant probability, our data structure is accurate on all possible queries in R d .
Revised: Theorem 1.4 is based on showing that with constant probability, our data structure is accurate on all possible queries in R d .

Original: In particular, we first show that our data structure is accurate on a sufficiently fine net of points through a standard union bound argument, which incurs the d overhead compared to the space required to handle a single query.
Revised: In particular, we first show that our data structure is accurate on a sufficiently fine net of points through a standard union bound argument, which incurs the d overhead compared to the space required to handle a single query.

Original: We then show that if the algorithm and the kernel function are both Lipschitz, which is true for sampling-based algorithms and a number of standard kernel functions, then accuracy on the net implies accuracy on all possible points in R d .
Revised: We then show that if the algorithm and the kernel function are both Lipschitz, which is true for sampling-based algorithms and a number of standard kernel functions, then accuracy on the net implies accuracy on all possible points in R d .

Original:  D YNAMIC R EGRESSION UNDER L ABEL U PDATES
Revised:  D YNAMIC R EGRESSION UNDER L ABEL U PDATES

Original: In this section, we consider the dynamic problem of maintaining the cost of the least-squares regression, where the labels receive adaptively chosen updates.
Revised: In this section, we consider the dynamic problem of maintaining the cost of the least-squares regression, where the labels receive adaptively chosen updates.

Original: Let A ∈ R n × d be the design matrix and b ∈ R n be the target label.
Revised: Let A ∈ R n × d be the design matrix and b ∈ R n be the target label.

Original: A classic problem in numerical linear algebra and optimization is to solve the ℓ 2 least-squares regression objective
Revised: A classic problem in numerical linear algebra and optimization is to solve the ℓ 2 least-squares regression objective

Original: [Equation]
Revised: [Equation]

Original: We consider a dynamic version of the problem, where the label receives adaptively chosen updates.
Revised: We consider a dynamic version of the problem, where the label receives adaptively chosen updates.

Original: We assume that each update can only affect K entries of the label vector.
Revised: We assume that each update can only affect K entries of the label vector.

Original: In this setting, we show:
Revised: In this setting, we show:

Original: Theorem 2.1 (Main theorem; dynamic maintenance of regression cost) .
Revised: Theorem 2.1 (Main theorem; dynamic maintenance of regression cost) .

Original: Let ε ∈
Revised: Let ε ∈

Original: (0 , 1 / 4) be an error parameter and b (1) be the initial target label.
Revised: (0 , 1 / 4) be an error parameter and b (1) be the initial target label.

Original: Given ε, A , b (1) , a stream of T adaptively chosen,
Revised: Given ε, A , b (1) , a stream of T adaptively chosen,

Original: K -sparse updates to the label, Algorithm 4 outputs an estimate (cid:98) F i such that (cid:98) F i
Revised: K -sparse updates to the label, Algorithm 4 outputs an estimate (cid:98) F i such that (cid:98) F i

Original: =
Revised: =

Original: (1 ± ε ) F ( A , b ( i ) ) for all i with high probability.
Revised: (1 ± ε ) F ( A , b ( i ) ) for all i with high probability.

Original: Furthermore, the algorithm requires a preprocessing step in time (cid:101)
Revised: Furthermore, the algorithm requires a preprocessing step in time (cid:101)

Original: O
Revised: O

Original: (nnz( A ) + poly( d )) .
Revised: (nnz( A ) + poly( d )) .

Original: The amortized update time of the algorithm is
Revised: The amortized update time of the algorithm is

Original: [Equation]
Revised: [Equation]

Original: We defer the technical details to Appendix 3.
Revised: We defer the technical details to Appendix 3.

Original: Here, we describe the main ideas of the algorithm.
Revised: Here, we describe the main ideas of the algorithm.

Original: At a high-level, our algorithm implements a sketch-and-solve strategy.
Revised: At a high-level, our algorithm implements a sketch-and-solve strategy.

Original: First, in pre-processing, the algorithm samples a leverage score sketching matrix S ∈ R k × n , where k
Revised: First, in pre-processing, the algorithm samples a leverage score sketching matrix S ∈ R k × n , where k

Original: = O (cid:0) d log d/ε 2 (cid:1) .
Revised: = O (cid:0) d log d/ε 2 (cid:1) .

Original: This provides a (1 + ε ) ℓ 2 subspace embedding for A .
Revised: This provides a (1 + ε ) ℓ 2 subspace embedding for A .

Original: Standard results in sketching imply that it suffices to solve for the sketched objective of min x ∈ R d ∥ SAx − Sb ∥ 22 (Sarlos (2006); Clarkson & Woodruff (2013; 2017); Woodruff (2014)).
Revised: Standard results in sketching imply that it suffices to solve for the sketched objective of min x ∈ R d ∥ SAx − Sb ∥ 22 (Sarlos (2006); Clarkson & Woodruff (2013; 2017); Woodruff (2014)).

Original: Let (cid:98)
Revised: Let (cid:98)

Original: A = SA .
Revised: A = SA .

Original: A (1 + ε ) optimal solution is thus given by (cid:98) A † b .
Revised: A (1 + ε ) optimal solution is thus given by (cid:98) A † b .

Original: Moreover, our goal is to maintain the regression cost, rather than this solution vector.
Revised: Moreover, our goal is to maintain the regression cost, rather than this solution vector.

Original: Hence, we can apply the Johnson–Lindenstrauss lemma and focus on
Revised: Hence, we can apply the Johnson–Lindenstrauss lemma and focus on

Original: [Equation]
Revised: [Equation]

Original: where G ∈ R O ( log n/ε 2 ) × n is a JL sketch.
Revised: where G ∈ R O ( log n/ε 2 ) × n is a JL sketch.

Original: To track the cost value dynamically, the algorithm first computes and stores M = GA ( SA ) † in pre-processing.
Revised: To track the cost value dynamically, the algorithm first computes and stores M = GA ( SA ) † in pre-processing.

Original: In the 1 -st step, given the initial target label b (1) , the algorithm computes Sb ( i ) ,
Revised: In the 1 -st step, given the initial target label b (1) , the algorithm computes Sb ( i ) ,

Original: M (cid:0) Sb (1) (cid:1) and Gb (1) .
Revised: M (cid:0) Sb (1) (cid:1) and Gb (1) .

Original: Then it outputs (cid:98) F 1 = (cid:13)(cid:13) MSb (1) − Gb (1) (cid:13)(cid:13) 2 2 as an estimate of the regression cost.
Revised: Then it outputs (cid:98) F 1 = (cid:13)(cid:13) MSb (1) − Gb (1) (cid:13)(cid:13) 2 2 as an estimate of the regression cost.

Original: For the later steps, we show how to maintain Gb ( i ) ,
Revised: For the later steps, we show how to maintain Gb ( i ) ,

Original: Sb ( i ) efficiently, by exploiting the sparsity of the updates.
Revised: Sb ( i ) efficiently, by exploiting the sparsity of the updates.

Original: Finally, to remain robust under adaptive inputs, we aggregate multiple copies using private median and carefully balance the parameters to achieve the run-time guarantee.
Revised: Finally, to remain robust under adaptive inputs, we aggregate multiple copies using private median and carefully balance the parameters to achieve the run-time guarantee.

Original:  D ETAILS ON D YNAMIC R EGRESSION
Revised:  D ETAILS ON D YNAMIC R EGRESSION

Original: In this section, we consider the dynamic problem of maintaining the cost of the least-squares regression, where the labels receive adaptively chosen updates.
Revised: In this section, we consider the dynamic problem of maintaining the cost of the least-squares regression, where the labels receive adaptively chosen updates.

Original: We first introduce the basic setting of the problem in Section 3.1.
Revised: We first introduce the basic setting of the problem in Section 3.1.

Original: In Section 3.2, we design a key subroutine under non-adaptive updates.
Revised: In Section 3.2, we design a key subroutine under non-adaptive updates.

Original: The data structure enjoys a nearly linear update time.
Revised: The data structure enjoys a nearly linear update time.

Original: This allows us to aggregate multiple copies of the procedure and thereby efficiently ensure adversarial robustness against an adaptive adversary.
Revised: This allows us to aggregate multiple copies of the procedure and thereby efficiently ensure adversarial robustness against an adaptive adversary.

Original: The argument is via an application of differential privacy and detailed subsequently in Section 3.3.
Revised: The argument is via an application of differential privacy and detailed subsequently in Section 3.3.

Original: 3. B ASIC S ETTING
Revised: 3. B ASIC S ETTING

Original: Let A ∈ R n × d be the design matrix and b ∈ R n be the target label.
Revised: Let A ∈ R n × d be the design matrix and b ∈ R n be the target label.

Original: A classic problem in numerical linear algebra and optimization is to solve the ℓ 2 least-squares regression objective
Revised: A classic problem in numerical linear algebra and optimization is to solve the ℓ 2 least-squares regression objective

Original: [Equation]
Revised: [Equation]

Original: We consider a dynamic version of the problem, where the design matrix A remains unchanged.
Revised: We consider a dynamic version of the problem, where the design matrix A remains unchanged.

Original: However, at each step (at most) K entries of b undergo an update.
Revised: However, at each step (at most) K entries of b undergo an update.

Original: Moreover, we assume that the updates are chosen adaptively by an adversary in the following manner.
Revised: Moreover, we assume that the updates are chosen adaptively by an adversary in the following manner.

Original: • The algorithm starts by receiving the input A ∈ R n × d and b (1) ∈ R n .
Revised: • The algorithm starts by receiving the input A ∈ R n × d and b (1) ∈ R n .

Original: In the i -th step, the algorithm outputs an estimate (cid:98) F i of the cost F ( A , b ( i ) ) , where b ( i ) is the target label corresponding to the step.
Revised: In the i -th step, the algorithm outputs an estimate (cid:98) F i of the cost F ( A , b ( i ) ) , where b ( i ) is the target label corresponding to the step.

Original: • The adversary observes (cid:98) F i and updates at most K labels to form b ( i ) .
Revised: • The adversary observes (cid:98) F i and updates at most K labels to form b ( i ) .

Original: Let b (1) , b (2) , . . .
Revised: Let b (1) , b (2) , . . .

Original: , b ( T ) ∈ R n be the resulting sequence of labels over T steps.
Revised: , b ( T ) ∈ R n be the resulting sequence of labels over T steps.

Original: The goal of the algorithm is to output a (1 + ε ) approximation to the optimal cost at every step, while minimizing the update time.
Revised: The goal of the algorithm is to output a (1 + ε ) approximation to the optimal cost at every step, while minimizing the update time.

Original: 3. D YNAMIC A LGORITHM FOR O BLIVIOUS I NPUTS
Revised: 3. D YNAMIC A LGORITHM FOR O BLIVIOUS I NPUTS

Original: In this section, we provide a key subroutine that maintains a data structure under oblivious updates.
Revised: In this section, we provide a key subroutine that maintains a data structure under oblivious updates.

Original: On a high-level, the data structure aims to enable a sketch-and-solve strategy dynamically.
Revised: On a high-level, the data structure aims to enable a sketch-and-solve strategy dynamically.

Original: The main ideas are two fold: (1) apply randomized sketching to reduce dimensionality and therefore the run-time, and (2) exploit the sparsity of the updates to argue that the regression costs can be maintained efficiently.
Revised: The main ideas are two fold: (1) apply randomized sketching to reduce dimensionality and therefore the run-time, and (2) exploit the sparsity of the updates to argue that the regression costs can be maintained efficiently.

Original: Before delving into the technical details, we give an overview of the algorithm.
Revised: Before delving into the technical details, we give an overview of the algorithm.

Original: Overview of the algorithm.
Revised: Overview of the algorithm.

Original: We start by assuming that the algorithm has access to D LS (via
Revised: We start by assuming that the algorithm has access to D LS (via

Original: Lemma A.11), the row leverage score sampling data structure for A .
Revised: Lemma A.11), the row leverage score sampling data structure for A .

Original: In preprocessing, the algorithm samples a leverage score sketching matrix S ∈ R k × n from D LS , where k
Revised: In preprocessing, the algorithm samples a leverage score sketching matrix S ∈ R k × n from D LS , where k

Original: = O
Revised: = O

Original: ( d log d/ε 2 ) .
Revised: ( d log d/ε 2 ) .

Original: This provides a (1 + ε ) ℓ 2 subspace embedding for A .
Revised: This provides a (1 + ε ) ℓ 2 subspace embedding for A .

Original: Standard results in sketching imply that it suffices to solve for the sketched objective of min x ∈ R d
Revised: Standard results in sketching imply that it suffices to solve for the sketched objective of min x ∈ R d

Original: ∥ SAx − Sb ∥ 22 Sarlos (2006); Clarkson & Woodruff (2013; 2017); Woodruff (2014).
Revised: ∥ SAx − Sb ∥ 22 Sarlos (2006); Clarkson & Woodruff (2013; 2017); Woodruff (2014).

Original: Let (cid:98) A = SA .
Revised: Let (cid:98) A = SA .

Original: Then a (1 + ε ) optimal solution is thus given by (cid:98) A † b .
Revised: Then a (1 + ε ) optimal solution is thus given by (cid:98) A † b .

Original: Moreover, our goal is to maintain the regression cost, rather than this solution vector.
Revised: Moreover, our goal is to maintain the regression cost, rather than this solution vector.

Original: Hence, we can apply Johnson–Lindenstrauss lemma and focus on
Revised: Hence, we can apply Johnson–Lindenstrauss lemma and focus on

Original: [Equation]
Revised: [Equation]

Original: where G ∈ R O (log n/ε 2 ) × n is a JL sketch.
Revised: where G ∈ R O (log n/ε 2 ) × n is a JL sketch.

Original: Next, we describe how to track the cost value dynamically.
Revised: Next, we describe how to track the cost value dynamically.

Original: We stress that the sketching matrices S and
Revised: We stress that the sketching matrices S and

Original: G are sampled upfront in the preprocessing stage and remain fixed afterwards.
Revised: G are sampled upfront in the preprocessing stage and remain fixed afterwards.

Original: The algorithm stores
Revised: The algorithm stores

Original: G and M = GA ( SA ) † , both computed in preprocessing.
Revised: G and M = GA ( SA ) † , both computed in preprocessing.

Original: Meanwhile, it maintains Gb ( i ) ,
Revised: Meanwhile, it maintains Gb ( i ) ,

Original: Sb ( i ) , initialized at i = 1 .
Revised: Sb ( i ) , initialized at i = 1 .

Original: In the first step, given the initial target label b (1) , the algorithm computes Sb ( i ) ,
Revised: In the first step, given the initial target label b (1) , the algorithm computes Sb ( i ) ,

Original: M (cid:0) Sb (1) (cid:1) and Gb (1) .
Revised: M (cid:0) Sb (1) (cid:1) and Gb (1) .

Original: Then it outputs (cid:98) F 1 = (cid:13)(cid:13) MSb (1) − Gb (1) (cid:13)(cid:13) 2 2 as an estimate of the regression cost.
Revised: Then it outputs (cid:98) F 1 = (cid:13)(cid:13) MSb (1) − Gb (1) (cid:13)(cid:13) 2 2 as an estimate of the regression cost.

Original: Let’s consider the i -th step, where the label is updated to b ( i ) .
Revised: Let’s consider the i -th step, where the label is updated to b ( i ) .

Original: First, we read the K labels that get changed and update Sb ( i − 1) to Sb ( i ) accordingly.
Revised: First, we read the K labels that get changed and update Sb ( i − 1) to Sb ( i ) accordingly.

Original: This can be done in O ( K ) time.
Revised: This can be done in O ( K ) time.

Original: Finally, we simply compute M ( Sb ( i ) ) and Gb ( i ) and output (cid:98) F i =
Revised: Finally, we simply compute M ( Sb ( i ) ) and Gb ( i ) and output (cid:98) F i =

Original: (cid:13)(cid:13) MSb ( i )
Revised: (cid:13)(cid:13) MSb ( i )

Original: − Gb ( i ) (cid:13)(cid:13) 2 2 .
Revised: − Gb ( i ) (cid:13)(cid:13) 2 2 .

Original: We store Gb ( i ) for the next iteration.
Revised: We store Gb ( i ) for the next iteration.

Original: We now describe the algorithm formally, followed by an analysis of its run-time and accuracy.
Revised: We now describe the algorithm formally, followed by an analysis of its run-time and accuracy.

Original: Formal description of the algorithm.
Revised: Formal description of the algorithm.

Original: We assume D LS for A is given.
Revised: We assume D LS for A is given.

Original: The data structure is initialized by drawing the sketching matrices G and S .
Revised: The data structure is initialized by drawing the sketching matrices G and S .

Original: We also compute M = GSA ( SA ) † in preprocessing.
Revised: We also compute M = GSA ( SA ) † in preprocessing.

Original: This matrix is stored explicitly throughout.
Revised: This matrix is stored explicitly throughout.

Original: Algorithm 1 Initialize the data structure, i.e., preprocessing
Revised: Algorithm 1 Initialize the data structure, i.e., preprocessing

Original: Input: Design matrix A ∈ R n × d , initial label b (1) ∈ R n
Revised: Input: Design matrix A ∈ R n × d , initial label b (1) ∈ R n

Original: a (
Revised: a (

Original: score row sampling
Revised: score row sampling

Original: S
Revised: S

Original: R k × n
Revised: R k × n

Original: A from D LS .
Revised: A from D LS .

Original: Sample a JL sketch matrix G
Revised: Sample a JL sketch matrix G

Original: R Cε − 2 log n × n , for a sufficiently large C , by Theorem A.6.
Revised: R Cε − 2 log n × n , for a sufficiently large C , by Theorem A.6.

Original: Compute and store M = GA ( SA ) † .
Revised: Compute and store M = GA ( SA ) † .

Original: At each step, the algorithm computes Sb ( i ) by reading all K entries of b i − 1 that are updated in the step.
Revised: At each step, the algorithm computes Sb ( i ) by reading all K entries of b i − 1 that are updated in the step.

Original: After that, compute M ( Sb ( i ) ) and Gb ( i ) and output (cid:13)(cid:13)
Revised: After that, compute M ( Sb ( i ) ) and Gb ( i ) and output (cid:13)(cid:13)

Original: Mb ( i ) − Gb ( i ) (cid:13)(cid:13) 2 2 .
Revised: Mb ( i ) − Gb ( i ) (cid:13)(cid:13) 2 2 .

Original: The algorithm is formally given by Algorithm 2.
Revised: The algorithm is formally given by Algorithm 2.

Original: Input: Matrices G ∈ R
Revised: Input: Matrices G ∈ R

Original: Cε − 2 log n × n , S ∈ R k × n , M ∈ R (cid:101) O
Revised: Cε − 2 log n × n , S ∈ R k × n , M ∈ R (cid:101) O

Original: ( 1 /ε 2 ) × k and the label b ( i )
Revised: ( 1 /ε 2 ) × k and the label b ( i )

Original: Output: Estimate of the regression cost F (cid:0) A , b
Revised: Output: Estimate of the regression cost F (cid:0) A , b

Original: ( i ) (cid:1) 1: Compute Sb ( i ) by reading all K entries of b ( i − 1) that are updated.
Revised: ( i ) (cid:1) 1: Compute Sb ( i ) by reading all K entries of b ( i − 1) that are updated.

Original: 2: Compute M (cid:0) Sb i ) (cid:1) and Gb i ) .
Revised: 2: Compute M (cid:0) Sb i ) (cid:1) and Gb i ) .

Original: 3: Output (cid:98) F i = (cid:13)(cid:13) MSb i ) − Gb
Revised: 3: Output (cid:98) F i = (cid:13)(cid:13) MSb i ) − Gb

Original: Analysis of the algorithm.
Revised: Analysis of the algorithm.

Original: We now analyze the run-time of the algorithm.
Revised: We now analyze the run-time of the algorithm.

Original: First, consider the preprocessing stage performed by Algorithm 1.
Revised: First, consider the preprocessing stage performed by Algorithm 1.

Original: Lemma 3.1 (Preprocessing time) .
Revised: Lemma 3.1 (Preprocessing time) .

Original: Assuming access to the leverage score sampling data structure
Revised: Assuming access to the leverage score sampling data structure

Original: D LS , the preprocessing time of Algorithm 1 is
Revised: D LS , the preprocessing time of Algorithm 1 is

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By Lemma A.11, the guarantee of the sampling data structure D LS , it takes O ( k log( nd )) time to obtain a leverage score sample S of size k .
Revised: By Lemma A.11, the guarantee of the sampling data structure D LS , it takes O ( k log( nd )) time to obtain a leverage score sample S of size k .

Original: Drawing the JL sketch is straightforward, and standard constructions such as i.i.d.
Revised: Drawing the JL sketch is straightforward, and standard constructions such as i.i.d.

Original: Gaussian entries require O ( k log n/ε 2 ) times to form G .
Revised: Gaussian entries require O ( k log n/ε 2 ) times to form G .

Original: Finally, we need to compute M .
Revised: Finally, we need to compute M .

Original: Computing GA requires O (cid:16) nnz( A ) εlog n (cid:17) time by sparse matrix multiplication.
Revised: Computing GA requires O (cid:16) nnz( A ) εlog n (cid:17) time by sparse matrix multiplication.

Original: Moreover, since GA is a matrix of O (cid:16) log n ε 2 (cid:17) rows, then computing ( GA )( SA ) † reduces to O (cid:16) log n ε 2 (cid:17) number of linear system solves with respect to SA ∈ R k × d .
Revised: Moreover, since GA is a matrix of O (cid:16) log n ε 2 (cid:17) rows, then computing ( GA )( SA ) † reduces to O (cid:16) log n ε 2 (cid:17) number of linear system solves with respect to SA ∈ R k × d .

Original: By conjugate gradient type methods, since κ ( SA ) = (1 ± ε ) κ
Revised: By conjugate gradient type methods, since κ ( SA ) = (1 ± ε ) κ

Original: ( A ) , each solve can be achieved to high accuracy in O (cid:16)(cid:112) κ
Revised: ( A ) , each solve can be achieved to high accuracy in O (cid:16)(cid:112) κ

Original: ( A ) log(1 /ε )
Revised: ( A ) log(1 /ε )

Original: (cid:17) number of matrix-vector products with respect to A Golub & Van Loan (2013).
Revised: (cid:17) number of matrix-vector products with respect to A Golub & Van Loan (2013).

Original: In total, this gives a run-time of O (cid:16)(cid:112) κ ( A ) nnz( A ) log(1 /ε ) (cid:17) .
Revised: In total, this gives a run-time of O (cid:16)(cid:112) κ ( A ) nnz( A ) log(1 /ε ) (cid:17) .

Original: Lemma 3.2 (Update time) .
Revised: Lemma 3.2 (Update time) .

Original: The update time of Algorithm 2 is O (cid:0)
Revised: The update time of Algorithm 2 is O (cid:0)

Original: Kε 2 log n (cid:1) per step.
Revised: Kε 2 log n (cid:1) per step.

Original: Proof.
Revised: Proof.

Original: First, the algorithm reads the K entries that are updated and compute the Sb ( i ) from Sb ( i − 1) .
Revised: First, the algorithm reads the K entries that are updated and compute the Sb ( i ) from Sb ( i − 1) .

Original: This step takes O ( K ) time, since we just need to update the entries that lie in the support of the row sampling matrix S .
Revised: This step takes O ( K ) time, since we just need to update the entries that lie in the support of the row sampling matrix S .

Original: Similarly, in step 2 of Algorithm 2 we can update Gb ( i − 1) to Gb ( i ) in
Revised: Similarly, in step 2 of Algorithm 2 we can update Gb ( i − 1) to Gb ( i ) in

Original: O ( K log n/ε 2 ) time.
Revised: O ( K log n/ε 2 ) time.

Original: Since S is a row sampling matrix and b ( i ) only has K entries updated, then
Revised: Since S is a row sampling matrix and b ( i ) only has K entries updated, then

Original: Sb ( i ) has at most K entries updated as well.
Revised: Sb ( i ) has at most K entries updated as well.

Original: It follows that given M (cid:0)
Revised: It follows that given M (cid:0)

Original: Sb ( i − 1) (cid:1) from the prior round,
Revised: Sb ( i − 1) (cid:1) from the prior round,

Original: M (cid:0) Sb ( i ) (cid:1) can be updated in O (cid:0)
Revised: M (cid:0) Sb ( i ) (cid:1) can be updated in O (cid:0)

Original: Kε 2 log n (cid:1) time.
Revised: Kε 2 log n (cid:1) time.

Original: Lemma 3.3 (Accuracy) .
Revised: Lemma 3.3 (Accuracy) .

Original: Given a stream of T =
Revised: Given a stream of T =

Original: O ( d 2 ) non-adaptive updates and error parameter ε ∈ (0 , 1 / 4) , Algorithm 2 outputs an estimate (cid:98) F i of the regression cost F ( A , b ( i ) )
Revised: O ( d 2 ) non-adaptive updates and error parameter ε ∈ (0 , 1 / 4) , Algorithm 2 outputs an estimate (cid:98) F i of the regression cost F ( A , b ( i ) )

Original: such that (cid:98) F i =
Revised: such that (cid:98) F i =

Original: (1 ± ε ) F ( A , b ( i ) ) for all i with high probability.
Revised: (1 ± ε ) F ( A , b ( i ) ) for all i with high probability.

Original: Proof.
Revised: Proof.

Original: First, we apply the subspace embedding property of S .
Revised: First, we apply the subspace embedding property of S .

Original: This implies that with high probability,
Revised: This implies that with high probability,

Original: [Equation]
Revised: [Equation]

Original: Apply the JL lemma (Theorem A.6), where we consider the collection of O (cid:0) d 2 (cid:1) (1 + ε ) optimal predictions { y ∗ i } Ti =1 with y ∗ i
Revised: Apply the JL lemma (Theorem A.6), where we consider the collection of O (cid:0) d 2 (cid:1) (1 + ε ) optimal predictions { y ∗ i } Ti =1 with y ∗ i

Original: = A ( SA ) † b ( i ) .
Revised: = A ( SA ) † b ( i ) .

Original: Via union bound, we have that with high probability for all i ∈
Revised: Via union bound, we have that with high probability for all i ∈

Original: [ T ]
Revised: [ T ]

Original: [Equation]
Revised: [Equation]

Original: Our algorithm precisely solves for y ∗ i each iteration.
Revised: Our algorithm precisely solves for y ∗ i each iteration.

Original: Combining the two equations above finishes the proof.
Revised: Combining the two equations above finishes the proof.

Original: 3. D YNAMIC A LGORITHM WITH A DVERSARIAL R OBUSTNESS
Revised: 3. D YNAMIC A LGORITHM WITH A DVERSARIAL R OBUSTNESS

Original: To put everything together and ensure adversarial robustness, we use a standard approach of
Revised: To put everything together and ensure adversarial robustness, we use a standard approach of

Original: Hassidim et al.
Revised: Hassidim et al.

Original: Beimel et al.
Revised: Beimel et al.

Original: ; Attias et al.
Revised: ; Attias et al.

Original: Our full algorithm maintains Γ =
Revised: Our full algorithm maintains Γ =

Original: O (cid:16) √ T log( nT ) (cid:17) independent copies of the key subroutine for T
Revised: O (cid:16) √ T log( nT ) (cid:17) independent copies of the key subroutine for T

Original: = O (cid:16) nnz( A ) ε 2 K (cid:17) .
Revised: = O (cid:16) nnz( A ) ε 2 K (cid:17) .

Original: Then at each step, we output the private median of the outputs of these copies.
Revised: Then at each step, we output the private median of the outputs of these copies.

Original: Advanced composition of DP ensures robustness up to T rounds.
Revised: Advanced composition of DP ensures robustness up to T rounds.

Original: Afterwards, the algorithm reboots by rebuilding the copies, using fresh randomness independently for sampling and computing the sketching matrices.
Revised: Afterwards, the algorithm reboots by rebuilding the copies, using fresh randomness independently for sampling and computing the sketching matrices.

Original: independent instances structure
Revised: independent instances structure

Original: Algorithm 1.
Revised: Algorithm 1.

Original: PrivMed the Γ instances with privacy parameter
Revised: PrivMed the Γ instances with privacy parameter

Original: = O (cid:16)
Revised: = O (cid:16)

Original: √ log( probability δ = 1 poly ( m,T ) .
Revised: √ log( probability δ = 1 poly ( m,T ) .

Original: For each query, return the output of PrivMed .
Revised: For each query, return the output of PrivMed .

Original: Theorem 3.4.
Revised: Theorem 3.4.

Original: [Main theorem; dynamic maintenance of regression cost] Let ε ∈ (0 , 1 / 4) be an error parameter and b (1) be the initial target label.
Revised: [Main theorem; dynamic maintenance of regression cost] Let ε ∈ (0 , 1 / 4) be an error parameter and b (1) be the initial target label.

Original: Given ε, A , b (1) , a stream of T adaptively chosen,
Revised: Given ε, A , b (1) , a stream of T adaptively chosen,

Original: K -sparse updates to the label, Algorithm 4 outputs an estimate (cid:98) F i such that (cid:98) F i
Revised: K -sparse updates to the label, Algorithm 4 outputs an estimate (cid:98) F i such that (cid:98) F i

Original: =
Revised: =

Original: (1 ± ε ) F ( A , b ( i ) ) for all i with high probability.
Revised: (1 ± ε ) F ( A , b ( i ) ) for all i with high probability.

Original: Furthermore, the algorithm requires a preprocessing step in time (cid:101)
Revised: Furthermore, the algorithm requires a preprocessing step in time (cid:101)

Original: O
Revised: O

Original: (nnz( A ) + poly( d )) .
Revised: (nnz( A ) + poly( d )) .

Original: The amortized update time of the algorithm is
Revised: The amortized update time of the algorithm is

Original: [Equation]
Revised: [Equation]

Original: per round.
Revised: per round.

Original: We defer the proof of Theorem 3.4 and a discussion on a deterministic algorithm to Section B.
Revised: We defer the proof of Theorem 3.4 and a discussion on a deterministic algorithm to Section B.

Original: A CKNOWLEDGEMENTS
Revised: A CKNOWLEDGEMENTS

Original: Sandeep Silwal is supported by an NSF Graduate Research Fellowship under Grant No. 1745302, and
Revised: Sandeep Silwal is supported by an NSF Graduate Research Fellowship under Grant No. 1745302, and

Original: NSF TRIPODS program (award DMS-2022448), NSF award CCF-2006798, and Simons Investigator
Revised: NSF TRIPODS program (award DMS-2022448), NSF award CCF-2006798, and Simons Investigator

Original: Award (via Piotr Indyk).
Revised: Award (via Piotr Indyk).

Original:  David P. Woodruff is partially supported  by a Simons Investigator Award and by the National Science Foundation under Grant No.
Revised: This work was done in part while David P. Woodruff was at Google Research and supported in part by a Simons Investigator Award and by the National Science Foundation under Grant No.

Original: CCF-1815840.
Revised: CCF-1815840.

Original: Fred Zhang is supported by ONR grant N00014-18-1-2562.
Revised: Fred Zhang is supported by ONR grant N00014-18-1-2562.

Original: This work was done in part while Samson Zhou was at Carnegie Mellon University and supported in part by a Simons Investigator Award and by the National Science Foundation under Grant No.
Revised: This work was done in part while Samson Zhou was at Carnegie Mellon University and supported in part by a Simons Investigator Award and by the National Science Foundation under Grant No.

Original: CCF-1815840.
Revised: CCF-1815840.

Original: A P RELIMINARIES
Revised: A P RELIMINARIES

Original: Notations In this paper, we use [ n ] for a positive integer n > 0 to denote the set { 1 , . . . , n } .
Revised: Notations In this paper, we use [ n ] for a positive integer n > 0 to denote the set { 1 , . . . , n } .

Original: We use poly( n ) to denote a fixed polynomial in n .
Revised: We use poly( n ) to denote a fixed polynomial in n .

Original: We say an event occurs with high probability if it occurs with probability 1 − 1 poly( n ) .
Revised: We say an event occurs with high probability if it occurs with probability 1 − 1 poly( n ) .

Original: For real numbers a, b and positive ε , we say a = (1 ± ε ) b if (1 − ε ) b ≤ a ≤ (1 + ε ) b .
Revised: For real numbers a, b and positive ε , we say a = (1 ± ε ) b if (1 − ε ) b ≤ a ≤ (1 + ε ) b .

Original: Let e i ∈
Revised: Let e i ∈

Original: R
Revised: R

Original: n be the i ’th standard basis vector.
Revised: n be the i ’th standard basis vector.

Original: Let X + denote the Moore-Penrose pseudo-inverse of matrix X .
Revised: Let X + denote the Moore-Penrose pseudo-inverse of matrix X .

Original: Let ∥ X ∥ denote the operator norm of X .
Revised: Let ∥ X ∥ denote the operator norm of X .

Original: Let κ ( X ) =
Revised: Let κ ( X ) =

Original: ∥ X + ∥
Revised: ∥ X + ∥

Original: ∥
Revised: ∥

Original: X ∥ denote the condition number of X .
Revised: X ∥ denote the condition number of X .

Original: A. D IFFERENTIAL P RIVACY
Revised: A. D IFFERENTIAL P RIVACY

Original: Much of our technical results leverage tools from DP.
Revised: Much of our technical results leverage tools from DP.

Original: We recall its definition and several key statements.
Revised: We recall its definition and several key statements.

Original: Definition A.1 (Differential privacy, Dwork et al.
Revised: Definition A.1 (Differential privacy, Dwork et al.

Original: Given ε > 0 and δ ∈ (0 , 1) , a randomized algorithm A : X ∗ → Y is ( ε, δ ) -differentially private if, for every neighboring datasets S and S ′ and for all E ⊆ Y ,
Revised: Given ε > 0 and δ ∈ (0 , 1) , a randomized algorithm A : X ∗ → Y is ( ε, δ ) -differentially private if, for every neighboring datasets S and S ′ and for all E ⊆ Y ,

Original: [Equation]
Revised: [Equation]

Original: Theorem A.2 (Amplification via sampling, e.g., Bun et al.
Revised: Theorem A.2 (Amplification via sampling, e.g., Bun et al.

Original: Let A be an ( ε, δ ) -differentially private algorithm for ε ≤ 1 , δ ∈ (0 , 1) .
Revised: Let A be an ( ε, δ ) -differentially private algorithm for ε ≤ 1 , δ ∈ (0 , 1) .

Original: Given a database S of size n , let A ′ be the algorithm that constructs a database T ⊂ S by subsampling (with replacement) s ≤ n 2 rows of S and outputs A ( T ) .
Revised: Given a database S of size n , let A ′ be the algorithm that constructs a database T ⊂ S by subsampling (with replacement) s ≤ n 2 rows of S and outputs A ( T ) .

Original: Then A ′ is ( ε ′ , δ ′ )
Revised: Then A ′ is ( ε ′ , δ ′ )

Original: -differentially private for
Revised: -differentially private for

Original: [Equation]
Revised: [Equation]

Original: Theorem A.3 (Private median, e.g., Hassidim et al.
Revised: Theorem A.3 (Private median, e.g., Hassidim et al.

Original: Given a database D ∈ X ∗ , there exists an ( ε, 0) -differentially private algorithm PrivMed that outputs an element x ∈ X such that with probability at least 1 − δ , there are at least | S | 2 − k elements in S that are at least x , and at least | S | 2 − k elements in S in S that are at most x , for k
Revised: Given a database D ∈ X ∗ , there exists an ( ε, 0) -differentially private algorithm PrivMed that outputs an element x ∈ X such that with probability at least 1 − δ , there are at least | S | 2 − k elements in S that are at least x , and at least | S | 2 − k elements in S in S that are at most x , for k

Original: = O (cid:16) 1 ε log | X | δ (cid:17) .
Revised: = O (cid:16) 1 ε log | X | δ (cid:17) .

Original: Theorem A.4 (Advanced composition, e.g., Dwork et al. (2010)) .
Revised: Theorem A.4 (Advanced composition, e.g., Dwork et al. (2010)) .

Original: Let ε, δ ′ ∈ (0 , 1] and let δ ∈ [0 , 1] .
Revised: Let ε, δ ′ ∈ (0 , 1] and let δ ∈ [0 , 1] .

Original: Any mechanism that permits k adaptive interactions with mechanisms that preserve ( ε, δ ) -differential privacy guarantees ( ε ′ , kδ + δ ′ )
Revised: Any mechanism that permits k adaptive interactions with mechanisms that preserve ( ε, δ ) -differential privacy guarantees ( ε ′ , kδ + δ ′ )

Original: -differential privacy, where ε ′ =
Revised: -differential privacy, where ε ′ =

Original: (cid:113) 

Revised: (cid:113)

Original: k
Revised: k

Original: ln 1 δ ′ · ε + 2 kε 2 .
Revised: ln 1 δ ′ · ε + 2 kε 2 .

Original: Theorem A.5 (Generalization of DP, e.g., Dwork et al.
Revised: Theorem A.5 (Generalization of DP, e.g., Dwork et al.

Original: ; Bassily et al.
Revised: ; Bassily et al.

Original: Let ε ∈
Revised: Let ε ∈

Original: (0 , 1 / 3) , δ ∈ (0 , ε/ 4) , and n ≥ 1 ε 2 log 2 εδ .
Revised: (0 , 1 / 3) , δ ∈ (0 , ε/ 4) , and n ≥ 1 ε 2 log 2 εδ .

Original: Suppose A : X n → 2 X is an ( ε, δ ) -differentially private algorithm that curates a database of size n and produces a function h : X → { 0 , 1 } .
Revised: Suppose A : X n → 2 X is an ( ε, δ ) -differentially private algorithm that curates a database of size n and produces a function h : X → { 0 , 1 } .

Original: Suppose D is a distribution over X and S is a set of n elements drawn independently and identically distributed from D .
Revised: Suppose D is a distribution over X and S is a set of n elements drawn independently and identically distributed from D .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: A. N UMERICAL L INEAR A LGEBRA
Revised: A. N UMERICAL L INEAR A LGEBRA

Original: Our results on dynamic regression relies upon some tools in numerical linear algebra.
Revised: Our results on dynamic regression relies upon some tools in numerical linear algebra.

Original: We first recall the dimensionality reduction techniques.
Revised: We first recall the dimensionality reduction techniques.

Original: Theorem A.6 (Johnson-Lindenstrauss transformation, ε -JL) .
Revised: Theorem A.6 (Johnson-Lindenstrauss transformation, ε -JL) .

Original: Given ε > 0 , there exists a family of random maps Π m,d ∈ R m ×
Revised: Given ε > 0 , there exists a family of random maps Π m,d ∈ R m ×

Original: d
Revised: d

Original: with m =
Revised: with m =

Original: O (cid:0) 

Revised: O (cid:0)

Original: ε 2 (cid:1) such that for any x ∈ R d , we have
Revised: ε 2 (cid:1) such that for any x ∈ R d , we have

Original: [Equation]
Revised: [Equation]

Original: Moreover, Π x takes O (cid:0) dε 2 (cid:1) time to compute.
Revised: Moreover, Π x takes O (cid:0) dε 2 (cid:1) time to compute.

Original: Theorem A.7 (Fast JL) .
Revised: Theorem A.7 (Fast JL) .

Original: Given ε > 0 , there exists a family of random maps Π m,d ∈ R m ×
Revised: Given ε > 0 , there exists a family of random maps Π m,d ∈ R m ×

Original: d
Revised: d

Original: with m =
Revised: with m =

Original: O (cid:16) log d ε 2 (cid:17) such that for any x ∈ R d , we have
Revised: O (cid:16) log d ε 2 (cid:17) such that for any x ∈ R d , we have

Original: [Equation]
Revised: [Equation]

Original: Moreover, Π x takes O (cid:16) log d
Revised: Moreover, Π x takes O (cid:16) log d

Original: ε 2 + d log d (cid:17) time to compute.
Revised: ε 2 + d log d (cid:17) time to compute.

Original: A row sampling matrix S has rows that are multiples of natural basis vectors, so that SA is a (weighted) sample of the rows of A .
Revised: A row sampling matrix S has rows that are multiples of natural basis vectors, so that SA is a (weighted) sample of the rows of A .

Original: A column sampling matrix is defined similarly.
Revised: A column sampling matrix is defined similarly.

Original: The size of a row/column sampling matrix is defined as the number of rows/columns it samples.
Revised: The size of a row/column sampling matrix is defined as the number of rows/columns it samples.

Original: The leverage score of the i th row a ⊤ i of A is
Revised: The leverage score of the i th row a ⊤ i of A is

Original: [Equation]
Revised: [Equation]

Original: For a survey on leverage score and applications, we refer the reader to Mahoney (2011).
Revised: For a survey on leverage score and applications, we refer the reader to Mahoney (2011).

Original: Definition A.8 (Leverage score sampling) .
Revised: Definition A.8 (Leverage score sampling) .

Original: Let u be a vector of leverage score overestimates, i.e., τ i ( A ) ≤ u i .
Revised: Let u be a vector of leverage score overestimates, i.e., τ i ( A ) ≤ u i .

Original: Let α be a sampling rate parameter and
Revised: Let α be a sampling rate parameter and

Original: c be a fixed positive constant.
Revised: c be a fixed positive constant.

Original: For each row, we define a sampling probability p i = min { 1 , α · u i c log d } .
Revised: For each row, we define a sampling probability p i = min { 1 , α · u i c log d } .

Original: The leverage score sampling matrix is a row sampling matrix S with independently chosen entries such that S ii = 1 √ p i with probability p
Revised: The leverage score sampling matrix is a row sampling matrix S with independently chosen entries such that S ii = 1 √ p i with probability p

Original: i and 0 otherwise.
Revised: i and 0 otherwise.

Original: Definition A.9 (Subspace embedding) .
Revised: Definition A.9 (Subspace embedding) .

Original: A (1 ± ε ) ℓ 2 subspace embedding for the column space of an n × d matrix A is a matrix S for which for all x ∈ R d
Revised: A (1 ± ε ) ℓ 2 subspace embedding for the column space of an n × d matrix A is a matrix S for which for all x ∈ R d

Original: [Equation]
Revised: [Equation]

Original: Theorem A.10 (Leverage sampling implies subspace embedding, Theorem 17 of Woodruff (2014)) .
Revised: Theorem A.10 (Leverage sampling implies subspace embedding, Theorem 17 of Woodruff (2014)) .

Original: Let α = ε − 2 and c be a sufficiently large constant.
Revised: Let α = ε − 2 and c be a sufficiently large constant.

Original: With high probability, the leverage score sampling matrix is a (1 ± ε ) ℓ 2 subspace embedding.
Revised: With high probability, the leverage score sampling matrix is a (1 ± ε ) ℓ 2 subspace embedding.

Original: Furthermore, it has size O (cid:0) d log d/ε 2 (cid:1) .
Revised: Furthermore, it has size O (cid:0) d log d/ε 2 (cid:1) .

Original: The approximate leverage scores can be computed in input-sparsity time.
Revised: The approximate leverage scores can be computed in input-sparsity time.

Original: Afterwards, repeated sampling from the leverage score distribution can be done efficiently using the binary tree data structure in quantum-inspired numerical linear algebra.
Revised: Afterwards, repeated sampling from the leverage score distribution can be done efficiently using the binary tree data structure in quantum-inspired numerical linear algebra.

Original: Lemma A.11 (Leverage score computation and sampling data structure; see Woodruff (2014);
Revised: Lemma A.11 (Leverage score computation and sampling data structure; see Woodruff (2014);

Original: Chepurko et al.
Revised: Chepurko et al.

Original: Let A ∈ R n × d .
Revised: Let A ∈ R n × d .

Original: There exists an algorithm that given A outputs a vector of row leverage score overestimates with high probability and in run-time (cid:101) O (nnz( A ) + poly( d )) .
Revised: There exists an algorithm that given A outputs a vector of row leverage score overestimates with high probability and in run-time (cid:101) O (nnz( A ) + poly( d )) .

Original: Furthermore, there exists a sampling data structure D LS that stores the row leverage scores of
Revised: Furthermore, there exists a sampling data structure D LS that stores the row leverage scores of

Original: A such that given a positive integer m ≤ n , returns a leverage score sample of A of size m in
Revised: A such that given a positive integer m ≤ n , returns a leverage score sample of A of size m in

Original: O ( m log( mn )) time.
Revised: O ( m log( mn )) time.

Original: In total, the pre-processing takes O (nnz( A ) + poly( d )) time.
Revised: In total, the pre-processing takes O (nnz( A ) + poly( d )) time.

Original: B A DDITIONAL D ETAILS ON D YNAMIC R EGRESSION
Revised: B A DDITIONAL D ETAILS ON D YNAMIC R EGRESSION

Original: Theorem 3.4.
Revised: Theorem 3.4.

Original: [Main theorem; dynamic maintenance of regression cost] Let ε ∈ (0 , 1 / 4) be an error parameter and b (1) be the initial target label.
Revised: [Main theorem; dynamic maintenance of regression cost] Let ε ∈ (0 , 1 / 4) be an error parameter and b (1) be the initial target label.

Original: Given ε, A , b (1) , a stream of T adaptively chosen,
Revised: Given ε, A , b (1) , a stream of T adaptively chosen,

Original: K -sparse updates to the label, Algorithm 4 outputs an estimate (cid:98) F i such that (cid:98) F i
Revised: K -sparse updates to the label, Algorithm 4 outputs an estimate (cid:98) F i such that (cid:98) F i

Original: =
Revised: =

Original: (1 ± ε ) F ( A , b ( i ) ) for all i with high probability.
Revised: (1 ± ε ) F ( A , b ( i ) ) for all i with high probability.

Original: Furthermore, the algorithm requires a preprocessing step in time (cid:101)
Revised: Furthermore, the algorithm requires a preprocessing step in time (cid:101)

Original: O
Revised: O

Original: (nnz( A ) + poly( d )) .
Revised: (nnz( A ) + poly( d )) .

Original: The amortized update time of the algorithm is
Revised: The amortized update time of the algorithm is

Original: [Equation]
Revised: [Equation]

Original: per round.
Revised: per round.

Original: Proof.
Revised: Proof.

Original: We focus on any fixed epoch of T iterations.
Revised: We focus on any fixed epoch of T iterations.

Original: Let {A i } Γ i =1 be the collection of Γ data structures maintained by the Algorithm 4 and T i be the transcript between Algorithm 4 and the adversary at round i , consisting of the algorithm’s output and the update requested by the adversary.
Revised: Let {A i } Γ i =1 be the collection of Γ data structures maintained by the Algorithm 4 and T i be the transcript between Algorithm 4 and the adversary at round i , consisting of the algorithm’s output and the update requested by the adversary.

Original: To handle a sequence of T adaptive queries, consider the transcript T ( R )
Revised: To handle a sequence of T adaptive queries, consider the transcript T ( R )

Original: = {T 1 , . . .
Revised: = {T 1 , . . .

Original: , T T } , where R denotes the internal randomness of Algorithm 4.
Revised: , T T } , where R denotes the internal randomness of Algorithm 4.

Original: Note that for a fixed iteration, T i is (cid:16)
Revised: Note that for a fixed iteration, T i is (cid:16)

Original: O (cid:16)
Revised: O (cid:16)

Original: [Equation]
Revised: [Equation]

Original: (cid:17) , 0 (cid:17) -differentially private with respect to the algorithms
Revised: (cid:17) , 0 (cid:17) -differentially private with respect to the algorithms

Original: A 1 , . . . , A Γ , since the private median algorithm PrivMed is (cid:16) O (cid:16) 1 √ T log( nT ) (cid:17) , 0 (cid:17) -differentially private.
Revised: A 1 , . . . , A Γ , since the private median algorithm PrivMed is (cid:16) O (cid:16) 1 √ T log( nT ) (cid:17) , 0 (cid:17) -differentially private.

Original: By the advanced composition of differential privacy, i.e., Theorem A.4, the transcript T is (cid:16) O (1) ,
Revised: By the advanced composition of differential privacy, i.e., Theorem A.4, the transcript T is (cid:16) O (1) ,

Original: [Equation]
Revised: [Equation]

Original: poly
Revised: poly

Original: [Equation]
Revised: [Equation]

Original: -differentially private with respect to the algorithms
Revised: -differentially private with respect to the algorithms

Original: A 1 , . . . , A Γ .
Revised: A 1 , . . . , A Γ .

Original: Algorithm 4 runs Γ instances of the data structure with error parameter ε .
Revised: Algorithm 4 runs Γ instances of the data structure with error parameter ε .

Original: For any given round i ∈ [ T ] , we say that an instance j ∈ [Γ] is correct if its output f i,j is within a (1 ± ε ) factor of F ( A , b ( i ) ) and incorrect otherwise.
Revised: For any given round i ∈ [ T ] , we say that an instance j ∈ [Γ] is correct if its output f i,j is within a (1 ± ε ) factor of F ( A , b ( i ) ) and incorrect otherwise.

Original: For a fixed i , let Y j be the indicator variable for whether f i,j is correct.
Revised: For a fixed i , let Y j be the indicator variable for whether f i,j is correct.

Original: From the generalization properties of differential privacy, i.e., Theorem A.5, we have that for any fixed iteration i ,
Revised: From the generalization properties of differential privacy, i.e., Theorem A.5, we have that for any fixed iteration i ,

Original: Pr
Revised: Pr

Original: [Equation]
Revised: [Equation]

Original: where Y denotes the indicator random variable for whether a random instance of the algorithm A (not necessarily restricted to the m instances maintained by the algorithm) is correct at the given round i .
Revised: where Y denotes the indicator random variable for whether a random instance of the algorithm A (not necessarily restricted to the m instances maintained by the algorithm) is correct at the given round i .

Original: Since a random instance A has randomness that is independent of the adaptive update, then E [ Y ] ≥ 34 .
Revised: Since a random instance A has randomness that is independent of the adaptive update, then E [ Y ] ≥ 34 .

Original: Therefore, by a union bound over all T rounds, we have
Revised: Therefore, by a union bound over all T rounds, we have

Original: [Equation]
Revised: [Equation]

Original: which implies that the output on the i th round is correct with probability at least 1 − 1 poly( m,T ) , since
Revised: which implies that the output on the i th round is correct with probability at least 1 − 1 poly( m,T ) , since

Original: T = d .
Revised: T = d .

Original: Then by a union bound over i ∈ [ T ] for all T rounds within an epoch, we have that the data structure answers all T queries with probability 1 − 1 m 2 , under the adaptively chosen updates.
Revised: Then by a union bound over i ∈ [ T ] for all T rounds within an epoch, we have that the data structure answers all T queries with probability 1 − 1 m 2 , under the adaptively chosen updates.

Original: Finally, by a union bound over all m updates, we have that the algorithm succeeds with probability at least 1 − 1 m .
Revised: Finally, by a union bound over all m updates, we have that the algorithm succeeds with probability at least 1 − 1 m .

Original: We now analyze the run-time of the algorithm.
Revised: We now analyze the run-time of the algorithm.

Original: The preprocessing time follows from the guarantee of Lemma A.11.
Revised: The preprocessing time follows from the guarantee of Lemma A.11.

Original: For update time, we amortize over each epoch.
Revised: For update time, we amortize over each epoch.

Original: Within an epoch, we invoke Γ =
Revised: Within an epoch, we invoke Γ =

Original: O (cid:16) √ T log( nT ) (cid:17) copies of the data structure in Section 3.2, and so we consider the preprocessing and update time from there and amortize over the epoch length T .
Revised: O (cid:16) √ T log( nT ) (cid:17) copies of the data structure in Section 3.2, and so we consider the preprocessing and update time from there and amortize over the epoch length T .

Original: By Lemma 3.1, each copy takes β
Revised: By Lemma 3.1, each copy takes β

Original: =
Revised: =

Original: O (cid:16)(cid:112)
Revised: O (cid:16)(cid:112)

Original: [Equation]
Revised: [Equation]

Original: A
Revised: A

Original: [Equation]
Revised: [Equation]

Original: log
Revised: log

Original: [Equation]
Revised: [Equation]

Original: nnz
Revised: nnz

Original: [Equation]
Revised: [Equation]

Original: log n + dε 2 log n (cid:17) time to pre-process.
Revised: log n + dε 2 log n (cid:17) time to pre-process.

Original: For every step of update, each copy takes O (cid:0)
Revised: For every step of update, each copy takes O (cid:0)

Original: Kε 2 log n (cid:1) time by Lemma 3.2.
Revised: Kε 2 log n (cid:1) time by Lemma 3.2.

Original: Therefore, the amortized update time for every epoch of length T
Revised: Therefore, the amortized update time for every epoch of length T

Original: =
Revised: =

Original: O (cid:16) nnz( A ) ε 2 K (cid:17) is
Revised: O (cid:16) nnz( A ) ε 2 K (cid:17) is

Original: [Equation]
Revised: [Equation]

Original: This completes the proof.
Revised: This completes the proof.

Original: B. A N E XACT AND D ETERMINISTIC A LGORITHM
Revised: B. A N E XACT AND D ETERMINISTIC A LGORITHM

Original: We now give a simple deterministic algorithm for the dynamic regression problem based on an SVD trick.
Revised: We now give a simple deterministic algorithm for the dynamic regression problem based on an SVD trick.

Original: Let A =
Revised: Let A =

Original: UΣV ⊤ be the SVD of A , where U ∈ R n × d , Σ ∈ R d × d and V ∈ R d × d .
Revised: UΣV ⊤ be the SVD of A , where U ∈ R n × d , Σ ∈ R d × d and V ∈ R d × d .

Original: The starting observation is that for any solution vector x , we can write the regression cost as
Revised: The starting observation is that for any solution vector x , we can write the regression cost as

Original: [Equation]
Revised: [Equation]

Original: since U is orthonormal.
Revised: since U is orthonormal.

Original: The goal is the maintain the solution vector x
Revised: The goal is the maintain the solution vector x

Original: = A † b and the associated right-side quantity (cid:13)(cid:13)
Revised: = A † b and the associated right-side quantity (cid:13)(cid:13)

Original: ΣV ⊤ x − U ⊤ b (cid:13)(cid:13) .
Revised: ΣV ⊤ x − U ⊤ b (cid:13)(cid:13) .

Original: Now suppose we compute
Revised: Now suppose we compute

Original: A † ∈ R d × n and U ⊤ ∈ R d × n in pre-processing, and A † b (1) and U ⊤ b (1) in the first round.
Revised: A † ∈ R d × n and U ⊤ ∈ R d × n in pre-processing, and A † b (1) and U ⊤ b (1) in the first round.

Original: Then since all subsequent updates to b are all K -sparse, we only pay O ( dK ) time per step to maintain A † b ( i ) and U ⊤ b ( i ) .
Revised: Then since all subsequent updates to b are all K -sparse, we only pay O ( dK ) time per step to maintain A † b ( i ) and U ⊤ b ( i ) .

Original: The algorithm is formally given by Algorithm 5.
Revised: The algorithm is formally given by Algorithm 5.

Original: Observe that the algorithm always maintains the exact optimal regression cost.
Revised: Observe that the algorithm always maintains the exact optimal regression cost.

Original: Moreover, the procedure does not require any randomness, and therefore it is adversarially robust to adaptive inputs.
Revised: Moreover, the procedure does not require any randomness, and therefore it is adversarially robust to adaptive inputs.

Original: We formally claim the following guarantees of the algorithm.
Revised: We formally claim the following guarantees of the algorithm.

Original: Theorem B.1 (Deterministic maintenance of regression costs) .
Revised: Theorem B.1 (Deterministic maintenance of regression costs) .

Original: Given A , b (1) and a stream of adaptively chosen, K -sparse updates to the label, Algorithm 5 takes O ( dK ) time to update and maintain the exact regression cost F ( A , b ( i ) ) at all iterations i .
Revised: Given A , b (1) and a stream of adaptively chosen, K -sparse updates to the label, Algorithm 5 takes O ( dK ) time to update and maintain the exact regression cost F ( A , b ( i ) ) at all iterations i .

Original: The pre-processing requires an SVD of A , in O ( n 2 d ) time.
Revised: The pre-processing requires an SVD of A , in O ( n 2 d ) time.

Original: C A F RAMEWORK FOR A DVERSARIAL R OBUSTNESS
Revised: C A F RAMEWORK FOR A DVERSARIAL R OBUSTNESS

Original: In this section, we describe the benchmark framework that enables Q adaptive queries to a data structure by using (cid:101) O (cid:0) √ Q (cid:1) copies of a non-adaptive data structure.
Revised: In this section, we describe the benchmark framework that enables Q adaptive queries to a data structure by using (cid:101) O (cid:0) √ Q (cid:1) copies of a non-adaptive data structure.

Original: The framework and corresponding analysis of correctness are simply compartmentalizations of the techniques in Hassidim et al.
Revised: The framework and corresponding analysis of correctness are simply compartmentalizations of the techniques in Hassidim et al.

Original: Beimel et al.
Revised: Beimel et al.

Original: ; Attias et al.
Revised: ; Attias et al.

Original: For the sake of completeness, we include them here and discuss additional applications.
Revised: For the sake of completeness, we include them here and discuss additional applications.

Original: Namely, we show that through advanced composition of differential privacy, the private median of (cid:101) O (cid:0) √ Q (cid:1) copies protects the internal randomness of each non-adaptive data structure while still adding sufficiently small noise to guarantee accuracy.
Revised: Namely, we show that through advanced composition of differential privacy, the private median of (cid:101) O (cid:0) √ Q (cid:1) copies protects the internal randomness of each non-adaptive data structure while still adding sufficiently small noise to guarantee accuracy.

Original: Moreover, we use amplification of privacy by sampling to only consider a small subset of the (cid:101) O (cid:0) √ Q (cid:1) non-adaptive data structures to further improve the runtime.
Revised: Moreover, we use amplification of privacy by sampling to only consider a small subset of the (cid:101) O (cid:0) √ Q (cid:1) non-adaptive data structures to further improve the runtime.

Original: 1: r ← O (cid:0) √ Q log 2 ( nQ ) (cid:1) , k ← O (log( nQ )) 2: for i ∈ [ r ] do 3:
Revised: 1: r ← O (cid:0) √ Q log 2 ( nQ ) (cid:1) , k ← O (log( nQ )) 2: for i ∈ [ r ] do 3:

Original: Implement data structure D i on the input 4: for each query q i , i ∈
Revised: Implement data structure D i on the input 4: for each query q i , i ∈

Original: [ Q ] do 5:
Revised: [ Q ] do 5:

Original: Let S be a set of k indices sampled (with replacement) from [ r ] 6:
Revised: Let S be a set of k indices sampled (with replacement) from [ r ] 6:

Original: For each j ∈ [ k ] , let d i,j be the output of D S j on query q i 7: d i ← PrivMed ( { d i,j } j ∈ [ k ] ) , where PrivMed is (1 , 0) -DP
Revised: For each j ∈ [ k ] , let d i,j be the output of D S j on query q i 7: d i ← PrivMed ( { d i,j } j ∈ [ k ] ) , where PrivMed is (1 , 0) -DP

Original: We first argue that Algorithm 6 maintains accuracy against Q rounds of interaction with an adaptive adversary.
Revised: We first argue that Algorithm 6 maintains accuracy against Q rounds of interaction with an adaptive adversary.

Original: Let R =
Revised: Let R =

Original: { R (0) , R (1) , . . .
Revised: { R (0) , R (1) , . . .

Original: , R ( r ) } ,
Revised: , R ( r ) } ,

Original: where R (1) , . . .
Revised: where R (1) , . . .

Original: , R ( r ) denotes the random strings used by the oblivious data structures D 1 , . . .
Revised: , R ( r ) denotes the random strings used by the oblivious data structures D 1 , . . .

Original: , D r and R (0) denotes the additional randomness used by
Revised: , D r and R (0) denotes the additional randomness used by

Original: Algorithm 6, such as in the private median subroutine PrivMed .
Revised: Algorithm 6, such as in the private median subroutine PrivMed .

Original: Consider a transcript T ( R )
Revised: Consider a transcript T ( R )

Original: =
Revised: =

Original: { T 1 , . . .
Revised: { T 1 , . . .

Original: , T Q } such that for each i ∈ [ Q ] , we define T i =
Revised: , T Q } such that for each i ∈ [ Q ] , we define T i =

Original: ( q i , d i ) to be the ordered pair consisting of the query
Revised: ( q i , d i ) to be the ordered pair consisting of the query

Original: q i and the corresponding answer d i by Algorithm 6 using the random string R (0) , as well as the oblivious data structures D 1 , . . .
Revised: q i and the corresponding answer d i by Algorithm 6 using the random string R (0) , as well as the oblivious data structures D 1 , . . .

Original: , D r with random strings R (1) , . . .
Revised: , D r with random strings R (1) , . . .

Original: , R ( r ) .
Revised: , R ( r ) .

Original: We remark that d i is a random variable due to the randomness of each data structure, as well as the randomness of the private median subroutine PrivMed .
Revised: We remark that d i is a random variable due to the randomness of each data structure, as well as the randomness of the private median subroutine PrivMed .

Original: We will first argue that the transcript T R is differentially private with respect to R .
Revised: We will first argue that the transcript T R is differentially private with respect to R .

Original: We emphasize that similar arguments were made in the streaming model by Hassidim et al.
Revised: We emphasize that similar arguments were made in the streaming model by Hassidim et al.

Original: (2020) and in the dynamic model Beimel et al.
Revised: (2020) and in the dynamic model Beimel et al.

Original: ; Attias et al.
Revised: ; Attias et al.

Original: Lemma C.1.
Revised: Lemma C.1.

Original: For a fixed iteration, T i is (cid:16) O (cid:16) 1 √ Q log( nQ ) (cid:17) , 0 (cid:17) -differentially private with respect to
Revised: For a fixed iteration, T i is (cid:16) O (cid:16) 1 √ Q log( nQ ) (cid:17) , 0 (cid:17) -differentially private with respect to

Original: R .
Revised: R .

Original: Proof.
Revised: Proof.

Original: We first observe that PrivMed is (1 , 0) -differentially private on the outputs of the r =
Revised: We first observe that PrivMed is (1 , 0) -differentially private on the outputs of the r =

Original: O (cid:0) √ Q log 2 ( nQ ) (cid:1) data structures.
Revised: O (cid:0) √ Q log 2 ( nQ ) (cid:1) data structures.

Original: Algorithm 6 samples k
Revised: Algorithm 6 samples k

Original: = O (log( nQ )) groups of data structures from the r total data structures.
Revised: = O (log( nQ )) groups of data structures from the r total data structures.

Original: Thus by amplification via sampling, i.e., Theorem A.2, PrivMed is (cid:16) O (cid:16) 1 √ Q log( nQ ) (cid:17) , 0 (cid:17) -differentially private.
Revised: Thus by amplification via sampling, i.e., Theorem A.2, PrivMed is (cid:16) O (cid:16) 1 √ Q log( nQ ) (cid:17) , 0 (cid:17) -differentially private.

Original: Therefore, T i is (cid:16) O (cid:16) 1 √ Q log( nQ ) (cid:17) , 0 (cid:17) -differentially private with respect to R .
Revised: Therefore, T i is (cid:16) O (cid:16) 1 √ Q log( nQ ) (cid:17) , 0 (cid:17) -differentially private with respect to R .

Original: We next argue that the entire transcript is differentially private with respect to the randomness R .
Revised: We next argue that the entire transcript is differentially private with respect to the randomness R .

Original: Lemma C.2.
Revised: Lemma C.2.

Original: T is (cid:16) O (1) , 1 poly( nQ ) (cid:17) -differentially private with respect to R .
Revised: T is (cid:16) O (1) , 1 poly( nQ ) (cid:17) -differentially private with respect to R .

Original: Proof.
Revised: Proof.

Original: By Lemma C.1, for each fixed iteration i ∈ [ Q ] , the transcript T i is (cid:16) O (cid:16)
Revised: By Lemma C.1, for each fixed iteration i ∈ [ Q ] , the transcript T i is (cid:16) O (cid:16)

Original: [Equation]
Revised: [Equation]

Original: √ Q log( nQ ) (cid:17) , 0 (cid:17) - differentially private with respect to R .
Revised: √ Q log( nQ ) (cid:17) , 0 (cid:17) - differentially private with respect to R .

Original: Note that the transcript T is an adaptive composition of the transcripts T 1 , . .
Revised: Note that the transcript T is an adaptive composition of the transcripts T 1 , . .

Original: . , T Q .
Revised: . , T Q .

Original: Thus, by the advanced composition of differential privacy, i.e., Theorem A.4, the transcript T is (cid:16) O (1) ,
Revised: Thus, by the advanced composition of differential privacy, i.e., Theorem A.4, the transcript T is (cid:16) O (1) ,

Original: [Equation]
Revised: [Equation]

Original: differentially private with respect to R .
Revised: differentially private with respect to R .

Original: We now prove the correctness of our unifying framework.
Revised: We now prove the correctness of our unifying framework.

Original: Proof of Theorem 1.2: For a fixed query q i with i ∈
Revised: Proof of Theorem 1.2: For a fixed query q i with i ∈

Original: [ Q ] , let S be the corresponding set of k indices sampled from [ r ] .
Revised: [ Q ] , let S be the corresponding set of k indices sampled from [ r ] .

Original: Let V be the set of valid answers on query q i .
Revised: Let V be the set of valid answers on query q i .

Original: Let I j be an indicator variable for whether the output d i,j on query
Revised: Let I j be an indicator variable for whether the output d i,j on query

Original: q i by D S j is correct, so that I j
Revised: q i by D S j is correct, so that I j

Original: = 1 if d i,j ∈ V and I j = 0 if d i,j / ∈ V .
Revised: = 1 if d i,j ∈ V and I j = 0 if d i,j / ∈ V .

Original: By assumption, we have that for each j ∈ [ k ] ,
Revised: By assumption, we have that for each j ∈ [ k ] ,

Original: Pr [ I j = 1] ≥ 3 4 , so that E [ I j ] ≥ 34 .
Revised: Pr [ I j = 1] ≥ 3 4 , so that E [ I j ] ≥ 34 .

Original: We define the random variable I = 1 k (cid:80) j ∈ [ k ] I j so that by linearity of expectation,
Revised: We define the random variable I = 1 k (cid:80) j ∈ [ k ] I j so that by linearity of expectation,

Original: E
Revised: E

Original: [ I ] = 1 k (cid:80) j ∈ [ k ] E
Revised: [ I ] = 1 k (cid:80) j ∈ [ k ] E

Original: [ I j ] ≥ 34 .
Revised: [ I j ] ≥ 34 .

Original: To handle a sequence of Q adaptive queries, we consider the transcript T ( R )
Revised: To handle a sequence of Q adaptive queries, we consider the transcript T ( R )

Original: =
Revised: =

Original: { T 1 , . . .
Revised: { T 1 , . . .

Original: , T Q } for the randomness R =
Revised: , T Q } for the randomness R =

Original: { R (0) , R (1) , . . .
Revised: { R (0) , R (1) , . . .

Original: , R ( r ) } previously defined, i.e., for each i ∈
Revised: , R ( r ) } previously defined, i.e., for each i ∈

Original: [ Q ] , T i =
Revised: [ Q ] , T i =

Original: ( q i , d i ) is the ordered pair consisting of the query
Revised: ( q i , d i ) is the ordered pair consisting of the query

Original: q i and the corresponding answer d i by Algorithm 6 using the random string R (0) , as well as the oblivious data structures D 1 , . . .
Revised: q i and the corresponding answer d i by Algorithm 6 using the random string R (0) , as well as the oblivious data structures D 1 , . . .

Original: , D r with random strings
Revised: , D r with random strings

Original: R (1) , . . .
Revised: R (1) , . . .

Original: , R ( r ) .
Revised: , R ( r ) .

Original: By Lemma C.2, we have that T is (cid:16) O (1) , 1 poly( nQ ) (cid:17) -differentially private with respect to R .
Revised: By Lemma C.2, we have that T is (cid:16) O (1) , 1 poly( nQ ) (cid:17) -differentially private with respect to R .

Original: For j ∈ [ k ] , we define the function success( R ( S j ) ) to be the indicator variable for whether the output d i,S j by data structure D S
Revised: For j ∈ [ k ] , we define the function success( R ( S j ) ) to be the indicator variable for whether the output d i,S j by data structure D S

Original: j is successful on query q i .
Revised: j is successful on query q i .

Original: For example, if D is supposed to answer queries within (1 + α ) -approximation, then we define success( R ( S j ) ) to be one if d i,S j is within a (1 + α ) -approximation to the true answer on query q i , and zero otherwise.
Revised: For example, if D is supposed to answer queries within (1 + α ) -approximation, then we define success( R ( S j ) ) to be one if d i,S j is within a (1 + α ) -approximation to the true answer on query q i , and zero otherwise.

Original: From the generalization properties of differential privacy, i.e., Theorem A.5, we have
Revised: From the generalization properties of differential privacy, i.e., Theorem A.5, we have

Original: Pr
Revised: Pr

Original: [Equation]
Revised: [Equation]

Original: j ∈ [ k ] success
Revised: j ∈ [ k ] success

Original: [Equation]
Revised: [Equation]

Original: success
Revised: success

Original: [Equation]
Revised: [Equation]

Original: (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)
Revised: (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)

Original: [Equation]
Revised: [Equation]

Original: for sufficiently small O (1) .
Revised: for sufficiently small O (1) .

Original: Therefore, by a union bound over all Q queries, we have
Revised: Therefore, by a union bound over all Q queries, we have

Original: [Equation]
Revised: [Equation]

Original: which implies that d i is correct on query q i .
Revised: which implies that d i is correct on query q i .

Original: Then by a union bound over i ∈
Revised: Then by a union bound over i ∈

Original: [ Q ] for all Q adaptive queries, we have that the data structure answers all Q adaptive queries with high probability.
Revised: [ Q ] for all Q adaptive queries, we have that the data structure answers all Q adaptive queries with high probability.

Original: □
Revised: □

Original: D A PPLICATIONS OF O UR F RAMEWORK
Revised: D A PPLICATIONS OF O UR F RAMEWORK

Original: Theorem 1.2 has applications to a number of central problems in data science and machine learning.
Revised: Theorem 1.2 has applications to a number of central problems in data science and machine learning.

Original: In this section, we formally describe the range queries, point queries, matrix-vector norm queries, and linear regression problems; we defer discussion of adaptive distance estimation, kernel density estimation, and nearest neighbor search to the the appendix.
Revised: In this section, we formally describe the range queries, point queries, matrix-vector norm queries, and linear regression problems; we defer discussion of adaptive distance estimation, kernel density estimation, and nearest neighbor search to the the appendix.

Original: D. A PPLICATION : M ATRIX -V ECTOR N ORM Q UERIES
Revised: D. A PPLICATION : M ATRIX -V ECTOR N ORM Q UERIES

Original: In the matrix-vector norm query problem, we are given a matrix A ∈ R n × d
Revised: In the matrix-vector norm query problem, we are given a matrix A ∈ R n × d

Original: and we would like to handle Q adaptive queries x (1) , . . .
Revised: and we would like to handle Q adaptive queries x (1) , . . .

Original: , x ( Q ) for an approximation parameter ε > 0 by outputting a (1 + ε ) -approximation to ∥
Revised: , x ( Q ) for an approximation parameter ε > 0 by outputting a (1 + ε ) -approximation to ∥

Original: Ax ( i )
Revised: Ax ( i )

Original: ∥ p for each query x
Revised: ∥ p for each query x

Original: ( i ) ∈ R d with i ∈
Revised: ( i ) ∈ R d with i ∈

Original: [ Q ] .
Revised: [ Q ] .

Original: Here we define ∥ v ∥ pp = (cid:80) i ∈
Revised: Here we define ∥ v ∥ pp = (cid:80) i ∈

Original: [ d ] | v i | p for a vector v ∈ R d .
Revised: [ d ] | v i | p for a vector v ∈ R d .

Original: Observe that computing Ax ( i ) explicitly and then computing its p -norm requires O ( nd ) time.
Revised: Observe that computing Ax ( i ) explicitly and then computing its p -norm requires O ( nd ) time.

Original: Thus for n ≫ d , a much faster approach is to produce a subspace embedding, i.e., to compute a matrix M ∈ R m × d with m ≪ n , such that for all x ∈ R d ,
Revised: Thus for n ≫ d , a much faster approach is to produce a subspace embedding, i.e., to compute a matrix M ∈ R m × d with m ≪ n , such that for all x ∈ R d ,

Original: [Equation]
Revised: [Equation]

Original: However, because subspace embeddings must be correct over all possible queries, the number of rows of M is usually m = Ω (cid:0) dε 2 (cid:1) due to requiring correctness over an ε -net.
Revised: However, because subspace embeddings must be correct over all possible queries, the number of rows of M is usually m = Ω (cid:0) dε 2 (cid:1) due to requiring correctness over an ε -net.

Original: Theorem D.1 (Indyk (2006); Li (2008)) .
Revised: Theorem D.1 (Indyk (2006); Li (2008)) .

Original: Given A ∈ R n × d , p ∈ (0 , 2] , and an accuracy parameter ε > 0 , there exists an algorithm that creates a data structure that uses O (cid:0) 1 ε 2 log n (cid:1) bits of space and outputs a (1 + ε ) -approximation to ∥ Ax ∥ p for a query x ∈ R d , with high probability, in time
Revised: Given A ∈ R n × d , p ∈ (0 , 2] , and an accuracy parameter ε > 0 , there exists an algorithm that creates a data structure that uses O (cid:0) 1 ε 2 log n (cid:1) bits of space and outputs a (1 + ε ) -approximation to ∥ Ax ∥ p for a query x ∈ R d , with high probability, in time

Original: O (cid:0) d ε 2 log n (cid:1) .
Revised: O (cid:0) d ε 2 log n (cid:1) .

Original: Theorem D.1 essentially creates a matrix R ∈
Revised: Theorem D.1 essentially creates a matrix R ∈

Original: R m × n of random variables sampled from p -stable distribution (Zolotarev, 1986)
Revised: R m × n of random variables sampled from p -stable distribution (Zolotarev, 1986)

Original: and then stores the matrix RA .
Revised: and then stores the matrix RA .

Original: Once the query x arrives, the data structure then outputs a (1 + ε ) -approximation to ∥
Revised: Once the query x arrives, the data structure then outputs a (1 + ε ) -approximation to ∥

Original: Ax ∥ p by computing a predetermined function on RAx .
Revised: Ax ∥ p by computing a predetermined function on RAx .

Original: The restriction on p ∈ (0 , 2] is due to the fact that p -stable distributions only exist for p ∈ (0 , 2] .
Revised: The restriction on p ∈ (0 , 2] is due to the fact that p -stable distributions only exist for p ∈ (0 , 2] .

Original: From Theorem D.1 and Theorem 1.2, we have the following:
Revised: From Theorem D.1 and Theorem 1.2, we have the following:

Original: Theorem D.2.
Revised: Theorem D.2.

Original: Given A ∈ R n × d , p ∈ (0 , 2] , and an accuracy parameter ε > 0 , there exists an algorithm that creates a data structure that uses O (cid:16) √ Qε 2 log 2 ( nQ ) (cid:17) bits of space and outputs a (1 + ε ) -approximation to ∥
Revised: Given A ∈ R n × d , p ∈ (0 , 2] , and an accuracy parameter ε > 0 , there exists an algorithm that creates a data structure that uses O (cid:16) √ Qε 2 log 2 ( nQ ) (cid:17) bits of space and outputs a (1 + ε ) -approximation to ∥

Original: Ax ( i ) ∥
Revised: Ax ( i ) ∥

Original: p with i ∈
Revised: p with i ∈

Original: [ Q ] for Q adaptive queries x (1) , . . .
Revised: [ Q ] for Q adaptive queries x (1) , . . .

Original: , x ( Q ) ∈ R d , with high probability, in time (cid:101) O (cid:0) dε 2 log 2 ( nQ ) + log 3 ( nQ ) (cid:1) .
Revised: , x ( Q ) ∈ R d , with high probability, in time (cid:101) O (cid:0) dε 2 log 2 ( nQ ) + log 3 ( nQ ) (cid:1) .

Original: D. A PPLICATION : L INEAR R EGRESSION
Revised: D. A PPLICATION : L INEAR R EGRESSION

Original: In the linear regression problem, we are given a fixed matrix A ∈ R n × d
Revised: In the linear regression problem, we are given a fixed matrix A ∈ R n × d

Original: and we would like to handle Q adaptive queries b (1) , . . .
Revised: and we would like to handle Q adaptive queries b (1) , . . .

Original: , b ( Q ) , for an approximation parameter ε > 0 , by outputting a (1 + ε ) -approximation to min x ∈ R d ∥
Revised: , b ( Q ) , for an approximation parameter ε > 0 , by outputting a (1 + ε ) -approximation to min x ∈ R d ∥

Original: Ax − b ( i ) ∥ 2 for each query b ( i ) ∈
Revised: Ax − b ( i ) ∥ 2 for each query b ( i ) ∈

Original: R n with i ∈
Revised: R n with i ∈

Original: [ Q ] .
Revised: [ Q ] .

Original: For linear regression, we can again compute a subspace embedding M = SA ∈ R m × n and answer a query b ( i ) by approximately solving min x ∈ R d
Revised: For linear regression, we can again compute a subspace embedding M = SA ∈ R m × n and answer a query b ( i ) by approximately solving min x ∈ R d

Original: ∥ SAx − Sb ( i ) ∥ 2 , where S is a sketching matrix (Clarkson & Woodruff, 2013).
Revised: ∥ SAx − Sb ( i ) ∥ 2 , where S is a sketching matrix (Clarkson & Woodruff, 2013).

Original: Theorem D.3 (Clarkson & Woodruff (2013)) .
Revised: Theorem D.3 (Clarkson & Woodruff (2013)) .

Original: Given A ∈ R n × d , b ∈ R n , and an accuracy parameter ε > 0 , there exists an algorithm that creates a data structure that uses O (cid:16) d 2 ε 2 log 2 ( nQ ) (cid:17) bits of space and outputs a (1 + ε ) -approximation to min x ∈ R d ∥
Revised: Given A ∈ R n × d , b ∈ R n , and an accuracy parameter ε > 0 , there exists an algorithm that creates a data structure that uses O (cid:16) d 2 ε 2 log 2 ( nQ ) (cid:17) bits of space and outputs a (1 + ε ) -approximation to min x ∈ R d ∥

Original: Ax − b ∥ 2 with high probability.
Revised: Ax − b ∥ 2 with high probability.

Original: However, this may fail for multiple interactions with the data structure.
Revised: However, this may fail for multiple interactions with the data structure.

Original: For example, suppose the adversary learns the kernel of S .
Revised: For example, suppose the adversary learns the kernel of S .

Original: Then the adversary could query some vector b ( i ) in the kernel of S so that Sb ( i ) is the all zeros vector, so that the output is the all zeros vector of dimension d , which could be arbitrarily bad compared to the actual minimizer.
Revised: Then the adversary could query some vector b ( i ) in the kernel of S so that Sb ( i ) is the all zeros vector, so that the output is the all zeros vector of dimension d , which could be arbitrarily bad compared to the actual minimizer.

Original: Thus the na¨ıve approach is to maintain
Revised: Thus the na¨ıve approach is to maintain

Original: Q subspace embeddings, one for each query, resulting in a data structure with space (cid:101) O (cid:16) Qdε 2 (cid:17) .
Revised: Q subspace embeddings, one for each query, resulting in a data structure with space (cid:101) O (cid:16) Qdε 2 (cid:17) .

Original: By comparison, Theorem D.3 and Theorem 1.2 yield the following:
Revised: By comparison, Theorem D.3 and Theorem 1.2 yield the following:

Original: Theorem D.4.
Revised: Theorem D.4.

Original: Given A ∈ R n × d and an accuracy parameter ε > 0 , there exists an algorithm that creates a data structure that uses O (cid:16) √ Qd 2 ε 2 log 3 ( nQ ) (cid:17) bits of space and with high probability, outputs (1 + ε ) -approximations to min x ∈ R d ∥
Revised: Given A ∈ R n × d and an accuracy parameter ε > 0 , there exists an algorithm that creates a data structure that uses O (cid:16) √ Qd 2 ε 2 log 3 ( nQ ) (cid:17) bits of space and with high probability, outputs (1 + ε ) -approximations to min x ∈ R d ∥

Original: Ax − b ( i ) ∥ 2 for Q adaptive queries b (1) , . . .
Revised: Ax − b ( i ) ∥ 2 for Q adaptive queries b (1) , . . .

Original: , b ( Q ) .
Revised: , b ( Q ) .

Original: D. A PPLICATION : H ALF -S PACE Q UERIES
Revised: D. A PPLICATION : H ALF -S PACE Q UERIES

Original: Given a set P of n points in R d , the range query or search problem asks us to pre-process P so that given a region R , chosen from a predetermined family, one can quickly count or return the points in
Revised: Given a set P of n points in R d , the range query or search problem asks us to pre-process P so that given a region R , chosen from a predetermined family, one can quickly count or return the points in

Original: P ∩ R .
Revised: P ∩ R .

Original: This is an extremely well-studied class of problems in computational geometry Toth et al.
Revised: This is an extremely well-studied class of problems in computational geometry Toth et al.

Original: (2017) and the case where the regions R are hyperplanes (also called half-spaces) is of special interest since many algebraic constraints can be “lifted” to be hyperplanes in a higher dimension.
Revised: (2017) and the case where the regions R are hyperplanes (also called half-spaces) is of special interest since many algebraic constraints can be “lifted” to be hyperplanes in a higher dimension.

Original: Unfortunately, exact versions of the problem are known to have the “curse of dimensionality” and suffer from exponential dependence on d in the query time (Br¨onnimann et al., 1993; Chazelle, 2000).
Revised: Unfortunately, exact versions of the problem are known to have the “curse of dimensionality” and suffer from exponential dependence on d in the query time (Br¨onnimann et al., 1993; Chazelle, 2000).

Original: Nonetheless, Chazelle et al.
Revised: Nonetheless, Chazelle et al.

Original: (2008) gave a data structure capable of answering hyperplane queries approximately with polynomial query time.
Revised: (2008) gave a data structure capable of answering hyperplane queries approximately with polynomial query time.

Original: Their notion of approximation is as follows: given a set of points P in the unit ℓ 2 ball, hyperplane R , and ε > 0 , we return the number of points that are on a given side of the hyperplane R up to additive error equal to the number of points in P which lie within distance ε of the boundary of R .
Revised: Their notion of approximation is as follows: given a set of points P in the unit ℓ 2 ball, hyperplane R , and ε > 0 , we return the number of points that are on a given side of the hyperplane R up to additive error equal to the number of points in P which lie within distance ε of the boundary of R .

Original: We will refer to this query as an ε -approximate hyperplane query.
Revised: We will refer to this query as an ε -approximate hyperplane query.

Original: Chazelle et al.
Revised: Chazelle et al.

Original: (2008) proved the following theorem.
Revised: (2008) proved the following theorem.

Original: Theorem D.5 (Chazelle et al.
Revised: Theorem D.5 (Chazelle et al.

Original: Given a set of points P that lie in the unit ℓ 2 ball, there exists a data structure that pre-processes P using space (cid:101) O (cid:16) dn O ( ε − 2 ) (cid:17) such that any ε -approximate hyperplane range query is answered correctly with high probability.
Revised: Given a set of points P that lie in the unit ℓ 2 ball, there exists a data structure that pre-processes P using space (cid:101) O (cid:16) dn O ( ε − 2 ) (cid:17) such that any ε -approximate hyperplane range query is answered correctly with high probability.

Original: The query time is (cid:101) O (cid:0) d/ε 2 (cid:1) .
Revised: The query time is (cid:101) O (cid:0) d/ε 2 (cid:1) .

Original: The data structure of Chazelle et al.
Revised: The data structure of Chazelle et al.

Original: (2008) is randomized and in particular employs randomized dimensionality reduction.
Revised: (2008) is randomized and in particular employs randomized dimensionality reduction.

Original: Thus, it is feasible that queries might fail for multiple adaptive interactions with the data structure.
Revised: Thus, it is feasible that queries might fail for multiple adaptive interactions with the data structure.

Original: By utilizing our framework of Section C and Theorem 1.2, we can obtain the following robust guarantee.
Revised: By utilizing our framework of Section C and Theorem 1.2, we can obtain the following robust guarantee.

Original: Theorem D.6.
Revised: Theorem D.6.

Original: Given a set of points P that lie in the unit ℓ 2 ball, there exists a data structure which pre-processes P using space (cid:101) O (cid:16) √ Qdn O ( ε − 2 ) (cid:17) such that Q adaptive ε -hyperplane range queries are answered correctly with high probability.
Revised: Given a set of points P that lie in the unit ℓ 2 ball, there exists a data structure which pre-processes P using space (cid:101) O (cid:16) √ Qdn O ( ε − 2 ) (cid:17) such that Q adaptive ε -hyperplane range queries are answered correctly with high probability.

Original: The query time is (cid:101) O (cid:0) d/ε 2 (cid:1) .
Revised: The query time is (cid:101) O (cid:0) d/ε 2 (cid:1) .

Original: D. A PPLICATION : P OINT Q UERIES ON T URNSTILE S TREAMS
Revised: D. A PPLICATION : P OINT Q UERIES ON T URNSTILE S TREAMS

Original: In the problem of point queries on turnstile streams, there exists a stream of m updates.
Revised: In the problem of point queries on turnstile streams, there exists a stream of m updates.

Original: Each update specifies a coordinate i ∈ [ n ] of an underlying frequency vector f ∈ R n and changes f i by some amount between ∆ i ∈ [ − ∆ , ∆] , where ∆ = poly( n ) .
Revised: Each update specifies a coordinate i ∈ [ n ] of an underlying frequency vector f ∈ R n and changes f i by some amount between ∆ i ∈ [ − ∆ , ∆] , where ∆ = poly( n ) .

Original: Given any constant accuracy parameter ε >any time t ∈ [ m ] , we define f ( t ) to be the frequency vector implicitly defined after the first t updates.
Revised: Given any constant accuracy parameter ε >any time t ∈ [ m ] , we define f ( t ) to be the frequency vector implicitly defined after the first t updates.

Original: Then the point query problem is to output f ( t ) i for various choices of t ∈ [ m ] and i ∈ [ n ] within an additive error of ε ∥ f ( t ) ∥ 1 .
Revised: Then the point query problem is to output f ( t ) i for various choices of t ∈ [ m ] and i ∈ [ n ] within an additive error of ε ∥ f ( t ) ∥ 1 .

Original: Theorem D.7 (Alman & Yu (2020)) .
Revised: Theorem D.7 (Alman & Yu (2020)) .

Original: There exists an algorithm that uses space O (cid:0) log 2 n (cid:1) bits, worst-case update time O (cid:0) log 0 .
Revised: There exists an algorithm that uses space O (cid:0) log 2 n (cid:1) bits, worst-case update time O (cid:0) log 0 .

Original: 582 n (cid:1) , and query time O (cid:0) log 1 .
Revised: 582 n (cid:1) , and query time O (cid:0) log 1 .

Original: 582 n (cid:1) , that supports point queries with ε = 0 .
Revised: 582 n (cid:1) , that supports point queries with ε = 0 .

Original: 1 with high probability.
Revised: 1 with high probability.

Original: An important quality of Theorem D.7 is that it significantly improves the update time over previous data structures, e.g., Charikar et al.
Revised: An important quality of Theorem D.7 is that it significantly improves the update time over previous data structures, e.g., Charikar et al.

Original: (2004), at a cost in query time.
Revised: (2004), at a cost in query time.

Original: By applying Theorem 1.2, we can avoid a blow-up in query time while still enjoying the update time improvements:
Revised: By applying Theorem 1.2, we can avoid a blow-up in query time while still enjoying the update time improvements:

Original: Theorem D.8.
Revised: Theorem D.8.

Original: There exists an algorithm that uses space O (cid:0) √ Q log 3 ( nQ ) (cid:1) bits, has worst-case update time O (cid:0) √ Q log 1 .
Revised: There exists an algorithm that uses space O (cid:0) √ Q log 3 ( nQ ) (cid:1) bits, has worst-case update time O (cid:0) √ Q log 1 .

Original: 582 ( nQ ) (cid:1) and query time (cid:101) O (cid:0) log 3 ( nQ ) (cid:1) , and supports Q adaptive point queries with ε = 0 .
Revised: 582 ( nQ ) (cid:1) and query time (cid:101) O (cid:0) log 3 ( nQ ) (cid:1) , and supports Q adaptive point queries with ε = 0 .

Original: 1 and with high probability.
Revised: 1 and with high probability.

Original: E A DAPTIVE D ISTANCE E STIMATION
Revised: E A DAPTIVE D ISTANCE E STIMATION

Original: In the adaptive distance estimation problem, there exists a set X
Revised: In the adaptive distance estimation problem, there exists a set X

Original: = { x (1) , . . .
Revised: = { x (1) , . . .

Original: , x ( n ) } of n points in R d .
Revised: , x ( n ) } of n points in R d .

Original: Given an accuracy parameter ε > 0 , the goal is to output a (1 + ε ) -approximation to ∥ x ( i ) − q ∥ p for each query q across all points x ( i ) ∈ X , while minimizing the space, query time, or pre-processing time for the corresponding data structures.
Revised: Given an accuracy parameter ε > 0 , the goal is to output a (1 + ε ) -approximation to ∥ x ( i ) − q ∥ p for each query q across all points x ( i ) ∈ X , while minimizing the space, query time, or pre-processing time for the corresponding data structures.

Original: The trivial solution stores all n points and computes all n distances to each query point and thus can handle an unlimited number of queries.
Revised: The trivial solution stores all n points and computes all n distances to each query point and thus can handle an unlimited number of queries.

Original: Since each point has dimension d , the trivial solution uses space and query time O ( nd ) .
Revised: Since each point has dimension d , the trivial solution uses space and query time O ( nd ) .

Original: Cherapanamjeri &
Revised: Cherapanamjeri &

Original: Nelson (2020) first improved the query time to (cid:101) O (cid:0) n + d ε 2 (cid:1) at the cost of using (cid:101) O (cid:16) ( n + d ) d ε(cid:17) space and (cid:101) O (cid:16) nd 

Revised: Nelson (2020) first improved the query time to (cid:101) O (cid:0) n + d ε 2 (cid:1) at the cost of using (cid:101) O (cid:16) ( n + d ) d ε(cid:17) space and (cid:101) O (cid:16) nd

Original: ε 2 (cid:17) pre-processing time.
Revised: ε 2 (cid:17) pre-processing time.

Original: Like the trivial solution, the algorithm of Cherapanamjeri & Nelson (2020) also permits an arbitrary number of queries.
Revised: Like the trivial solution, the algorithm of Cherapanamjeri & Nelson (2020) also permits an arbitrary number of queries.

Original: In this section, we first apply our framework to show a data structure that can handle Q queries of approximate distances from a specified point in X , using query time (cid:101) O (cid:0) n + d ε 2 (cid:1) , pre-processing time (cid:101) O (cid:16) nd √ Q ε 2 (cid:17) , and space (cid:101) O (cid:16) ( n + d ) √ Q ε(cid:17) .
Revised: In this section, we first apply our framework to show a data structure that can handle Q queries of approximate distances from a specified point in X , using query time (cid:101) O (cid:0) n + d ε 2 (cid:1) , pre-processing time (cid:101) O (cid:16) nd √ Q ε 2 (cid:17) , and space (cid:101) O (cid:16) ( n + d ) √ Q ε(cid:17) .

Original: Hence for d ≫ n √ Q , our data structure already improves on the work of Cherapanamjeri & Nelson (2020).
Revised: Hence for d ≫ n √ Q , our data structure already improves on the work of Cherapanamjeri & Nelson (2020).

Original: However in this setting, each of the Q queries returns only the approximate distance between a query point and a single point in X .
Revised: However in this setting, each of the Q queries returns only the approximate distance between a query point and a single point in X .

Original: By comparison, Cherapanamjeri & Nelson (2020) outputs approximate distances to all points in X and moreover, follow-up work by Cherapanamjeri & Nelson (2022) improved the pre-processing time to (cid:101) O (cid:0) ndε 2 (cid:1) .
Revised: By comparison, Cherapanamjeri & Nelson (2020) outputs approximate distances to all points in X and moreover, follow-up work by Cherapanamjeri & Nelson (2022) improved the pre-processing time to (cid:101) O (cid:0) ndε 2 (cid:1) .

Original: Therefore, we address these two shortcomings of our framework by giving a data structure that (1) handles the case where we return the approximate distances of all points in X from Q adaptive query points and (2) achieves pre-processing time (cid:101) O (cid:0) ndε 2 (cid:1) .
Revised: Therefore, we address these two shortcomings of our framework by giving a data structure that (1) handles the case where we return the approximate distances of all points in X from Q adaptive query points and (2) achieves pre-processing time (cid:101) O (cid:0) ndε 2 (cid:1) .

Original: For completeness, we now show correctness of our algorithm across all Q adaptive queries, though we remark that the proof can simply be black-boxed into Theorem 1.2.
Revised: For completeness, we now show correctness of our algorithm across all Q adaptive queries, though we remark that the proof can simply be black-boxed into Theorem 1.2.

Original: Theorem E.1.
Revised: Theorem E.1.

Original: With high probability, we have
Revised: With high probability, we have

Original: [Equation]
Revised: [Equation]

Original: 1: r ← O (cid:0) √ Q log 2 ( nQ ) (cid:1) , k ← O (log( nQ )) 2: Let Π 1 , . . .
Revised: 1: r ← O (cid:0) √ Q log 2 ( nQ ) (cid:1) , k ← O (log( nQ )) 2: Let Π 1 , . . .

Original: , Π r ∈ R m × d be a JL transformation matrix (see Theorem A.6 or Theorem A.7) 3: for j ∈ [ r ] do 4:
Revised: , Π r ∈ R m × d be a JL transformation matrix (see Theorem A.6 or Theorem A.7) 3: for j ∈ [ r ] do 4:

Original: Compute Π j x i 5: for each query ( y , i ) with y ∈ R d , i ∈
Revised: Compute Π j x i 5: for each query ( y , i ) with y ∈ R d , i ∈

Original: [ n ] do ▷ Adaptive queries 6:
Revised: [ n ] do ▷ Adaptive queries 6:

Original: Let S be a set of k indices sampled (with replacement) from [ r ] 7: for j ∈ [ k ] do 8: d i,j ← ∥ Π S j ( x i − y )
Revised: Let S be a set of k indices sampled (with replacement) from [ r ] 7: for j ∈ [ k ] do 8: d i,j ← ∥ Π S j ( x i − y )

Original: ∥9:
Revised: ∥9:

Original: d i ← PrivMed ( { d i,j } j ∈ [ k ] ) , where PrivMed is (1 , 0) -DP.
Revised: d i ← PrivMed ( { d i,j } j ∈ [ k ] ) , where PrivMed is (1 , 0) -DP.

Original: 10: return d i for all q ∈ [ Q ] .
Revised: 10: return d i for all q ∈ [ Q ] .

Original: Proof.
Revised: Proof.

Original: Fix query
Revised: Fix query

Original: ( y q , i q ) with q ∈ [ Q ] and i q ∈ [ n ] .
Revised: ( y q , i q ) with q ∈ [ Q ] and i q ∈ [ n ] .

Original: Let S be a set of k indices sampled (with replacement) from [ r ] .
Revised: Let S be a set of k indices sampled (with replacement) from [ r ] .

Original: By Theorem A.6 or Theorem A.7, then we have for each j ∈ [ k ] ,
Revised: By Theorem A.6 or Theorem A.7, then we have for each j ∈ [ k ] ,

Original: [Equation]
Revised: [Equation]

Original: Let I j be an indicator variable so that I j = 1 if (1 − ε )
Revised: Let I j be an indicator variable so that I j = 1 if (1 − ε )

Original: ∥ x i q − y q ∥ 2 ≤ ∥ Π S j ( x i q − y q ) ∥ 2 ≤ (1 + ε ) ∥ x i q − y q ∥ 2 and I j = 0 otherwise, so that we have Pr [ I j = 1] ≥ 34 , or equivalently,
Revised: ∥ x i q − y q ∥ 2 ≤ ∥ Π S j ( x i q − y q ) ∥ 2 ≤ (1 + ε ) ∥ x i q − y q ∥ 2 and I j = 0 otherwise, so that we have Pr [ I j = 1] ≥ 34 , or equivalently,

Original: E
Revised: E

Original: [ I j ] ≥ 34 .
Revised: [ I j ] ≥ 34 .

Original: Let I =
Revised: Let I =

Original: 1 k (cid:80) j ∈ [ k ] I j so that by linearity of expectation, E [ I ] = 1 k (cid:80) j ∈ [ k ] E
Revised: 1 k (cid:80) j ∈ [ k ] I j so that by linearity of expectation, E [ I ] = 1 k (cid:80) j ∈ [ k ] E

Original: [ I j ] ≥ 34 .
Revised: [ I j ] ≥ 34 .

Original: To address adaptive queries, we first note that PrivMed is (1 , 0) -differentially private on the outputs of the r Fast JL transforms.
Revised: To address adaptive queries, we first note that PrivMed is (1 , 0) -differentially private on the outputs of the r Fast JL transforms.

Original: Since we sample k = O (log( nQ )) groups from the r = O (cid:0) √ Q log 2 ( nQ ) (cid:1) groups with replacement, then by amplification via sampling, i.e., Theorem A.2, PrivMed is (cid:16)
Revised: Since we sample k = O (log( nQ )) groups from the r = O (cid:0) √ Q log 2 ( nQ ) (cid:1) groups with replacement, then by amplification via sampling, i.e., Theorem A.2, PrivMed is (cid:16)

Original: O (cid:16) 1 √ Q log( nQ ) (cid:17) , 0 (cid:17) -differentially private.
Revised: O (cid:16) 1 √ Q log( nQ ) (cid:17) , 0 (cid:17) -differentially private.

Original: Thus, by the advanced composition of differential privacy, i.e., Theorem A.4, the mechanism permits Q adaptive queries and is (cid:16) O (1) , 1 poly( nQ ) (cid:17) - differentially private.
Revised: Thus, by the advanced composition of differential privacy, i.e., Theorem A.4, the mechanism permits Q adaptive queries and is (cid:16) O (1) , 1 poly( nQ ) (cid:17) - differentially private.

Original: By the generalization properties of differential privacy, i.e., Theorem A.5, we have
Revised: By the generalization properties of differential privacy, i.e., Theorem A.5, we have

Original: [Equation]
Revised: [Equation]

Original: for sufficiently small O (1) .
Revised: for sufficiently small O (1) .

Original: Thus we have
Revised: Thus we have

Original: [Equation]
Revised: [Equation]

Original: which implies that (1 − ε )
Revised: which implies that (1 − ε )

Original: ∥ x i q − y q ∥ 2 ≤ d i ≤ (1+ ε ) ∥ x i q − y q ∥ 2 .
Revised: ∥ x i q − y q ∥ 2 ≤ d i ≤ (1+ ε ) ∥ x i q − y q ∥ 2 .

Original: Therefore, by a union bound across
Revised: Therefore, by a union bound across

Original: Q adaptive queries ( y q , x i q ) with q ∈ [ Q ] , we have that (1 − ε )
Revised: Q adaptive queries ( y q , x i q ) with q ∈ [ Q ] , we have that (1 − ε )

Original: ∥ x i q − y q ∥ 2 ≤ d i ≤ (1+ ε ) ∥ x i q − y q ∥for all q ∈ [ Q ] with high probability.
Revised: ∥ x i q − y q ∥ 2 ≤ d i ≤ (1+ ε ) ∥ x i q − y q ∥for all q ∈ [ Q ] with high probability.

Original: Theorem E.2.
Revised: Theorem E.2.

Original: There exists an algorithm that answers Q adaptive distance estimation queries within a factor of (1 + ε ) .
Revised: There exists an algorithm that answers Q adaptive distance estimation queries within a factor of (1 + ε ) .

Original: [Table]
Revised: [Table]

Original: Proof.
Revised: Proof.

Original: By Theorem A.7, each fast JL transform uses O (cid:16) log d
Revised: By Theorem A.7, each fast JL transform uses O (cid:16) log d

Original: ε 2 + d log d (cid:17) runtime and stores m =
Revised: ε 2 + d log d (cid:17) runtime and stores m =

Original: O (cid:16) log d ε 2 (cid:17) rows.
Revised: O (cid:16) log d ε 2 (cid:17) rows.

Original: On the other hand, by Theorem A.6, each JL transform uses O (cid:0) dε 2 + d log d (cid:1) runtime and stores m =
Revised: On the other hand, by Theorem A.6, each JL transform uses O (cid:0) dε 2 + d log d (cid:1) runtime and stores m =

Original: O (cid:16) log d ε 2 (cid:17) rows.
Revised: O (cid:16) log d ε 2 (cid:17) rows.

Original: By comparison, Cherapanamjeri & Nelson (2020) uses O (cid:16) nd log n ε(cid:17) words of space and O (cid:0) dε 2 (cid:1) query time.
Revised: By comparison, Cherapanamjeri & Nelson (2020) uses O (cid:16) nd log n ε(cid:17) words of space and O (cid:0) dε 2 (cid:1) query time.

Original: E. F ASTER P RE - PROCESSING T IME FOR A DAPTIVE D ISTANCE E STIMATION
Revised: E. F ASTER P RE - PROCESSING T IME FOR A DAPTIVE D ISTANCE E STIMATION

Original: In this section, we present an improved algorithm for Adaptive Distance Estimation, which allows the release of distances to all n points in the dataset for a single query, matching the query time of Cherapanamjeri & Nelson (2020) with an improved space complexity of O ( ε − 2 √ Qn ) .
Revised: In this section, we present an improved algorithm for Adaptive Distance Estimation, which allows the release of distances to all n points in the dataset for a single query, matching the query time of Cherapanamjeri & Nelson (2020) with an improved space complexity of O ( ε − 2 √ Qn ) .

Original: Our results utilize a class of structured randomized linear transformations based on Hadamard matrices recursively defined below:
Revised: Our results utilize a class of structured randomized linear transformations based on Hadamard matrices recursively defined below:

Original: [Equation]
Revised: [Equation]

Original: The associated class of randomized linear transformations are now defined below:
Revised: The associated class of randomized linear transformations are now defined below:

Original: [Equation]
Revised: [Equation]

Original: Note that for any vector z , h ( z ) may be computed in time O ( md log d ) due to the recursive definition of the Hadamard transform.
Revised: Note that for any vector z , h ( z ) may be computed in time O ( md log d ) due to the recursive definition of the Hadamard transform.

Original: We now let ϕ and Φ denote the pdf and cdf of a standard normal random variable, Quant α ( { a i } i ∈
Revised: We now let ϕ and Φ denote the pdf and cdf of a standard normal random variable, Quant α ( { a i } i ∈

Original: [ l ] ) the α th quantile of a multi-set of real numbers { a i } i ∈ [ l ] for any l ∈ N and define ψ r as follows:
Revised: [ l ] ) the α th quantile of a multi-set of real numbers { a i } i ∈ [ l ] for any l ∈ N and define ψ r as follows:

Original: [Equation]
Revised: [Equation]

Original: Through the remainder of the section, we condition on the event defined in the following lemma:
Revised: Through the remainder of the section, we condition on the event defined in the following lemma:

Original: Lemma E.3 (Claims 5.1 and 5.2 Cherapanamjeri & Nelson (2022)) .
Revised: Lemma E.3 (Claims 5.1 and 5.2 Cherapanamjeri & Nelson (2022)) .

Original: For any δ ∈ (cid:0)
Revised: For any δ ∈ (cid:0)

Original: 0 , 12 (cid:1) , with probability at least 1 − δ :
Revised: 0 , 12 (cid:1) , with probability at least 1 − δ :

Original: ∀ z s.t ∥
Revised: ∀ z s.t ∥

Original: z ∥ = 1 : 2 ≤ Quant α − β/ 4 (cid:0) { h ( z ) i } i ∈
Revised: z ∥ = 1 : 2 ≤ Quant α − β/ 4 (cid:0) { h ( z ) i } i ∈

Original: [ md ] (cid:1) ≤
Revised: [ md ] (cid:1) ≤

Original: Quant α + β/ 4 (cid:0) { h ( z ) i } i ∈
Revised: Quant α + β/ 4 (cid:0) { h ( z ) i } i ∈

Original: [ md ] (cid:1) ≤∀ z s.t ∥
Revised: [ md ] (cid:1) ≤∀ z s.t ∥

Original: z ∥ = 1 , r ≥ 4 (cid:112) log(1 /ε ) : (cid:16) 1 − ε 2 (cid:17) ≤ 1 md · (cid:114) π 2 · (cid:88)
Revised: z ∥ = 1 , r ≥ 4 (cid:112) log(1 /ε ) : (cid:16) 1 − ε 2 (cid:17) ≤ 1 md · (cid:114) π 2 · (cid:88)

Original: i ∈
Revised: i ∈

Original: [ md ] ψ r ( h i ( z )) ≤ (cid:16) 1 + ε 2 (cid:17)
Revised: [ md ] ψ r ( h i ( z )) ≤ (cid:16) 1 + ε 2 (cid:17)

Original: as long as m ≥ Cε − 2 log(2 /δ ) log 5 ( d/ε ) for some absolute constant C > 0 .
Revised: as long as m ≥ Cε − 2 log(2 /δ ) log 5 ( d/ε ) for some absolute constant C > 0 .

Original: We will additionally require the following technical result from Cherapanamjeri & Nelson (2022), where for any vector v ∈ R d and multiset S
Revised: We will additionally require the following technical result from Cherapanamjeri & Nelson (2022), where for any vector v ∈ R d and multiset S

Original: = { i j } j ∈
Revised: = { i j } j ∈

Original: k ] with i j ∈ [ d ] , v S denotes the vector [ v i 1 , . .
Revised: k ] with i j ∈ [ d ] , v S denotes the vector [ v i 1 , . .

Original: . , v i k ] :
Revised: . , v i k ] :

Original: Lemma E.4 (Theorem 1.4 Cherapanamjeri & Nelson (2022)) .
Revised: Lemma E.4 (Theorem 1.4 Cherapanamjeri & Nelson (2022)) .

Original: Assume h : R d → R md (SRHT) satisfies the conclusion of Lemma E.3.
Revised: Assume h : R d → R md (SRHT) satisfies the conclusion of Lemma E.3.

Original: Then, there is an algorithm, RetNorm , which satisfies for all x ∈ R d :
Revised: Then, there is an algorithm, RetNorm , which satisfies for all x ∈ R d :

Original: P S { (1 − ε ) ·
Revised: P S { (1 − ε ) ·

Original: ∥ x ∥ ≤ RetNorm( h ( x ) S ) ≤ (1 + ε ) · ∥ x ∥} ≥ 1 − δ for S
Revised: ∥ x ∥ ≤ RetNorm( h ( x ) S ) ≤ (1 + ε ) · ∥ x ∥} ≥ 1 − δ for S

Original: = { i j } j ∈ [ k ] with i j iid ∼ Unif([ md ]) when k ≥ Cε − 2 log(2 /ε )
Revised: = { i j } j ∈ [ k ] with i j iid ∼ Unif([ md ]) when k ≥ Cε − 2 log(2 /ε )

Original: log(2 /δ )
Revised: log(2 /δ )

Original: for some C > 0 .
Revised: for some C > 0 .

Original: Furthermore, RetNorm runs in time O ( k ) .
Revised: Furthermore, RetNorm runs in time O ( k ) .

Original: With these primitives, we will construct our data structure for adaptive distance estimation.
Revised: With these primitives, we will construct our data structure for adaptive distance estimation.

Original: Our constructions is formally described in Algorithm 8.
Revised: Our constructions is formally described in Algorithm 8.

Original: 1: m ← Cε − 2 log 6 (2 dn/ε ) 2: Let h be an SRHT as defined in SRHT ▷ Revealed to analyst 3: r ← C √ Q log 3 ( nd ) , k ← Cε − 2 log(2 /ε ) log(2 nd ) 4: for i ∈ [ n ] do 5:
Revised: 1: m ← Cε − 2 log 6 (2 dn/ε ) 2: Let h be an SRHT as defined in SRHT ▷ Revealed to analyst 3: r ← C √ Q log 3 ( nd ) , k ← Cε − 2 log(2 /ε ) log(2 nd ) 4: for i ∈ [ n ] do 5:

Original: Compute y i
Revised: Compute y i

Original: =
Revised: =

Original: h ( x i ) 6: for j ∈ [ r ] do 7:
Revised: h ( x i ) 6: for j ∈ [ r ] do 7:

Original: Let S i,j be a set of k indices sampled with replacement from [ md ] 8: l ← C log( nd ) 9: for j ∈ 1 : Q do ▷ Adaptive queries 10:
Revised: Let S i,j be a set of k indices sampled with replacement from [ md ] 8: l ← C log( nd ) 9: for j ∈ 1 : Q do ▷ Adaptive queries 10:

Original: Receive query q j 11: v j ← h ( q j ) 12: for i ∈ [ n ] do 13:
Revised: Receive query q j 11: v j ← h ( q j ) 12: for i ∈ [ n ] do 13:

Original: Let { t i,j,p } p ∈
Revised: Let { t i,j,p } p ∈

Original: [ l ] be a set of l indices sampled (with replacement) from [ r ] 14: for p ∈
Revised: [ l ] be a set of l indices sampled (with replacement) from [ r ] 14: for p ∈

Original: [ l ] do 15: d i,j,p ← RetNorm(( v j − y i ) S
Revised: [ l ] do 15: d i,j,p ← RetNorm(( v j − y i ) S

Original: i,ti,j,p ) 16:
Revised: i,ti,j,p ) 16:

Original: d i,j ← PrivMed ( { d i,j,p } p ∈ [ l ] ) , where PrivMed is ( O (1) , 0) -DP.
Revised: d i,j ← PrivMed ( { d i,j,p } p ∈ [ l ] ) , where PrivMed is ( O (1) , 0) -DP.

Original: 17: return { d i,j } i ∈ [ n ]
Revised: 17: return { d i,j } i ∈ [ n ]

Original: The proof of correctness of Algorithm 8 will follow along similar lines to that of Algorithm 6 with a more refined analysis of the privacy loss incurred due to the adaptivity of the data analyst.
Revised: The proof of correctness of Algorithm 8 will follow along similar lines to that of Algorithm 6 with a more refined analysis of the privacy loss incurred due to the adaptivity of the data analyst.

Original: In particular, each input query results in n different queries made to a differentially private mechanism
Revised: In particular, each input query results in n different queries made to a differentially private mechanism

Original: PrivMed leading to a total of nQ queries.
Revised: PrivMed leading to a total of nQ queries.

Original: A na ¨ ıve application of Theorem 1.2 would thus result in a data structure with space complexity scaling as (cid:101) O
Revised: A na ¨ ıve application of Theorem 1.2 would thus result in a data structure with space complexity scaling as (cid:101) O

Original: ( n 3 / 2 √ Q ) as opposed to the desired (cid:101)
Revised: ( n 3 / 2 √ Q ) as opposed to the desired (cid:101)

Original: O ( n √ Q ) and query complexity (cid:101)
Revised: O ( n √ Q ) and query complexity (cid:101)

Original: O ( ε − 2 nd ) .
Revised: O ( ε − 2 nd ) .

Original: The key insight yielding the improved result is the privacy loss incurred by a single query is effectively amortized across n independent differentially private algorithms each capable of answering Q adaptively chosen queries correctly with high probability.
Revised: The key insight yielding the improved result is the privacy loss incurred by a single query is effectively amortized across n independent differentially private algorithms each capable of answering Q adaptively chosen queries correctly with high probability.

Original: To start, we first condition on the event in Lemma E.3 and assume public access to the correspondingly defined SRHT h .
Revised: To start, we first condition on the event in Lemma E.3 and assume public access to the correspondingly defined SRHT h .

Original: We now use R to denote the randomness used to instantiate the multisets, S i,j , in Algorithm 8 and decompose it as follows R =
Revised: We now use R to denote the randomness used to instantiate the multisets, S i,j , in Algorithm 8 and decompose it as follows R =

Original: { R i } i ∈ [ n ] with R i =
Revised: { R i } i ∈ [ n ] with R i =

Original: { R i,j } j ∈ [ r ] where R i,j corresponds to the randomness used to generate the set S i,j and the random elements t
Revised: { R i,j } j ∈ [ r ] where R i,j corresponds to the randomness used to generate the set S i,j and the random elements t

Original: i,p .
Revised: i,p .

Original: As in the proof of Theorem 1.2, we define a transcript T =
Revised: As in the proof of Theorem 1.2, we define a transcript T =

Original: { T j } j ∈ [ Q ] with T j = ( q j , { d i,j } i ∈ [ n ] ) denoting the j th query and the responses returned by Algorithm 8 as a single transaction.
Revised: { T j } j ∈ [ Q ] with T j = ( q j , { d i,j } i ∈ [ n ] ) denoting the j th query and the responses returned by Algorithm 8 as a single transaction.

Original: Lemma E.5.
Revised: Lemma E.5.

Original: For all i ∈ [ n ] , j ∈ [ Q ] , T j is (cid:16) o (cid:16)
Revised: For all i ∈ [ n ] , j ∈ [ Q ] , T j is (cid:16) o (cid:16)

Original: [Equation]
Revised: [Equation]

Original: -differentially private with R i .
Revised: -differentially private with R i .

Original: Proof.
Revised: Proof.

Original: The proof is identical to that of Lemma C.1 with the observation that each transaction T j only results in a single query to a differentially private mechanism operating on R i .
Revised: The proof is identical to that of Lemma C.1 with the observation that each transaction T j only results in a single query to a differentially private mechanism operating on R i .

Original: Lemma E.6.
Revised: Lemma E.6.

Original: For all i ∈ [ n ] , T is (cid:16) o (1) , 1 poly
Revised: For all i ∈ [ n ] , T is (cid:16) o (1) , 1 poly

Original: [Equation]
Revised: [Equation]

Original: nQ
Revised: nQ

Original: [Equation]
Revised: [Equation]

Original: -differentially private with respect to R i .
Revised: -differentially private with respect to R i .

Original: Proof.
Revised: Proof.

Original: The proof is identical to Lemma C.2 and follows from Theorem A.4 and Lemma E.5.
Revised: The proof is identical to Lemma C.2 and follows from Theorem A.4 and Lemma E.5.

Original: We now prove the correctness of our improved procedure for adaptive distance estimation.
Revised: We now prove the correctness of our improved procedure for adaptive distance estimation.

Original: Proof of Theorem 1.3: We condition on the event in the conclusion of Lemma E.3 start by bounding the failure probability of a single query.
Revised: Proof of Theorem 1.3: We condition on the event in the conclusion of Lemma E.3 start by bounding the failure probability of a single query.

Original: The bound for the whole sequence of adaptively chosen queries follows by a union bound.
Revised: The bound for the whole sequence of adaptively chosen queries follows by a union bound.

Original: Now, fixing i ∈ [ n ] and j ∈ [ Q ] , note that the sub-transcript
Revised: Now, fixing i ∈ [ n ] and j ∈ [ Q ] , note that the sub-transcript

Original: T ( j ) =
Revised: T ( j ) =

Original: { T p } p ∈ [ j − 1] is (cid:16) o
Revised: { T p } p ∈ [ j − 1] is (cid:16) o

Original: [Equation]
Revised: [Equation]

Original: -differentially private with respect to R i .
Revised: -differentially private with respect to R i .

Original: Furthermore, define the indicator random variables:
Revised: Furthermore, define the indicator random variables:

Original: [Equation]
Revised: [Equation]

Original: Additionally, defining W := (cid:80) lp
Revised: Additionally, defining W := (cid:80) lp

Original: =1 W p , we get by the differential privacy of the sub-transcript, T ( j ) ,
Revised: =1 W p , we get by the differential privacy of the sub-transcript, T ( j ) ,

Original: Lemma E.4 and Theorem A.5:
Revised: Lemma E.4 and Theorem A.5:

Original: [Equation]
Revised: [Equation]

Original: Consequently, we get from Theorem A.3 and another union bound:
Revised: Consequently, we get from Theorem A.3 and another union bound:

Original: [Equation]
Revised: [Equation]

Original: A subsequent union bound over all i ∈ [ n ] , j ∈ [ Q ] yields:
Revised: A subsequent union bound over all i ∈ [ n ] , j ∈ [ Q ] yields:

Original: [Equation]
Revised: [Equation]

Original: A final union bound over the conclusion of Lemma E.3 concludes the proof.
Revised: A final union bound over the conclusion of Lemma E.3 concludes the proof.

Original: The runtime guarantees follow from the fact that for all z ∈ R d , h ( z ) is computable in time O
Revised: The runtime guarantees follow from the fact that for all z ∈ R d , h ( z ) is computable in time O

Original: ( md log d ) and the runtime guarantees of RetNorm .
Revised: ( md log d ) and the runtime guarantees of RetNorm .

Original: □
Revised: □

Original: F A DAPTIVE K ERNEL D ENSITY E STIMATION
Revised: F A DAPTIVE K ERNEL D ENSITY E STIMATION

Original: Kernel density estimation is an important problem in learning theory and statistics that has recently attracted significant interest, e.g., (Charikar & Siminelakis, 2017; Backurs et al., 2018; Charikar et al., 2020; Bakshi et al., 2022).
Revised: Kernel density estimation is an important problem in learning theory and statistics that has recently attracted significant interest, e.g., (Charikar & Siminelakis, 2017; Backurs et al., 2018; Charikar et al., 2020; Bakshi et al., 2022).

Original: In the adaptive kernel density estimation problem, the input is a set X
Revised: In the adaptive kernel density estimation problem, the input is a set X

Original: = { x (1) , . . .
Revised: = { x (1) , . . .

Original: , x ( n ) } of n points in R d .
Revised: , x ( n ) } of n points in R d .

Original: Given an accuracy parameter ε > 0 and a threshold parameter τ > 0 , the goal is to output a (1 + ε ) -approximation to the quantity 1 n (cid:80) i ∈
Revised: Given an accuracy parameter ε > 0 and a threshold parameter τ > 0 , the goal is to output a (1 + ε ) -approximation to the quantity 1 n (cid:80) i ∈

Original: [ n ] k ( x ( i ) , q ) , for a kernel function k under the promise that the output is at least τ .
Revised: [ n ] k ( x ( i ) , q ) , for a kernel function k under the promise that the output is at least τ .

Original: A standard approach is to sample O (cid:0) 1 τε 2 (cid:1) points and then use O (cid:0) dτε 2 (cid:1) query time to output the empirical kernel density for a specific query.
Revised: A standard approach is to sample O (cid:0) 1 τε 2 (cid:1) points and then use O (cid:0) dτε 2 (cid:1) query time to output the empirical kernel density for a specific query.

Original: Backurs et al.
Revised: Backurs et al.

Original: (2019) give an algorithm for kernel density estimation that uses O (cid:0) 1 τε 2 (cid:1) space and O (cid:16) d √ τε 2 (cid:17) query time, improving over the standard sampling approach.
Revised: (2019) give an algorithm for kernel density estimation that uses O (cid:0) 1 τε 2 (cid:1) space and O (cid:16) d √ τε 2 (cid:17) query time, improving over the standard sampling approach.

Original: Theorem F.1.
Revised: Theorem F.1.

Original: Backurs et al.
Revised: Backurs et al.

Original: Given ε, τ > 0 , there exists a data structure D that uses O (cid:0) 1 τε 2 (cid:1) space and O (cid:16) d ε 2 √ τ (cid:17) query time that outputs a (1 + ε ) -approximation D
Revised: Given ε, τ > 0 , there exists a data structure D that uses O (cid:0) 1 τε 2 (cid:1) space and O (cid:16) d ε 2 √ τ (cid:17) query time that outputs a (1 + ε ) -approximation D

Original: ( y )
Revised: ( y )

Original: to a kernel density estimation query
Revised: to a kernel density estimation query

Original: y that has value at least τ , i.e.,
Revised: y that has value at least τ , i.e.,

Original: [Equation]
Revised: [Equation]

Original: However, the analysis for both these algorithms fails for the adaptive setting, where there can be dependencies between the query and the data structure.
Revised: However, the analysis for both these algorithms fails for the adaptive setting, where there can be dependencies between the query and the data structure.

Original: By using the data structure of Backurs et al.
Revised: By using the data structure of Backurs et al.

Original: (2019) as a subroutine, our framework immediately implies an algorithm for adaptive kernel density estimation that uses (cid:101) O (cid:16) √ Q τε 2 (cid:17) space and O (cid:16) d log Q √ τε 2 (cid:17) query time to answer each of Q adaptive queries.
Revised: (2019) as a subroutine, our framework immediately implies an algorithm for adaptive kernel density estimation that uses (cid:101) O (cid:16) √ Q τε 2 (cid:17) space and O (cid:16) d log Q √ τε 2 (cid:17) query time to answer each of Q adaptive queries.

Original: Algorithm
Revised: Algorithm

Original: Input: Number Q of queries, accuracy ε , threshold τ 1: r ← O (cid:0) √ Q log 2 Q (cid:1) 2: for i ∈
Revised: Input: Number Q of queries, accuracy ε , threshold τ 1: r ← O (cid:0) √ Q log 2 Q (cid:1) 2: for i ∈

Original: [ r ] do ▷ Pre-processing 3:
Revised: [ r ] do ▷ Pre-processing 3:

Original: Let T i be a KDE data structure 4: for each query y q ∈ R d with q ∈ [ Q ] do ▷ Adaptive queries 5:
Revised: Let T i be a KDE data structure 4: for each query y q ∈ R d with q ∈ [ Q ] do ▷ Adaptive queries 5:

Original: Let S be a set of k indices sampled (with replacement) from [ r ] 6: for i ∈
Revised: Let S be a set of k indices sampled (with replacement) from [ r ] 6: for i ∈

Original: [ k ] do 7:
Revised: [ k ] do 7:

Original: Let D i be the output of T S i on query y q 8: return d q = PrivMed ( { D i } i ∈ [ k ] ) , where PrivMed is (1 , 0) -DP.
Revised: Let D i be the output of T S i on query y q 8: return d q = PrivMed ( { D i } i ∈ [ k ] ) , where PrivMed is (1 , 0) -DP.

Original: For completeness, we now show adversarial robustness of our algorithm across Q adaptive queries.
Revised: For completeness, we now show adversarial robustness of our algorithm across Q adaptive queries.

Original: Again we remark that the proof can simply be black-boxed into Theorem 1.2, though we include the specific kernel density details in the following proof as a warm-up for the following section.
Revised: Again we remark that the proof can simply be black-boxed into Theorem 1.2, though we include the specific kernel density details in the following proof as a warm-up for the following section.

Original: Lemma F.2.
Revised: Lemma F.2.

Original: Algorithm 9 answers Q adaptive kernel density estimation queries within a factor of (1 + ε ) , provided each query has value at least τ .
Revised: Algorithm 9 answers Q adaptive kernel density estimation queries within a factor of (1 + ε ) , provided each query has value at least τ .

Original: Proof.
Revised: Proof.

Original: Fix query y q ∈ R d with q ∈ [ Q ] .
Revised: Fix query y q ∈ R d with q ∈ [ Q ] .

Original: Let S be a set of k indices sampled (with replacement) from [ r ] .
Revised: Let S be a set of k indices sampled (with replacement) from [ r ] .

Original: Then by Theorem F.1, we have that for each j ∈ [ k ] ,
Revised: Then by Theorem F.1, we have that for each j ∈ [ k ] ,

Original: [Equation]
Revised: [Equation]

Original: Let I j be an indicator variable so that I j = 1 if (cid:12)(cid:12) D S
Revised: Let I j be an indicator variable so that I j = 1 if (cid:12)(cid:12) D S

Original: j ( y ) − KDE( X, y ) (cid:12)(cid:12) ≤ ε · KDE( X, y ) and
Revised: j ( y ) − KDE( X, y ) (cid:12)(cid:12) ≤ ε · KDE( X, y ) and

Original: I j = 0 otherwise, so that we have Pr [ I j = 1] ≥ 34 or equivalently, E
Revised: I j = 0 otherwise, so that we have Pr [ I j = 1] ≥ 34 or equivalently, E

Original: [ I j ] ≥ 34 .
Revised: [ I j ] ≥ 34 .

Original: Let I =
Revised: Let I =

Original: 1 k (cid:80) j ∈ [ k ] I j so that E [ I ] = 1 k (cid:80) j ∈ [ k ] E
Revised: 1 k (cid:80) j ∈ [ k ] I j so that E [ I ] = 1 k (cid:80) j ∈ [ k ] E

Original: [ I j ] ≥ 34 .
Revised: [ I j ] ≥ 34 .

Original: To handle adaptive queries, we first note that PrivMed is (1 , 0) -differentially private on the outputs of the r kernel density estimation data structures.
Revised: To handle adaptive queries, we first note that PrivMed is (1 , 0) -differentially private on the outputs of the r kernel density estimation data structures.

Original: We sample k
Revised: We sample k

Original: = O (log Q ) indices from the r = O (cid:0) √ Q log 2 Q (cid:1) data structures with replacement.
Revised: = O (log Q ) indices from the r = O (cid:0) √ Q log 2 Q (cid:1) data structures with replacement.

Original: Thus by amplification via sampling, i.e., Theorem A.2, PrivMed is (cid:16) O (cid:16)
Revised: Thus by amplification via sampling, i.e., Theorem A.2, PrivMed is (cid:16) O (cid:16)

Original: [Equation]
Revised: [Equation]

Original: Q (cid:17) , 0 (cid:17) -differentially private.
Revised: Q (cid:17) , 0 (cid:17) -differentially private.

Original: By the advanced composition of differential privacy, i.e., Theorem A.4, our algorithm can answer Q adaptive queries with (cid:16)
Revised: By the advanced composition of differential privacy, i.e., Theorem A.4, our algorithm can answer Q adaptive queries with (cid:16)

Original: O (1) , 1 poly( Q ) (cid:17) -differentially privacy.
Revised: O (1) , 1 poly( Q ) (cid:17) -differentially privacy.

Original: By the generalization properties of differential privacy, i.e.,
Revised: By the generalization properties of differential privacy, i.e.,

Original: Theorem A.5, we have
Revised: Theorem A.5, we have

Original: [Equation]
Revised: [Equation]

Original: 1 k (cid:88) j ∈ [ k ]
Revised: 1 k (cid:88) j ∈ [ k ]

Original: I j − E
Revised: I j − E

Original: [ I ] (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ≥ 1  < 0 .
Revised: [ I ] (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ≥ 1  < 0 .

Original: 01 , for sufficiently small constant O (1) in the private median algorithm PrivMed .
Revised: 01 , for sufficiently small constant O (1) in the private median algorithm PrivMed .

Original: Therefore,
Revised: Therefore,

Original: Pr   1 k (cid:88) i ∈
Revised: Pr   1 k (cid:88) i ∈

Original: [ k ]
Revised: [ k ]

Original: I i > 0 .
Revised: I i > 0 .

Original:  > 0 .
Revised:  > 0 .

Original: 99 , so that | d q − KDE( X, y q ) | ≤ ε · KDE( X, y ) across Q queries y q with q ∈ [ Q ] .
Revised: 99 , so that | d q − KDE( X, y q ) | ≤ ε · KDE( X, y ) across Q queries y q with q ∈ [ Q ] .

Original: Theorem F.3.
Revised: Theorem F.3.

Original: There exists an algorithm that uses O (cid:16) √ Q log 2 Q τε(cid:17) space and answers Q adaptive kernel density estimation queries within a factor of (1 + ε ) , provided each query has value at least τ .
Revised: There exists an algorithm that uses O (cid:16) √ Q log 2 Q τε(cid:17) space and answers Q adaptive kernel density estimation queries within a factor of (1 + ε ) , provided each query has value at least τ .

Original: Each query uses O (cid:16) d log( nQ ) ε 2 √ τ (cid:17) runtime.
Revised: Each query uses O (cid:16) d log( nQ ) ε 2 √ τ (cid:17) runtime.

Original: By comparison, random sampling, e.g., Charikar & Siminelakis (2017), uses Qτε 2 samples to answer
Revised: By comparison, random sampling, e.g., Charikar & Siminelakis (2017), uses Qτε 2 samples to answer

Original: Q queries and each query uses O (cid:0) dτε 2 (cid:1) runtime and using Q copies of the data structure by Backurs et al.
Revised: Q queries and each query uses O (cid:0) dτε 2 (cid:1) runtime and using Q copies of the data structure by Backurs et al.

Original: (2019) uses O (cid:16) Qτε 2 (cid:17) space and O (cid:16)
Revised: (2019) uses O (cid:16) Qτε 2 (cid:17) space and O (cid:16)

Original: d ε 2 √ τ (cid:17) runtime.
Revised: d ε 2 √ τ (cid:17) runtime.

Original: F. U NLIMITED A DAPTIVE Q UERIES FOR K ERNEL D ENSITY E STIMATION
Revised: F. U NLIMITED A DAPTIVE Q UERIES FOR K ERNEL D ENSITY E STIMATION

Original: In this section, we go beyond the limits of our framework and analyze the case where there may be an unbounded number of adversarial queries.
Revised: In this section, we go beyond the limits of our framework and analyze the case where there may be an unbounded number of adversarial queries.

Original: Theorem 1.4.
Revised: Theorem 1.4.

Original: Suppose the kernel function k is L -Lipschitz in the second variable for some L > 0 , i.e., | k
Revised: Suppose the kernel function k is L -Lipschitz in the second variable for some L > 0 , i.e., | k

Original: ( x , y ) − k ( x , z ) | ≤ L ∥ y − z ∥ 2 for all x , y , z ∈ R d .
Revised: ( x , y ) − k ( x , z ) | ≤ L ∥ y − z ∥ 2 for all x , y , z ∈ R d .

Original: Moreover, suppose that for all ∥ x − y ∥ 2 ≤ ρ , we have k ( x , y ) ≤ τ 3 .
Revised: Moreover, suppose that for all ∥ x − y ∥ 2 ≤ ρ , we have k ( x , y ) ≤ τ 3 .

Original: Then an algorithm that produces a kernel density estimation data structure D that is L -Lipschitz over a set X of points with diameter at most ∆ and outputs a (1+ ε ) -approximation to KDE queries with value at least τ with probability at least 1 − δ using space S ( n, ε, τ, log δ ) and query time T ( n, ε, τ, log δ ) , then there exists a KDE data structure that with probability at least 0 .
Revised: Then an algorithm that produces a kernel density estimation data structure D that is L -Lipschitz over a set X of points with diameter at most ∆ and outputs a (1+ ε ) -approximation to KDE queries with value at least τ with probability at least 1 − δ using space S ( n, ε, τ, log δ ) and query time T ( n, ε, τ, log δ ) , then there exists a KDE data structure that with probability at least 0 .

Original: 99 , outputs a (1 + ε ) -approximation to any number of KDE queries with value at least τ using space
Revised: 99 , outputs a (1 + ε ) -approximation to any number of KDE queries with value at least τ using space

Original: S (cid:16) n, O ( ε ) , O ( τ ) , O (cid:16) d log (∆+ ρ ) L ετ (cid:17)(cid:17) and query time T (cid:16) n, O ( ε ) , O ( τ ) , O (cid:16) d log (∆+ ρ ) L ετ (cid:17)(cid:17) .
Revised: S (cid:16) n, O ( ε ) , O ( τ ) , O (cid:16) d log (∆+ ρ ) L ετ (cid:17)(cid:17) and query time T (cid:16) n, O ( ε ) , O ( τ ) , O (cid:16) d log (∆+ ρ ) L ετ (cid:17)(cid:17) .

Original: Proof.
Revised: Proof.

Original: Given a set X ⊆ R d of n points with diameter ∆ , let N be an ετL -net over a ball of radius ∆ + ρ that contains X .
Revised: Given a set X ⊆ R d of n points with diameter ∆ , let N be an ετL -net over a ball of radius ∆ + ρ that contains X .

Original: More formally, let B be a ball of radius (∆ + ρ ) that contains X and for every y ∈ B , there exists a point z ∈ N such that ∥ y − z ∥ 2 ≤ ετL .
Revised: More formally, let B be a ball of radius (∆ + ρ ) that contains X and for every y ∈ B , there exists a point z ∈ N such that ∥ y − z ∥ 2 ≤ ετL .

Original: We can construct the net greedily so that |N| ≤ (cid:16) 2(∆+ ρ ) L ετ (cid:17) d .
Revised: We can construct the net greedily so that |N| ≤ (cid:16) 2(∆+ ρ ) L ετ (cid:17) d .

Original: We implement a data structure D that answers each (non-adaptive) kernel density estimation query with multiplicative approximation (cid:0) 1 + ε 3 (cid:1) for any kernel density estimation query with value at least τ 2 , with probability at least 1 − δ , where δ ≤ 1 100 |N| .
Revised: We implement a data structure D that answers each (non-adaptive) kernel density estimation query with multiplicative approximation (cid:0) 1 + ε 3 (cid:1) for any kernel density estimation query with value at least τ 2 , with probability at least 1 − δ , where δ ≤ 1 100 |N| .

Original: Then by a union bound, D correctly answers each kernel density estimation query in N with probability at least 0 .
Revised: Then by a union bound, D correctly answers each kernel density estimation query in N with probability at least 0 .

Original: Let q ∈ R
Revised: Let q ∈ R

Original: d be an arbitrary query such that KDE( X, q ) ≥ τ .
Revised: d be an arbitrary query such that KDE( X, q ) ≥ τ .

Original: By assumption, we have that ∥ q − x ∥ 2 ≤ ρ for some x ∈ X and thus q ∈ B .
Revised: By assumption, we have that ∥ q − x ∥ 2 ≤ ρ for some x ∈ X and thus q ∈ B .

Original: By the definition of N , there exists some y ∈ N such that ∥ q − y ∥ 2 ≤ ετ 3 L .
Revised: By the definition of N , there exists some y ∈ N such that ∥ q − y ∥ 2 ≤ ετ 3 L .

Original: Then since k is L -Lipschitz in the second variable, we have
Revised: Then since k is L -Lipschitz in the second variable, we have

Original: [Equation]
Revised: [Equation]

Original: Hence, KDE( X, q ) ≥ τ implies that KDE( X, y ) ≥ τ 2 .
Revised: Hence, KDE( X, q ) ≥ τ implies that KDE( X, y ) ≥ τ 2 .

Original: Let K y be the output of the data structure
Revised: Let K y be the output of the data structure

Original: D on query y .
Revised: D on query y .

Original: Then by correctness of D on N for any query with threshold at least τ 2 , we have
Revised: Then by correctness of D on N for any query with threshold at least τ 2 , we have

Original: [Equation]
Revised: [Equation]

Original: Let K q be the output of the data structure D on query y .
Revised: Let K q be the output of the data structure D on query y .

Original: Since the algorithm itself is L -Lipschitz, then
Revised: Since the algorithm itself is L -Lipschitz, then

Original: [Equation]
Revised: [Equation]

Original: Therefore by the triangle inequality, we have that
Revised: Therefore by the triangle inequality, we have that

Original: [Equation]
Revised: [Equation]

Original: Since KDE( X, y ) ≤ KDE( X, q ) + ετ 3 n , then it follows that
Revised: Since KDE( X, y ) ≤ KDE( X, q ) + ετ 3 n , then it follows that

Original: [Equation]
Revised: [Equation]

Original: for n ≥ 6 .
Revised: for n ≥ 6 .

Original: In particular, sampling-based algorithms for kernels that are Lipschitz are also Lipschitz.
Revised: In particular, sampling-based algorithms for kernels that are Lipschitz are also Lipschitz.

Original: Thus to apply Theorem 1.4, it suffices to identify kernels that are L -Lipschitz and use the data structure of Theorem F.1.
Revised: Thus to apply Theorem 1.4, it suffices to identify kernels that are L -Lipschitz and use the data structure of Theorem F.1.

Original: To that end, we note that the kernels k ( x , y ) =
Revised: To that end, we note that the kernels k ( x , y ) =

Original: C C + ∥ x − y ∥ 2 for C > 0 and k ( x , y )
Revised: C C + ∥ x − y ∥ 2 for C > 0 and k ( x , y )

Original: =
Revised: =

Original: Ce −∥ x − y ∥ 2 are both Lipschitz for some function of C .
Revised: Ce −∥ x − y ∥ 2 are both Lipschitz for some function of C .

Original: In particular, we have
Revised: In particular, we have

Original: [Equation]
Revised: [Equation]

Original: so k ( x , y ) =
Revised: so k ( x , y ) =

Original: C C + ∥ x − y ∥ 2 is 1 C -Lipschitz.
Revised: C C + ∥ x − y ∥ 2 is 1 C -Lipschitz.

Original: Similarly, since e − x is 1 -Lipschitz, then
Revised: Similarly, since e − x is 1 -Lipschitz, then

Original: [Equation]
Revised: [Equation]

Original: ∥ 2 , so k ( x , y ) =
Revised: ∥ 2 , so k ( x , y ) =

Original: Ce −∥ x − y ∥ 2 is C -Lipschitz.
Revised: Ce −∥ x − y ∥ 2 is C -Lipschitz.

Original: G E MPIRICAL E VALUATION
Revised: G E MPIRICAL E VALUATION

Original: We empirically demonstrate the space and query time efficiency of our framework of Section C.
Revised: We empirically demonstrate the space and query time efficiency of our framework of Section C.

Original: We consider the problem of ℓ 2 norm estimation where queries q 1 , q 2 , . . .
Revised: We consider the problem of ℓ 2 norm estimation where queries q 1 , q 2 , . . .

Original: are generated in an adaptive fashion and our goal is to output an estimate of ∥ q i ∥ 2 for all i .
Revised: are generated in an adaptive fashion and our goal is to output an estimate of ∥ q i ∥ 2 for all i .

Original: This setting is a special case of adaptive distance estimation and captures the essence of our adversarial robustness framework.
Revised: This setting is a special case of adaptive distance estimation and captures the essence of our adversarial robustness framework.

Original: In addition, this same setting was investigated empirically in prior works Cherapanamjeri & Nelson (2020).
Revised: In addition, this same setting was investigated empirically in prior works Cherapanamjeri & Nelson (2020).

Original: Experimental Setup.
Revised: Experimental Setup.

Original: Consider the setting of Algorithm 6: it creates r copies of an underlying randomized data structure and upon a query, it subsamples k of them and outputs an answer aggregated via the private median.
Revised: Consider the setting of Algorithm 6: it creates r copies of an underlying randomized data structure and upon a query, it subsamples k of them and outputs an answer aggregated via the private median.

Original: In our setting, the underlying algorithm will be the fast Johnson-Lindenstrauss (JL) transform which is defined as follows: it is the matrix PHD : R d → R m where D is a diagonal matrix with uniformly random ± 1 entries, H is the Hadamard transform, and P is a sampling matrix uniformly samples m rows of HD .
Revised: In our setting, the underlying algorithm will be the fast Johnson-Lindenstrauss (JL) transform which is defined as follows: it is the matrix PHD : R d → R m where D is a diagonal matrix with uniformly random ± 1 entries, H is the Hadamard transform, and P is a sampling matrix uniformly samples m rows of HD .

Original: Our algorithm will initialize r copies of this matrix where the sampling matrix P and diagonal D will be the randomness which is “hidden” from the adversary.
Revised: Our algorithm will initialize r copies of this matrix where the sampling matrix P and diagonal D will be the randomness which is “hidden” from the adversary.

Original: Upon query q , we sample k different Fast JL data structures, input q to all of them, and proceed as in Algorithm 6.
Revised: Upon query q , we sample k different Fast JL data structures, input q to all of them, and proceed as in Algorithm 6.

Original: Note that this setting exactly mimics the theoretical guarantees of Section C and is exactly Algorithm 7 of Section E.
Revised: Note that this setting exactly mimics the theoretical guarantees of Section C and is exactly Algorithm 7 of Section E.

Original: In our experiments, d = 4096 , m = 250 , r = 200 , and k = 5 .
Revised: In our experiments, d = 4096 , m = 250 , r = 200 , and k = 5 .

Original: These are exactly the parameters chosen in prior works Cherapanamjeri & Nelson (2020).
Revised: These are exactly the parameters chosen in prior works Cherapanamjeri & Nelson (2020).

Original: We will have 5000 adaptive queries q i which are described shortly.
Revised: We will have 5000 adaptive queries q i which are described shortly.

Original: Our experiments are done on a 2021 M
Revised: Our experiments are done on a 2021 M

Original: Macbook Pro with 32 gigabytes of RAM.
Revised: Macbook Pro with 32 gigabytes of RAM.

Original: We implemented all algorithms in Python 3.5 using Numpy.
Revised: We implemented all algorithms in Python 3.5 using Numpy.

Original: The Hadamard transform code is from Andoni et al.
Revised: The Hadamard transform code is from Andoni et al.

Original: and we use Google’s differential privacy library 2 for the private median implementation.
Revised: and we use Google’s differential privacy library 2 for the private median implementation.

Original: Baselines.
Revised: Baselines.

Original: We will consider three baselines.
Revised: We will consider three baselines.

Original: JL will denote a standard (Gaussian) JL map from dimension 4096 to 250 .
Revised: JL will denote a standard (Gaussian) JL map from dimension 4096 to 250 .

Original: Baseline 1 will denote the algorithm of Cherapanamjeri & Nelson (2020).
Revised: Baseline 1 will denote the algorithm of Cherapanamjeri & Nelson (2020).

Original: At a high level, it instantiates many independent copies of the standard Gaussian JL map and only feeds an incoming query into a select number of subsampled data structures.
Revised: At a high level, it instantiates many independent copies of the standard Gaussian JL map and only feeds an incoming query into a select number of subsampled data structures.

Original: Note that our experimental setting is mimicking exactly that of Cherapanamjeri & Nelson (2020) where the same parameters r (number of different underlying data structures) and k (number of subsampled data structures to use for a query) were used.
Revised: Note that our experimental setting is mimicking exactly that of Cherapanamjeri & Nelson (2020) where the same parameters r (number of different underlying data structures) and k (number of subsampled data structures to use for a query) were used.

Original: This ensures that both our algorithm and theirs have access to the same number of different JL maps and thus allows us to compare the two approaches on an equal footing.
Revised: This ensures that both our algorithm and theirs have access to the same number of different JL maps and thus allows us to compare the two approaches on an equal footing.

Original: The last baseline, denoted as Baseline 2 , is the main algorithm of Cherapanamjeri & Nelson (2022) which is the optimized version of Cherapanamjeri & Nelson (2020).
Revised: The last baseline, denoted as Baseline 2 , is the main algorithm of Cherapanamjeri & Nelson (2022) which is the optimized version of Cherapanamjeri & Nelson (2020).

Original: At a high level, their algorithm proceeds similarly to that of Cherapanamjeri & Nelson (2020), except they employ Hadamard transforms (after multiplying the query entry-wise by random Gaussians), rather than using Gaussian JL maps.
Revised: At a high level, their algorithm proceeds similarly to that of Cherapanamjeri & Nelson (2020), except they employ Hadamard transforms (after multiplying the query entry-wise by random Gaussians), rather than using Gaussian JL maps.

Original: Furthermore, instead of subsampling, their algorithm feeds an incoming query into all the different copies of the Hadamard transform, and subsamples the coordinates of the concatenated output for norm estimation.
Revised: Furthermore, instead of subsampling, their algorithm feeds an incoming query into all the different copies of the Hadamard transform, and subsamples the coordinates of the concatenated output for norm estimation.

Original: We again set the parameters of their algorithm to match that of our algorithm and
Revised: We again set the parameters of their algorithm to match that of our algorithm and

Original: Baseline 1 by using r copies of their Hadamard transform and subsampling mk total coordinates.
Revised: Baseline 1 by using r copies of their Hadamard transform and subsampling mk total coordinates.

Original: We refer to the respective papers for full details of their algorithms.
Revised: We refer to the respective papers for full details of their algorithms.

Original: Summary of adaptive queries.
Revised: Summary of adaptive queries.

Original: Our input queries are the same adaptive queries used in Cherapanamjeri & Nelson (2020).
Revised: Our input queries are the same adaptive queries used in Cherapanamjeri & Nelson (2020).

Original: To summarize, let Π denote the map used in the JL benchmark stated above.
Revised: To summarize, let Π denote the map used in the JL benchmark stated above.

Original: The i -th query for 1 ≤ i ≤ 5000 will be of the form q i = (cid:80) ij =1 ( − 1) W
Revised: The i -th query for 1 ≤ i ≤ 5000 will be of the form q i = (cid:80) ij =1 ( − 1) W

Original: i
Revised: i

Original: z i , which we then normalize to have unit norm.
Revised: z i , which we then normalize to have unit norm.

Original: The z i are standard Gaussian vectors.
Revised: The z i are standard Gaussian vectors.

Original: W i is the indicator variable for the event ∥
Revised: W i is the indicator variable for the event ∥

Original: Π( z i − e 1 ) ∥ 2 ≤ ∥ Π( z i + e 1 ) ∥ 2 where e 1 is the first standard basis vector.
Revised: Π( z i − e 1 ) ∥ 2 ≤ ∥ Π( z i + e 1 ) ∥ 2 where e 1 is the first standard basis vector.

Original: Intuitively, the queries become increasingly correlated with the matrix Π since we successively “augment” the queries in a biased fashion.
Revised: Intuitively, the queries become increasingly correlated with the matrix Π since we successively “augment” the queries in a biased fashion.

Original: See Section 5 of Cherapanamjeri & Nelson (2020) for a more detailed discussion of the adaptive inputs.
Revised: See Section 5 of Cherapanamjeri & Nelson (2020) for a more detailed discussion of the adaptive inputs.

Original: Results.
Revised: Results.

Original: Our results are shown in Figure 1.
Revised: Our results are shown in Figure 1.

Original: In Figure 1a, we plot the norm estimated by each of the algorithms in each of the queries across iterations.
Revised: In Figure 1a, we plot the norm estimated by each of the algorithms in each of the queries across iterations.

Original: We see that the na¨ıve JL map increasingly deviates from the true value of 1 . 0 .
Revised: We see that the na¨ıve JL map increasingly deviates from the true value of 1 . 0 .

Original: This is intuitive as the adaptive queries are increasingly correlated
Revised: This is intuitive as the adaptive queries are increasingly correlated

Original: [Figure]
Revised: [Figure]

Original: with the map Π .
Revised: with the map Π .

Original: The performance of all other algorithms are indistinguishable in Figure 1a.
Revised: The performance of all other algorithms are indistinguishable in Figure 1a.

Original: Thus, we only zoom into the performances of our algorithm and Baseline 1 and Baseline 2 , shown in
Revised: Thus, we only zoom into the performances of our algorithm and Baseline 1 and Baseline 2 , shown in

Original: Figure 1b.
Revised: Figure 1b.

Original: For these three algorithms, we plot a histogram of answers outputted by the respective algorithms across all iterations.
Revised: For these three algorithms, we plot a histogram of answers outputted by the respective algorithms across all iterations.

Original: We see that the algorithm of Cherapanamjeri & Nelson (2020), shown in the blue shaded histogram, is the most accurate as it has the smallest deviations from the true answer of 1 . 0 .
Revised: We see that the algorithm of Cherapanamjeri & Nelson (2020), shown in the blue shaded histogram, is the most accurate as it has the smallest deviations from the true answer of 1 . 0 .

Original: Our algorithm, shown in green, is noisier than Baseline 1 since it has a wider range of variability.
Revised: Our algorithm, shown in green, is noisier than Baseline 1 since it has a wider range of variability.

Original: This may be due to the fact that we use a differentially private median algorithm, which naturally incurs additional noise.
Revised: This may be due to the fact that we use a differentially private median algorithm, which naturally incurs additional noise.

Original: Lastly, Baseline 2 is also noisier than Baseline 1 and comparable to our algorithm.
Revised: Lastly, Baseline 2 is also noisier than Baseline 1 and comparable to our algorithm.

Original: This may be due to the fact that the algorithm of Cherapanamjeri & Nelson (2022) requires very fine-tuned constants in their theoretical bounds, which naturally deviate in practice.
Revised: This may be due to the fact that the algorithm of Cherapanamjeri & Nelson (2022) requires very fine-tuned constants in their theoretical bounds, which naturally deviate in practice.

Original: Lastly, Figure 1c shows the cumulative runtime of all three algorithms across all iterations.
Revised: Lastly, Figure 1c shows the cumulative runtime of all three algorithms across all iterations.

Original: Our algorithm, shown in green, is the fastest while Baseline 2 is the slowest.
Revised: Our algorithm, shown in green, is the fastest while Baseline 2 is the slowest.

Original: This is explained by the fact that Baseline 2 calculates many more Hadamard transforms than our algorithm does.
Revised: This is explained by the fact that Baseline 2 calculates many more Hadamard transforms than our algorithm does.