Title: Adaptive Data Analysis in a Balanced Adversarial Model
Venue: NeurIPS 2023
Accepted: yes
Keywords: Adaptive Data Analysis

Differential Privacy

Statistical Queries
Abstract: In adaptive data analysis, a mechanism gets $n$ i.i.d. samples from an unknown distribution $\cal{D}$, and
is required to provide accurate estimations to a sequence of adaptively chosen statistical queries with respect to $\cal{D}$.
Hardt and Ullman (FOCS 2014) and Steinke and Ullman (COLT 2015) showed that in general, it is computationally hard to answer more than $\Theta(n^2)$ adaptive queries, assuming the existence of one-way functions. 

However, these negative results strongly rely on an adversarial model that significantly advantages the adversarial analyst over the mechanism, as the analyst, who chooses the adaptive queries, also chooses the underlying distribution $\cal{D}$. 
This imbalance raises questions with respect to the applicability of the obtained hardness results -- an analyst who has complete knowledge of the underlying distribution $\cal{D}$ would have little need, if at all, to issue statistical queries to a mechanism which only holds a finite number of samples from $\cal{D}$.

We consider more restricted adversaries, called \emph{balanced}, where each such adversary consists of two separated algorithms: The \emph{sampler} who is the entity that chooses the distribution and provides the samples to the mechanism, and the \emph{analyst} who chooses the adaptive queries, but has no prior knowledge of the underlying distribution (and hence has no a priori advantage with respect to the mechanism). 
We improve the quality of previous lower bounds by revisiting them using an efficient \emph{balanced} adversary, under standard public-key cryptography assumptions. We show that these stronger hardness assumptions are unavoidable in the sense that any computationally bounded \emph{balanced} adversary that has the structure of all known attacks, implies the existence of public-key cryptography.

Adaptive Data Analysis in a Balanced Adversarial Model

Abstract

In adaptive data analysis, a mechanism gets n i.i.d. samples from an unknown distribution D , and is required to provide accurate estimations to a sequence of adaptively chosen statistical queries with respect to D . Hardt and Ullman [2014] and Steinke and Ullman [2015a] showed that, in general, it is computationally hard to answer more than Θ( n 2 ) adaptive queries, assuming the existence of one-way functions.

However, these negative results strongly rely on an adversarial model that significantly advantages the adversarial analyst over the mechanism, as the analyst, who chooses the adaptive queries, also chooses the underlying distribution D . This imbalance raises questions with respect to the applicability of the obtained hardness results - an analyst who has complete knowledge of the underlying distribution D would have little need, if at all, to issue statistical queries to a mechanism which only holds a finite number of samples from D .

We consider more restricted adversaries, called balanced , where each such adversary consists of two separate algorithms: The sampler who is the entity that chooses the distribution and provides the samples to the mechanism, and the analyst who chooses the adaptive queries, but has no prior knowledge of the underlying distribution (and hence has no a priori advantage with respect to the mechanism).

We improve the quality of previous lower bounds by revisiting them using an efficient balanced adversary, under standard public-key cryptography assumptions. We show that these stronger hardness assumptions are unavoidable in the sense that any computationally bounded balanced adversary that has the structure of all known attacks, implies the existence of public-key cryptography.

1 Introduction

Statistical validity is a widely recognized crucial feature of modern science. Lack of validity popularly known as the replication crisis in science poses a serious threat to the scientific process and also to the public's trust in scientific findings.

One of the factors leading to the replication crisis is the inherent adaptivity in the data analysis process. To illustrate adaptivity and its effect, consider a data analyst who is testing a specific research

hypothesis. The analyst gathers data, evaluates the hypothesis empirically, and often finds that their hypothesis is not supported by the data, leading to the formulation and testing of more hypotheses. If these hypotheses are tested and formed based on the same data (as acquiring fresh data is often expensive or even impossible), then the process is of adaptive data analysis (ADA) because the choice of hypotheses depends on the data. However, ADA no longer aligns with classical statistical theory, which assumes that hypotheses are selected independently of the data (and preferably before gathering data). ADA may lead to overfitting and hence false discoveries.

Statistical validity under ADA is a fundamental problem in statistics, that has received only partial answers. A recent line of work, initiated by Dwork et al. [2015c] and includes [Hardt and Ullman, 2014, Dwork et al., 2015a,b, Steinke and Ullman, 2015a,b, Bassily et al., 2016, Rogers et al., 2016, Russo and Zou, 2016, Smith, 2017, Feldman and Steinke, 2017, Nissim et al., 2018, Feldman and Steinke, 2018, Shenfeld and Ligett, 2019, Jung et al., 2020, Fish et al., 2020, Dagan and Kur, 2022, Kontorovich et al., 2022, Dinur et al., 2023, Blanc, 2023] has resulted in new insights into ADA and robust paradigms for guaranteeing statistical validity in ADA. A major objective of this line of work is to design optimal mechanisms M that initially obtain a dataset S containing n i.i.d. samples from an unknown distribution D , and then answers adaptively chosen queries with respect to D . Importantly, all of M 's answers must be accurate with respect to the underlying distribution D , not just w.r.t. the empirical dataset S . The main question is how to design an efficient mechanism that provides accurate estimations to adaptively chosen statistical queries, where the goal is to maximize the number of queries M can answer. This objective is achieved by providing both upper- and lower-bound constructions, where the lower-bound constructions demonstrate how an adversarial analyst making a small number of queries to an arbitrary M can invariably force M to err. The setting for these lower-bound proofs is formalized as a two-player game between a mechanism M and an adversary A as in Game 1.1.

Game 1.1 (ADA game between a mechanism M and an adversarial analyst A ) .

M gets a dataset S of n i.i.d. samples from an unknown distribution D over X .

For i = 1 , . . . , ℓ :

-A sends a query q i : X ↦→ [ -1 , 1] to M .

-M sends an answer y i ∈ [ -1 , 1] to A . (As A and M are stateful, q and y may depend on the previous messages.)

i i fails if ∃ i ∈ [ ℓ ] s.t. | y i -E x ∼D [ q i ( x )] | > 1 / 10 .

M

Aquestion that immediately arises from the description of Game 1.1 is to whom should the distribution D be unknown, and how to formalize this lack of knowledge. Ideally, the mechanism M should succeed with high probability for every unknown distribution D and against any adversary A .

In prior work, this property was captured by letting the adversary choose the distribution D at the outset of Game 1.1. Namely, the adversary A can be seen as a pair of algorithms ( A 1 , A 2 ) , where A 1 chooses the distribution D and sends a state st to A 2 (which may contain the entire view of A 1 ), and after that, M and A 2 ( st ) interacts in Game 1.1. In this adversarial model, Hardt and Ullman [2014] and Steinke and Ullman [2015a] showed that, assuming the existence of one way functions, it is computationally hard to answer more than Θ( n 2 ) adaptive queries. These results match the state-of-the-art constructions [Dwork et al., 2015c,a,b, Steinke and Ullman, 2015b, Bassily et al., 2016, Feldman and Steinke, 2017, 2018, Dagan and Kur, 2022, Blanc, 2023]. 1 In fact, each such negative result was obtained by constructing a single adversary A that fails all efficient mechanisms. This means that, in general, it is computationally hard to answer more than Θ( n 2 ) adaptive queries even when the analyst's algorithm is known to the mechanism. On the other hand, in each of these negative results, the adversarial analyst has a significant advantage over the mechanism - their ability to select the distribution D . This allows the analyst to inject random trapdoors in D (e.g., keys of an encryption scheme) which are then used in forcing a computationally limited mechanism to fail, as the mechanism does not get a hold of the trapdoor information.

For most applications, the above adversarial model seems to be too strong. For instance, a data analyst who is testing research hypotheses usually has no knowledge advantage about the distribution that the mechanism does not have. In this typical setting, even if the underlying distribution D happens to have a trapdoor, if the analyst recovers the trapdoor then the mechanism should also be able to recover it and hence disable its adversarial usage.

In light of this observation, we could hope that in a balanced setting, where the underlying distribution is unknown to both the mechanism and the analyst, it would always be possible for M to answer more than O ( n 2 ) adaptive queries. To explore this possibility, we introduce what we call a balanced adversarial model.

Definition 1.2 (Balanced Adversary) . A balanced adversary A consists of two isolated algorithms: The sampler A 1 , which chooses a distribution D and provides i.i.d. samples to the mechanism M , and the analyst A 2 , which asks the adaptive queries. No information is transferred from A 1 to A 2 . See Game 1.3.

Game 1.3 (The ADA game between a mechanism M and a balanced adversary A = ( A 1 , A 2 ) ) .

A 1 chooses a distribution D over X (specified by a sampling algorithm) and provides n i.i.d. samples S to M (by applying the sampling algorithm n times). /* A 1 does not provide A 2 with any information */

M and A 2 play Game 1.1 (with respect to D and S ). M fails if and only if it fails in Game 1.1.

Note that the difference between the balanced model and the previous (imbalanced) one is whether A 1 can send a state to A 2 after choosing the distribution D (in the imbalanced model it is allowed, in contrast to the balanced model). 2

We remark that the main advantage of the balanced model comes when considering a publicly known sampler A 1 (as we do throughout this work). This way, A 1 captures the common knowledge that both the mechanism M and the analyst A 2 have about the underlying distribution D .

Question 1.4. Do the lower-bounds proved in prior work hold also for balanced adversaries?

In this work we answer Question 1.4 in the positive. We do that using a publicly known analyst A 2 (which even makes it stronger than what is required for a lower bound). I.e., even though the sampler A 1 and the analyst A 2 are publicly known and cannot communicate with each other, they fail any computationally bounded mechanism. However, our lower-bound is based on stronger hardness assumptions than in prior work, namely, we use public-key cryptography.

1.1 Our Results

Our first result is a construction of a balanced adversary forcing any computationally bounded mechanim to fail in Game 1.3.

Theorem 1.5 (Computational lower bound, informal) . There exists an efficient balanced adversary A = ( A 1 , A 2 ) that fails any computationally bounded mechanism M using Θ( n 2 ) adaptive queries. Moreover, it does so by choosing a distribution over a small domain.

Our construction in Theorem 1.5 uses the structure of previous attacks of Hardt and Ullman [2014] and Steinke and Ullman [2015a], but relies on a stronger hardness assumption of public-key cryptography. We prove that this is unavoidable.

Theorem 1.6 (The necessity of public-key cryptography for proving Theorem 1.5, informal) . Any computationally bounded balanced adversary that follows the structure of all currently known attacks, implies the existence of public-key cryptography (in particular, a key-agreement protocol).

In Section 1.3 we provide proof sketches of Theorems 1.5 and 1.6, where the formal statements appear in Sections 3 and 4 (respectively) and the formal proofs appear in the supplementary material.

Potential Consequences for the Information Theoretic Setting. Theorem 1.6 has immediate implication to the information theoretic setting, and allow for some optimism regarding the possibility of constructing an inefficient mechanism that answers many adaptive queries.

It is known that an inefficient mechanism can answer exponentially many adaptive queries, but such results have a strong dependency on the domain size. For instance, the Private Multiplicative Weights √

algorithm of Hardt and Rothblum [2010] can answer 2 ˜ O n/ log |X| adaptive queries accurately. However, this result is not useful whenever n ≤ O ( √ log |X| ) . Indeed, Steinke and Ullman [2015a] showed that this dependency is unavoidable in general, by showing that large domain can be used for constructing a similar, unconditional, adversary that fails any computationally unbounded mechanism after Θ( n 2 ) queries. Our Theorem 1.6 implies that such an attack cannot be implemented in the balanced setting, which gives the first evidence that there might be a separation between the computational and information theoretic setting under the balanced adversarial model (in contrast with the imbalanced model).

(

)

Corollary 1.7. There is no balanced adversary that follows the structure of all currently known attacks, and fails any (computationally unbounded ) mechanism.

In order to see why Corollary 1.7 holds, suppose that we could implement such kind of attack using a balanced adversary. Then by Theorem 1.6, this would imply that we could construct an information-theoretic key agreement protocol (i.e., a protocol between two parties that agree on a key that is secret from the eyes of a computationally unbounded adversary that only sees the transcript of the execution). But since the latter does not exist, we conclude that such a balanced adversary does not exists either. In other words, we do not have a negative result that rules out the possibility of constructing an inefficient mechanism that can answer many adaptive queries of a balanced adversary, and we know that if a negative result exists, then by Theorem 1.6 it cannot follow the structure of Hardt and Ullman [2014], Steinke and Ullman [2015a].

1.2 Comparison with Elder [2016]

The criticism about the lower bounds of Hardt and Ullman [2014], Steinke and Ullman [2015a] is not new and prior work has attempted at addressing them with only partial success.

For example, Elder [2016] presented a similar 'balanced" model (called 'Bayesian ADA"), where both the analyst and the mechanism receive a prior P which is a family of distributions, and then the distribution D is drawn according to P (unknown to both the mechanism and the analyst).

From an information theoretic point of view, this model is equivalent to ours when the sampler A 1 is publicly known, since A 1 simply defines a prior. But from a computational point of view, defining the sampling process (i.e., sampling D and the i.i.d. samples from it) in an algorithmic way is better when we would like to focus on computationally bounded samplers.

Elder [2016] only focused on the information-theoretic setting. His main result is that a certain family of mechanisms (ones that only use the posterior means) cannot answer more than ˜ O ( n 4 ) adaptive queries. This, however, does not hold for any mechanism's strategy. In particular, it does not apply to general computationally efficient mechanisms. Our negative result is quantitatively stronger ( n 2 vs n 4 ) and it applies for all computationally efficient mechanisms. 3

Table 1 summarizes the comparison between Theorem 1.5 and the prior lower bounds (ignoring computational hardness assumptions).

1.3 Techniques

We follow a similar technique to that used in Hardt and Ullman [2014] and Steinke and Ullman [2015a], i.e., a reduction to a restricted set of mechanisms, called natural .

Definition 1.8 (Natural mechanism [Hardt and Ullman, 2014]) . A mechanism M is natural if, when given a sample S = ( x 1 , . . . , x n ) ∈ X n and a query q : X → [ -1 , 1] , M returns an answer that is a function solely of ( q ( x 1 ) , . . . , q ( x n )) . In particular, M does not evaluate q on other data points of its choice.

Hardt and Ullman [2014] and Steinke and Ullman [2015a] showed that there exists an adversarial analyst ˜ A that fails any natural mechanism M , even when M is computationally unbounded, and even when D is chosen to be the uniform distribution over { 1 , 2 , . . . , m = 2000 n } (I.e., D is known to everyone). While general mechanisms could simply use the knowledge of the distribution to answer any query, natural mechanisms are more restricted, and can only provide answers based on the n -size dataset S that they get. The restriction to natural mechanisms allowed Steinke and Ullman [2015a] to use interactive fingerprinting codes , which enable to reveal S using Θ( n 2 ) adaptive queries when the answers are accurate and correlated with S .

To construct an attacker A that fails any computationally bounded mechanism (and not just natural mechanisms), prior work forced the mechanism to behave naturally by using a private-key encryption scheme. More specifically, the adversary first samples m secret keys sk 1 , . . . , sk m , and then defines D to be the uniform distribution over the pairs { ( j, sk j ) } j m =1 . The adversary then simulates an adversary

˜ A which fails natural mechanisms as follows: a query ˜ q : [ m ] → [ -1 , 1] issued by ˜ A is translated by A to a set of m encryptions { ct j } j m =1 where each ct j is an encryption of ˜ q ( j ) under the key sk j . These encryptions define a new query q that on input ( j, sk ) , outputs the decryption of ct j under the key sk . However, since M is computationally bounded and has only the secret keys that are part of its dataset S , it can only decrypt the values of ˜ q on points in S , yielding that it effectively behaves naturally .

Note that the above attack A is imbalanced as it injects the secret keys sk 1 , . . . , sk m into D and then uses these keys when it forms queries. In other words, even though the attacker A is known to the mechanism, A is able to fail M by creating a secret correlation between its random coins and the distribution D .

1.3.1 Balanced Adversary via Identity Based Encryption Scheme

For proving Theorem 1.5, we replace the private-key encryption scheme with a public-key primitive called identity-based encryption (IBE) scheme [Shamir, 1984, Cocks, 2001, Boneh and Franklin, 2001]. Such a scheme enables to produce m secret keys sk 1 , . . . , sk m along with a master public key mpk . Encrypting a message to a speific identity j ∈ [ m ] only requires mpk , but decrypting a message for identity j must be done using its secret key sk j . Using an IBE scheme we can achieve a reduction to natural mechanisms via a balanced adversary A = ( A 1 , A 2 ) as follows: A 1 samples keys mpk , sk 1 , . . . , sk m according to the IBE scheme, and defines D to be the uniform distribution over the triplets { ( j, mpk , sk j ) } j m =1 . The analyst A 2 , which does not know the keys, first asks queries of the form q ( j, mpk , sk ) = mpk k for every bit k of mpk in order to reveal it. Then, it follows a strategy as in the previous section, i.e., it simulates an adversary ˜ A which foils natural mechanisms by translating each query query ˜ q : [ m ] → [ -1 , 1] issued by ˜ A by encrypting each ˜ q ( j ) for identity j using mpk . Namely, the IBE scheme allowed the analyst to implement the attack of Hardt and Ullman [2014] and Steinke and Ullman [2015a], but without having to know the secret keys sk 1 , . . . , sk m .

We can implement the IBE scheme using a standard public-key encryption scheme: in the sampling process, we sample m independent pairs of public and secret keys { ( pk j , sk j ) } j m =1 of the original scheme, and define mpk = ( pk 1 , . . . , pk m ) . When encrypting a message for identity j ∈ [ m ] , we could simply encrypt it using pk j (part of mpk ), which can only be decrypted using sk j . The disadvantage of this approach is the large master public key mpk that it induces. Applying the encryption scheme with security parameter of λ , the master key mpk will be of size λ · m and not just λ as the sizes of the secret keys. This means that implementing our balanced adversary with such an encryption scheme would result with a distribution over a large domain X , which would not rule out the possibility to construct a mechanism for distributions over smaller domains. Yet, Döttling and

Garg [2021] showed that it is possible to construct a fully secure IBE scheme using a small mpk of size only O ( λ · log m ) under standard hardness assumptions (e.g., the Computational Diffie Helman problem [Diffie and Hellman, 1976] 4 or the hardness of factoring ).

1.3.2 Key-Agreement Protocol via Balanced Adversary

In order to prove Theorem 1.6, we first explain what type of adversaries the theorem applies to. Recall that in all known attacks (including ours), the adversary A wraps a simpler adversary ˜ A that fails natural mechanisms. In particular, the wrapper A has two key properties:

A knows E x ∼D [ q ℓ ( x )] for the last query q ℓ that it asks (becuase it equals to 1 m ∑ m j =1 ˜ q ℓ ( j ) , where ˜ q ℓ is the wrapped query which is part of A 's view), and

If the mechanism attempts to behave accurately in the first ℓ -1 rounds (e.g., it answers the empirical mean 1 n ∑ x ∈S q ( x ) for every query q ), then A , as a wrapper of ˜ A , will be able to ask a last query q ℓ that would fail any computationally bounded last-round strategy for the mechanism.

We next show that any computationally bounded balanced adversary A that has the above two properties, can be used for constructing a key-agreement protocol. That is, a protocol between two computationally bounded parties P 1 and P 2 that enable them to agree on a value which cannot be revealed by a computationally bounded adversary who only sees the transcript of the execution. See Protocol 1.9.

Protocol 1.9 (Key-Agreement Protocol ( P 1 , P 2 ) via a balanced adversary A = ( A 1 , A 2 ) ) .

Input: 1 n . Let ℓ = ℓ ( n ) and X = X ( n ) be the number queries and the domain that is used by the adversary A .

Operation:

P 1 emulates A 1 on input n for obtaining a distribution D over X (specified by a sampling procedure), and samples n i.i.d. samples S .

P 2 initializes an emulation of A 2 on input n .

For i = 1 to ℓ :

P 2 receives the i th query q i from the emulated A 2 and sends it to P 1 .

P 1 computes y i = 1 n ∑ x ∈S q i ( x ) , and sends it to P 2 .

P 2 sends y i as the i th answer to the emulated A 2 .

P 1 and P 2 agree on E x ∼D [ q ℓ ( x )] .

The agreement of Protocol 1.9 relies on the ability of P 1 and P 2 to compute E x ∼D [ q ℓ ( x )] . Indeed, P 1 can accurately estimate it using the access to the sampling procedure, and P 2 can compute it based on the view of the analyst A 2 (follows by Property 1).

To prove the secrecy guarantee of Protocol 1.9, assume towards a contradiction that there exists a computationally bounded adversary G that given the transcript of the execution, can reveal E x ∼D [ q ℓ ( x )] . Now consider the following mechanism for the ADA game: In the first ℓ -1 queries, answer the empirical mean, but in the last query, apply G on the transcript and answer its output. By the assumption on G , the mechanism will be able to accurately answer the last query, in contradiction to Property 2.

We note that Property 1 can be relaxed by only requiring that A is able to provide a 'good enough" estimation of E x ∼D [ q ℓ ( x )] . Namely, as long as the estimation provided in Property 1 is better than the estimation that an adversary can obtain in Property 2 (we prove that an n Ω(1) multiplicative gap suffices), this would imply that Protocol 1.9 is a weak key-agreement protocol, which can be amplified to a fully secure one using standard techniques.

We also note that by requiring in Game 1.3 that A 1 samples from D according to the sampling procedure, we implicitly assume here that sampling from D can be done efficiently (because A 1 is

assumed to be computationally bounded). Our reduction to key-agreement relies on this property, since if sampling from D could not be done efficiently, then P 1 would not have been a computationally bounded algorithm.

1.4 Perspective of Public Key Cryptography

Over the years, cryptographic research has proposed solutions to many different cryptographic tasks under a growing number of (unproven) computational hardness assumptions. To some extent, this state of affairs is unavoidable, since the security of almost any cryptographic primitive implies the existence of one-way functions [Impagliazzo and Luby, 1989] (which in particular implies that P = NP ). Yet, all various assumptions can essentially be divided into two main types: private key cryptography and public key cryptography [Impagliazzo, 1995]. The former type is better understood: A series of works have shown that the unstructured form of hardness guaranteed by one-way functions is sufficient to construct many complex and structured primitives such as pseudorandom generators [Håstad et al., 1999], pseudorandom functions [Goldreich et al., 1986] and permutations [Luby and Rackoff, 1988], commitment schemes [Naor, 1991, Haitner et al., 2009], universal one-way hash functions [Rompel, 1990], zero-knowledge proofs [Goldreich et al., 1987], and more. However, reductions are much less common outside the one-way functions regime, particularly when constructing public-key primitives. In the famous work of Impagliazzo and Rudich [1989] they gave the first evidence that public key cryptography assumptions are strictly stronger than one-way functions, by showing that key-agreement, which enables two parties to exchange secret messages over open channels, cannot be constructed from one-way functions in a black-box way.

̸

Our work shows that a balanced adversary for the ADA game that has the structure of all known attacks, is a primitive that belongs to the public-key cryptography type. In particular, if public-key cryptography does not exist, it could be possible to construct a computationally bounded mechanism that can handle more than Θ( n 2 ) adaptive queries of a balanced adversary (i.e., we currently do not have a negative result that rules out this possibility).

1.5 Other Related Work

Nissim et al. [2018] presented a variant of the lower bound of Steinke and Ullman [2015a] that aims to reduce the number of queries used by the attacker. However, their resulting lower bound only holds for a certain family of mechanisms, and it does not rule out all computationally efficient mechanisms.

Dinur et al. [2023] revisited and generalized the lower bounds of Hardt and Ullman [2014] and Steinke and Ullman [2015a] by showing that they are a consequence of a space bottleneck rather than a sampling bottleneck. Yet, as in the works by Hardt, Steinke, and Ullman, the attack by Dinur et al. relies on the ability to choose the underlying distribution D and inject secret trapdoors in it, and hence it utilizes an imbalanced adversary.

Recently, lower bounds constructions for the ADA problem were used as a tool for constructing (conditional) lower bounds for other problems, such as the space complexity of adaptive streaming algorithms [Kaplan et al., 2021] and the time complexity of dynamic algorithms Beimel et al. [2022]. Our lower bound for the ADA problem is qualitatively stronger than previous lower bounds (as the adversary we construct has less power). Thus, our lower bound could potentially yield new connections and constructions in additional settings.

1.6 Conclusion and Open Problems

In this work we present the balanced adversarial model for the ADA problem, and show that the existence of a balanced adversary that has the structure of all previously known attacks is equivalent to the existence of public-key cryptography. Yet, we do not know what is the truth outside of the public-key cryptography world. Can we present a different type of efficient attack that is based on weaker hardness assumptions (like one-way functions)? Or is it possible to construct an efficient mechanism that answer more than Θ( n 2 ) adaptive queries assuming that public-key cryptography does not exist? We also leave open similar questions regarding the information theoretic case. We currently do not know whether it is possible to construct an unbounded mechanism that answers exponential number of queries for any distribution D (regardless of its domain size).

In a broader perspective, lower bounds such as ours show that no general solution exists for a problem. They often use unnatural inputs or distributions and rely on cryptographic assumptions. They are important as guidance for how to proceed with a problem, e.g., search for mechanisms that would succeed if the underlying distribution is from a "nice" family of distributions.

2 Preliminaries

2.1 Notations

We use calligraphic letters to denote sets and distributions, uppercase for random variables, and lowercase for values and functions. For n ∈ N , let [ n ] = { 1 , 2 , . . . , n } . Let neg( n ) stand for a negligible function in n , i.e., a function ν ( n ) such that for every constant c > 0 and large enough n it holds that ν ( n ) < n -c . For n ∈ N we denote by 1 n the n -size string 1 . . . 1 ( n times). Let PPT stand for probabilistic polynomial time. We say that a pair of algorithms A = ( A 1 , A 2 ) is PPT if both A 1 and A 2 are PPT algorithms.

2.2 Distributions and Random Variables

Given a distribution D , we write x ∼ D , meaning that x is sampled according to D . For a multiset S , we denote by U S the uniform distribution over S , and let x ← S denote that x ∼ U S . For a distribution D and a value n ∈ N , we denote by D n the distribution of n i.i.d. samples from D . For a distribution D over X and a query q : X → [ -1 , 1] , we abuse notation and denote q ( D ) := E x ∼D [ q ( x )] , and similarly for S = ( x 1 , . . . , x n ) ∈ X ∗ we abuse notation and denote q ( S ) := E x ←S [ q ( x )] = 1 n ∑ n i =1 x i .

2.3 Cryptographic Primitives

2.3.1 Key Agreement Protocols

The most basic public-key cryptographic primitive is a ( 1 -bit) key-agreement protocol, defined below.

Definition 2.1 (key-agreement protocol) . Let π be a two party protocol between two interactive PPT algorithms P 1 and P 2 , each outputs 1 -bit. Let π (1 n ) denote a random execution of the protocol on joint input 1 n (the security parameter), and let O 1 n , O 2 n and T n denote the random variables of P 1 's output, P 2 's output, and the transcript (respectively) in this execution. We say that π is an ( α, β ) -key-agreement protocol if the following holds for any PPT ('eavesdropper') A and any n ∈ N :

Agreement: Pr [ O 1 n = O 2 n ] ≥ α ( n ) , and

Secrecy: Pr [ A ( T n ) = O 1 n ] ≤ β ( n ) .

We say that π is a fully-secure key-agreement protocol if it is an (1 -neg( n ) , 1 / 2 + neg( n )) -keyagreement protocol.

2.3.2 Identity-Based Encryption

An Identity-Based Encryption (IBE) scheme [Shamir, 1984, Cocks, 2001, Boneh and Franklin, 2001] consists of four PPT algorithms ( Setup , KeyGen , Encrypt , Decrypt ) defined as follows:

Setup (1 λ ) : given the security parameter λ , it outputs a master public key mpk and a master secret key msk .

KeyGen ( msk , id ) : given the master secret key msk and an identity id ∈ [ n ] , it outputs a decryption key sk id .

Encrypt ( mpk , id , m ) : given the master public key mpk , and identity id ∈ [ n ] and a message m , it outputs a ciphertext ct .

Decrypt ( sk id , ct ) : given a secret key sk id for identity id and a ciphertext ct , it outputs a string m .

The following are the properties of such an encryption scheme:

Completeness: For all security parameter λ , identity id ∈ [ n ] and a message m , with probability 1 over ( mpk , msk ) ∼ Setup (1 λ ) and sk id ∼ KeyGen ( msk , id ) it holds that

Security: For any PPT adversary A = ( A 1 , A 2 ) it holds that:

where IND IBE A is shown in Experiment 2.2. 5

Experiment 2.2 ( IND IBE A (1 λ ) ) .

( mpk , msk ) ∼ Setup (1 λ ) .

( id ∗ , ( m 0 1 , . . . , m 0 k ) , ( m 1 1 , . . . , m 1 k ) , st ) ∼ A KeyGen ( msk , · ) 1 ( mpk ) where ∣ ∣ m 0 i ∣ ∣ = ∣ ∣ m 1 i ∣ ∣ for every i ∈ [ k ] and for each query id by A 1 to KeyGen ( msk , · ) we have that id = id ∗ .

Sample b ←{ 0 , 1 } .

Sample ct ∗ i ∼ Encrypt ( mpk , id ∗ , m b i ) for every i ∈ [ k ] .

̸

b ′ ∼ A KeyGen ( msk , · ) 2 ( mpk , ( ct ∗ 1 , . . . ct ∗ k ) , st ) where for each query id by A 2 to KeyGen ( msk , · ) we have that id = id ∗ .

Output 1 if b = b ′ and 0 otherwise.

Namely, the adversary chooses two sequences of messages ( m 0 1 , . . . , m 0 k ) and ( m 1 1 , . . . , m 1 k ) , and gets encryptions of either the first sequence or the second one, where the encryptions made for identity id ∗ that the adversary does not hold its key (not allowed to query KeyGen on input id ∗ ). The security requirement means that she cannot distinguish between the two cases (except with negligible probability).

Theorem 2.3 (Döttling and Garg [2021]) . Assume that the Computational Diffie-Hellman (CDH) Problem is hard. Then there exists an IBE scheme E = ( Setup , KeyGen , Encrypt , Decrypt ) for n identities such that given a security parameter λ , the master keys and each decryption key are of size O ( λ · log n ) . 6

2.4 Balanced Adaptive Data Analysis

As described in the introduction, the mechanism plays a game with a balanced adversary that consists of two (isolated) algorithms: a sampler A 1 , which chooses a distribution D over a domain X and provides n i.i.d. samples to M , and an analyst A 2 , which asks the adaptive queries about the distribution. Let ADA n,ℓ, X [ M , A = ( A 1 , A 2 )] denote Game 1.3 on public inputs n - the number of samples, ℓ - the number of queries, and X - the domain. We denote by output 1 the case that M fails in the game, and 0 otherwise. Since this work mainly deals with computationally bounded algorithms that we would like to model as PPT algorithms, we provide n and ℓ in unary representation. We also assume for simplicity that X is finite, which allows to represent each element as a binary vector of dimension ⌈ log |X|⌉ , and we provide the dimension in unary representation as well.

All previous negative results (Hardt and Ullman [2014], Steinke and Ullman [2015a], and Dinur et al. [2023]) where achieved by reduction to a restricted family of mechanisms, called natural mechanisms (Definition 1.8). These are algorithms that can only evaluate the query on the sample points they are given.

For natural mechanisms (even unbounded ones), the following was proven.

Theorem 2.4 (Hardt and Ullman [2014], Steinke and Ullman [2015a]) . There exists a pair of PPT algorithms ˜ A = ( ˜ A 1 , ˜ A 2 ) such that for every natural mechanism ˜ M and every large enough n and ℓ = Θ( n 2 ) it holds that

̸

In particular, ˜ A 1 always chooses the uniform distribution over [2000 n ] , and ˜ A 2 uses only queries over the range {-1 , 0 , 1 } .

3 Constructing a Balanced Adversary via IBE

We prove that if an IBE scheme exists, then there is an efficient reduction to natural mechanisms that holds against any PPT mechanism, yielding a general lower bound for the computational case.

Theorem 3.1 (Restatement of Theorem 1.5) . Assume the existence of an IBE scheme E that supports m = 2000 n identities with security parameter λ = λ ( n ) s.t. n ≤ poly( λ ) (e.g., λ = n 0 . 1 ) using keys of length k = k ( n ) . Then there exists a PPT balanced adversary A = ( A 1 , A 2 ) and ℓ = Θ( n 2 ) + k such that for every PPT mechanism M it holds that

The proof of the theorem is given in the supplementary material. Note that we use a domain X with log |X| = 2 k +log n + O (1) , and by applying Theorem 2.3, the lower bound holds for k = O ( λ · log n ) under the CDH hardness assumption.

4 Reduction to Natural Mechanisms Implies Key Agreement

We prove that any PPT balanced adversary A = ( A 1 , A 2 ) that has the structure of all known lower bounds (Hardt and Ullman [2014], Steinke and Ullman [2015a], Dinur et al. [2023] and ours in Section 3), can be used to construct a key-agreement protocol.

All known constructions use an adversary A that wraps the adversary ˜ A for the natural mechanisms case (Theorem 2.4) by forcing every mechanism M to behave naturally using cryptography. In particular, they all satisfy Properties 1 and 2 from Section 1.3.2.

The formal statement is given in the following theorem. The proof is provided in the supplementary material.

Theorem 4.1 (Restatement of Theorem 1.6) . Assume the existence of a PPT adversary A = ( A 1 , A 2 ) and functions ℓ = ℓ ( n ) ≤ poly( n ) and X = X ( n ) with log |X| ≤ poly( n ) such that the following holds: Let n ∈ N and consider a random execution of ADA n,ℓ, X [ M , A ] where M is the mechanism that given a sample S and a query q , answers the empirical mean q ( S ) . Let D n and Q n be the (r.v. 's of the) values of D and q = q ℓ (the last query) in the execution (respectively), let T n be the transcript of the execution between the analyst A 2 and the mechanism M (i.e., the queries and answers), and let V n be the view of A 2 at the end of the execution (without loss of generality, its input, random coins and the transcript). Assume that

∃ PPT algorithm F s.t. ∀ n ∈ N : Pr [ | F ( V n ) -Q n ( D n ) | ≤ n -1 / 10 ] ≥ 1 -neg( n ) , and

∀ PPT algorithm G and ∀ n ∈ N : Pr[ | G ( T n ) -Q n ( D n ) | ≤ 1 / 10] ≤ 1 / 4 + neg( n ) .

Then using A and F it is possible to construct a fully-secure key-agreement protocol.

Note that Assumption 1 in Theorem 4.1 formalizes the first property in which the analyst knows a good estimation of the true answer, and the PPT algorithm F is the assumed knowledge extractor. Assumption 2 in Theorem 4.1 formalizes the second property which states that the mechanism, which answers the empirical mean along the way, will fail in the last query, no matter how it chooses to act (this behavior is captured with the PPT algorithm G ), and moreover, it is enough to assume that this requirement only holds with respect to to transcript of the execution, and not with respect to the view of the mechanism.

We refer to the full version of the paper, given in the supplementary material, for all the missing proofs.

Acknowledgments and Disclosure of Funding

Kobbi Nissim is partially supported by NSF grant No. CNS-2001041 and a gift to Georgetown University. Uri Stemmer is partially supported by the Israel Science Foundation (grant 1871/19) and by Len Blavatnik and the Blavatnik Family foundation. Eliad Tsfadia is partially supported by the Fulbright Program and a gift to Georgetown University.

References

Bassily, K. Nissim, A. D. Smith, T. Steinke, U. Stemmer, and J. Ullman. Algorithmic stability for adaptive data analysis. In Proceedings of the 48th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2016 , pages 1046-1059, 2016.

Beimel, H. Kaplan, Y. Mansour, K. Nissim, T. Saranurak, and U. Stemmer. Dynamic algorithms against an adaptive adversary: generic constructions and lower bounds. In STOC , pages 1671-1684. ACM, 2022.

Blanc. Subsampling suffices for adaptive data analysis. CoRR , abs/2302.08661, 2023.

Boneh and M. K. Franklin. Identity-based encryption from the weil pairing. In J. Kilian, editor, Advances in Cryptology - CRYPTO 2001, 21st Annual International Cryptology Conference, Proceedings , volume 2139, pages 213-229, 2001.

C. Cocks. An identity based encryption scheme based on quadratic residues. In Cryptography and Coding, 8th IMA International Conference, Proceedings , volume 2260, pages 360-363, 2001.

Dagan and G. Kur. A bounded-noise mechanism for differential privacy. In Conference on Learning Theory , volume 178, pages 625-661. PMLR, 2022.

Diffie and M. E. Hellman. New directions in cryptography. IEEE Trans. Inf. Theory , 22(6): 644-654, 1976.

Dinur, U. Stemmer, D. P. Woodruff, and S. Zhou. On differential privacy and adaptive data analysis with bounded space. CoRR , abs/2302.05707, 2023.

Döttling and S. Garg. Identity-based encryption from the diffie-hellman assumption. J. ACM , 68 (3):14:1-14:46, 2021.

Dwork, V. Feldman, M. Hardt, T. Pitassi, O. Reingold, and A. Roth. Generalization in adaptive data analysis and holdout reuse. In Advances in Neural Information Processing Systems (NIPS) , 2015a.

Dwork, V. Feldman, M. Hardt, T. Pitassi, O. Reingold, and A. Roth. The reusable holdout: Preserving validity in adaptive data analysis. Science , 349(6248):636-638, 2015b.

Dwork, V. Feldman, M. Hardt, T. Pitassi, O. Reingold, and A. L. Roth. Preserving statistical validity in adaptive data analysis. In STOC , pages 117-126. ACM, 2015c.

Elder. Challenges in bayesian adaptive data analysis. CoRR , abs/1604.02492, 2016. URL http://arxiv.org/abs/1604.02492 .

Feldman and T. Steinke. Generalization for adaptively-chosen estimators via stable median. In Proceedings of the 30th Conference on Learning Theory, COLT 2017 , volume 65, pages 728-757. PMLR, 2017.

Feldman and T. Steinke. Calibrating noise to variance in adaptive data analysis. In Conference On Learning Theory, COLT 2018 , volume 75, pages 535-544. PMLR, 2018.

Fish, L. Reyzin, and B. I. P. Rubinstein. Sampling without compromising accuracy in adaptive data analysis. In Algorithmic Learning Theory, ALT 2020 , volume 117, pages 297-318. PMLR, 2020.

Goldreich, S. Goldwasser, and S. Micali. How to construct random functions. Journal of the ACM , 33(4):792-807, 1986.

Goldreich, S. Micali, and A. Wigderson. How to play any mental game or A completeness theorem for protocols with honest majority. In Proceedings of the 19th Annual ACM Symposium on Theory of Computing, STOC 1987 , pages 218-229, 1987.

Haitner, M. Nguyen, S. J. Ong, O. Reingold, and S. Vadhan. Statistically hiding commitments and statistical zero-knowledge arguments from any one-way function. SIAM Journal on Computing , 39(3):1153-1218, 2009.

Hardt and G. Rothblum. A multiplicative weights mechanism for privacy-preserving data analysis. In Proc. 51 st Foundations of Computer Science (FOCS) , pages 61-70. IEEE, 2010.

Hardt and J. Ullman. Preventing false discovery in interactive data analysis is hard. In FOCS , pages 454-463, 2014.

Håstad, R. Impagliazzo, L. A. Levin, and M. Luby. A pseudorandom generator from any one-way function. SIAM Journal on Computing , 28(4):1364-1396, 1999.

Impagliazzo. A personal view of average-case complexity. In Proceedings of the Tenth Annual Structure in Complexity Theory Conference , pages 134-147. IEEE Computer Society, 1995.

Impagliazzo and M. Luby. One-way functions are essential for complexity based cryptography (extended abstract). In 30th Annual Symposium on Foundations of Computer Science, FOCS 1989 , pages 230-235, 1989.

Impagliazzo and S. Rudich. Limits on the provable consequences of one-way permutations. In Annual ACM Symposium on Theory of Computing (STOC) , pages 44-61, 1989.

Jung, K. Ligett, S. Neel, A. Roth, S. Sharifi-Malvajerdi, and M. Shenfeld. A new analysis of differential privacy's generalization guarantees. In ITCS , volume 151 of LIPIcs , pages 31:1-31:17. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020.

Kaplan, Y. Mansour, K. Nissim, and U. Stemmer. Separating adaptive streaming from oblivious streaming using the bounded storage model. In CRYPTO (3) , volume 12827 of Lecture Notes in Computer Science , pages 94-121. Springer, 2021.

Kontorovich, M. Sadigurschi, and U. Stemmer. Adaptive data analysis with correlated observations. In ICML , volume 162 of Proceedings of Machine Learning Research , pages 11483-11498. PMLR, 2022.

Luby and C. Rackoff. How to construct pseudorandom permutations from pseudorandom functions. SIAM Journal on Computing , 17(2):373-386, 1988.

Naor. Bit commitment using pseudorandomness. Journal of Cryptology , 4(2):151-158, 1991.

Nissim, A. D. Smith, T. Steinke, U. Stemmer, and J. R. Ullman. The limits of post-selection generalization. In NeurIPS , pages 6402-6411, 2018.

M. Rogers, A. Roth, A. D. Smith, and O. Thakkar. Max-information, differential privacy, and post-selection hypothesis testing. In IEEE 57th Annual Symposium on Foundations of Computer Science, FOCS 2016 , pages 487-494, 2016.

Rompel. One-way functions are necessary and sufficient for secure signatures. In Annual ACM Symposium on Theory of Computing (STOC) , pages 387-394, 1990.

Russo and J. Zou. Controlling bias in adaptive data analysis using information theory. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, AISTATS 2016 , volume 51, pages 1232-1240. JMLR.org, 2016.

Shamir. Identity-based cryptosystems and signature schemes. In Advances in Cryptology, Proceedings of CRYPTO 1984 , volume 196, pages 47-53, 1984.

Shenfeld and K. Ligett. A necessary and sufficient stability notion for adaptive generalization. In NeurIPS , pages 11481-11490, 2019.

D. Smith. Information, privacy and stability in adaptive data analysis. CoRR , abs/1706.00820, 2017.

Steinke and J. Ullman. Interactive fingerprinting codes and the hardness of preventing false discovery. In COLT , pages 1588-1628, 2015a.

Steinke and J. Ullman. Between pure and approximate differential privacy. CoRR , abs/1501.06095, 2015b.

Peer reviews:
comments: SUMMARY:
The paper considers balanced adversaries that do not have access to the underlying distribution and instead interact with an oracle adversary that chooses the distribution and samples points from it. The authors show that with this more realistic adversarial setting, they can improve the previous known lower bounds for adaptive data analysis.

STRENGTHS:
The authors consider a more realistic balanced adversary that does not have access to the underlying distribution. This adversarial setting is novel and allows for better bounds on the adaptive data analysis.

WEAKNESSES:
While considering this balanced adversary is more practical, I would suggest the authors to include a summary table comparing their setting, assumptions and their bounds with respect to the prior works. This would better highlight the differences and focus more on the key contribution of the paper.

QUESTIONS:
I am confused why the authors have the value 2000n in Theorem 2.4. Is there something particular about this value 2000, and if so how do they arrive at this? Can this be a parameter k where k does not take large values?

LIMITATIONS:
I don't feel there is any direct or evident negative societal impact of this work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAdaptive data analysis considers the setting where a mechanism and an analyst play a game against each other, where for $k$ sequential rounds, the analyst issues queries to a mechanism which holds a dataset of size $n$ sampled from a distribution, the mechanism returns a response, and the analyst chooses the next query accordingly. The mechanism wins the game if all responses are close to the expected value of the queries on the dataset.\n\nUsing tools developed in the context of differential privacy, it was proven that a mechanism can respond to up to $k = O(n^{2})$ queries, while winning the game with high probability. At the same time, it was proven that if the number of queries exceeds $O(n^{2})$, there exists an analyst that can win the game. This lower bound assumes the analyst can choose the distribution, and is based on the cryptographic assumption of the existence of fingerprinting codes. This imposes an asymmetry between the power and knowledge of the analyst and mechanism which is unnatural, since an analyst that knows the distribution has no reason to query the mechanism.\n\nThe current work solves this problem, by proving a similar lower bound for a balanced setting, where the distribution is chosen by a third player. This proof is based on the stronger cryptographic assumption of the existence of public-key cryptography. The authors also prove that the lower bound does not hold, without this assumption, proving a separation between the balanced and unbalanced settings.\n\nSTRENGTHS:\nThe deep asymmetry that was used in the proof the previous lower bounds was undesired, and left an open question whether it is essential, since it is not justified in most natural settings. Solving this open question is an important step in understanding the dynamics of adaptive data analysis. They are based on a novel addition to the existing lower bound framework, and are presented in a clear and formal way.\n\nThe distinction between the two cryptographic assumptions required in both cases, might hold implications for cryptography as well.\n\nWEAKNESSES:\nI have only one minor comment to add. It seems like the authors use another implicit assumption throughout the paper, that the mechanism knows the analyst it is playing against. If this was not the case, the separation of the analyst into two analysts (one choosing the distribution and the other issuing the queries) will be meaningless, as we could always consider the case where the first analyst always chooses the same distribution, and so the second one can use that information while the mechanism cannot. \n\nI do not bring this up as a limitation of the work, but as a recommendation to clarify the setting, so to avoid confusion by the reader.\n\n=======\n\n**Edit after rebuttal discussion:**\n\nI have no further concerns.\n\nQUESTIONS:\nIn section 2.4 the authors mention the sample size, number of queries and dimensions are provided in unary representation. I would appreciate if the authors can clarify how this affects the computational complexity, and how they justify this decision.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 9, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-04"}

comments: SUMMARY:
The paper proposes to consider balanced adversaries for ADA. Under the public-key assumptions, the paper proves that

STRENGTHS:
1. The paper assumes a more practical threat model for ADA.

2. The theoretical results in this paper seem to be well-supported. I am not an expert in this research direction, so I am not sure about my judgement.

WEAKNESSES:
1. There is no discussion about the real-world machine learning applications of the results.

2. There is no conclusion section to summarize the paper. The paper ends after Section 4 and makes me feel that the paper is not complete.

3. The theoretical results are based on the assumption that public-key cryptography exists.

4. I am not an expert in this research direction, and I feel that this paper is not easy to follow because there are many terms. I suggest that the paper could add a table to explain the terms used in the paper to improve the readability.

QUESTIONS:
1. Is there any real-world machine learning application of the results?

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies limitations of \"adaptive data analysis\" with a focus on \"computational assumptions\" that are used for proving such limitations (i.e., lower bounds).\n\nA mechanism $M$ answers $m=m(n)$ queries (almost) correctly, if it succeeds in the following game for every distribution $D$ and PPT adversary $A$. The mechanism is given n iid samples from D. Then the adversary $A$ asks $m$ adaptive statistical queries $q_1,\\dots,q_m$ from the mechanism $M$. The mechanism succeeds if all of its answers are within $\\pm 1/10$ of the true answers with respect to distribution $D$.\n\nPrevious work has shown that at most $m=\\Theta(n^2)$ queries can be answered adaptively by computationally bounded mechanisms, assuming one-way functions exist. Their proof uses private-key encryption schemes (whose existence is equivalent to one-way functions) and a reduction from a class of mechanisms, called \"natural\". The previous lower bound also implicitly assumes a rather powerful adversary who both chooses the distribution D as well as the queries $q_1,...q_m$.\n\nThis paper starts by making the (interesting) observation that if an adversary already knows the distribution $D$, then it won't have a meaningful interest in making the queries. So, in some sense, the previous lower bounds are not as meaningful as one wishes in real world scenarios; this raises the hope for getting positive results that go around the lower bounds by leveraging this observation.\n\nIn summary, the main result of the paper is bootstrap the previous lower bound to a model to a weaker adversarial model.\nIn more detail, the contributions of this paper are as follow:\n\n1. A new formal model of security called the \"balanced model\" is defined, in which the adversary is split into two entities that do not communicate. The first adversary $A_1$ picks the distribution $D$ and the iid samples. The second adversary $A_2$ picks the adaptive queries.\n2. It is shows that assuming the stronger primitive of secure public-key encryption (as opposed to just OWFs) one can still extend the previous lower to the balanced model.\n3. The assumption of public-key encryption is somehow justified, by showing any lower bound with minimal properties (that hold for current lower bound proofs) would lead to key agreement protocols (which is a close primitive/assumption to public key encryption).\n\nSTRENGTHS:\nThe main strengths of the paper:\n\n* excellent writing quality. my only criticism is that the definition used in *previous* work is not presented formally, and one cannot determine the exact details from the informal text. this is needed for making a fully detailed comparison between the new definition and the old one, because this is a major part of this paper.\n\n* revisiting a previous lower bound for an important problem from a natural practical perspective.\n\nWEAKNESSES:\nMy criticisms of the paper are the following:\n\n1. About the model (Item 1 above): I am not sure if the new split (balanced) model is adding much value actually. suppose we fix a mechanism $M$ and look for a balanced 2-part polynomial-size adversary $(A_1,A_2)$ against $M$. In this setting, it seems meaningless to say that only $A_1$ knows the distribution D, because one can fix the distribution $D$ that $A_1$ picks to its \"best possible\" distribution (against $M$). This way, $A_2$ also would be aware of this distribution implicitly, as it can depend on it. In other words, one can see this argument as giving shared coin to $A_1,A_2$, and then fixing it to its best value (which leads to no communication between $A_1$ and $A_2$). \n\nI shall add that, from a game theoretic perspective, it would seems a preferred and stronger lower bound if we could use *one* fixed adversary strategy that bounds the power of every computationally limited mechanism. However, for the sake of proving lower bounds, it is fine if we'd pick the adversary $A_M$ based on the mechanism $M$. \n\n2. About the technical depth of the lower bound (Item 2 above). This proof seems like a rather straightforward adaptation of the previous proof. but using IBE (or a bunch of public key encryption schemes) instead of Secret Key Encryption schemes. This is a mild criticism, as I am in general in favor of simpler proofs. But this proof seems like a simple proof for a new question rather than a simple proof for a question that existed before, so I am a bit torn here.\n\n3. About necessity of Key Agreement (Item 3 above): I am not a fan of investing in weakening assumptions that are used for *lower bounds*. as I believe there is a fundamental difference between weakening assumptions behind positive results vs those of negative results. When we weaken the assumption behind, say, public key encryption from an assumption $P_1$ to a weaker assumption $P_2$, it means that the schemes that would be deployed *right now* would not break at an unknown time (with potential catastrophic consequences) even if $P_1$ turns to be false. However, when we weaken the (still widely believed) computational assumption behind a a *lower bound*, is is a completely different story, because even using an assumption as strong as IO to prove an impossibility has the same practical consequence as using OWFs. The two scenarios would diverge (and a weaker assumption would start to look interesting) after breaking every IO scheme, which is most likely never gonna happen, and the weakening assumption changes nothing right now. So, in summary, when it comes to lower bounds I am totally fine with using something as strong as any assumption that seems strong enough for the foreseeable future.\n\nupdate: the issues above were discussed and I am happy with author(s)' response.\n\nQUESTIONS:\nMy main question is about the meaningfulness of the balanced attack model which is described above.\n\nLIMITATIONS:\nThe limitations are also elaborated in the \"weakness\" section of the review above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}