Paper: Differentially Pivate Per-Instance Additive Noise Mechanism: A Game Theoretic Approach (ICLR-2024)

Reviewer comment (Experiments):
The experiments conducted in this study were carried out with a relatively low level of privacy protection, specifically $\epsilon = \{1,2,4,8\}$. It is worth noting that, in practical scenarios, an even lower value of $\epsilon$ (i.e., $\epsilon<1$) is often preferred. I would greatly appreciate it if the authors could either provide further justification for their choice of $\epsilon$ or consider conducting additional experiments with smaller values of $\epsilon$ to address this concern comprehensively.

Author response:
We sincerely appreciate this suggestion. The authors kindly note that $\epsilon>1$ is widely used for practical DP use cases. For example, [Apple](https://www.apple.com/privacy/docs/Differential_Privacy_Overview.pdf) is known to use $\epsilon$-DP with epsilon values ranging between 2 and 8.

However, some previous papers have provided results for $\epsilon<1$ in extreme cases. Therefore, we conducted additional experiments for the cases where $\epsilon=0.1$ and $\epsilon=0.3$.

- For additional experiments, please refer to Figures 6, 7, 8, 9, and 10. Based on the newly added results, we found that the proposed NVO game still better preserves utility while guaranteeing $\epsilon$-pDP. In these cases, the Laplace mechanism provides a uniform distribution, with almost zero utility. We added the results in Appendices D and E, because of the page limit.

Reviewer comment (Evaluation):
I am genuinely curious about the computational cost associated with this approach. The datasets used in the experiments consisted of roughly one thousand samples, and the variance set was defined as relatively small. For the best response dynamics (BRD) approach, it would be valuable if the authors could present the computational time required.

Author response:
Thank you for providing new suggestions. The reviewer raised questions about computational complexity of BRD approach and robustness of privacy preserving in AE approach. The bellows are one-by-one responses to the suggestions.

**Computational complexity of BRD approach**: First of all, we kindly remind you that the wall-clock time of the BRD algorithm is available in Tables 1, 3, 4, and 5 for various datasets, including a newly added large dataset (credit profile dataset). Moreover, for computational complexity analysis, we first bring a fact from proof of Theorem 4.1 that in each round of the game, at least one player has a change of incrementally guaranteeing the pDP of a data instance. Thus, for privacy preserving, the maximum number of required steps is $|\mathcal{D}|$. Since the computational complexity of the payoff computation in each step of a round is $O(|\mathcal{D}|)$, the worst computational complexity of the BRD algorithm is $O(|\mathcal{D}|^2)$. 

For extremely large datasets, the required computation time becomes a limitation of our work. In such scenarios, it might be necessary to consider methods such as grouping data points with identical query output values to reduce computation by adding the same noises. 
Yet, additional future work is required to develop a better solution.

To comply with the comment, we have revised the manuscript.
- Added statements in **Limitation** paragraph in Section 7.
- Add a paragraph **Convergence of BRD toward NE** in Section 5.1 

**Failure of AE to obtain NE point**:
With ample time, the AE algorithm has the capability to reach a level of performance comparable to that of the exact enumeration algorithm, achieving an NE point. We extended our experiments for an extended duration to ensure convergence. However, there's no theoretical assurance of attaining an NE point within a polynomial time frame. In our revised manuscript, we relocated the detailed explanations of the AE algorithm to Appendix B. It's worth noting that we included this algorithm to assess the effectiveness of a non-greedy approach within our proposed NVO game when provided with ample computational resources.

We strongly agree that we should address the exact solution that satisfies pDP. Thus, we additionally added more discussions of convergence in the BRD algorithm.

- To reflect the comment, we moved the AE algorithm part into Appendix B on page 14, since AE is no longer our main focus.

Reviewer comment (Theory):
Regarding the approximate enumeration (AE) approach, I have some reservations about its privacy protection level, as Theorem 4.1 can only guarantee privacy protection for a Nash Equilibrium (NE) point. However, there is a possibility that AE may fail to obtain any NE point, which raises questions about the robustness of privacy protection in this context.

Author response:
Thank you for providing new suggestions. The reviewer raised questions about computational complexity of BRD approach and robustness of privacy preserving in AE approach. The bellows are one-by-one responses to the suggestions.

**Computational complexity of BRD approach**: First of all, we kindly remind you that the wall-clock time of the BRD algorithm is available in Tables 1, 3, 4, and 5 for various datasets, including a newly added large dataset (credit profile dataset). Moreover, for computational complexity analysis, we first bring a fact from proof of Theorem 4.1 that in each round of the game, at least one player has a change of incrementally guaranteeing the pDP of a data instance. Thus, for privacy preserving, the maximum number of required steps is $|\mathcal{D}|$. Since the computational complexity of the payoff computation in each step of a round is $O(|\mathcal{D}|)$, the worst computational complexity of the BRD algorithm is $O(|\mathcal{D}|^2)$. 

For extremely large datasets, the required computation time becomes a limitation of our work. In such scenarios, it might be necessary to consider methods such as grouping data points with identical query output values to reduce computation by adding the same noises. 
Yet, additional future work is required to develop a better solution.

To comply with the comment, we have revised the manuscript.
- Added statements in **Limitation** paragraph in Section 7.
- Add a paragraph **Convergence of BRD toward NE** in Section 5.1 

**Failure of AE to obtain NE point**:
With ample time, the AE algorithm has the capability to reach a level of performance comparable to that of the exact enumeration algorithm, achieving an NE point. We extended our experiments for an extended duration to ensure convergence. However, there's no theoretical assurance of attaining an NE point within a polynomial time frame. In our revised manuscript, we relocated the detailed explanations of the AE algorithm to Appendix B. It's worth noting that we included this algorithm to assess the effectiveness of a non-greedy approach within our proposed NVO game when provided with ample computational resources.

We strongly agree that we should address the exact solution that satisfies pDP. Thus, we additionally added more discussions of convergence in the BRD algorithm.

- To reflect the comment, we moved the AE algorithm part into Appendix B on page 14, since AE is no longer our main focus.

Reviewer comment (Theory):
I think the game theoretic framing is not meaningful. Perhaps it was included as a way to justify the optimization procedure. I would prefer if it was removed because I believe it does not add anything to the paper, but makes things more confusing.

Author response:
Thank you for your valuable feedback on our manuscript. In light of your comments, we have added an explanation about our choice to use game-theoretic interpretations. While there are various approaches to introduce our work, such as optimization theory, we opted for the method we believed would be most accessible and familiar to our audience. We acknowledge that our initial explanation, particularly regarding Figure 1, had certain shortcomings. 
Also, we change the vector-form data representation $\mathbf{d}_i$ into a scalar representation $d_i$ for simplicity. Accordingly, we have revised the manuscript for improved clarity and a more effective presentation of our approach. We earnestly hope that these modifications will meet the reviewer's expectations and enhance the overall quality of our manuscript. In addition, we revised the fatal errors you pointed out.

---

**1. Use of game theory**
> Comment: Why did you decide to use a game-theoretic presentation?

We are sincerely sorry for the complicated interpretation. 
We use game-theoretic presentation because we think it is familiar to the audience.
As a result, the proposed algorithm is a greedy algorithm for updating each data instance’s additive noise.
Here, we kindly note that we mainly contributed by showing the NE point of the NVO game guarantees the $\epsilon$-pDP for all data instances, where the greedy algorithm cannot guarantee $\epsilon$-pDP if Equation 9 does not hold.

Also, we note that the proposed work has the following challenges in optimization perspective.

We can represent the target optimization as a constraint problem.

$$\min U(\mathcal{D}, \mathcal{M}(\mathcal{D})) ~~~ \textnormal{s.t.} \left|\ln\frac{\Pr[\mathcal{M}(\mathcal{D}) \in S]}{\Pr[\mathcal{M}\textcolor{blue}{(\mathcal{D} \setminus \{\mathbf{d}\})} \in S]}\right| \leq \epsilon, \forall \mathbf{d} \in \mathcal{D}, \forall S\in\mathnormal{Range}(\mathcal{M})$$

- Initially, the continuous nature of the domain $S$ poses a unique challenge, differing from that in $\epsilon$-DP. In this context, it's necessary to confirm the $\epsilon$-pDP condition for each instance and each point within $S$, an endeavor impractical to accomplish in polynomial time. To address this issue, we have employed data categorization in our approach.
- Furthermore, the variable $\mathbf{b}_i$ exhibits both continuity and a complex interdependence. While gradient descent is a common technique for finding local optima in standard optimization problems, our situation is complicated by the constraint being a non-differentiable function, which makes typical projection methods ineffective. As a solution, we have opted to moderate the variance of $b$, thereby simplifying the problem.

Regrettably, despite the desire to pivot the problem definition from game theory to optimization theory, substantial portions had already been developed centered on game theory. Fully acknowledging the difficulty in comprehending game theory-centric explanations, we extensively revised Section 4 of the paper, although every change cannot be highlighted. We made substantial modifications to enhance the clarity of explanations. We earnestly hope that these revisions bring greater satisfaction to your understanding.

We sincerely apologize for confusion, and we hope these revisions significantly enhance the paper's comprehensibility.

Reviewer comment (Novelty):
My interpretation of what you are doing is setting two objective functions for both the utility and privacy, then you are optimizing their sum in a greedy way. With this understanding, I think this work falls below the standard for this venue.

Author response:
Thank you for your valuable feedback on our manuscript. In light of your comments, we have added an explanation about our choice to use game-theoretic interpretations. While there are various approaches to introduce our work, such as optimization theory, we opted for the method we believed would be most accessible and familiar to our audience. We acknowledge that our initial explanation, particularly regarding Figure 1, had certain shortcomings. 
Also, we change the vector-form data representation $\mathbf{d}_i$ into a scalar representation $d_i$ for simplicity. Accordingly, we have revised the manuscript for improved clarity and a more effective presentation of our approach. We earnestly hope that these modifications will meet the reviewer's expectations and enhance the overall quality of our manuscript. In addition, we revised the fatal errors you pointed out.

---

**1. Use of game theory**
> Comment: Why did you decide to use a game-theoretic presentation?

We are sincerely sorry for the complicated interpretation. 
We use game-theoretic presentation because we think it is familiar to the audience.
As a result, the proposed algorithm is a greedy algorithm for updating each data instance’s additive noise.
Here, we kindly note that we mainly contributed by showing the NE point of the NVO game guarantees the $\epsilon$-pDP for all data instances, where the greedy algorithm cannot guarantee $\epsilon$-pDP if Equation 9 does not hold.

Also, we note that the proposed work has the following challenges in optimization perspective.

We can represent the target optimization as a constraint problem.

$$\min U(\mathcal{D}, \mathcal{M}(\mathcal{D})) ~~~ \textnormal{s.t.} \left|\ln\frac{\Pr[\mathcal{M}(\mathcal{D}) \in S]}{\Pr[\mathcal{M}\textcolor{blue}{(\mathcal{D} \setminus \{\mathbf{d}\})} \in S]}\right| \leq \epsilon, \forall \mathbf{d} \in \mathcal{D}, \forall S\in\mathnormal{Range}(\mathcal{M})$$

- Initially, the continuous nature of the domain $S$ poses a unique challenge, differing from that in $\epsilon$-DP. In this context, it's necessary to confirm the $\epsilon$-pDP condition for each instance and each point within $S$, an endeavor impractical to accomplish in polynomial time. To address this issue, we have employed data categorization in our approach.
- Furthermore, the variable $\mathbf{b}_i$ exhibits both continuity and a complex interdependence. While gradient descent is a common technique for finding local optima in standard optimization problems, our situation is complicated by the constraint being a non-differentiable function, which makes typical projection methods ineffective. As a solution, we have opted to moderate the variance of $b$, thereby simplifying the problem.

Regrettably, despite the desire to pivot the problem definition from game theory to optimization theory, substantial portions had already been developed centered on game theory. Fully acknowledging the difficulty in comprehending game theory-centric explanations, we extensively revised Section 4 of the paper, although every change cannot be highlighted. We made substantial modifications to enhance the clarity of explanations. We earnestly hope that these revisions bring greater satisfaction to your understanding.

We sincerely apologize for confusion, and we hope these revisions significantly enhance the paper's comprehensibility.

Reviewer comment (Theory):
There are some errors, even in very important mathematical statements, for example, as written, the LHS of (1) is always 0 since $z \in \mathcal{Z}$. I had to read the original reference to understand what pDP was. Please address this.

Author response:
**2. Correction of Equation 1**
> Comment: What does (1) mean, as the current statement seems meaningless?

We deeply apologize for an oversight in a crucial definition that permeates the entire paper. We sincerely appreciate your interest in our work.
We have rectified a fatal error in Definition 3.1 as follows.

Definition 3.1 ($\epsilon$-pDP)
A randomized mechanism, denoted by $\mathcal{M}$, has a range $\mathcal{R}(\mathcal{M})$. For a fixed dataset $\mathcal{Z}$ and a fixed data instance $z\in\mathcal{Z}$, the mechanism $\mathcal{M}$ meets $\epsilon$-pDP, if the following condition holds:

$$
\bigg|\ln\frac{\Pr[\mathcal{M}(\mathcal{Z}) \in S]}{\Pr[\textcolor{blue}{\mathcal{M}(\mathcal{Z} \setminus \{z\})} \in S]}\bigg| \leq \epsilon, ~ \forall S \subseteq \textnormal{Range}(\mathcal{M}).
$$


We have uniformly rectified other sections to maintain consistency. 

**Remark:** The variance in mathematical notations compared to Wang (2019) lies in Definition 3.1. Consequently, it deals with the difference in probability of query outputs based on the presence or absence of a single data point, thus remaining equivalent meaning.

Reviewer comment (Writing):
What does (1) mean, as the current statement seem meaningless?

Author response:
**2. Correction of Equation 1**
> Comment: What does (1) mean, as the current statement seems meaningless?

We deeply apologize for an oversight in a crucial definition that permeates the entire paper. We sincerely appreciate your interest in our work.
We have rectified a fatal error in Definition 3.1 as follows.

Definition 3.1 ($\epsilon$-pDP)
A randomized mechanism, denoted by $\mathcal{M}$, has a range $\mathcal{R}(\mathcal{M})$. For a fixed dataset $\mathcal{Z}$ and a fixed data instance $z\in\mathcal{Z}$, the mechanism $\mathcal{M}$ meets $\epsilon$-pDP, if the following condition holds:

$$
\bigg|\ln\frac{\Pr[\mathcal{M}(\mathcal{Z}) \in S]}{\Pr[\textcolor{blue}{\mathcal{M}(\mathcal{Z} \setminus \{z\})} \in S]}\bigg| \leq \epsilon, ~ \forall S \subseteq \textnormal{Range}(\mathcal{M}).
$$


We have uniformly rectified other sections to maintain consistency. 

**Remark:** The variance in mathematical notations compared to Wang (2019) lies in Definition 3.1. Consequently, it deals with the difference in probability of query outputs based on the presence or absence of a single data point, thus remaining equivalent meaning.

Reviewer comment (Writing):
What is $\mathbf{b}_j$ in the inner for loop of Algorithm 1?

Author response:
**3. Correction of Algorithm 1**
> Comment: What does it mean to take a union of vectors as in Algorithm 1?
> Comment: What is bj in the inner for loop of Algorithm 1?

We sincerely appreciate your corrections. The authors agree that the notation is really confusing. We deeply apologize for this typo in the explanation of the main algorithm. We corrected it to make the notations clearer.

**Union of vectors in Algorithm 1**: 
Thank you for this question. We have a mistake in this notation, we forgot to exploit the set notation. We replace $\mathcal{D}=\bigcup_{i=1}^{|\mathcal{D}|}d_{i}$ by $\mathcal{D}=\set{d_{i}|i=1,...,m}$ 

and $\mathcal{V} =\bigcup_{i=1}^{|\mathcal{V}|}{v_{i}}$ into $\mathcal{V}=\set{v_{i}|i=1,...,n}$.


**$b_j$ in the inner for loop of Algorithm 1**: We apologize sincerely for the error in depicting $b$ (now $v$) as a vector when it should have been a scalar. To clarify, $b_j$ (now $v_j$) refers to the variance value corresponding to the $j$-th element in the variance set $\mathcal{V} = \set{v_{1}, v_{2}, …, v_{n}}$.

Reviewer comment (Theory):
Lack of problem formulation. A central premise of the paper is that "ensuring pDP for a particular data instance is inherently dependent on the noise distribution of other instances". However the claim is not formulated in mathematical terms, making the paper difficult to follow for those readers who are not already convinced of this claim before reading the paper. The lack of problem formulation also makes it confusing why existing noise mechanisms (for example the per instance Gaussian mechanism in Wang (2019)) are not desirable.

Author response:
**1. Problem formulation should be addressed.**
> Comment: Can you formulate the difficulty of optimizing noise distribution for pDP in mathematical terms? What target is being optimized? What is the relation of your work to, for example, the Gaussian mechanism in Wang (2019)?

Answer: Thank you for this suggestion. To comply with the comment, we first added the following optimization problem in the main paper.

$$\min U(\mathcal{D}, \mathcal{M}(\mathcal{D})) ~~~ \textnormal{s.t.} \left|\ln\frac{\Pr[\mathcal{M}(\mathcal{D}) \in S]}{\Pr[\mathcal{M}(\mathcal{D} \setminus \{d\}) \in S]}\right| \leq \epsilon, \forall d \in \mathcal{D}, \forall S\in\mathrm{Range}(\mathcal{M})$$

where the objective function minimized represents the distributional difference between $q(\mathcal{D})$ and $q(\mathcal{M}(\mathcal{D}))$, with various metrics available. 

Our primary focus is on addressing the constraint of the problem, namely $\epsilon$-pDP. Consider the term $\left|\ln\frac{\Pr[\mathcal{M}(\mathcal{D}) \in S]}{\Pr[\mathcal{M}(\mathcal{D} \setminus \{d\}) \in S]}\right|$.

- First, the domain of $S$ is continuous, which presents a challenge distinct from that in $\epsilon$-DP. Here, we must verify the $\epsilon$-pDP condition for each instance and each point within $S$, a task that is not feasible within polynomial time. To overcome this, our method incorporates data categorization.
- Second, the variance parameter $v$ is continuous and intricately interlinked. In typical optimization problems, gradient descent is utilized to identify a locally optimal solution. However, in our case, the constraint represents a non-differentiable function, rendering common projection methods unsuitable. Therefore, we relax the diversity of $v$ to simplify and solve the problem effectively.

We propose an NVO game, designed based on a relaxed version of our initial problem. Despite this relaxation, ensuring $\epsilon$-pDP remains a significant challenge. In Theorem 4.1, we demonstrate that our proposed NVO game successfully achieves $\epsilon$-pDP, provided that the condition (9) is met.

**Relation to $\epsilon$-pDP study :**
Wang's (2019) study drew attention to the varying levels of privacy protection in data instances when the same noise is added to each query output. However, it primarily focused on identifying and analyzing these issues, **without offering concrete solutions**. In contrast, our paper represents a pioneering effort in this field. We **formulate and address a problem that involves optimizing the addition of distinct noises to individual data instances**. This approach ensures a uniform level of pDP throughout the dataset while maintaining its statistical utility. We appreciate your feedback highlighting this critical aspect of our work. Accordingly, we have revised the paper to more prominently feature this significant contribution."

- We have revised statements to reflect this discussion. (please refer to Section 1 on page 1, and Section 4.1 on page 4)

Reviewer comment (Writing):
Ambiguous scope. The first 2.5 pages of the paper give an impression that the new noise addition mechanism, similar to the Laplace mechanism for DP, is suitable for general queries, but on page 3 it is then stated that "we focus on the random sampling query" and $q$ is defined to be the random sampling query. Later, in Remark 4.1 on page 4, the utility function uses $q$ to refer to generic queries. It is ambiguous whether the paper's results are applicable to queries other than the random sampling query.

Author response:
**2. Extensibility to other queries.**
> Comment: Is your work, in the present form, limited to the random sampling query?

Answer: **It is definitely possible. The random sampling query is a fundamental query that encompasses all possible statistical queries.** More specifically, the random sampling query can capture the statistical distribution of a dataset. Therefore, based on the post-processing theorem, our proposed NVO game can answer all statistical queries that rely on this distribution. 

**Example:** Let's consider the NBA player height dataset. Suppose the query asks for the proportion of players with a height of 203 cm or above. In this scenario, we can derive the response from the distribution of the modified dataset through random sampling queries. In the original height dataset, the proportion of players with a height of 203 cm or above is 49.16%. If one player who is 210 cm tall is removed from the dataset, the proportion changes to 49.08%, with the difference being only 0.08%. However, after applying our algorithm, the calculated proportions are 48.84% with the player and 48.80% without the player.

- To comply with the comment, we have added Remark 3.1 on page 3, and a new discussion in Section 7 on page 9 (paragraph **Extensibility to other queries**).

Reviewer comment (Novelty):
In "Relation to ($\epsilon, \delta$)-DP" under Section 2, the citation for Gaussian mechanism is incorrect: Dwork and Roth (2014) is an expository work; the Gaussian mechanism appeared in the literature much earlier.

Author response:
**4. Minor issues**
> Comment: In "Relation to (e, delta)-DP" under Section 2, the citation for Gaussian mechanism is incorrect: Dwork and Roth (2014) is an expository work; the Gaussian mechanism appeared in the literature much earlier.
Answer: We have revised the sentence to comply with the comment. 
- Page 2: The pioneer of DP, Dwork & Roth (2014), defined (ϵ, δ)-DP, demonstrating that the Gaussian mechanism can achieve this definition.

Reviewer comment (Theory):
The definition of pDP is imprecise. Compared to the original definition in Wang (2019), Definition 3.1 in this paper does not explicitly fix the data set $\mathcal Z$.

Author response:
**4. Minor issues**
> Comment: The definition of pDP is imprecise. Compared to the original definition in Wang (2019), Definition 3.1 in this paper does not explicitly fix the data set Z.

Answer: Thank you for your careful comment. We added **fixed** dataset $\mathcal{Z}$ in the Definition 3.1.

- Please refer to Definition 3.1 to see the changes.

Reviewer comment (Theory):
There is an incorrect statement above Definition 3.2: pDP holding for "every $z$ within $\mathcal Z$" does not guarantee DP, if the data set $\mathcal Z$ is fixed.

Author response:
**4. Minor issues**
> Comment: There is an incorrect statement above Definition 3.2: pDP holding for "every $z$ within $\mathcal{Z}$" does not guarantee DP, if the data set $\mathcal{Z}$ is fixed.

Answer: Thank you for this clarification again. To correct the statement, we have added a new condition “for all dataset $\mathcal{Z}$”.
- Please refer to statements below Equation 1 and Equation 9.

Reviewer comment (Theory):
Can you formulate the difficulty of optimizing noise distribution for pDP in mathematical terms? What target is being optimized? What is the relation of your work to, for example, the Gaussian mechanism in Wang (2019)?

Author response:
**1. Problem formulation should be addressed.**
> Comment: Can you formulate the difficulty of optimizing noise distribution for pDP in mathematical terms? What target is being optimized? What is the relation of your work to, for example, the Gaussian mechanism in Wang (2019)?

Answer: Thank you for this suggestion. To comply with the comment, we first added the following optimization problem in the main paper.

$$\min U(\mathcal{D}, \mathcal{M}(\mathcal{D})) ~~~ \textnormal{s.t.} \left|\ln\frac{\Pr[\mathcal{M}(\mathcal{D}) \in S]}{\Pr[\mathcal{M}(\mathcal{D} \setminus \{d\}) \in S]}\right| \leq \epsilon, \forall d \in \mathcal{D}, \forall S\in\mathrm{Range}(\mathcal{M})$$

where the objective function minimized represents the distributional difference between $q(\mathcal{D})$ and $q(\mathcal{M}(\mathcal{D}))$, with various metrics available. 

Our primary focus is on addressing the constraint of the problem, namely $\epsilon$-pDP. Consider the term $\left|\ln\frac{\Pr[\mathcal{M}(\mathcal{D}) \in S]}{\Pr[\mathcal{M}(\mathcal{D} \setminus \{d\}) \in S]}\right|$.

- First, the domain of $S$ is continuous, which presents a challenge distinct from that in $\epsilon$-DP. Here, we must verify the $\epsilon$-pDP condition for each instance and each point within $S$, a task that is not feasible within polynomial time. To overcome this, our method incorporates data categorization.
- Second, the variance parameter $v$ is continuous and intricately interlinked. In typical optimization problems, gradient descent is utilized to identify a locally optimal solution. However, in our case, the constraint represents a non-differentiable function, rendering common projection methods unsuitable. Therefore, we relax the diversity of $v$ to simplify and solve the problem effectively.

We propose an NVO game, designed based on a relaxed version of our initial problem. Despite this relaxation, ensuring $\epsilon$-pDP remains a significant challenge. In Theorem 4.1, we demonstrate that our proposed NVO game successfully achieves $\epsilon$-pDP, provided that the condition (9) is met.

**Relation to $\epsilon$-pDP study :**
Wang's (2019) study drew attention to the varying levels of privacy protection in data instances when the same noise is added to each query output. However, it primarily focused on identifying and analyzing these issues, **without offering concrete solutions**. In contrast, our paper represents a pioneering effort in this field. We **formulate and address a problem that involves optimizing the addition of distinct noises to individual data instances**. This approach ensures a uniform level of pDP throughout the dataset while maintaining its statistical utility. We appreciate your feedback highlighting this critical aspect of our work. Accordingly, we have revised the paper to more prominently feature this significant contribution."

- We have revised statements to reflect this discussion. (please refer to Section 1 on page 1, and Section 4.1 on page 4)

Reviewer comment (Theory):
Is the algorithm in Section 5.2 trying to find an approximate instead of an exact solution? If so, does the pDP guarantee hold for the approximate solution? It appears that we only know from Theorem 4.1 that the exact solution satisfies pDP.

Author response:
**3. Guarantees of AE algorithms for the NE.**

> Comment: Is the algorithm in Section 5.2 trying to find an approximate instead of an exact solution? If so, does the pDP guarantee hold for the approximate solution? It appears that we only know from Theorem 4.1 that the exact solution satisfies pDP.

**AE Algorithm**: With **ample time**, the AE algorithm has the potential to match the performance of the exact enumeration algorithm and attain an NE point. Our experiments continued for an extended period to ensure convergence. Nevertheless, there is **no theoretical guarantee** that an **NE point is achievable within polynomial time**. In the updated version of our manuscript, we have relocated the specifics of **the AE algorithm to Appendix B**. It is important to mention that this algorithm was included to evaluate the performance of a non-greedy approach in our proposed NVO game when given ample computational time.

We strongly agree that we should address the exact solution that satisfies pDP. Thus, we additionally added more discussions of convergence in the BRD algorithm.

- To reflect the comment, we moved the AE algorithm part into Appendix B on page 14 and added this discussion.