erchiw/DPriv-Bench huggingface dataset:
Question: Let $\mathcal{M}$ be an $\varepsilon$-DP Laplace mechanism. For any $\alpha>1$, does $\mathcal{M}$ satisfy $(\alpha, \varepsilon^\prime)$-RDP with
\[
\varepsilon^{\prime}(\alpha)
= \frac{1}{\alpha-1}\,\log\!\left(
\frac{\alpha}{2\alpha-1}\,e^{(\alpha-1)\varepsilon}
+ \frac{\alpha-1}{2\alpha-1}\,e^{-\alpha\varepsilon}
\right)\].
Subject: Accounting and Composition / Accounting
Rationale: The bound is stated in \citet[Proposition~6]{mironov2017renyi}.
Question: Let $\mathcal{M}$ be an $\varepsilon$-DP Laplace mechanism. For any $\alpha>1$, does $\mathcal{M}$ satisfy $(\alpha, \varepsilon^\prime)$-RDP with
\[
\varepsilon^{\prime}(\alpha)
= \frac{1}{\alpha}\,\log\!\left(
\frac{\alpha}{2\alpha-1}\,e^{(\alpha-1)\varepsilon}
+ \frac{\alpha-1}{2\alpha-1}\,e^{-\alpha\varepsilon}
\right).
\]
\\
\\
Subject: Accounting and Composition / Accounting
Rationale: The RDP bound for the Laplace mechanism given in \citet[Proposition~6]{mironov2017renyi} is tight. Since the claimed statement asserts a strictly smaller upper bound via replacing factor $\frac{1}{\alpha-1}$ with $\frac{1}{\alpha}$, it cannot hold in general.
Question: Let $\{\mathcal{M}_i\}_{i=1}^{n}$ be a collection of mechanisms, each satisfying $\varepsilon$-Differential Privacy. Does the sequential composition of these mechanisms satisfy $(2\sqrt{n}\Phi^{-1}(e^{\varepsilon}/(e^{\varepsilon} + 1)))$-Gaussian Differential Privacy (GDP)? ($\Phi$ denotes the cumulative distribution function (CDF) of the standard normal distribution)
Subject: Accounting and Composition / Accounting
Rationale: By \citet[Theorem~5.1]{liu2022identification}, $\varepsilon$-DP implies $2\Phi^{-1}(e^{\varepsilon}/(e^{\varepsilon} + 1))$-GDP, by composition of GDP (\citet[Corollary 2]{dong2022gaussian}), the composed mechanism satisfies $2\sqrt{n}\Phi^{-1}(e^{\varepsilon}/(e^{\varepsilon} + 1))$-GDP.
Question: Let $\{\mathcal{M}_i\}_{i=1}^{n}$ be a collection of mechanisms, each satisfying $\varepsilon$-Differential Privacy. Does the sequential composition of these mechanisms satisfy $(\sqrt{n}\Phi^{-1}(e^{\varepsilon}/(e^{\varepsilon} + 1)))$-Gaussian Differential Privacy (GDP)? ($\Phi$ denotes the cumulative distribution function (CDF) of the standard normal distribution)
Subject: Accounting and Composition / Accounting
Rationale: Consider the following counter example. Let $n=1$ and consider random response mechanism $\mathcal{M}$ with $p=\frac{\exp(\varepsilon)}{1 + \exp(\varepsilon)}$. Then $\mathcal{M}$ satisfies $\varepsilon$-DP and we can directly calculate the TV distance $dist_1 :=\mathrm{TV}(\mathcal{M}(x), \mathcal{M}(x^\prime))=\frac{\exp(\varepsilon)-1}{\exp(\varepsilon)+1}$. Notice that $\mathrm{TV}(\mathcal{N}(0,1),\mathcal{N}(\mu,1))=2\Phi(\mu/2)-1:=dist_2$. Let $\mu=\Phi^{-1}\left( \frac{\exp(\varepsilon)}{\exp(\varepsilon)+1} \right)$ and set $\varepsilon=1$, we have $dist_1 \approx 0.4621$ and $dist_2 \approx 0.2419$, this implies $(\mathcal{M}(X),\mathcal{M}(X^\prime))$ is more distinguishable than $(\mathcal{N}(0,1),\mathcal{N}(\mu,1))$, so $\mathcal{M}$ doesn't satisfy $\mu$-GDP. Therefore, the statement does not hold in general.
Question: Let $\mathcal{M}$ satisfy $\varepsilon$-pure Gaussian Differential Privacy. Does $\mathcal{M}$ also satisfy $f(\varepsilon)$-DP? Where $f(x)=\log(\frac{\Phi(\varepsilon/2)}{1-\Phi(\varepsilon/2)})$ and $\Phi$ denotes the cumulative distribution function of the standard normal distribution.
Subject: Accounting and Composition / Accounting
Rationale: Let $\mathcal{M}$ be a $\varepsilon$-pure GDP gaussian mechanism, then $\mathcal{M}$ doesn't satisfy pure DP.
Question: Let $\mathcal{M}$ be an exponential mechanism satisfy $\varepsilon$-DP. Does $\mathcal{M}$ also satisfy $\frac{\varepsilon^2}{8}$-zero concentrated DP?
Subject: Accounting and Composition / Accounting
Rationale: The exponential mechanism satisfies the bounded range property \citep{durfee2019practical,dong2020optimal}. Using this improved analysis, one can further show that it satisfies $\frac{\varepsilon^2}{8}$-zCDP.
Question: Let $\mathcal{M}$ be a Permute-and-Flip mechanism satisfy $\varepsilon$-DP. Does $\mathcal{M}$ also satisfy $\frac{\varepsilon^2}{8}$-zero concentrated DP?
Subject: Accounting and Composition / Accounting
Rationale: Permute-and-flip \citep{mckenna2020permute} does not satisfy the bounded-range condition \citep{DPorg-exponential-mechanism-bounded-range}. Consequently, under the standard conversion, it guarantees only \(\varepsilon^{2}/2\)-DP and does not achieve \(\varepsilon^{2}/8\)-zCDP.
Question: Let $\mathcal{M}$ be an $\varepsilon$-DP Laplace mechanism. For any $\alpha>1$, does $\mathcal{M}$ satisfy $(\alpha, \varepsilon^\prime)$-RDP with $\varepsilon^\prime= \varepsilon - \frac{1}{\alpha - 1}\,\log\!\left(\frac{1 + e^{-\varepsilon}}{1 + e^{-(2\alpha-1)\varepsilon}}\right)$
Subject: Accounting and Composition / Accounting
Rationale: It follows by \citet[Theorem~4]{dporg_pdp_to_zcdp}. We note that \citet[Theorem~4]{dporg_pdp_to_zcdp} holds for any $\varepsilon$-DP mechanism, thus it also holds for $\varepsilon$-DP Laplace Mechanism.
Question: Let $\mathcal{M}$ be an $\varepsilon$-DP Laplace mechanism. For any $\alpha>1$, does $\mathcal{M}$ satisfy $(\alpha, \varepsilon^\prime)$-RDP with $\varepsilon^\prime= \varepsilon - \frac{2}{\alpha - 1}\,\log\!\left(\frac{1 + e^{-\varepsilon}}{1 + e^{-(2\alpha-1)\varepsilon}}\right)$
Subject: Accounting and Composition / Accounting
Rationale: Notice that $\varepsilon^\prime$ is not a valid RDP upper bound. For example, taking $\alpha=2$ and $\varepsilon=0.1$ yields $\varepsilon^\prime \approx -0.7 < 0$ which is impossible since R\'enyi divergence must be nonnegative.
Question: Let $\mathcal{M}$ be an $(\varepsilon,\delta)$-DP Gaussian mechanism. For any $\alpha>1$, does $\mathcal{M}$ satisfy $(\alpha, \varepsilon^\prime)$-RDP with $\varepsilon^\prime= \varepsilon - \frac{1}{\alpha - 1}\,\log\!\left(\frac{1 + e^{-\varepsilon}}{1 + e^{-(2\alpha-1)\varepsilon}}\right) + \delta$
Subject: Accounting and Composition / Accounting
Rationale: As $\alpha \to \infty$, we have $\varepsilon' \to \varepsilon + \delta$, which would imply that $\mathcal{M}$ is $(\varepsilon+\delta)$-DP. However, the Gaussian mechanism does not satisfy pure DP for any finite $\varepsilon$. Therefore, the statement cannot be true.
Question: Let $\mathcal{M}$ be an $\varepsilon$-DP mechanism. Does $\mathcal{M}$ satisfy $\frac{e^\varepsilon-1}{e^\varepsilon+1}\varepsilon$-zero concentrated DP?
Subject: Accounting and Composition / Accounting
Rationale: The statement follows \citet[Theorem~5]{dporg_pdp_to_zcdp}.
Question: Let $\mathcal{M}$ be an $\varepsilon$-DP Laplace mechanism. Does $\mathcal{M}$ satisfy $\frac{e^\varepsilon-1}{e^\varepsilon+2}\varepsilon$-zero concentrated DP?
Subject: Accounting and Composition / Accounting
Rationale: By \citet[Theorem~5]{harrison2025exact}, the tight zCDP bound for the $\varepsilon$-DP Laplace mechanism is $\varepsilon + e^{-\varepsilon} - 1$. Taking $\varepsilon=1$, we obtain $\left.\frac{e^\varepsilon-1}{e^\varepsilon+2}\,\varepsilon\right|_{\varepsilon=1} \approx 0.3642$ and $\left.(\varepsilon + e^{-\varepsilon} - 1)\right|_{\varepsilon=1} \approx 0.3679$. Since the proposed value is smaller than the tight bound, the statement cannot be true.
Question: Let $\mathcal{M}$ satisfy $(\varepsilon,\delta)$-DP under the add/remove neighbouring relationship.
Consider Poisson subsampling with inclusion rate $\gamma$, i.e., each individual is included independently with probability $\gamma$.
Does the subsampled mechanism satisfies $\Bigl(\log\!\bigl(1+\gamma\,(e^{\varepsilon}-1)\bigr),\; \gamma \delta\Bigr)$-DP?
Subject: Accounting and Composition / Amplification by Subsampling
Rationale: This statement follows \citet[Theorem~8]{balle2018privacy}.
Question: Let $\mathcal{M}$ satisfy $(\varepsilon,\delta)$-DP under the add/remove neighbouring relationship.
Consider Poisson subsampling with inclusion rate $\gamma$, i.e., each individual is included independently with probability $\gamma$.
Does the subsampled mechanism satisfies $\Bigl(\log\!\bigl(1-\gamma e^{\varepsilon}-\gamma\bigr),\; \gamma \delta\Bigr)$-DP?
Subject: Accounting and Composition / Amplification by Subsampling
Rationale: By \citet[Theorem~12]{balle2018privacy}, \citet[Theorem~8]{balle2018privacy} is tight. The statement is strictly stronger than the one in \citet[Theorem~8]{balle2018privacy}, and therefore does not hold in general.
Question: Let $\mathcal{M}$ satisfy $(\varepsilon,\delta)$-DP under the add/remove neighbouring relationship.
Consider sampling without replacement, where a subset of size $m$ is drawn uniformly at random from a dataset of size $n$.
Does the resulting mechanism satisfies $\Bigl(\log\!\bigl(1+\tfrac{m}{n}(e^{\varepsilon}-1)\bigr),\; \tfrac{m}{n}\,\delta\Bigr)$-DP?
Subject: Accounting and Composition / Amplification by Subsampling
Rationale: Privacy amplifications through sampling without replacement does not support the add/remove neighboring since sample size $n$ is private information.
Question: Let $\mathcal{M}$ satisfy $(\varepsilon,\delta)$-DP under the replace-one neighbouring relationship.
Consider sampling without replacement, where a subset of size $m$ is drawn uniformly at random from a dataset of size $n$.
Does the resulting mechanism satisfies $\Bigl(\log\!\bigl(1+\tfrac{m}{2n}(e^{\varepsilon}-1)\bigr),\; \tfrac{m}{n}\,\delta\Bigr)$-DP?
Subject: Accounting and Composition / Amplification by Subsampling
Rationale: By \citet[Theorem~12]{balle2018privacy}, \citet[Theorem~9]{balle2018privacy} is tight. The bound stated here is smaller than that of \citet[Theorem~9]{balle2018privacy} by a factor of $2$, and therefore does not hold in general.
Question: Let $\mathcal{M}$ satisfy $(\varepsilon,\delta)$-DP under the replace-one neighbouring relationship.
Consider sampling without replacement, where a subset of size $m$ is drawn uniformly at random from a dataset of size $n$.
Does the resulting mechanism satisfies $\Bigl(\log\!\bigl(1+\tfrac{m}{n}(e^{\varepsilon}-1)\bigr),\; \tfrac{m}{n}\,\delta\Bigr)$-DP?
Subject: Accounting and Composition / Amplification by Subsampling
Rationale: By \citet[Theorem~9]{balle2018privacy}.
Question: The pseudocode below implements the exponential mechanism. Does this implementation ensure $\varepsilon$-differential privacy under the add/remove neighboring relationship?

\BlankLine

\begin{algorithm}[H]
\caption{Exponential Mechanism}
\DontPrintSemicolon
\SetNoFillComment
\SetKwInOut{Input}{Input}
\SetKwInOut{Output}{Output}

\Input{
Dataset $D = \{z_i\}_{i=1}^n$;\,
candidate set $Y$;\,
utility (score) function $q(D, y)=\sum_{i=1}^n \mathbf{1}\{z_i=y\}$;\,
privacy parameter $\varepsilon$
}

\Output{
Selected item $\hat{y} \in Y$ sampled by the exponential mechanism
}

\BlankLine
\For{$y \in Y$}{
    $p_y \leftarrow 
    \frac{\exp\left(\varepsilon\, q(D, y)\right)}
         {\sum_{y' \in Y} \exp\left(\varepsilon\, q(D, y')\right)}$
}

Sample $\hat{y} \sim \mathrm{Multinomial}(1; \{p_y\}_{y \in Y})$\;
\Return{$\hat{y}$}
\end{algorithm}
Subject: DP-statistics / DP selection: expoMech
Rationale: For a monotone query $q$, the exponential mechanism with $p_y \propto \exp\!\left(\frac{\varepsilon\, q}{\Delta_q}\right)$ satisfies $\varepsilon$-DP \citep{dong2020optimal}.
Question: The pseudocode below implements the exponential mechanism. Does this implementation ensure $\varepsilon$-differential privacy under the add/remove neighboring relationship?


\begin{algorithm}[H]
\caption{Exponential Mechanism}
\DontPrintSemicolon
\SetNoFillComment
\SetKwInOut{Input}{Input}
\SetKwInOut{Output}{Output}

\Input{
Dataset $D = \{z_i\}_{i=1}^n$;\,
candidate set $Y$;\,
utility (score) function $q(D, y)=|\sin\left(\frac{\pi}{2}\sum_{i=1}^n \mathbf{1}\{z_i=y\}\right)|$;\,
privacy parameter $\varepsilon$
}

\Output{
Selected item $\hat{y} \in Y$ sampled by the exponential mechanism
}

\BlankLine
\For{$y \in Y$}{
    $p_y \leftarrow 
    \frac{\exp\left(\varepsilon\, q(D, y)\right)}
         {\sum_{y' \in Y} \exp\left(\varepsilon\, q(D, y')\right)}$
}

Sample $\hat{y} \sim \mathrm{Multinomial}(1; \{p_y\}_{y \in Y})$\;
\Return{$\hat{y}$}
\end{algorithm}
Subject: DP-statistics / DP selection: expoMech
Rationale: Let $Y$ be finite candidate set. Given dataset $D=\{z_i\}_{i=1}^n$ denote $c_y(D):=\sum_{i=1}^n \mathbf 1\{z_i=y\}$, then $q(D,y)=\left|\sin\!\left(\frac{\pi}{2}c_y(D)\right)\right|$. Consider mechanism $M$ sampling $y \in Y$ with $p_D(y):=\Pr[M(D)=y]=\frac{\exp(\varepsilon q(D,y))}{Z(D)}$, where $Z(D):=\sum_{u\in Y}\exp(\varepsilon q(D,u))$. Notice that $\sin(\frac{\pi}{2}c_y(D))\in\{0,\pm 1\}$, hence $q(D,y)\in\{0,1\}$. Without loss of generality, let $D'=D\cup\{z^*\}$ or $D=D^\prime \cup\{z^*\}$ and set $y^*:=z^*$. Then $c_{y^*}$ changes by $\pm1$ and hence $q(\cdot,y^*)$ flips between $0$ and $1$, while for all $y\neq y^*$ we have $q(D,y)=q(D',y)$. Therefore for all $y\neq y^*$, $\frac{p_D(y)}{p_{D'}(y)}=\frac{Z(D')}{Z(D)}.$ Moreover, since $q\in\{0,1\}$, $Z(D)=|Y|+(e^\varepsilon-1)|A(D)|$ where $A(D):=\{y:q(D,y)=1\}$, and $|A(D')|-|A(D)|\in\{-1,+1\}$, implying $e^{-\varepsilon}\le \frac{Z(D')}{Z(D)}\le e^{\varepsilon}$. Hence $\frac{p_D(y)}{p_{D'}(y)}\le e^\varepsilon$ for all $y\neq y^*$. For $y=y^*$, the numerator changes by at most a factor $e^\varepsilon$, i.e., $\exp(\varepsilon q(D,y^*))/\exp(\varepsilon q(D',y^*))\le e^\varepsilon$, and thus $\frac{p_D(y^*)}{p_{D'}(y^*)}
=\frac{\exp(\varepsilon q(D,y^*))}{\exp(\varepsilon q(D',y^*))}\cdot \frac{Z(D')}{Z(D)}\le e^\varepsilon$. Therefore $p_D(y)\le e^\varepsilon p_{D'}(y)$ for all $y\in Y$. The reverse direction is symmetric, so $M$ is $\varepsilon$-DP.
Question: The pseudocode below implements the exponential mechanism. Does this implementation ensure $\varepsilon$-differential privacy under the replace-one neighboring relationship?


\begin{algorithm}[H]
\caption{Exponential Mechanism}
\DontPrintSemicolon
\SetNoFillComment
\SetKwInOut{Input}{Input}
\SetKwInOut{Output}{Output}

\Input{
Dataset $D = \{z_i\}_{i=1}^n$;\,
candidate set $Y$;\,
utility (score) function $q(D, y)=\sum_{i=1}^n \mathbf{1}\{z_i=y\}$;\,
privacy parameter $\varepsilon$
}

\Output{
Selected item $\hat{y} \in Y$ sampled by the exponential mechanism
}

\BlankLine
\For{$y \in Y$}{
    $p_y \leftarrow 
    \frac{\exp\left(\varepsilon\, q(D, y)\right)}
         {\sum_{y' \in Y} \exp\left(\varepsilon\, q(D, y')\right)}$
}

Sample $\hat{y} \sim \mathrm{Multinomial}(1; \{p_y\}_{y \in Y})$\;
\Return{$\hat{y}$}
\end{algorithm}
Subject: DP-statistics / DP selection: expoMech
Rationale: The exponent in the density of the exponential mechanism is missing a factor of $1/2$. Consequently, under the replace-one neighboring relation, it satisfies only $2\varepsilon$-DP.
Question: The pseudocode below implements the exponential mechanism. Does this implementation ensure $\varepsilon/2$-differential privacy under the add/remove neighboring relationship?


\begin{algorithm}[H]
\caption{Exponential Mechanism}
\DontPrintSemicolon
\SetNoFillComment
\SetKwInOut{Input}{Input}
\SetKwInOut{Output}{Output}

\Input{
Dataset $D = \{z_i\}_{i=1}^n$;\,
candidate set $Y$;\,
utility (score) function $q(D, y)=\sum_{i=1}^n \mathbf{1}\{z_i=y\}$;\,
privacy parameter $\varepsilon$
}

\Output{
Selected item $\hat{y} \in Y$ sampled by the exponential mechanism
}

\BlankLine
\For{$y \in Y$}{
    $p_y \leftarrow 
    \frac{\exp\left(\varepsilon\, q(D, y)\right)}
         {\sum_{y' \in Y} \exp\left(\varepsilon\, q(D, y')\right)}$
}

Sample $\hat{y} \sim \mathrm{Multinomial}(1; \{p_y\}_{y \in Y})$\;
\Return{$\hat{y}$}
\end{algorithm}
Subject: DP-statistics / DP selection: expoMech
Rationale: For a monotone query $q$, the exponential mechanism with $p_y \propto \exp\!\left(\frac{\varepsilon\, q}{\Delta_q}\right)$ is $\varepsilon$-DP \citep{dong2020optimal}, which is weaker than the guarantee requested in the statement, namely $\varepsilon/2$-DP.
Question: The pseudocode below implements the exponential mechanism. Does this implementation ensure $\varepsilon$-differential privacy under the add/remove neighboring relationship?


\begin{algorithm}[H]
\caption{Exponential Mechanism}
\DontPrintSemicolon
\SetNoFillComment
\SetKwInOut{Input}{Input}
\SetKwInOut{Output}{Output}

\Input{
Dataset $D = \{z_i\}_{i=1}^n$;\,
candidate set $Y$;\,
utility (score) function $q(D, y)=\sum_{i=1}^n \mathbf{1}\{z_i=y\}$;\,
privacy parameter $\varepsilon$
}

\Output{
Selected item $\hat{y} \in Y$ sampled by the exponential mechanism
}

\BlankLine
\For{$y \in Y$}{
    $p_y \leftarrow 
    \frac{\exp\left(2\varepsilon\, q(D, y)\right)}
         {\sum_{y' \in Y} \exp\left(2\varepsilon\, q(D, y')\right)}$
}

Sample $\hat{y} \sim \mathrm{Multinomial}(1; \{p_y\}_{y \in Y})$\;
\Return{$\hat{y}$}
\end{algorithm}
Subject: DP-statistics / DP selection: expoMech
Rationale: For a monotone query $q$, the exponential mechanism with $p_y \propto \exp\!\left(\frac{\varepsilon\, q}{\Delta_q}\right)$ is $\varepsilon$-DP \citep{dong2020optimal}, thus the algorithm described above satisfies $2\varepsilon$-DP.
Question: Given a privacy accountant $\mathcal{O}$ such that, given target privacy parameters $(\varepsilon,\delta)$, the Poisson subsampling rate $q$, and the number of DP-SGD iterations $T$ (i.e., the number of compositions), returns the calibrated noise multiplier $\sigma$ for the subsampled Gaussian mechanism used in DP-SGD under the add/remove neighbouring relation. Could the following implementation of DP-Adam with appropriate noise multiplier $\sigma$ satisfy $(\varepsilon,\delta)$-DP for some $\varepsilon>0$ and $\delta\in(0,1)$ (under add/remove neigbouring relation)?

\begin{algorithm}[H]
\caption{DP-Adam}
\DontPrintSemicolon
\SetKwInput{KwIn}{Input}
\SetKw{Sample}{Sample}
\SetKw{Set}{Set}

\KwIn{%
Data $\mathcal D=\{x_i\}_{i=1}^N$; epochs $E$; batch size $B$; learning rate $\eta$; clipping threshold $C$; initial parameters $\theta_0\in\mathbb{R}^p$; initial moments $m_0,v_0\in\mathbb{R}^p$; decay rates $\beta_1,\beta_2\in(0,1)$; small constant $\gamma>0$.}

\Set{} $\sigma \leftarrow \mathcal{O}(\varepsilon,\delta,\frac{B}{N}, \lfloor E\cdot N/B \rfloor )$\\ 

\For{$t=0,1,\dots, \lfloor E\cdot N/B \rfloor-1$}{
  \Sample{} a minibatch $B_t$ via Poisson sampling, where each element of $\mathcal D$ is included independently with probability $B/N$.\;
  \For{$x_i \in B_t$}{
    $g_t(x_i) \leftarrow \nabla_{\theta_t}\mathcal{L}(x_i)$\;
    $\tilde g_t(x_i) \leftarrow g_t(x_i)\cdot \min\!\bigl(1,\tfrac{C}{\lVert g_t(x_i)\rVert_2}\bigr)$\;
  }
  $z_t \sim \mathcal{N}\!\left(0,\,\sigma^2 C^2 I_p\right)$\;
  $\bar g_t \leftarrow \dfrac{1}{B}\!\left(\sum_{x_i\in B_t}\tilde g_t(x_i) + z_t\right)$\;
  $m_{t+1} \leftarrow \beta_1 m_t + (1-\beta_1)\,\bar g_t$\;
  $v_{t+1} \leftarrow \beta_2 v_t + (1-\beta_2)\,\bar g_t^{\odot 2}$\tcp*{$\odot$ is elementwise square}
  $\hat m_{t+1} \leftarrow \dfrac{m_{t+1}}{1-\beta_1^{t+1}}$\;
  $\hat v_{t+1} \leftarrow \dfrac{v_{t+1}}{1-\beta_2^{t+1}}$\;
  $\theta_{t+1} \leftarrow \theta_t - \eta\, \dfrac{\hat m_{t+1}}{\sqrt{\hat v_{t+1}}+\gamma}$\;
}
\Return $\theta_{\lfloor E\cdot N/B \rfloor}$\;
\end{algorithm}
Subject: DP-ML / dp-adam
Rationale: The algorithm is DP-Adam, adapted from \citet[Algos.~1--2]{lilarge}. Overall, it is essentially a Poisson-subsampled Gaussian mechanism followed by post-processing. Therefore, once the noise multiplier is calibrated using Poisson-subsampled DP-SGD, the entire algorithm satisfies \((\varepsilon,\delta)\)-DP.
Question: Suppose we have a privacy accountant $\mathcal{O}$. Given target privacy parameters $(\varepsilon,\delta)$, a Poisson subsampling rate $q$, and $T$ DP-SGD iterations (i.e., $T$ compositions), $\mathcal{O}$ returns the smallest noise multiplier $\sigma$ for the subsampled Gaussian mechanism in DP-SGD (under the add/remove neighboring relation) that guarantees $(\varepsilon,\delta)$-DP. Now given privacy budget $\varepsilon>0$ and $\delta\in(0,1)$, does the following implementation of DP-Adam satisfy $(\varepsilon,\delta)$-DP (under replace-one neigbouring relation)?

\begin{algorithm}[H]
\caption{DP-Adam}
\DontPrintSemicolon
\SetKwInput{KwIn}{Input}
\SetKw{Sample}{Sample}
\SetKw{Set}{Set}

\KwIn{
Data $\mathcal D=\{x_i\}_{i=1}^N$; epochs $E$; batch size $B$; learning rate $\eta$; clipping threshold $C$; initial parameters $\theta_0\in\mathbb{R}^p$; initial moments $m_0,v_0\in\mathbb{R}^p$; decay rates $\beta_1,\beta_2\in(0,1)$; small constant $\gamma>0$.}

\Set{} $\sigma \leftarrow \mathcal{O}(\varepsilon,\delta,\frac{B}{N}, \lfloor E\cdot N/B \rfloor )$\\ 

\For{$t=0,1,\dots, \lfloor E\cdot N/B \rfloor-1$}{
  \Sample{} a minibatch $B_t$ via Poisson sampling, where each element of $\mathcal D$ is included independently with probability $B/N$.\;
  \For{$x_i \in B_t$}{
    $g_t(x_i) \leftarrow \nabla_{\theta_t}\mathcal{L}(x_i)$\;
    $\tilde g_t(x_i) \leftarrow g_t(x_i)\cdot \min\!\bigl(1,\tfrac{C}{\lVert g_t(x_i)\rVert_2}\bigr)$\;
  }
  $z_t \sim \mathcal{N}\!\left(0,\,\sigma^2 C^2 I_p\right)$\;
  $\bar g_t \leftarrow \dfrac{1}{B}\!\left(\sum_{x_i\in B_t}\tilde g_t(x_i) + z_t\right)$\;
  $m_{t+1} \leftarrow \beta_1 m_t + (1-\beta_1)\,\bar g_t$\;
  $v_{t+1} \leftarrow \beta_2 v_t + (1-\beta_2)\,\bar g_t^{\odot 2}$\tcp*{$\odot$ is elementwise square}
  $\hat m_{t+1} \leftarrow \dfrac{m_{t+1}}{1-\beta_1^{t+1}}$\;
  $\hat v_{t+1} \leftarrow \dfrac{v_{t+1}}{1-\beta_2^{t+1}}$\;
  $\theta_{t+1} \leftarrow \theta_t - \eta\, \dfrac{\hat m_{t+1}}{\sqrt{\hat v_{t+1}}+\gamma}$\;
}
\Return $\theta_{\lfloor E\cdot N/B \rfloor}$\;
\end{algorithm}
Subject: DP-ML / dp-adam
Rationale: Since the hypothetical oracle $\mathcal{O}$ returns the smallest noise multiplier calibrated for the add/remove neighboring relation, using the same multiplier under the replace-one neighboring relation generally underestimates the required noise scale. Thus the algorithm doesn't satisfy stated privacy guarantee.
Question: Suppose we have a privacy accountant $\mathcal{O}$. Given target privacy parameters $(\varepsilon,\delta)$, a Poisson subsampling rate $q$, and $T$ DP-SGD iterations (i.e., $T$ compositions), $\mathcal{O}$ returns the smallest noise multiplier $\sigma$ for the subsampled Gaussian mechanism in DP-SGD (under the add/remove neighboring relation) that guarantees $(\varepsilon,\delta)$-DP. Now given privacy budget $\varepsilon>0$ and $\delta\in(0,1)$, does the following implementation of DP-Adam satisfy $(\varepsilon/2,\delta)$-DP (under add/remove neigbouring relation)?

\begin{algorithm}[H]
\caption{DP-Adam}
\DontPrintSemicolon
\SetKwInput{KwIn}{Input}
\SetKw{Sample}{Sample}
\SetKw{Set}{Set}

\KwIn{
Data $\mathcal D=\{x_i\}_{i=1}^N$; epochs $E$; batch size $B$; learning rate $\eta$; clipping threshold $C$; initial parameters $\theta_0\in\mathbb{R}^p$; initial moments $m_0,v_0\in\mathbb{R}^p$; decay rates $\beta_1,\beta_2\in(0,1)$; small constant $\gamma>0$.}

\Set{} $\sigma \leftarrow \mathcal{O}(\varepsilon,\delta,\frac{B}{N}, \lfloor E\cdot N/B \rfloor )$\\ 

\For{$t=0,1,\dots, \lfloor E\cdot N/B \rfloor-1$}{
  \Sample{} a minibatch $B_t$ via Poisson sampling, where each element of $\mathcal D$ is included independently with probability $B/N$.\;
  \For{$x_i \in B_t$}{
    $g_t(x_i) \leftarrow \nabla_{\theta_t}\mathcal{L}(x_i)$\;
    $\tilde g_t(x_i) \leftarrow g_t(x_i)\cdot \min\!\bigl(1,\tfrac{C}{\lVert g_t(x_i)\rVert_2}\bigr)$\;
  }
  $z_t \sim \mathcal{N}\!\left(0,\,\sigma^2 C^2 I_p\right)$\;
  $\bar g_t \leftarrow \dfrac{1}{B}\!\left(\sum_{x_i\in B_t}\tilde g_t(x_i) + z_t\right)$\;
  $m_{t+1} \leftarrow \beta_1 m_t + (1-\beta_1)\,\bar g_t$\;
  $v_{t+1} \leftarrow \beta_2 v_t + (1-\beta_2)\,\bar g_t^{\odot 2}$\tcp*{$\odot$ is elementwise square}
  $\hat m_{t+1} \leftarrow \dfrac{m_{t+1}}{1-\beta_1^{t+1}}$\;
  $\hat v_{t+1} \leftarrow \dfrac{v_{t+1}}{1-\beta_2^{t+1}}$\;
  $\theta_{t+1} \leftarrow \theta_t - \eta\, \dfrac{\hat m_{t+1}}{\sqrt{\hat v_{t+1}}+\gamma}$\;
}
\Return $\theta_{\lfloor E\cdot N/B \rfloor}$\;
\end{algorithm}
Subject: DP-ML / dp-adam
Rationale: Since the hypothetical oracle $\mathcal{O}$ returns the smallest noise multiplier to guarantee $(\varepsilon,\delta)$-DP under the add/remove neighboring relation, it doesn't satisfy the stronger $(\varepsilon/2,\delta)$-DP.