Trustworthy Machine Learning through Data-Specific Indistinguishability
Authors: Hanshen Xiao
Zhen Yang
G. Edward Suh
Venue: ICML 2025 poster (2025)
Keywords: Differential trustworthiness
data-specific indistinguishability
memorization
backdoor attacks
copyright
Primary area: social_aspects

Abstract: This paper studies a range of AI/ML trust concepts, including memorization, data poisoning, and copyright, which can be modeled as constraints on the influence of data on a (trained) model, characterized by the outcome difference from a processing function (training algorithm). In this realm, we show that provable trust guarantees can be efficiently provided through a new framework termed Data-Specific Indistinguishability (DSI) to select trust-preserving randomization tightly aligning with targeted outcome differences, as a relaxation of the classic Input-Independent Indistinguishability (III). We establish both the theoretical and algorithmic foundations of DSI with the optimal multivariate Gaussian mechanism. We further show its applications to develop trustworthy deep learning with black-box optimizers. The experimental results on memorization mitigation, backdoor defense, and copyright protection show both the efficiency and effectiveness of the DSI noise mechanism.

Trustworthy Machine Learning through Data-Specific Indistinguishabiiiy
Hanshen Xiao 12
Zhen Yang? 2
G. Edward Suh"
Abstract
reference inputs (datasets). We formalize the framework of Differential Trust (DT) as follows. Given processing func- tion F, a target input U, and group of reference datasets Rj:mm] = {R1, ... Rm), }, (some of) the outputs F(R;) de- rived from the reference datasets are assumed to be safe; our objective is to regulate the divergence between the output F(U) generated from U and the outputs F(R,) from the references. Several concrete examples are listed below.
This paper studies a range of AI/MI. trust con- cepts, including memorization, data poisoning and copyright, which can be modeled as con- straints on the influence of data on (trained) model, characterized by the outcome difference from a processing function (training algorithm). In this realm, we show that provable trust guar- antees can be efficiently provided through a new framework termed Data-Specific Indistinguisha- bility (DSI) to select trust-preserving random- ization tightly aligning with targeted outcome differences, relaxation of the classic Input- Independent Indistinguishabiliyy (III). We estab- lish both the theoretical and algorithmic founda- tions of DSI with the optimal multivariate Gaus- sian mechanism. We further show its applica- tions to develop trustworthy deep learning with black-box optimizers. The experimental results on memorization mitigation, backdoor defense, and copyright protection show both the efficiency and effectiveness of the DSI noise mechanism.
Unintended memorization is common is- sue in large models, where the model learns specific details or features from individual examples rather than capturing the common patterns of the underlying data distribution (Tirumala et al., 2022). In the context of language mod- els (LMs), mitigating memorization typically involves de- fending against queries that attempt to regurgitate sensitive training data verbatim (Zanella- Béguullin et al., 2020). For instance, (Tirumala et al., 2022; Carlini et al., 2022) define memorization behaviors as follows: given a concatenated context from the input training set in a form [p]s], the LM is able to exactly reproduce s when prompted with p. Suppose [p|s] is uniquely contained in a sample u E U, and F is a training algorithm. By selecting reference R= U U\ u, the resulting model F(R) can be considered safe against the memorization of [pls]. as this feature is never seen by the model during training (Carlini et al., 2021).
Memorization:
1. Introduction
The remarkable advances in artificial intelligence (AI) and machine learning (ML) in recent years have been largely driven by optimizing loss functions based on predictive accuracy. Despite this success, the widespread deployment of AI has raised a range of societal and ethical concerns, particularly related to the use of data and its impact on trained models. These concerns have spurred the emergence of research on trustworthy data processing.
By injecting specially- crafted training data, an attacker can manipulate the training process to embed hidden, malicious behavior into the model while maintaining its performance on normal data. For ex- ample, adding specific patch to an image might cause the model to misclassify it when a particular sticker is present (Gu et al., 2017: Tran et al., 2018). Existing empirical de- fense largely relies on filtering suspicious samples (Jagielski et al., 2018; Baracaldo et al., 2017) or robust gradient aggre- gator (Tran et al., 2018: Kane et al., 2024), while providing provable guarantees remains challenging. In the DT frame- work, consider a (virtual) distributed setup where U consists of data collected from m sources, with one of them being poisoned. Let R; represent the subset of U after excluding all data provided by the i-th source. Among the m models F(R,), E [1 : m] = 2,, m). there is guaranteed to be at least one model trained entirely on benign data.
Data Poisoning (Backdoor) Attack:
In this paper, we present a unified and systematic framework for a critical subset of trust concepts. These concepts can be modeled as constraints on the differences between the outputs of an algorithm (a machine learning training algo- rithm) for a given input (dataset) and an additional set of
NVVIII 2 Department of Computer Science, Purdue Univer- sity. Correspondence to: Hanshen Xiao <hsxiao@@ purduc.edus
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Copyright and Contribution: Ensuring training samples

---

Trustworthy Machine Learning through Data-Specific Indistinguishabilty
on an Internet scale entirely free of copyrighted materials is often impractical (Vyas al., 2023). In addition, the recent success of generative models has raised growing concerns, particularly among the artist community where generated images or music can closely mimic specific artistic styles, potentially infringing on intellectual property rights (Shan et al., 2023; Dhariwal et al., 2020). Furthermore, in col- laborative learning (Sim et al., 2020) where a model is trained using data from multiple parties, how to quantify their individual contributions to acknowledge ownership or determine compensation remains open. In these use cases, reference R can be selected as the subset of U that excludes all samples from a particular party (e.g., an artist, author, or data source). Consequently, F(R) serves as a reference that excludes influence from the particular party's data.
Input data U
St
Federated learning algorithm F
Noks
Polsoned data
c(U)
F(U)
Clean
Clean
Figure I. Illustration of applications of DSI noise in defending against poisoning attacks. Provided an input training data U col- lected from three data sources including an unknown malicious entity, one reference subset S2 is fully clean; the injected noise e(U) ensures that produced model F(U) + e(U) is statistically close to the noisy model F(S2) +E(U) trained on fully clean data.
1.2. From to Data-Specific Indistinguishabiliiy (DSI)
Indistinguishabiiity (III)
Although both privacy protection - focused on confidential- ity protection as in DP and PAC Privacy (Xiao & Devadas, 2023; Xiao et al., 2024; Sridhar et al., 2025) - and DT leverage indistinguishability, their underlying motivations are different. A key insight we will highlight in this work is that input independence is not necessary for many DT applications, where we only need indistinguishability with respect to specific set of safe reference inputs, enabling much sharpened utility-trust tradeoff.
1.1. Indistinguishability
To build provable DT guarantees, we aim ensure our target output F(U) is indistingguishable from the safe ref- erences F(R,), i.e., probabilistically, F(U) is distributed sufficiently close to each F(R,). In the context of above examples, this presents a guarantee to either simultaneously protect copyright or prevent memorization for each differ- ential data subset U \ R2 or maintain robustness against backdoor attacks provided the model remains close to the one that is only trained with a benign dataset R,
When Input Independence is Needed: A widely accepted approach to measuring privacy risk to quantify the addi- tional knowledge gained through leakage--peeiffiallly how much the leakage alters the adversary's 's prior belief and aids their inference about sensitive information. III (Definition 1.1) requires satisfactory indistinguishability (1) holds for arbitrary input pairs. This is necessary if we want to up- per bound the additional knowledge' for an adversary with arbitrary belief and optimal strategy (Shannon, 1949).
Historically, the idea of (input-independentt) indistinguisha- bility traces back to Shannon's 's pioncering work on perfecrt secrecy (Shannon, 1949), which laid the foundation for both Differential Privacy (DP) (Dwork et al., 2006b) and modern cryptography (Goldwasser & Micali, 1984). We summarize this framework below.
Definition 1.1 ((5, v) Input-Independent Indistinguishabil- ity (IID). Given a divergence measure v/, a mechanism : + o satisfies (5, --IIII if for arbitrary two input selections Ū and O' differing some objective feature,
For confidentiality protection, to ensure a processing func- tion F with satisfactory III guarantees, the introduced mod- ification, (e.g., noise), should not leak information about the secret input U itself. Therefore, most III randomization approaches, including exponential mechanism (McSherry & Talwar, 2007) and Laplace/Gaussian noise mechanism (Dwork al., 2014) in DP, are impu--nndependen" and only determined by the public information of F to privatize.
sup & (PM(O))llPMO)) MO)) 55 0,0
(1)
(1) characterizes, given the output M(U) for some input U, the hardness to distinguish U from two arbitrary candi- dates U and O' in the worst case. By properly selecting the objective feature as the sensitive part of a secret input U to protect, Definition 1.1 can capture many classic privacy def- initions. For example, DP aims to obscure the participation of an individual record and we may select Ú and O' as a pair of adjacent datasets differing in one datapoint; accordingly, when we select db varying from maximal, Hockey- -Stck and Rényi divergence, (1) leads to pure cDP (Dwork et al., 2006b), approximate (e, 8D- (Dwork et al., 2006a) and Rényi DP (Mironov, 2017). Recent works have also shown applications of DP to mitigate memorization (Carlini et al., 2021) or approximate unlearning (Gupta et al., 2021).
As global guarantee, clearly III shows a sufficient condition to ensure formal mitigation for DT. However, from a practice standpoint, it faces two challenges:
i) a worst-case bound (1) may fail to account for varying in- distinguishability demands among references. For instance,
'Alternatively, the additional knowledge can be defined by the optimal posterior success rate of an adversarial inference on the sensitive information after observing the leakage minus the optimal priorí success rate (Xiao Devadas, 2023).
AA a clarification, input-dependen randomness could still lead to III, for example, noise calibrated with smooth sensitivity (Nissim et al., 2007), but our proposed input-dependent Gaussian as defined later in Definition 1.2 generally cannot ensure III.
2

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiliy
in the example of data contribution mentioned earlier, user may wish to allocate different budgets based on data quality or preferences for the i-th provider's 's data usage;
As reasonable relaxation, DSI also systematically ad- dresses the two above-mentioned challenges of applying III to DT: i) by taking the target input U and its references as parameters in the noise mechanism, we allow a user to freely adjust the distinguishability levels % across refer- ences; ii) from utility standpoint, the optimal DSI noise is independent of dimensionality and can be upper bounded by the number m of references, which makes tigh-dimensional trustworthy processing feasible, as elaborated in Section 3.
ii) III is also notorious for its heavy utility trade-offs. The required level of randomness to achieve III has been ex- tensively studied and is often constrained by the curse of dimensionality (Hardt & Talwar, 2010: Bassily et al.. 2014: Xiao et al., 2023a), where the utility compromise increases with the dimensionality of the output.
1.3. Contribution and Paper Organization
With above understanding, in applications where our inputs are public, for example artists are willing to publish and allow others to have access to their works once their intellectual property is properly protected, or in the scenarios where we do not target infor- mation leakage but the governance of AI models against specific adversarial strategies, for example memorization mitigation resisting prompt elicitation to reproduce specific input contexts, both F(U) and safe references F(R.) can be freely used as public parameters in mechanism designs, different from wewacc-presrriingg operations. Additionally, we only require Spec-Sp Indistinguissaaiiiiy (DSI) between each pair (())))((R)) for i = 1,2, m..
DSI and Noise Mechanism:
In this paper, we initiate the study Differential Trust (DT) through DSI-based defense with fourfold contributions from concept, algorithm, theory and application perspectives.
1. DT and DSI Framework: We formalize DT as unified way to define a wide range of trust concepts and propose the concept of DSI to support quantitative studies and guarantees.
2. Optimal Gaussian Mechanism: We show, for s being an arbitrary (non-decreasing function of) f-divergence, determining the ligh-dimensional Gaussian noise with minimal variance satisfying Definition 1.2 can be re- duced to a convex optimization (Lemma 3.1) over m non-negative Lagrangian multipliers (Theorem 3.3), which is efficiently solvable (Algorithm 1). Of inde- pendent interest, the proposed noise mechanism also addresses the operational challenge in controlling per- instance or individual privacy (Wang, 2019; Feldman & Zrnic, 2021; Thudi et al., 2024) (Appendix A).
Operationally, since a given (possibly black-box) processing function F may not inherently satisfy all differential require- ments, we consider introducing a (multivariate) Gaussian noise& e for required DSI such that for each pair (U, R,), i E [1 : mz]. the noisy distributions F(U)+ + e and F(R:) +e are sufficiently close, as formalized below.
3. Properties of DSI: We comprehensively study the properties of DSI guarantees, including post- processing immunity, probabilistic interpretation, com- position (Theorem 3.4) and grouping (Lemma 3.5), to facilitate its application.
Definition 1.2 ((7)1=m)), +b) Spta-Specific Indistinguishabil- ity (DSI) Gaussian Mechanism). Given processing func- tion F and an input U with m associated reference datasets Rismi(UU = ( E [1 m/] possibly determined by U, for some divergence measure v), an (711=]] v/) DSI Gaussian mechanism returns a zero-mean Gaussian distri- bution N (0. Σ(U)) whose covariance Σ(U) can be selected dependent on input U and references RI::m) (U) such that for noise e(U independently sampled from (0,
4. DSI Deep Learning: Stemmed from Differentially Private Stochastic Gradient Descent (DP-SGD) (Abadi et al., 2016), we propose a more general framework (AI- gorithm 2) to iteratively incorporate optimal DSI noise into a deep learning procedure with black-box optimiz- ers. We show aimeenioonaltyyiddeeeddde and the algorithmic flexibility, which allows local up- dates without artificial clipping, significantly sharpen the utility compromise (Table 1) especially when scal- ing models. We show its applications in cutting-edge applications for various trust concepts, including mem- orization mitigation in large language models (LLM), resistance backdoor attacks, and copyright protec- tion (Section 5).
v' (P (()))())()))) ||P (())))( S %. i = 1,--- ,, (2)
DSI noise
where P. represents the distribution of random variable
The dependeee or data-specifie aspect in Definition 1.2
is two-fold. First, the DSI in (2) only requires satisfactory indistingusshabiliyy regarding specific output pairs produced by our target input U and references Ri (U). Second, both the references R1:mm] (U) and the injected noise e(U) can be adaptively selected dependent on the input U. In (2), smaller Y implies a stronger mitigation on the differential effect between F(U) and F(R;) (their noisy versions are statistically closer and harder to distinguish), which typi- cally requires a larger noise variance along the direction of difference F(U)- - F(R.). An application of backdoor defense with DSI noise mechanism is illustrated in Fig. 1.
2. Preliminaries
Divergence and Probability: One possible selection of the measurement şi in Definition 1.2 is f-divergence, as defined below.
3

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiity
Definition 2.1 (/-Divergence). Let ʃ (0, +x) R
and properties of DSI guarantees. We select t an arbi- trary f-divergence, while our following results also work for v being some non-decreasing function of a f-divergence, such as Rényi Divergence. Temporally, we assume a deter- ministic processing function F l1" + Rd whose output is a d-dimensional real vector; the generalization to a random- ized function can be found in Appendix D. In the following, we simplify the notations of the Gaussian noise, its covari- ance and the m references by c, E and R R[ respectively, which inherently can be dependent on the input U. We will also use MM.) = F() + e to denote the noisy version of F.
be convex function with f(1) 0. Let P and Q be two distributions on some measurable space, and the f- divergence D between P and Q is defined as
()(P)IQ) = Bo
(3)
Here, we use dP (dQ) to represent the probability density (or mass) function of probability distribution P (Q).
Many commonly used statistical divergence measurements are special cases of f-divergence. In Definition 2.1, when we select ft) t log(t)) (3) becomes the Kullback-Leibler (KL) divergence; when s(() = |1 11/2, (3) becomes the Total Variation (TV) Drv (PIIQ) = 1/2- I IdP dQ).
Let 2; = =(R,) F(U) be the output difference between applying the processing function F(-) on U and Rg. For any given f-divergence Dj and Gaussian noise e ~ M/ E), the objective divergence in (2) can be expressed
f f-divergence enjoys multiple desirable properties to facil- itate both the interpretation and usage of DSI guarantees. As detailed in Appendix B, DSI guarantees measured in f- divergence is robust to post- postsproesssing (Lemma B. That is to say. suppose we have found proper noise e(U) such that (2) holds for the noisy version of F(.), then the same security parameters still hold for the output G(F(.)) + e(-)) after applying a post-procsssing operator G. Other use- ful properties including shift invarfiance (Definition B.2) and joint convexity (Definition B.3), which will be used in determining the optimal noise. Additionally, many other divergences enjoying tially a non-decreasing function of some f f-divergence. One representative is Rényi Divergence defined below.
Dj (P(PM (PM(R)) Dr
Σ))
(N(O, E)I((, E)).
Here, we utilize the shift-invariance property of ʃ- divergence (Lemma B.2). Thus, mathematically, determin- ing the optimal d-dimensional Gaussian noise in terms of minimal expected 12-norm square E[]||[]] can be framed as the following optimization with constraints.
properties are essen-
min Σ
Ee = min Tr(E) Σ
(5)
s.t. (V Σ)) 27,i 1,2, m. (6)
Definition 2.2 (a Rényi Divergence). For a the a Rényi Divergence Ra between two distributions P and Q with identical support sets is defined
In (5), it is noted that the expected 42 4--ormm squared of the noise corresponds to the trace of its covariance matrix Σ. As for the constraints (6), we show the following results as a universal simplification.
R., (PI[Q) = 1/ox- log (Eq (dP//(()))
Lemma 3.1 (f-Divergence between Gaussians). For an arbitrary f f-divergence and two (multivariate) Gaussian dis- tributions with identical covariance N N(a, Σ) and (b, E), there exists some non-decreasing function Hy(()) such that
R., can be viewed as logarithm of an f f-divergence with selecting [() = 1a Divergences are also useful to bound the probability difference of some event in two distributions, and we take Rényi Divergence as an example.
Lemma 2.3 (Probability Difference Bound (Langlois et al., 2014)). For two distributions P and Q with identical support sets and an arbitrary event A, the probability
D,( 2)) HHy(lla ([[[labb]] bll-..).
(7)
Here, || = -1--16" represents the Mahalanobis norm.
P(A) < Q(A) -ا
(PIQ)) 1/a
(4)
Lemma 3.1 implies that any fddivergence between two Gaussians with identical covariance can be expressed as some monotone function of the Mahalanobis norm with respect to the difference of their mean. By Lemma 3.1, let H.7 be the inverse function of H, and % HH, which is uniquely determined, (6) can be rewritten as
Additional Notations: For a positive semi-definite (PSD)
matrix C, suppose its singular value decomposition (SVD)
a form C = P- Q- pt where P is a unitary matrix and Q is a diagonal matrix; we will use 1/2 P. P-Q pr to denote its factorization such that C (C)1/2. (C) Tr(C') represents the trace of matrix C' (sum of diagonals).
-F(U)=F(5)))3. 11 S 5r, i [1 [1:1m]] ml] (8)
3. DSI Gaussian Mechanism and Properties
Now, to address (5) under the constraints (8), a straight- forward approach might involve directly optimizing the d2 parameters of the noise covariance matrix Σ. However, this approach faces at least two significant challenges:
In this section, we present the main algorithm to determine the minimal DSI Gaussian noise and study the interpretation

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiity
First, an implicit constraint in (5) requires that Σ,, as a co- variance matrix, must be positive semi-definite. Notably, the set of all positive semi-definite matrices is convex but not continuous, making the implementation of projection optimization non-trivial. Second, in high-dimensional set- tings-the primary focus of this paper-tho the dimensionality d, which can reach billions in practical applications, is often much larger than the number of references m. Optimiz- ing d² parameters under such conditions becomes compu- tationally prohibitive. To overcome these challenges, we reformulate the objective problem into an alternative form.
Algorithm 1 Optimal DSI Gaussian Mechanism
Input: Output differences {21, 22, 2m } with its ma-
trix form 2 by rows, error threshold K, security parame- ters 7[]=m] initialized À = Apmmm) and step size 1).
baased based
1: if m S d then
2: Determine subspace basis II and the expression X' of
Z under II by Lemma 3.2. > {subspace embedding)
3: else
4: 11 = Laxd and X = Z.
5: end if
6: while true do
3.1. Algorithm for Optimal Gaussian Noise
7: À < max {A
77 . V V(()), 0} with gradient (
from (14).
D {GD with projection onto 1
Our proposed algorithm to determine the minimal noise consists of two components: subspace embedding and La- grangian multiplier gradient descent (GD), which is finally reduced to optimize only m parameters.
8: Compute Σ(A) = (Ei A; x xz 1/2
9: if || VR. c(A)] S K for i = 1.2,... ... m then
10: break
11: end if
Subspace Embedding: In the high- -ighddimessional case when m S d. it is noted that we only need to calibrate the noise e within the subspace spanned by [[1=m] to obfuscate each difference z; rather than the entire output space IR., More importantly, this subspace is of rank up to m, and even in the worst case when [[1mm] are all orthogomal to each other, the required noise will only scale in rate O( ((/mmn d)) which fundamentally avoids the curse of dimensionality in III mentioned earlier. Therefore, in the digh-iimeniiona case, we only need to determine the noise form in this subspace and afterwards project it back to Rdd
12:
end while
13: Sample E ~ ~NO, E(A)) and project it back to IRd as
e e II.
Output: e E Rd
basis II. With the optimum Σx in (10), we can sample (O E*) E Rh and project it back to Rd by ē II.
As a final remark, in the cases when m > d, there is no need to implement above-mentioned subspace embedding, which is equivalent to IT = Laxd: the d x d identity matrix, and X = Z, while all following analysis still works.
In the following, we use E Rdd to represent the matrix form of 2[1=m], where each row corresponds a 24, and we assume the rank of 2 is m with S S d.
Lagrangian Multiplier GD: We consider the equivalent mix-max dual problem of (10) with Lagrange multipliers x Apammp A; 2 O,
Lemma 3.2 (Subspace Basis). Consider the SVD of 2- ZT,
m Σ X,(1,, - -)). (II)
2. ZT Pz QZ. P/,, where Pz is a unitary matrix whose columns correspond to eigenvectors and Qz is diagonal of eigenvalues. Let Q2 E Rm and Pz Bdd be the submatrix Q2 and Pz after excluding zero eigenvalues with its eigenvectors.
(9)
min max Σ A 20
Tr(2)= +
1=1
It is not hard to verify that (11) is convex regarding E while linear (concave) with respect to A, implying that the opti- mum
of (11) is unique. The following theorem presents further insights that the optimum Σ* can be ex- pressed by the Lagrange multipliers \p=m) and X' and the optimization over Σ is thereby reduced to determining A Theorem 3.3 (Optimal Covariance from Optimal Lagrange Multipliers). Given a selection of X \pp=m) define
Then, the rows of II = Qz 1/2 PT. Z form an orthogonal unit basis of the subspuce spanned by the rows of Z.
Lemma 3.2 shows operationally how to find a basis I1 of the target subspace spanned by 2|1:m]- It is noted that each row of X' = Pz 21/ 1/2 now represents the expression of Z under the selected basis II. In addition, for any m- di-dimeniional Gaussian noise ह ~ (N, Σ) for E E Rm E | Galle-11f since Tr(E)= = Therefore, the entire problem can be reduced to finding a m m covariance matrix Σ to
Σ(A) = Ai o x;) 1/2
(12)
= Tr
The optimal solution Σn to (10) is unique in a form Σ. Σ(x) where X* arg minx20 L(X) is the unique op- timum of the function (()), defined below, under non- negative constraints X 2 0.
min Tr(E), s.t. | 1 S Si i i= 1,2, m.m. (10)
Σ
((A) = - (Tr(E(A)) + Σ Ar A1(12 - (.)). (13)
Here, ny is the i-th row of X, the expression of 2; under
5

---

Trustworthy Machine Learning through Data-Specific Indistinguisaabiity
Moreover, the gradient of (() in (13) has a closed form
Algorithm 2 DSI Deep Learning Framework
ac())
Input: a (black-box) optimizer O(w, B) starting from
0x -2, x + 9--
(14)
model weight w with data batch B, an input set U
{u[p: of n datapoints, m reference subsets R₂ C U
From Theorem 3.3, the entire problem is reduced to con- vex optimization (13) over m parameters À under a simple non-negative constraints, which is solvable using standard projected gradient descent (PGD). We summarize this op- timal Gaussian mechanism as Algorithm I and we discuss additional optimization trick in Appendix C.4.
round number T, T batches of samples B("), ,.., B B(t)) C U for E [] T7, security budget e, for [] m], and initialization w()
I: With a composition accounting (Theorem 3.4), deter- mine per-round security budget 6;
(!) for 1. E [1 : T)
given a global budget 6; for each i E [1: m] in (2).
3.2. Interpretation and Properties of DSI Guarantees
2: for 1 = 1, .... .odo
3: Determine R( R, n n()( for E [1 : m].
(i) One-Sided v.s. Two-Sided: Though our Gaussian mechanism ensures a symmetric DSI guarantee, we need to stress that this symmetry does not hold in general, i.e. (F M(U; M M(R.)) * (y MIR) M(U)). f- divergence technically is not a symmetric metric between two distributions. But one- ondes DSI (T (T())M is usually sufficient to provide desired sematic interpreta- tion. For example, recall the memorization or the backdoor applications discussed in Introduction section:
4: For each E [1 ml. compute the outcome difference
= Ow 1)) R()) Owwll-1), B()). (16)
Apply Algorithm 1 on (t)
[[1mm] with budgets a(t) [1mm]
to
determine the noise e(t).
6: Update weights w(()) ((w(t-1),B8 B(t)) + elt)
7:
end for Output: w((
R, represents U excluding some target samples to mitigate memorizing or data from malicious sources; if the reference model (RR)) trained on R; is of a high success rate to be ro- bust against either memorization or backdoor attacks, then the one-sided DST D MCU) |P M(R))) can be translated into an upper bound of the success rate for M(U), follow- ing results such as Lemma 2.3. However, such one-sided measurement the reference models in DSI is conceptu- ally different from III guarantees, such as DP, where the measurement must be symmetrie; otherwise the asymmetry itself could incur leakage.
implemented Fi, and becomes complicated to even ana- lyze the output distribution from M due to infinite possible instances of e1- Fortunately, we show an sequential imple- mentation of Algorithm 1, iteratively conditional on the noise sampled for preceding processing functions, is sulfi- cient to produce light composite DSI bound.
Theorem 3.4 (Adaptive Composition under a- e--iieereece). Let Dr, be (non-normalized) o-divergence with f(() = 1 in Definition 2.1. Suppose e is produced by Algorishm I on F1 such that D& (P ([() (ertteelFFF.rr [[F,()R)+))] I S%i (() for each pair (U, R,), RU,R.), i E [1 : ml. and conditional on e1, we implement Algorithm I on ()()+ e1) which returns e2 such that
(iii Postprocessing Immunity: The processing inequality (Lemma B. of f-divergence also straightforwardly implies that DSI guarantees robust to any post-process post-processing. One concrete example is that imagine the processing function F is some deep learning algorithm and if the noisy model weights satisfies (2), so do its the inference results produced.
Da (T ((TT))))))()))t)) (RR)) S 7
(2) then for
the joint mechanism M defined in (15).
Da (F M (I M(R)) sx( (1)
(2)
(17)
There are many practical applications where we may need to sequentially implement numerous, possibly-adaptive processing functions, where the output from the preceding function can be the input for the sub- sequent one; and we need to keep track of the cumulative divergence across multiple outputs generated. To be spe- cific, imagine two processing functions Fi and F2; in the context of DSI noise mechanism, our goal is to determine two noises ej and 62 such that for the joint mechanism M
(iii) Composition:
The equality (17) holds when the noises ei and e2 are light to locally produce 7 (1) and (2) for Fi and Fz, respectively.
Theorem 3.4 shows that under a-divergence, the adaptive composition can be simply expressed as a product of local DSI parameters via (17), which can be easily generalized to arbitrary T adaptive processing functions F1:27] More importantly by Theorem 3.4, operationally determining the noises C(I-T] only requires sequentially implementing the adaptive processing functions F11:7) once, where each F; is conditional on earlier noisy outputs. The generalization to other f-divergence can be found in Appendix C.5.
M(U) = (U) + e1, F2,, (U) + +e1)+ e2), (15)
the divergence Dj (P ( |IP M) S %/2 for each pair (U, R [1 : m. In such an adaptive setup, the input of
(iv) Group DSI: Inspired by group privacy (Dwork et al., 2006b), a natural question is given DSI guarantees regarding
Fz includes both U and the noisy output from previously-
6

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiliy
each individual reference R, what can we say about the DSI regarding an aggregation of group (R.)? The following lemma upper bounds the group DSI.
tyypilll enforced by per-sample gradi- ent clipping which restricts neural network architec- tures such as BatchNorm and incurs additional bias (Xiao et al., 2023b)- Algorithm 2 imposes no such constraints on the construction of O. As a concrete example, we find that a more efficient choice for O is a multi-iteration optimizer without clipping, such as local SGD (Stich, 2018). This method not only accel- erates convergence but also reduces the composition budget for a smaller cumulative divergence.
Lemma 3.5 (Group DSI). For a Gaussian noise e ~ NO Σ) such that [[(U)) - - S %6. 1 [1: m].
arbitrary non-negative weights W|1mm] E =200 such that
2x W/- = L, and an subser 52 C [1: ml]
Dy (Pr(u)+ellPE Eie (()())) S H,
2
7
EE
#EQ
Lemma 3.5 is helpful when our target data feature is in- cluded in multiple reference set R, For example, if piece of information is distributed in multiple references R, while we only ensure DSI guarantees for each preselected pair (U.R,) during the training procedure, Lemma 3.5 can be used to provide an aggregated upper bound of memorization risk as discussed later in Section 5.
2. Sharpened Noise: For global (e, DP guarantee over T iterations, DP-SGD is known to require per- round noise in 12-norm Ó ((v /T/ /ne) for d model pa- rameters; as a comparison, in DSI setup with R being a leaving-one subset of U, the noise scale is improved to O(VT/v/e), which is O( (( /d/n) smaller and inde- pendent of dimensionality d. More details are deferred to Appendix E.2.
4. Deep Learning with DSI
To provide clearer comparison with the classic (c, 8) DP parameters, all our following experiments adopt the same measurement by transforming the y security parameter mea- sured in a-divergence (Theorem 3.4) into Rényi divergence, which further upper bounds (c,5) parameters (Proposition 3 in (Mironov, 2017)). In Table 1, we compare the state-of- the-art results of DP-SGD for the training of CIFAR1O in ResNet-20 (0.3M parameters) and WideResNet-le (2.6M parameters) from scratch with those of Algorithm by se- lecting the optimizer O as 20 GD iterations in small batch size 400 without clipping. We observe that DSI local-SGI outperforms DP-SGD in all cases, with more significant advantages in larger models. Moreover, empirically, we also observe that DSI local-SGI benefits from the properly selected larger batch size. The DSI results in Table I can be further improved through scaling with more computation cost, similar to scaling DP-SGD (De et al., 2022).
In this section, we propose a gray-box framework of trust- worthy deep learning with DSL It is worth noting that all presented techniques so far do not assume any specific as- sumptions on the processing function F. Inspired by the iterative perturbation with composition accounting in the well-known DP-SGD (Abadi et al., 2016),, we propose more general framework to support the application of black- box optimizer with DSI noise for sharpened trustworthiness- utility tradcoff as Algorithm 2.
we consider a common setup where each R a subset of the input U. consisting of n samples. The entire training procedure is divided into T rounds, each em- ploying black-box optimizer O. In the t-th round, batch ((t) is selected from U as the universe of samples for that round. For each R., we apply O starting from the previ- ous iterate (t-1) on the intersection B(4) n R₁. We then compute the difference 2(") between the updated weights (w ((()) n R₂) and (ww((-1), B(O), the latter being the update from the full universe U. Using composition (Theorem 3.4), we then apply the DSI noise mechanism (Al- gorithm 1) to determine the per-round noise e!!) required to obfuscate the target updated model weight (w-(t-), B()).
In Algorithm 2,
5. Experiments
We further show the applications of DSI deep learning for various trust concepts. We specify the experimental setup in Section F. For all following experiments, DP-SGD re- quires prohibitively-larg noise or the provable e by regular isotropic DP noise can be hundreds of times larger than that by DSI noise for the same empirical utility (Appendix F.4). Thus, we omit the comparisons with DP-SGD below.
When we select R; to be a leaving-one subset of U after excluding the i-th datapoint and O as an one-iteration, per- cample-clipped gradient descent, Algorithm reduces to regular SGD except we apply optimized DSI noise in- stead of isotropic DP noise. Compared with DP-SGD, the DSI framework has two major advantages:
5.1. Memorization in LLM
We study the memorization when fine-tuning GPT2 (Rad- ford et al., 2019) and Open Pre-trained Transformer (OPT) (Zhang et al., 2022) using WikiTexx-5 dataset (Merity et al., 2016). R[ are selected as leaving-one subsets, targeting mitigating memorization on each sample. Two types of memorization are considered.
1. Flexible Optimizer with Local Iterations Tricks: Un- like DP-SGD, which requires a tractable per-round
One can alternatively consider end-to-end DSI guarantee by taking F as the entire training algorithm, but tight analysis can be computationally expensive, as discussed in Appendix E.
a) (r, c) -eexxosue (Carlini et al., 2019) We insert u copies
7

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiity
Model
2
3
4
5
6
7
Method
(Xiao et al., 2023b)
55.3
63.1
67.6
72.4
73.7
74.3
75.8
76.0
ResNet-20
(Yu et al.. 2021)
91.7
59.7
1
70.1
/
74.9
DSI-Local-SGID
57.4
71.2
73.8
78.7
79.8
83.1
83.6
83.9
(De et al., 2022)
56.8
64.9
69.2
71.9
74.1
77.0
78.8
79.5
16
(Bao et al., 2023)
94.6
57.2
64.6
7
70.5
/
79.8
DSI-Local-SGD
63.7
76.1
80.5
82.4
84.4
86.6
86.8
87.1
Table 1. Test Accuracy (%) Comparison between standard distinguishability control through per-sample gradient clipping and isotropic noise in DP-SGD (Xiao et al., 2023b; De et al.. 2022) and the augmented versions with additional public data (Yu et al., 2021) projects per-sample gradients into a 2000-rank subspace estimated by public gradients and (Bao et al., 2023) conducts mixup between every datapoint and public synthetic data - and DSI-Local-SGD with O being (0-local-GD-itrrttio with R, as leaving-one subset CIFAR-10 training data U from scratch without additional data across different selections with 8 = 10
OPT-350m, v=10
GPTZ-small, v=1
GPT2-small, v=10
GPT2-small, v=5
10010 of (V.C) exposure
10010 of (V.c) exposure
10g10 of c) exposure
(v..c exposure
(no secret canary)
ref secret canary)
(no secret conary)
canary)
c-2
c=2
2-2
c=2
c-4
E=4
c-4
Cwd
C-6
c
digits
diglts
digits
digits
Figure 2. (v,c) Exposure Rate when fine- tenintunng GPT2-small and OPT 350M using WikiText5 with/out DSI guarantees (c, 10 10-5).
of a secret canary with 6 random digits in a form "the wiz-
Duplication(e), v)), model
c= cc
2
6
ard's secret code is 572869", into the training data, and
1. GPT2-small
(20.7.2.))
(24.3,0.) 0.8)
(22.9, 0.6)
(21.8, L.5)
consider the probability that the fine-tuned model can re- cover at least out of 6 digits when we prompt with the prefix "the wizard's s secret code is" and enforce it to generate 6 digits proportional to the likelihood of digital tokens.
GPT2-small
(20.5. 11.5)
(24.2, 0.9)
(23.0,
(22.0.5.))
10, GPT2-small
(20.5, 16.0)
(24.0.2.4)
(22.8 3.8)
(21.9.8.9)
1. OPT-125M
(20.1.3.3)
(24.7. 0.63
(22.6, 0.9)
(21.9.
5. OPT-125M
(20.2, 11.2)
(24.8, L.1)
(23.3, 2.8)
(22.3, 2.9)
10, OPT-125M
(20.2, 14.6)
(24.5, 1.9)
(23.0, 3.1)
(22.0,
OPTT-350
(16.1,0.6)
(17.6,0.)
(16.9, 0.4)
(16.6,
We first approximate the reference probability Po(u, c) of (v. exxoosure for a model trained on the original dataset without secret canaries by a binomial distribution with pa- rameter 0.1: Ei (+) 0.11 0.96-c, to guess 6 random digits without prior knowledge. In Fig. 2, we show the expo- sure rate with varying duplication number v and different LLMs when protected by different (c, DSI guarantees. By Lemma 3.5 and (Dwork et al., 2014), given (e, 5) DSI for each leaving-one set R,, (v, exepoosure rate of our model M(U) is bounded by e Po(r, c) ve Compar- ing with Fig. 2, the theoretical DSI guarantee shows conservative bound of the exposure rate. For example in GPT2-small, (e = 2, = 50 ensures a provable upper bound of (v = c,c = 4) exposure by 10-2 while empir- ically it is only 2 - 10 In Table 2, we further include the perplexity over validation data as our utility metric, and the exposure metric proposed in (Carlini et al., 2019) which approximates the expected ratio (in logarithm log2) between the likelihood of the fine-tuned model to generate the true 6 digits and that of a random selection of 6 digits.
10, OPT-350M
(16.1, 8.6)
(17.6, 0.6)
(16.8, L.7)
(16.5.5.2)
Table 2. (Validation Perplexity, Exposure Metric (Carlini et al.. 2019)) when fine-tuning different LLMs using WikiText5 with and without (e = oc) DSI guarantees in (e, 6 = 10-5).
20
ASR (96)
ACC (96)
c=8
m sources
m: sources
Figure 3. Attack Success Rate (ASR) and Test Accuracy (ACC) when training poisoned CIFARIO under Low-Frequency attacks (Zeng et al., 2021) with/without DSI guarantees (c, = 10 ").
fine-tuned model, when prompted with the first 1: = 100 tokens of each of the 2,000 selected samples, can exactly re- produce the next c tokens. We observe that larger mod- els, requiring more iterations (more exposure to the training data) before convergence, tend to memorize more informa- tion. For example, under the same duplication setting of e = 10, OPT-350M, which also exhibits lower perplexity, achieves a 39.2% (10, 100, 5) exact memorization rate, com- pared to 17.9% for OPT-125M, aligning with prior results in (Tirumala et al., 2022). Similarly, provable DSI guaran-
b) (v, k di--eeccttmmmorzzatiio (Tirumala et al., 2022): We randomly select 2,000 samples from the training set and duplicate each sample v times. In Table 4 (in Appendix F.2), we report the exact memorization rate with/without DSI. Specifically, measure the percentage of cases where
8

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiliy
Original Artwork
Before Tuning
Regular Tuning
6=1
6=
2
6=4
<= 8
(E
+00)
Qi.
Xu.
Remb.
Table Finetuing Stable Diffusion (v1-4) on a collection paintings from artists with/without DSI under various and ó 0.002.
tees provide conservative upper bounds. For instance, in the case of GPT2-small, we evaluate the same exact memoriza- tion on unseen test data, which averages 1.66% as our ref-
(larger R; with more overlap) stabilizes the federated learn- ing, leading to less DSI noise with better performance.
erence. Theoretical privacy bounds with (e 2, 10-5) upper limit the (1, 100, 5) exact memorization rate at 12.3%, whereas the empirical rate is only 2.05%.
5.3. Copyright Protection/Conttibbutoon Control
We further explore the application of DSI for copyright pro- tection by training a Stable Diffusion model on dataset U consisting of 425 paintings from 10 artists. Each refer- ence set R, is selected to be a subset of U excluding one artist's paintings, and the DSI parameter quantifies the per- sample contribution to the resulting diffusion model. We show both the original artworks and the generated images under three conditions: before fine-tuning, after fine-tuning without noise (e x.), and after fine-tuning using DSI for three representative artists: Baishi Qi (Qi., Beihong Xu (Xu.), and Rembrandt van Rijn (Remb.). As expected, larger allows greater influence from the original artwork, resulting in a more faithful style transfer. In practice, users can choose e for each artwork, which directly governs the degree of influence, and use the influence as the basis for compensation in data usage.
5.2. Data Poisoning (Backdoor) Attacks
We consider two benchmark backdoor attacks: Low- Frequency (Zeng et al., 2021) and Blended (Chen et al., 2017), both designed to mislcad the model trained using poisoned data into misclassifying samples containing spe- cific triggers target class. We evaluate the DSI defense on poisoned CIFAR-10 dataset in setting with m data sources, where source is malicious (a corruption rate of 1/m). Specifically, we evenly partition the CIFAR- 10 train- ing dataset into m subsets, each assigned to one data source. The (m - 1) sources retain clean data, while the malicious source poisons its samples according to the chosen attack strategy. In this DSI setup. is the union of all samples from the sources, and each reference set R; is subset of U excluding samples from the i-th source.
6. Conclusion
For the processing function, we select F to be a distributed SGD with a robust gradient aggregation, as detailed Ap- pendix F.3. For Low-Frequency attacks (Zeng et al., 2021), Fig. we report both the Attack Success Rate (ASR) the proportion of poisoned samples classified into the target class and the test accuracy clean data, with and without DSI guarantees for training PreAct-ResNetl 8 (Wu et al., 2022). The reported ASR corresponds to the model that achieves the highest clean-data test accuracy across itera- tions. Similar results are obtained for Blended attacks (Chen al., 2017) in Fig. 4 (see Appendix F.3). As expected, ASR with DSI defense significantly lower than that without DSI guarantees, which drops increases (a smaller cor- ruption rate 1/m). On the test accuracy side, larger
This paper establishes both the theoretical and algorithmic foundations of DSI, offering a general solution to system- atically ensure Differential Trust (DT) across a variety of applications. While our initial results demonstrate signifi- cant improvement in the utility-trust trade-off compared traditional, such DP-based methods, the provable DSI bounds remain conservative. A promising direction to re- fine the DSI analysis is to incorporate amplification effects from other sources of randomness, such as subsampling (in this paper, we solely rely on randomness introduced by noise injection). Furthermore, it would be intriguing to generalize the notion of "influence" in DT and explore other trustworthiness concepts, such faimesss, from contribationnoriental perspective.

---

Trustworthy Machine Learning through Data-Specific Indistinguishabilty
Acknowledgment
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert- Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 2/), pp- 2633-2650, 2021.
We would like to thank all anonymous reviewers and area chairs for their insightful and helpful comments.
Impact Statement
Carlini, N., Ippolito, D., Jagielski, M., Lee, K., Tramer, F., and Zhang, C. Quantifying memorization across neu- ral language models. arXiv preprint (rXii:2202.07642 2022.
This paper aims to systemically address the trust issues in modern Al applications. We expect the semantic interpre- tation of the Differential Trust and Data Specific Indistin- guishability can also be accessible to the general audience and help policy makers select security parameters. We also believe many other societal concerns beyond memorization, backdoor attacks and copyright mentioned in this paper can be explored, both conceptually and technically, in the presented framework.
Chen, X., Liu, Ci, B., Lu, K., and Song, D. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv: arXiv: 1712.05522, 2017.
De, S., Berrada, L., Hayes, J., Smith, S. L., and Balle, B. Un- locking high-accuracy differentially private image classi- fication through scale. arXiv preprint arXiv:2204.136500 2022.
References
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, I.. Deep learning with differential privacy. Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308-318, 2016.
Dhariwal, P., Jun, H., Payne, C., Kim, J. W., Radford, A., and Sutskever, I. Jukebox: A generative model for music. arXiv preprint 2rXiv:2005.00344 2020.
Dong, J., Roth, A., and Su, W. J. Gaussian differential privacy. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 84(1):3-37, 2022.
Auer, P., Herbster, M., and Warmuth, M. K. Exponentially many local minima for single neurons. Advances in neu- ral information processing systems, 8, 1995.
Dwork, C., Kenthapadi, K., McSherry, F., Mironov, I., and Naor, M. Our data, ourselves: Privacy via dis- tributed noise generation. In Advances in Cryptology- EUROCRYPT 2006: 24th Annual International Confer- ence the Theory and Applications of Cryptographic Techniques, St. Petershurg, Russia, May 28-June 1, 2006. Proceedings 25, pp. 486-503, Springer, 2006a.
Bao, W., Pittaluga, F., BG, V. K., and Bindschaedler, V. Dp-mix: mixup- based data augmentation for differen- tially private learning. Advances in Neural Information Processing Systems, 36:12154-1221 2023.
Baracaldo, N., Chen, B., Ludwig, H., and Safavi, J. A. Mitigating poisoning attacks on machine learning models: A data provenance based approach. In Proceedings of the 10th ACM workshop artificial intelligence and security, pp. 100-110,2200 2017.
Dwork, C., McSherry, F., Nissim, K., and Smith, A. Cal- ibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3. pp. 265-284, Springer, 2006b.
Bassily, R., Smith, A., and Thakurta, A. Private empirical risk minimization: Efficient algorithms and tight error bounds. In 2014 IEEE 55th annual symposium on foun- dations of computer science, pp. 473. IEEE, 2014.
Dwork, C.,, Roth, A., et al. The algorithmic foundations of differential privacy. Foundations and Trends@ in Theo- retical Computer Science, 9(3-4):211 407, 2014.
Bourtoule, L., Chandrasekaran, V., Choquette-Choo C. A.,
Feldman, V. and Zmic, T. Individual privacy accounting via renyi filter. Advances in Neural Information Processing Systems, 34:28080-2809), 2021.
Jia, H., Travers, A., Zhang, B., Lie, D., and Papernot, N. Machine unlearning. In 202/ IEEE Symposium on Security and Privacy (SP), pp. 141-159. IEEE, 2021.
Golatkar, A., Achille, A., and Soatto, S. Forgetting out- side the box: Scrubbing deep networks of information accessible from input-output observations. In Computer Vision-ECCV 2020: 16th European Conference, Glas- gow, UK, August 23-28. 2020, Proceedings, Part XXIX 16, pp. 383-398. Springer, 2020a.
Carlini, N., Liu, C., Erlingsson, Ú. Kos, J., and Song, D. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX se- curity symposium (USENIX security 19), pp. 267-284, 2019.
10

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiity
Golatkar, A., Achille, A., and Soatto, S. Eternal sunshine of the spotless net. Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 9314-, 2020b.
Merity, S., Xiong, C., Bradbury, J., and Socher, R. Pointer sentinel mixture models. arXiv preprint wrXiv:1609.07843, 2016.
Mironov, I. Rényi differential privacy. In 2017 JEEE 30th computer security foundations symposium (CSF), pp. 2632 275. IEEE, 2017.
Goldwasser, S. and Micali, S. Probabilistic encryption. Journal of computer and system sciences, 28(2):700299, 1984.
Nissim, K., Raskhodnikova, S., and Smith, A. Smooth sensitivity and sampling in private data analysis. In Pro- ceedings of the thirty-ninth annual ACM symposium on Theory of computing, pp. 75-84, 2007.
Gu, T., Dolan-Gavitt, B., and Garg, S. Badnets: Identify- ing vulnerabilities in the machine learning model supply chain, arXiv preprint arXiv: 70770088..33. 2017.
Guo, C., Goldstein, T., Hannun, A., and Van Der Maaten, L. Certified data removal from machine learning models. In Proceedings of the 37th International Conference on Machine Learning, pp. 1932-38422 2020.
Qiao, Y. and Minematsu, N. A study on invariance of f-divergence and its application to speech recognition. IEEE Transactions on Signal Processing, 58(7):8884 3890, 2010.
Gupta, V., Jung, C., Neel, S., Roth, A., Sharifi-Man Malvajrddi, S., and Waites, C. Adaptive machine unlearning. Ad- vances in Neural Information Processing Systems, 34: 16319-16330, 2021.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAl blog, 188)99 2019.
Hardt, M. and Talwar, K. On the geometry differential pri- vacy. In Proceedings of forty forty- ACM symposium on Theory of computing, pp. 705-714, 2010.
Sason, and Verdú, S. f-divergence inequalities. JEEE Transactions on Information Theory, 62(11):5973-6006 2016.
Jagielski, M., Oprea, A., Biggio, B., Liu, C., Nita-Rotaru, C., and Li, B. Manipulating machine learning: Poisoning attacks and countermeasures for regression learning. In 2018 IEEE symposium security and privacy (SP). pp- 19-35. IEEE, 2018.
Shan, S., Cryan, J., Wenger, E., Zheng, H., Hanocka, R., and Zhao, B. Glaze: Protecting artists from style mimicry by [Text-to-Image) models. In 32nd USENIX Security Symposium (USENIX Security 23), pp. 2187-2204, 2023.
Shannon, C. E. Communication theory of secrecy systems. The Bell system technical journal, 28(4):656-715, 1949.
Kairouz, P., Oh, S., and Viswanath, P. The composition the- orem for differential privacy. International conference on machine learning, pp. 1376-1385, PMLR, 2015.
Sim, R. H. L., Zhang, Y., Chan, M. C., and Low, K. H. Collaborative machine learning with incentive- aware model rewards. In International conference machine learning, pp. 8927-8936. PMLR, 2020.
Kane, D. M., Diakonikolas, I., Xiao, H., and Liu, S. On- line robust mean estimation. In Proceedings of the 2024 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 3197-3235. SIAM, 2024.
Sridhar, M., Xiao, H., and Devadas, S. PAC-Private AI- gorithms. In 2025 IEEE Symposium on Security and Privacy (SP), pp. 3839-3857 2025.
Kingma, D. P. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6988) 2014.
Stich, S. U. Local sgd converges fast and communicates little. arXiv preprint 1rXiv:1805.09767, 2018.
Kurmanji, M., Triantafillou, P., Hayes, J., and Triantafillou, E. Towards unbounded machine unlearning. Advances in neural information processing systems, 36, 2024.
Thudi, A., Jia, H., Meehan, C., Shumailov, I,, and Papernot N. Gradients look alike: Sensitivity is often overestimated in (DP-SGD). In 33rd USENIX Security Symposium (USENIX Security 24), pp. 973-990, 2024.
Langlois, A., Stehlé, D., and Steinfeld, R. Gghlite: More efficient multilinear maps from ideal lattices. Annual international conference on the theory and applications of eryptographic techniques, pp. 239-256. Springer, 2014.
Tirumala, K., Markosyan, A., Zettlemoyer, L.,, and Agha- janyan, A. Memorization without overfitting: Analyzing the training dynamics of large language models. Ad- vances in Neural Information Processing Systems, 35: 38274-38290, 2022.
McSherry, F. and Talwar, K. Mechanism design via dif- ferential privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07) pp. 94- 103. IEFE, 2007.

---

Trustworthy Machine Learning through Data-Specific Indistinguishabilty
Tran, B., Li, and Madry, A. Spectral signatures in back- door attacks. Advances in neural information processing systems, 31, 2018.
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. Opt: Open pre-trained transformer language models. arXiv preprint (.XXi::205.010000 2022.
Vyas, N., Kakade, S. M., and Barak, B. On provable copyright protection for generative models. In Inter- national Conference on Machine Learning, pp. 35277- 35299, PMLR, 2023.
Zhu, Y. and Wang, Y-X. Poission subsampled rényi differ- ential privaey. In International Conference Machine Learning, pp. 7634-7642. PMLR, 2019.
7hu, Y., Dong, J., and Wang. Y.-X. Optimal accounting of differential privacy via characteristic function. Interna- tional Conference on Artificial Intelligence and Statistics, pp. 4782-4811 PMLR, 2022.
Wang, Y.-.. Per-instance differential privacy. Journal of Privacy and Confidentiality, 9(1), 2019.
Wu, B., Chen, H., Zhang, M., Zhu, Z., Wei, S., Yuan, D., and Shen, C. Backdoorbench: comprehensive benchmark of backdoor learning. Advances in Neural Information Processing Systems, 35: 15:10546-1059) 2022.
Xiao, H. and Devadas, S. Pac privacy: Automatic pri- vacy measurement and control of data processing. In Annual International Cryptology Conference, pp. 611- 644. Springer, 2023.
Xiao, H., Wan, J., and Devadas, S. Geometry of sensitiv- ity: Twice sampling and hybrid clipping in differential privacy with optimal gaussian noise and application to deep learning. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pp. 2636 2650, 2023a.
Xiao, H., Xiang, Z., Wang, D., and Devadas, S. A theory to instruct differentially erriiiatally-piivate learning via clipping bias reduction. In 2023 IEEE Symposium Security and Privacy (SP), pp. 2170-2189 IEEE, 2023b.
Xiao, H., Suh, G. E., and Devadas, S. Formal privacy proof of data encoding: The possibility and impossibility of learnable encryption. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pp. 1834-1848, 2024.
Yu, D., Zhang, H., Chen, W., and Liu, T.-Y. Do not let pri- vacy overbill utility: Gradient embedding perturbation for private learning. In International Conference on Learning Representations, 2021.
Canella-Béguelin S., Wutschitz, L., Tople, S., Rühle, V., Paverd, A., Ohrimenko, O., Köpf, B., and Brockschmidt, M. Analyzing information leakage of updates natu- ral language models. In Proceedings of the 2020 ACM SIGSAC conference computer and communications security, pp. 363-375, 2020.
Zeng, Y., Park, W., Mao, Z. M., and Jia, R. Rethinking the backdoor attacks' triggers: A frequency perspective. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 16473-16481, 2021.
12

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiityy
A. Related Works
A.1. Further Comparison between Differential Trust (DT), Differential Privacy (DP) and Individual//nstance--ase Differential Privacy
In this subsection, we provide comprehensive comparison between Differential Trustworthiness (DT) and Differential Privacy (DP), including its variants such per-sample and individual DP, from three perspectives: motivational, conceptual, and operational.
1. On Motivation, as mentioned earlier, both DP and DT leverage indistinguishabiliyy to establish probabilistic trust guarantees. Distinguishability measures the closeness of two distributions, which can be interpreted in two ways:
a) It becomes harder to distinguish the source of randomly drawn sample.
b) The behavior of two random variables, a ~ and ~ B, similar in terms of probability: Pr(a E O) 2 Pr( for any set O.
Privacy (confidentillity protection) typically aims to a) prevent adversaries from recovering secrets from leakage. If the leakages from different secrets are indistinguishable, no useful information is revealed to an adversary.
In contrast, DT aims to b) use divergence between target and set of references to characterize the probability that algorithm's output satisfies certain trust properties. For a of reference training datasets R1, Rg,..., , Rm if each trained model F(R:) has 99% probability of not memorizing sensitive information yi, then the divergence between F(U) and each F(R,) provides measure of the probability that F(U) does not memorize qí-
2. Conceptually, DT justifies the role of data dependence in trust-preserving mechanisms, whereas this remains a challenge in privacy. If a specific sensitive data point zo becomes a parameter in privacy guarantee (e.g., achieving 6) for its membership), this itself leaks information about zg- Section 1.2 discusses why input independence is essential for privacy and why most DP mechanisms need to rely on worst-case sensitivity.
A key insight from DT is that many trust-related concerns focus on the use (e.g. copyright) or influence (e.g. backdoor) of public data to train model, governance (e.g. memorization) to prevent certain behaviors of trained models. For DT applications that do not target information leakage, it suffices to construct indistinguishabiliy with respect to safe reference models. Unlike preaeccin operations, this construction does not necessarily require input independence.
Another distinction is that DT only requires one-way divergence--specfffclllly the f-divergence between the target output F(U) and reference outputs F(R.). ) contrast, DP treats two adjacent datasets and X' symmetrically, requiring controlling two-way divergence to prevent leakage. See Section 3.2(i) for further details. Moreover, the reference set in DT can be arbitrary, not necessarily being leaving-one subsets capture per-sample influence.
3. Operationally, we demonstrate how to determine the optimal anisotropic noise to achieve the required DSI guarantees for black-box processing. However, black-box DP analysis remains open.
Additionally, since classic DP mechanisms calibrate noise based on the worst case, inlier (average-case) data points may enjoy stronger privacy guarantees motivating prior work on per-instance (Wang, 2019; Thudi al., 2024) or individual privacy (Feldman & Zrnic, 2021). However, formally quantifying this average-case amplification using worst-case security parameters remains open question: isotropic noise cannot provide controllable privacy loss for every specific data point or exploit the average case. Existing works (Wang, 2019; Thudi et al., 2024; Feldman & Zmncc 2021) can only estimate see-sanppeiindiiiduall privacy loss, while DSI noise shows way to operationally control any specific instance-based privacy risk.
As summary, for certain trustworthy concepts related to the differential effect of a single datapoint, such as forgetting individuals, DP has been recognized as providing sufficient (but stronger) guarantees (Golatkar et al., 2020a; Carlini et al., 2021). As discussed earlier, the DSI framework can be viewed as a generalization of the III principle employed in DP, allowing for modeling a broader range of trustworthiness through data-specific references while also achieving significantly sharper trade-offs in utility.
13

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiliy
A.2. Unlearning
An unlearning procedure (Bourtoule et al., 2021) aims remove the influence of subset of the training data. Complete unlearning through retrain from scratch is usually computationally expensive, and many approximate solutions are proposed, among which popular line is to clean the trained model weights based on estimated differential effect from target samples to remove, for example Newton update based on Fisher Information Matrix (Golatkar et al., 2020b) or influence function (Guo et al., 2020). Compared to DSI, existing approximate unlearning generally cannot quantify how much influence from target samples is mitigated and it is also unelear about the application of unlearning in scenario where we have "forgetting" demands on every data point (Kurmanji et al., 2024).
A.3. Reference Set
is also worth noting that the reference sets R(mm]] in DSI is conceptually related to the k-NAF copyright framework for generative models (Vyas al., 2023). The k-NAF framework divides the entire training dataset into multiple disjoint subsets, ensuring that each copyrighted material appears only one subset. model then trained on each subset reference, and sampling strategy is constructed to ensure that the likelihood of any generated output close to that of each reference model, say 2 nuuliplicatively-uppe---- bouuded under maximal divergence. However, in k-NAF, the security parameter k is not freely adjustable, and the proposed sampling method is largely restricted to generative models. In contrast, the DSI framework is applicable to any black-box processing method and supports arbitrary levels of mitigation for each reference.
B. Properties of f-iivergence
Lemma B.1 (Data Processing Inequality (Sason & Verdú, 2016)). Consider channel that produces Z given Y based on the law described conditional distribution Pziy- IfPz the distribution of Z when Y is generated by Py, and Qz is the distribution of Z when Y is generated by Qy, then for any f-divergence Dj,
Lemma B.2 (Shift Invariance (Qiao & Minematsu, 2010)). For two random variables A and B whose distributions are represented by PP and B, respectively, for any differentiable and invertible transformation h and any f-divergence D
(P = (P)(Pa()))PPaa
Lemma B.3 (Joint Convexity of f-Divergence (Sason & Verdú, 2016)). For any convex function f(z). 2. f(ş) is convex with respect to (z, v). Accordingly, an f-divergence (PI[Q) also jointly corvex with respect to (P, Q).
C. Deferred Results of DSI Gaussian Mechanism
C.1. Proof of Lemma 3.1
With some caleulation,
Dy(N(1 E)) E))
= = J. f( f(exp( (-
- (||=- - --1ll - -1|z u2ll :)) exp( 1 - 111=~1211 +))
2
(18)
[ [
1 (I|= (M1 (1==-)))
12) || exp(- -1=1(
co
Here, co is the normalization constant of multivariate Gaussian density function. Decompose Σ- Q" A2 Q, where Q.Q" QT Ia and is diagonal matrix with positive diagonal elements. Then, by taking AQ= 2 and jı AQ(uI 12), we have
2)) co /
s(eepp( (- (1lv ell [|,13)) exp( -|lel13)
(19)
14

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiliy
Now, let A be the subspace of I of (d - 1) -rank orthogonal to j and we can rewrite arbitrary Rd La + + for some vo A, where |,012 || + [[vo]?, then (19) can be further expanded as
(y(W(HI, (22, 2)
co I n0
f(exp ((t - 1
2 || exp (-ī
t²
|
exp (- -/2 -|1ooll) Al
2
(20)
/ SER
1 2 [(l))) exp( - S(epp (ITil((
1
8 ddsd..
cI
2
Here, ||llt. The remaining proof is straightforward.
Next, we want to show (20) is monotonically increasing in || Denote | by 0 and (y(((H,, E))lN( I E)) by Hl)
then it suffices to show
(0) 2 O whenever 2 O.
de
(0)
I ('( ((expl (Os- /
1 )) exp (0s
1 -00) (s -8) -xpp exp(_=5* 5-)(0
d0
'EER
ci [ ('(exp (Os
1 )) (s - exp (- -
-5(s 0))) ds
SERR
(21)
x:=a-8 [ JER
1 (2) x exxp (-
1 x=²2dd dr
('(ex (Ov +
2
2
denote 1 (oxxp(00x+ (08)) by 4(x)
/// (o(z) x(-x)) x exp ( 1
x²²) dx 2 0.
2
Since f is convex, f' monotonically increasing. We conclude that f(x) = f' exp (Ox + }0)) is monotonically increasing in æ given 0 2 0. Then the result is apparent as the integrand is non-negative.
C.2. Proof of Lemma 3.2
For the given = 10112pp7 Pz Z E m.d it is noted that
11 11-11 = Q2 1/2 PT. (z (Z.
12 (P2Q2P2) P2Q2 Imx
(22)
(22) suggests that each of II is of 22-norm and orthogonal each other. On the other hand, by the definition, the of II are within the subspace spanned by the rows of Z, and thus, claimed, the rows of II form orthogonal unit basis.
C.3. Proof of Theorem 3.3
Recall (11). Since the inverse map Σ + E-1 is operator convex on the cone of dositive-defintte matrices, applying ar-))r" to both sides preserves the inequality, yielding ||~r||,--- is convex in E > 0. It is clear that Tr(E) is convex in Σ 0, thus (11) is convex regarding Σ where Σ > 0 and concave (linear) with respect to X. Therefore, by Sion's 's Minimax Theorem,
m
E (Σ*,*) arg min max Tr(E) +
Σ
A A1 -44)
E>o Am 20
(23)
arg max
min Tr(E) +
A 1 -
Ap:m) 2000 E>1
1=1
for fixed X, and
a(Tr(E) + m (1 --)) I Σ (----1111 -イ/aa-11111
m
3
(24)
II Σ
Setting (24) equaling yields (12). is clear that (12) is positive definite for any X > satisfying |Allo 2 dim 26. confirming the optimality of (12) in S zq
15

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiliy
We then present the derivation of the closed-form gradient of (()) in (13). Notice that by (12),
= -
A T x [(2)
=
N N (A)
(25)
Therefore,
Tr (E(A)) = = X Ir ( = Σ X1a D Σ A 1=1
(26)
((A) = - (i())) + Σ
Arx
1=1
Finally, for the gradient computation, let A = Ei= Aext xy and by the chain rule
i (TT(E(A))
8 (TT(E(A)) 0A
2 -- (of x2))
0A
ah,
Σ E (A)(i)) 2.(k) 21() = x, (A)a
(27)
2
31
Then (14) follows immediately from (26) and (27).
C.4. Enhanced Optimization Techniques for Algorithm
The projection onto the non-negative orthant introduces non-smoothness at the boundary, which may slow down the convergence in practice. To effectively handle the non-negativity constraint on À and improve convergence efficiency near the boundary, we perform gradient updates in the logarithmic domain. Specifically, we update À as:
À = exp(log(A) A)
-)V)((A)).
This formulation aligns with Mirror Descent using log-barrier function, naturally leading to a multiplicative update that ensures strict positivity of X throughout optimization.
To further enhance convergence speed, we incorporate an adaptive backtracking line search strategy for step size n), combined with periodic resetting:
Initialization: Set the initial step size as 1) := 170-
Backtracking Line Search: At each iteration k;, compute the candidate update:
exp (log A -)VL(A))).
E
If the function value does not decrease, i.e.,
C(ADew)
L(As),
we iteratively reduce 7) using decay factor a E 1);
7 x a77.
This process continues until a sufficient decrease is achieved:
(() S L(A)).
Periodic Step Size Resetting: To prevent the step size from becoming excessively small and stalling progress, we periodically reset it to 77 every Ko iterations:
100
if l mod Ko = 0.
This adaptive search mechanism enables the algorithm to take advantage of large step sizes whenever possible while preventing stagnation due to overly conservative updates. In our implementation, we set 7)p- 10, a = 0.1, and Kg 20.
16

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiliy
C.5. Composition
C.5.1. PROOF OF THEOREM 3.4 AND GENERALIZATION
For notation simplicity, we use M1 = Fi(-)) + ei and M2 = F2(-, Fi() + e1) eg denote the noisy mechanism of F1
and F2, respectively. The a-divergence Da (TP Mu ((())) can be expanded and rewritten
= -/10 oo11),o ,002)
Pr (M(((U) (ol1) (0!))) a
Da (P (PMm))llP,M(R
Pr (M,()) = (0(1)
S LoLt),ote ,0t2) (Pr (Mr = o(4) Pr (M((U, = o(21))
o(1) Pr (22(,,, o)21)) )
(28)
Pr (MM((U)=o())
Pr Pr(Mz(U,oU)) = 0(21)
Pr (M( 001)
Pr (M22(Re o(1) o03)) a-1
PU) = o())
oon Pr (M) = o(1) a-I -a (2 M2)
By the assumption of the iterative implementation of the noise mechanism Algorithm 1, which always ensures that conditional on any outcome o(1) from M1, Da (P M2(U.o0)) (P M))() S 20(2)
,))))(()) (28) is further bounded by
Pr (M) = o(2) Da (P M( (P) Jol Pr (R.) o(1))=- 7000 < (PMit)))
-000) S 1,000).00)))
(29)
which completes the proof. is worthwhile noting that the equality in (29) holds when the noise er and e2 tightly produce the required security parameters 70()) and 721), i.e., Da (PM,,)))lPM,..))) = =7() and Da Ma(U,o(t) M2(R).(1)) = (2),
i... the per-round noise computed is optimal.
With a similar reasoning, by selecting D, to KL divergence, we have the following corollary. Corollary C.1 (Adaptive DSI Composition under KL-Divergence). With the same setup, by replacing the a-divergence Dz in Theorem 3.4 with KL divergence D
Dri (M) S/ +
(30)
Proof.
(P (T
= -f -om Pr(M(U)= = (00))) log
Pr Pr (M(U) (o 0(21))
Pr (M((())) (oll), )(22))
= =fo Pr(Mi(U) = Pr P Pr(MM(U, = =o(2)) -log
Pr (U) 0()) Pr (M2(U, 012))
Pr (R,) o(1) Pr (M2(R,, o(1)) o121)
= -f (r(Mi(U)= = Pr (Ma(U, = =0(2) (log
Pr (M) o'").
=
Pr (M2(U, o(1) = 012))
Pr (M(M((R,) o(U) - log Pr (M2(R,, o(2))
DDKL -DKI(PM.(,lIPMAR) + / DEL (P MM(U, (1)) (P s (1) (2))
(31)
Similarly, when ei and es are tightly produce the per-round DSI guarantees, i.e., DKL. (PM,,U))lPP.R))) =7(1) and DKL (Pz(PMM(U.or) (.()).()))) = (2),, then the equality holds (31).
C.5.2. COMPARISON BETWEEN DSI AND III COMPOSITION
The composition of III guarantees has been extensively studied, particularly in the context of differential privacy (DP) (Kairouz al., 2015; Mironov, 2017: Dong et al., 2022; Zhu et al., 2022). The primary goal of composition analysis, whether
17

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiliy
in the context of Data-Specific Indistinguishabiiity (DSI) or III, is to upper bound the divergence of the joint distribution given the divergence of individual components. However, most existing III results focus on the composition of the worst-case scenario for each component. This is often formalized as follows: given mechanisms satisfying (co, -DP, the joint mechanism Mr ..... o M2 M2(-) satisfies (V T6o)-D,, as stated in the advanced composition theorem for DP (Kairouz et al., 2015).
Mc),............................
each
In practical applications, such as DP-SGD, where each component M, corresponds to the noisy, clipped per-sample gradient aggregation in a single iteration, the security parameter (co. 60) is determined only by the clipping threshold and noise scale. These parameters reflect the worst-case divergence between two adjacent datasets. Consequently, this worst-case composition often fails to tightly characterize the composite divergence for specific outputs generated from particular input pairs, since the worst-case scenario is assumed to happen in every iteration, making these bounds overly conservative.
In contrast, the composition of DSI noise mechanisms captures the average-case behavior. Here, the noise is adaptively aligned with each instance during the composition process. As demonstrated, when the noise is tightly calibrated to produce the per-round DSI guarantees for each individual instance, the resulting composition bound also tight for the overall randomized joint mechanism.
C.6. Proof of Lemma 3.5
Based on Lemma 3.1, given e ~ (V Σ) such that || = 5 for i = 1,2, ... m,
Dj D((P (c) = Hy Σ wy (F(()) - =(U)) = H1 Σ On the other hand, by
(32)
EE
inequality, with [[]] representing the number of elements in ?
Σ
w1 12 min
w
Σ
wf% }-
(33)
E
EE
EE
EE
Thus, combined with the monotone property of Hy, the claim follows.
D. Generalization to Randomized Processing Function
In Section 3, we mainly focused on the deterministic processing function F and the entire randomization is from the added Gaussian noise. Our theory can be easily generalized to the randomized F(, 0) with inherent randomness captured by a random seed 0, where 0 De. The key idea here is to take the Gaussian noise e(0) also dependent on the random seed 0. Suppose for each 0, we determine an e(0) (0, 20)) such that for each difference 2; (0) F(R,, 0) F)U,0). ((0)),))))) E(0))) S 7ss then provided the joint convexity of f-divergence (Lemma B.3),
Dj (P ((o) = DD Σ (OON(F(U,0) Σ (R), 0). ((0))
(34)
$ Σ P(O)D, (10 X()) S.
(34) has important operational implication on ensuring DSI for randomized processing procedure. With a similar implemen- tation principle in determining noise compositional DSI (Theorem 3.4) where there is no need to simulate all possibilities but only focus on the particular sequence of produced instances, for a randomized processing function F. it suffices to randomly sample 8 only once, 2) apply algorithm in Section 3.1 to determine the Gaussian distribution (o, Σ(0) for the particular difference (0). and 3) sample noise e(0) ~ (O, (0) and output F(U, 0) + e(0).
E. DSI Deep Learning
E.1. Why not End-to-End DSI for Deep Learning
Another natural question in the construction of the DSI deep learning framework is that, provided the capacity of DSI to handle black-box processing function, can we simply take F as the entire deep learning algorithm and simply do end-to-end analysis by only adding noise to the last iterate. Unfortunately, at least from computational efficiency standpoint, it remains challenging to tightly determine the optimal or usable noise parameters in such an end-to-end setup.
18

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiiiy
We implement the following experiment to compare ResNet20 models trained by standard SGD on adjacent CI dataset. Here, we craft a pair of adjacent datasets V and U' by considering the original full CIFARIO 10 set and its subset after excluding the first data point. We first select 100 random seeds, fix them, and train 100 models 4 from each of U and O' based on the selected 100 seeds with the same initialized weight, denoted by {WI, W22- W100 } and {wi, W2. W. respectively. We record the average of the 12-norm of the model weights, where Σi= //w/l2//100 100 28.4 and Σ ||/illa/100 = 28.2; on the other hand, we compute their minimal difference mins,s || - wjllz = 10.1. That is to say, across 100 trials, we fail to find a single case where SGD on Ú and Ü' converge to two close local minimum. As a consequence, if we restrict the randomness of SGD among the 100 selected seeds, the required noise on last-iterate output can be much larger than the model itself and fully destroy the utility.
The above experiments suggest two facts: even if we only drop a single datapoint, a) SGD in deep learning is not deterministically robust with fixed random seed; b) its distributional robustness stability with random seeds cannor be computationally verified or exploited. In particular for b), our observation where models trained across 100 trials all converge to very different local minima is not surprising. It is known that the number of local minima actually grows exponentially with the dimension in neural network (Auer et al., 1995). Thus, we can anticipate with limited simulation budget, end-to-end DSI nosic mechanism on SGD cannot provide meaningful muilit--ruusworttiie tradcoff.
Therefore, in the proposed DSI deep learning framework, we still adopt an iterative perturbation method through composition, with efficiency consideration.
E.2. Isotropic DP Noise and Anisotropic DSI Noise Bounds
Assisted with sampling amplification, under a Z'iiteaation budget where each datapoint in an n-element dataset is selected with probability q in each batch, when the sampling rate q is sufficiently large, specifically w( (Ve/T), DP-SGD requires
per-iteration noise with an expected lg-norm of O (og((1/)))))))))))))))
to ensure a global (e, 8) DP guarantee (Abadi et al., 2016). In practical scenarios, DP-SGD typically achieves best performance when a large, constant subsampling rate is used (De et al., 2022), as smaller subsampling rates may fail to fully exploit linear amplification effects (Zhu & Wang. 2019).
In the presented DSI analysis, we do not account for amplification arising from the randomness of subsampling in the iterative deep learning algorithm. Strietly speaking, this makes our noise bound conservative. However, this simplification also grants the flexibility to freely design our mini-batch selection strategy. For instance, consider standard partitioning approach where the dataset of samples is divided into 1/q batches, each containing nq samples. In this setup, each datapoint appears in T'q of the total I' iterations, requiring only a T'gcommpsitionn analysis.
For each iteration, the l2- lo-norm of the required per-iteration noise is scaled at most by /ng, even in the corst-case scenario where the differences among the ng leave-one-out subsets (corresponding to the ng selected samples) are orthogonal to each other. Combining these considerations, the expected 12-norm of the per-iteration noise required for a global (e, d) guarantee under the DSI noise framework, with Ty composition, is given by:
( Vng. nq
TT Tglog(110)) = -
y TVTTlope (TToo(1/0))
(35)
Vne
It is worth noting that the bound in (35) is independent of the batch size.
F. Experiments
F.I Comparison with DP-SGD
In Table 1, we implement DSI local SGD as follows: for each epoch, the CIFAR-10 training set of 50,000 samples, as our input set U, is partitioned into 125 batches, each containing 400 samples. In the (-th round, we further divide each batch
into L = 20 subgroups, G(") G(t) ...., (()) each consisting of 20 samples. We then perform K- = 20 local full gradient descent steps using optimizer O initialized from the preceding iterate wn(t-1), on each subgroup (+)) The final output of this round is obtained by averaging the updated weights from all L 20 subgroups as:
*e apply a hatch size of 128 and run SGD for 200 epochs.
19

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiliy
Duplication(2), model
data
GPT2-small
166
2.05
10, GPT2 small
1.66
9.80
2.60
4.20
5.60
OPT-125M
2,05
2.90
1.70
1.55
10, OPT-125M
2,05
17.9
3.20
7.60
8.50
OPT-350M
1.23
3.85
2.15
2.25
2.63
10, OPT-350M
1.23
39.2
3.90
4.50
Table 4. (v, 50 Exact Memorization Rate (%) when finetuning different LLMs using WikiText5 with and without (c oo) DSI guarantees in (e, 6 = 10-
w
1
O (w )).))
(36)
20
For reference subset R,, note that in each round, the computation over R; differs from that over U in at most one local gradient descent step speciipciiicllly in the subgroup containing the differing datapoint U R, Thus, to compute the noise in each iteration, suffices to determine 2j for j = 1.2,..., ,000, where 2, corresponds to the outcome difference:
2j 20 (0( (,,,)) (( G. (uy)),
(37)
for each uj included in subgroup G
This subgrouping strategy is primarily motivated by implementation efficiency. Under this setup, determining the noise per round requires only 201 executions of 20 local gradient descent steps over 10 samples, compared to 201 executions over 200 samples, significantly reducing computational overhead.
As for the hyperparameter selection, for e = 1. e = 2, E [3: : 4]. and E [5 : 8]. we select the epoch number to be 3, 6, 10 and 15, respectively. We uniformly select the learning rate of the local SGD to be 1.25 10-4.
F22 Memorization
We construct the WikiText-5 dataset by taking the first 5/103 portion of text examples from WikiText- 103 (Merity et al., 2016), resulting a training set of approximately 5.7 million tokens. The samples are then reformatted into 5,640 sequences, each of length 1,024, with each sequence treated as individual sample.
For different models, we determine the optimal number of training epochs based validation perplexity, selecting the bighest-performing epoch before overfitting occurs given the chosen learning rate. Specifically:
For GPT-2 small, we use a learning rate of 3 x 10-4 and train for epoch.
For OPT-125M, we use learning rate of 1.5 x 1 and train for epoch.
For OPT-350M, we use learning rate of 10-5 and train for 5 epochs.
For all DSI-related results presented in Fig. and Tables 2 and 4, employ standard Adam (Kingma, 2014), with a batch size of 20 without local iterations, as our optimizer O within the proposed DSI deep learning framework.
E3. Backdoor Attacks
We consider the following setup: for sources/entities where one of them is malicious, we evenly split the CIFARI0 training dataset, totally 50,000 samples, into m disjoint subsets, each with 50000/m m samples; for the objective poisoning strategy, the malicious entity will transform their assigned samples accordingly, while all other (m 1) users hold the original clean samples assigned. The PreAct-ResNet11 is pre-trained using CIFAR100 data.
The federate learning algorithm F is defined as follows. The entire training procedure consists of T phases, and each phase, captured by O in Algorithm 2, is formed by K local iterations, In each iteration, the central server asks each participated entity to subsample 2000/ m many samples from their local data and accordingly compute and send the gradient computed on current iterate. The server then robustly aggregate the received gradients in three steps:
1. Clipping: Suppose the server receives gradients 91, 92, ... In from each participated entity. They first determine
20

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiliy
==
==
8
:= 4
ACC
ASR
80
Performance (%)
60
40
20
0
10
20
40
80
10
20
40
80
m sources
m sources
Figure 4. Test Accuracy (ACC) and Attack Success Rate(ASR) when training poisoned CIFARIO under Blended attacks (Chen al.,
2017) with/without DSI guarantees (e,8 = 10 10"").
the medium of the 12-norm of the n entity-level gradient, denoted by s. Then, each gradient 9i is clipped to gf +
gi min(1, |g:
2. Cosine-Similarity Projection: For clipped gradients F(mmj) the server computes their mean, denoted by ÿ 1/n Σ of and determine the angle 0g between and J. where cos(f)) 1l Similarly, we compute the medium of [1 denoted by 0. We then further project each clipped s/1.0]] to a cone such that its projected version will not be than 0 cosine-away from g. To be formal,
(o
(g' g) 11g. |
g = arg min lgg Aill. s.t. || = |l arccos(
2
(38)
3. Aggregation and Update: Finally, the server takes the average di.m as the update and move forwards to the next iteration.
The
operations ensure that the gradient collected from each entity cannot be arbitrarily large or pointing out some direction dramatically different from the majority. This simple robust aggregation helpful in applying DSI noise to avoid the trivial attack where the adversary may just send large outliers enforce adding a huge noise to destroy the utility.
In particular, for the prper-parameter selection in our experiments shown in Fig. 3 and 4 we select T = 50 and K = 10 with step size (learning rate) 10-3, As mentioned before, each reference set R; is selected to be the entire training data excluding the samples from i-th entity. Thus, for each phase, to determine the outcome difference 2,, we will implement O across and Rioij with computation overhead (m?))
F44. Comparison between DSI Noise and Isotropic Noise in Defending Backdoor Attacks
In this subsection, we further include the comparison the empirical efficiency between classic isotropic noise and the optimized anisotropic noise in defending backdoor attacks.
Matching Utility Performance: We first adjust the variance of isotropic noise match the test accuracy obtained with the
DSI framework under 4 and 8 with 8 fixed to be 10-5, We then compare adversarial success rate (ASR), showing
'When implement O each R, each local iteration only takes the gradient collected from (m 1) entities; while when implement O on U, each local iteration the gradient collected from m entities.
21

---

Trustworthy Machine Learning through Data-Specific Indistinguishabiliy
that for the same empirical utility, DSI noise on average has better efficiency compared to isotropic noise in defending against backdoor attacks, as shown in Table 5, 6.
Evaluating Provable Indistinguishabilit:: We also report the provable (c, guarantees achievable by isotropic noise in this setup. As expected, when the number of sources m is small, worst-case sensitivity remains high. Even with m 80, for blended attacks, the resulting bound (c = 179, 5 = 10-9) is too weak to provide meaningful guarantees, to match the same performance from DSI framework with (E 4, 5 = 10-
Table Comparison on indistinguishabilty control and defense efficiency Adversarial Success Rate (ASR) against Now-Frequency Attacks (Zeng et al., 2021) between DSI Noise and Isotropic Noise (Iso.) with fixed test accuracy on clean data.
(a). 10 Sources
(b). 20 Sources
Test ACC (%)
Ind. Guarantee
ASR (%)
Test ACC (%)
Ind. Guarantee
ASR (%)
(c 8, 10 5) (DSI)
13.9
8, 6 10-5) (DSI)
8.4
75.6
79.1
(c 2019, = 10 (Iso.)
25.6
(c 1661, 10 5) (Iso.)
16.2
(c = 4, 10 5) (DSI)
2.0
(e = 4, 8 = 10 5) (DSI)
5.3
65.5
70.9
(c 575, 10 (Iso.)
20.2
(c 479,
(Iso.)
11.5
(c). 40 Sources
80 Sources
Test ACC (%)
Ind. Guarantee
ASR (%)
Test ACC (%)
Ind. Guarantee
ASR (%)
(c 8, 10 (DSI)
6.4
(c 8, 5 10 50 (DSI)
8.9
78.4
79.5
(e 627, 10 (Iso.)
7.3
(e 350, 6 = 10 (Iso.)
8.2
(c (c= 4,6 10 10-5)
(DSI)
6.6
(c = 4, 5- 10 (DSI)
6.1
73.6
74.1
(e 312,
10 10-5) 5) (Iso.)
7.2
(e = 114, 6 = 10 (Iso.)
7.7
Table Comparison on indistinguisaabiliy control and defense efficiency in Adversarial Success Rate (ASR) against Blended Attacks (Chen al., 2017) between DSI Noise and Isotropic Noise with fixed test accuracy on clean data.
(a). 10 Sources
(b). 20 Sources
Test ACC (%)
Ind. Guarantee
ASR (%)
Test ACC (%)
Ind. Guarantee
ASR (%)
(c 8, 10-5) (DSI)
15.7
(c 8, 8 10-5) (DSI)
7.7
73.9
77.9
(e 2019, 5 = 10 10-3) (Iso.)
9.9
(e 1661, 10 5) (Iso.)
8.8
(c = 4, 10-5) (DSI)
0.1
(e = 4, 6 = 10-5) (DSI)
1.1
57.5
67.1
(e = 350, 10 (Iso.)
9.8
(c 241, = (Iso.)
2.4
(c). m 40 Sources
(d). m Sourees
Test ACC (%)
Ind. Guarantee
ASR (%)
Test ACC (%)
Ind. Guarantee
ASR (%)
(E = 8, 10-5) (DSI)
7.1
(e 8, 8 10-5) (DSI)
3.4
78.8
79.3
(e 627, 10 5) (Iso.)
4.1
(e 350, 8 10 (Iso.)
3.9
(e ===00 4, = 10-5) (DSI)
12.3
(e = 4, = 10 10-5) (DSI)
4.7
73.5
73.7
(c = 312, 10 (Iso.)
5.1
(c = 179, 5 = 10
(Iso.)
3.6
22


Reviews:
[{"review_id": "3AvYxc05At", "reviewer": "Reviewer_53pn", "summary": "The idea of DSI is equivalent to per-instance privacy (see \"Y.-X. Wang. Per-instance differential privacy. Journal of Privacy and \nConfidentiality,\" or the equivalent individual privacy of \"V. Feldman and T. Zrnic. Individual privacy accounting via a renyi filter. Advances  in Neural Information Processing Systems\") when the reference is between two datasets and removing a datapoint. The case of two arbitrary datasets was also explored in \"Thudi, Anvith, et al. \"Gradients look alike: Sensitivity is often overestimated in {DP-SGD}\". No mention of \nthese existing and related works is mentioned in the paper, and differentiating to them is recommended.\n\nOn the technical aspects, unlike the past work the paper notes when just focusing on the Gaussian mechanism (and not the sampled Gaussian mechanism which could complicate the analysis and was analyzed in the aforementioned prior work) one can optimize the returned noise vector given the subspace of differences. This seems like the main novel \ncontribution.On the composition theorem, the method may be improved with the expectation based composition theorem of \"Gradients look alike: Sensitivity is often overestimated in {DP-SGD}\" which would seem to imply it's enough for each step to satisfy a $\\gamma$ constraint.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments and suggestions!\n\n**1. Comparison with (Per-Sample/Individual) Differential Privacy (DP)**\n\nWe believe there are at least three fundamental differences between Differential Trustworthiness (DT) and Differential Privacy (DP).  \n\n1-1) First, on **motivation**, though both DT and DP leverage distinguishability or statistical divergence, they target very different problems. Distinguishability measures the closeness of two distributions, which can be interpreted in two ways: \na) It is hard to distinguish the source of a randomly drawn sample; b) The behavior of two random variables, $a \\sim A$ and $b \\sim B$, is similar in probability: $\\Pr(a \\in O) \\approx \\Pr(b \\in O)$ for any set $O$.\n\nPrivacy (confidential protection) typically adopts a) to model preventing adversaries from recovering secrets from leakage. If the leakages from different secrets are indistinguishable, no useful information is revealed to an adversary.\n\nIn contrast, DT adopts b), using divergence between a target and a set of safe references to characterize the probability that an algorithm’s output satisfies certain trust properties. Imagine, for a set of reference training datasets $R_1, R_2, \\dots, R_m$, if each trained model $\\mathcal{F}(R_i)$ has a 99\\% probability of not memorizing sensitive information $q_i$, then the divergence between $\\mathcal{F}(U)$ and each $\\mathcal{F}(R_i)$ provides a bound on the probability that $\\mathcal{F}(U)$ does not memorize all $q_i$ **simultaneously**.\n\n1-2) **Conceptually**, DT justifies the role of data dependence in trust-preserving mechanisms, whereas this **remains a challenge in privacy**. If a specific sensitive data point $x_0$ becomes a parameter in a privacy guarantee (e.g., achieving $(\\epsilon, \\delta)$-DP for the membership of $x_0$), this statement itself already leaks information about $x_0$. Section 1.2 discusses why input independence is essential for privacy and why most DP mechanisms rely on worst-case sensitivity.\n\nSince DP mechanisms calibrate noise based on the worst case, regular (average-case) data points may enjoy stronger privacy guarantees—motivating prior work on per-sample [1,2] or individual privacy [3]. However, formally quantifying this average-case amplification from the provable, global security parameters remains an **open** question. Existing works [1,2,3] can only estimate per-sample/individual privacy loss, which itself is sensitive and cannot be disclosed.\n\nA key insight from DT is that many trust concerns focus on the use (e.g. copyright) or influence (e.g. backdoor) of public data to train a model, or governing models (e.g. memorization) to prevent certain behaviors, and it suffices to ensure indistinguishability with respect to safe reference models. More importantly, unlike privacy-preserving operations, when leakage is not a primary concern, input-independence is **not** necessary, and thus we propose the more efficient DSI.\n\nAnother distinction is that DT only requires one-way divergence—specifically, the $f$-divergence from the target output $\\mathcal{F}(U)$ to the reference outputs $\\mathcal{F}(R_i)$. In contrast, DP treats two adjacent datasets $X$ and $X'$ symmetrically, requiring two-way divergence to prevent leakage. See Section 3.2(i) for further details.\n\n1-3). **Operationally**, we demonstrate how to determine the optimal anisotropic noise to achieve the required DSI guarantees. However, isotropic noise in prior works [1,2,3] cannot provide **controllable** per-sample/individual privacy loss. Moreover, our composition theorem (Theorem 3.4) is tight, whereas the composition approach in [2] involves complex expectation-based estimates, and [3] still relies on worst-case individual guarantees.\n\nWe will incorporate those comparisons in our revision. \n\n**2. Comparison with State-of-the-art Results**\n\nPlease refer to Table 1. We **have** compared with the DP-SGD benchmarks (De et al., 2022, https://arxiv.org/pdf/2204.13650) you suggested. By optimizing DSI noise, we achieve **7-10\\%** improvement in test accuracy in all cases with varying indistinguishability budget. \n\n**3. Subsampling Amplification** \n\nWe appreciate your insightful comments and acknowledge that our current results do not exploit subsampling-based randomness, which could further enhance the trust-utility tradeoff—particularly in applications like SGD. We highlight this as a promising future direction in Section 6. But it is also noteworthy that even **without** subsampling amplification, DSI mechanisms already significantly outperform best-known DP methods that has incorporated subsampling (Table 1).\n\nFinally, we highly appreciate if you could let us know whether we addressed your concerns or you have additional questions. \n\n[1] Y.-X. Wang. Per-instance differential privacy.\n \n[2] V. Feldman and T. Zrnic, Individual privacy accounting via a Renyi filter.\n\n[3] Thudi, Anvith, et al. \"Gradients look alike: Sensitivity is often overestimated in DP-SGD.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes, it maybe better to use more recent baselines, e.g. from https://arxiv.org/pdf/2204.13650", "relation_to_broader_scientific_literature": "Some related work is not mentioned, see above\n\nAfter rebuttal:\n\nI think prior work captured some of what is discussed in the paper. Conceptually, it was already discussed that input dependence is ok for per-instance privacy for certain applications (e.g. see unlearning and memorization implications from Thudi et al.). Operationally, I understand the novelty and the setting -- optimizing noise for the specific datasets. I feel connection to prior work and equivalences need to be further explored.", "essential_references_not_discussed": "See above", "other_strengths_and_weaknesses": "-", "other_comments_or_suggestions": "-", "questions_for_authors": "Please compare to the papers mentioned above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OfPewoF53w", "reviewer": "Reviewer_pwit", "summary": "This paper proposes a unified framework for Differential Trustworthiness (DT), which models trust-related concerns in machine learning algorithms, such as memorization, data poisoning, and copyright issues. The framework aims to regulate the divergence between the outputs of a model trained on a target dataset and those trained on reference datasets, ensuring trustworthy model behavior. Specifically, it introduces the concept of Data-Specific Indistinguishability (DSI) and its implementation through a Gaussian noise mechanism to mitigate differential effects and protect sensitive information. The paper also provides a thorough exploration of the theoretical properties, algorithmic solutions, and practical applications of DSI, particularly in areas like large language models and computer vision, addressing challenges such as privacy, model robustness, and utility trade-offs.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive assessment and insightful questions.\n\n**1. Comparison to other Noise-based Solutions** \n\nWe appreciate your references to prior works that apply noise for trustworthy machine learning. Existing noise-based solutions typically face two key challenges: **a) High utility overhead** and **b) Lack of provable guarantees**.\n\nAs a representative of (a) and the primary baseline in our paper, Differential Privacy (DP) mechanisms can be used to ensure provable indistinguishability, which can further imply other trust guarantees such as memorization prevention and unlearning. However, ensuring a meaningful indistinguishability bound (i.e., small security parameters $\\epsilon, \\delta$) usually requires adding prohibitively-large isotropic DP noise, which is well-known to impose a significant utility cost.\n\nOn the other hand, a long line of empirical work explores noise-based defenses **without** formal guarantees. For example, [1] proposes adding noise to training data as a general defense against unknown textual backdoor attacks; [2] introduces random smoothing, which augments training samples with noisy versions to improve adversarial robustness of trained models; [3] adds small noise to gradients during training to resist membership inference attacks. While these approaches demonstrate strong empirical success, they lack theoretical bounds on how much noise is needed to provably prevent attacks or ensure trust guarantees.\n\nOur Data-Specific Indistinguishability (DSI) noise framework overcomes these limitations by determining the **minimal necessary** noise to ensure **provable** trustworthy guarantees while maintaining low utility overhead. The key idea can be summarized below. \n\n- First, given an algorithm $\\mathcal{F}$, we identify a safe reference set $R$ whose output $\\mathcal{F}(R)$ is highly likely to satisfy the desired trust properties. For example, if $\\mathcal{F}$ is a standard deep learning algorithm and $R$ is a clean dataset, then the trained model $\\mathcal{F}(R)$ is likely robust to backdoor attacks.\n\n- We then control distinguishability by adding minimal noise to reduce the statistical divergence between the target model and the reference model. This provably ensures that the target model has a high probability of satisfying the desired trustworthiness properties (see Lemma 2.3 and Section 3.2(i)). More importantly, unlike DP, which adds noise isotropically, DSI selectively adds noises only in necessary directions to minimize utility overhead. \n\nDSI also enjoys operational benefits, which enables black-box algorithm analysis and the noise only needs to be added to the final output of a data processing procedure. \n\n**2. Additional Experiments on Copyright/Contribution** \n\nWe have added a set of experiments for the copyright or contribution control using our DSI framework. Please refer to **Experiment 1 in attachment (https://anonymous.4open.science/r/4722-1FD5).** In this experiment, we finetune a stable diffusion model (v1-4 from https://huggingface.co/docs/diffusers/v0.8.0/en/training/text2image) on a collection $U$ of 425 paintings from 10 artists. We select the reference sets $R_i$ as $U$ excluding one painting, i.e., leaving-one subsets. Thus, the $(\\epsilon, \\delta)$ DSI parameters characterize the **per-sample contribution** to the trained model.\n\nIn the figure, we present: (1) the original painting, (2) the generated image from the pre-trained diffusion model **before** fine-tuning, (3) the generated image **after fine-tuning without noise**, and (4) the generated images **after fine-tuning with DSI noise** in different indistinguishability budgets ($\\epsilon$). From the plot, it is easy to observe that with a larger $\\epsilon$, the fine-tuned diffusion learns the art style better and the generated pictures look more similar to the training data.\n\nIn practice, the DSI parameters can be used to quantify and determine the contribution of each sample or data source, which provides economic solutions to data usage in collaborative learning.\n\nFinally, we highly appreciate if you could let us know whether we addressed your concerns or you have additional questions.\n\n[1] Zhai, Shengfang, et al. \"NCL: Textual Backdoor Defense Using Noise-Augmented Contrastive Learning.\"\n\n[2] Cohen, Jeremy, Elan Rosenfeld, and Zico Kolter. \"Certified Adversarial Robustness via Randomized Smoothing.\"\n\n[3] Aerni, Michael, Jie Zhang, and Florian Tramèr. \"Evaluations of Machine Learning Privacy Defenses Are Misleading.\"", "claims_and_evidence": "1.The writing of the abstract is somewhat difficult to understand and should be revised and expanded according to the logic of the introduction.\n\n2.The proposed method shows significant theoretical contributions, but questions arise regarding its practical applications (e.g., defending against backdoor attacks). How does it differ from existing noise-based methods, such as those proposed in [1] and other works?\n\n3.Experiments conducted on large language models are convincing. However, why are copyright-related experiments not included?\n\n[1] NCL: Textual Backdoor Defense Using Noise-augmented Contrastive Learning, ICASSP 2023", "methods_and_evaluation_criteria": "The DSI framework is novel, and its application to memorization, data poisoning, and copyright concerns is meaningful.", "theoretical_claims": "The theory appears to be sound.", "experimental_designs_or_analyses": "See Claim and Evidence point 3.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "Perhaps works related to memorization, data poisoning, and copyright issues concerning noise should be cited.", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jVDsOM3yZo", "reviewer": "Reviewer_cmFq", "summary": "The paper aims at combining various privacy preserving mechanism for machine learning into a unified framework, for instance reducing memorization, providing copyright protection, differential privacy and so on. The main mathematical technique is to use Data Specific Indistinguishability which aims at providing privacy protection to be dependent on the data (training data) and its restrictions instead of the worst case bounds in DP. The propose an algorithm to achieve this and discuss its properties like composition, post processing etc.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive assessment and helpful suggestions.\n\n**1. Relationship Between Privacy and Differential Trustworthiness (DT)**\n\nWe fully agree with your insightful comment that (differential) privacy is recognized as a stronger guarantee, which, as a **sufficient** condition, can produce many other (differential) trust guarantees (e.g., reducing memorization and protecting copyright). However, privacy is costly (we also acknowledge your point that \"privacy\" is a broad concept; here, we specifically use \"privacy\" to refer to confidential protection). This motivates us to explore more efficient solutions to address “weaker” trust guarantees—those aimed at controlling the influence of training data on models without necessarily preventing data leakage.\n\nStrictly speaking, privacy is **not** directly comparable to the DT (memorization, backdoors, copyright). One key insight we highlight (see Sections 1.1 and 1.2) is that privacy based on **Input-Independent Indistinguishability (III)** form a sufficient **but not necessary** solution: when the goal is not to prevent information leakage, input independence is **unnecessary** and technically only indistinguishability guarantee is needed to build provable trust. Thus, we propose and justify a relaxed version, Data-Specific Indistinguishability (DSI), and construct the optimal DSI noise mechanism. Due to space constraints, please refer to our Response 1 to Reviewer 53pn for more details on why input independence is necessary for privacy, how DSI compares to per-sample/individual Differential Privacy (DP).\n\n**2. Comparison to Distinguishability Control Tools rather than DP Definition**\n\nOur key motivation and contribution are **not** to improve or relax existing DP frameworks but rather to develop better methods for constructing indistinguishability for DT guarantees. As demonstrated above, DSI is strictly weaker than III and is also incomparable to DP guarantees. What our experiments actually target is the **efficiency of different mechanisms in achieving indistinguishability**.\n\nAs the baseline, the classic method to control distinguishability is through **clipping (sensitivity control) combined with isotropic noise**, as widely explored in DP literature. DP-SGD is a representative. To compare, we propose a new **anisotropic** noise mechanism to determine the minimal noise necessary for specific indistinguishability. Our framework also enables black-box algorithm analysis without requiring sensitivity control, thereby eliminating clipping bias. Furthermore, we establish accounting methods, such as composition (Theorem 3.4) and grouping (Lemma 3.5), to more tightly convert indistinguishability guarantees into probabilistic DT guarantees.\n\nIn summary, our baseline comparison focuses on the efficiency of existing DP methods in achieving the required indistinguishability or statistical divergence bounds **rather than their privacy guarantees themselves**. To clarify, we will revise claims such as \"comparison with DP-SGD\" to \"comparison with clipping + isotropic noise methods in DP-SGD.\"\n\n**3. Comparison to other Works** \n\n- PAC Privacy [1]:  PAC Privacy leverages secret entropy and models privacy risk by an adversary’s posterior success rate in recovering the secret. Different from examining the correlation between secrets and leakage from a privacy perspective, DT and DSI do **not** assume or rely on any input distribution. Instead, we optimize noise based on reference safe models.\n\n- Data-Dependent DP: Sticking to classical DP definitions, [2] utilizes public knowledge of graph model structures to optimize privacy budget allocation across multiple releases. However, the noise mechanism for each release remains the standard Laplace mechanism.\n\n- Sub-space and Public-private DP:  Assisted by public data, existing works have considered optimizing clipping strategies, subspace projection, or data augmentation. Still, all existing approaches apply standard **isotropic** noise. **In Exp2 ( https://anonymous.4open.science/r/4722-1FD5),  we include a more detailed comparison** and show **even without** assuming public data, DSI-SGD has outperformed all prior benchmarks training on CIFAR10 from scratch. In addition, public data tricks can also save DSI noise, e.g. mixing training data with public samples [3] can reduce the divergence between the target and reference models. \n\n**4. Limitations** \n\nWe will expand the discussion on limitations. As you noted, when both provable privacy and DT guarantees are required simultaneously, the weaker DSI is not applicable and III remains the only known method.\n\nFinally, please do not hesitate to let us know if we have addressed your concerns or you have additional questions. \n\n[1] PAC Privacy: Automatic Privacy Measurement and Control of Data Processing\n\n[2] Data-Dependent Differentially Private Parameter Learning for Directed Graphical Model\n\n[3] DP-Mix: Mixup-based Data Augmentation for Differentially Private Learning", "claims_and_evidence": "The main claim is providing a DSI procedure which can encompass various privacy preserving mechanisms into one definition and making it data dependent more specifically.\n\nSection 3 discusses the DSI gaussian mechanism, which aims to add gaussian noise as a way of providing privacy, and providing data dependent results i.e. removing the data independence assumption used in DP\n\nIn Section 3.2 the paper discusses various properties, and shows how the proposed method satisfies them.\n\nIn Algorithm 2, the paper proposes a framework for optimizing deep learning model with black box optimizers by modifying the gradients in similar spirit to DP-SGD", "methods_and_evaluation_criteria": "The proposed method is relevant and interesting, however, for the experiments / baselines, it appears that the paper does not compare against the right baselines. For instance, there have been several papers on data dependent DP, which exploit subspaces to improve the utility of the underlying privacy mechanism. However, the only experiment / baseline was DP-SGD\n\nTheoretically, it seems to be a relaxation of DP-SGD where the gradient clipping is by-passed as part of the definition, and the addition of noise is data dependent.\n\nIn the conclusion the authors mention that their approach is significantly better than DP, however, they both are aimed at doing different things. And to make a fair comparison it necessary to compare to the right papers. \n\nSome relevant baselines:\nhttps://arxiv.org/pdf/2311.14632\nhttps://arxiv.org/pdf/2210.00036\nhttps://proceedings.mlr.press/v202/bu23a/bu23a.pdf\nhttps://arxiv.org/abs/2212.00328\nhttps://arxiv.org/pdf/2203.11481", "theoretical_claims": "The definitions and theorems seem to be sound.", "experimental_designs_or_analyses": "I think the baselines used are not up to the mark, and its not the best to compare against DP-SGD. There are works on subspace identification in DP, where we can minimize the damage caused by clipping and gaussian noise. \n\nThe major concern I have is that the related works have not been correctly used to build upon. There have been plenty of works for improving DP.\n\nHere are few examples:\nhttps://arxiv.org/pdf/2007.03813\nhttps://cdn.aaai.org/ojs/20315/20315-13-24328-1-2-20220628.pdf\nhttps://arxiv.org/abs/1707.07708\nhttps://arxiv.org/pdf/2203.11481", "relation_to_broader_scientific_literature": "The paper aims provide a framework for differential trustworthiness, and provide a data dependent definition for it. It is well know in the privacy community that DP implies reduced memorization, copyright protection (using NAF), unlearning, and robustness to membership attacks. However, its difficult to make DP work in practice at scale, as the amount of noise scales rapidly with size, and thus the models trained with DP have reduced utility. \n\nThe paper aims to relax this and provide a simple procedure where we add gaussian noise which is tailored to the training data. While the aim and direction of the paper is interesting, I think there are a lot of closely related works which do something similar.\n\nAlso, the aim is at providing differential trustworthiness, however, its eventually compared against DP-SGD", "essential_references_not_discussed": "Some essential references which are missing:\n\nPer-instance DP:\n- https://arxiv.org/abs/1707.07708\n- https://arxiv.org/abs/2111.02281\n- https://openreview.net/pdf?id=ESt7ECoWpn\n\nSubspace based DP:\n- https://arxiv.org/abs/2108.11527\n- https://www.math.uci.edu/~rvershyn/papers/hsvz-subspaceprivacy.pdf\n\nPublic-Private DP:\n- https://arxiv.org/pdf/2306.15056\n- https://arxiv.org/pdf/2203.11481\n- https://proceedings.mlr.press/v202/nasr23a/nasr23a.pdf\n\nPAC Privacy:\n- https://arxiv.org/abs/2210.03458\n- https://arxiv.org/abs/2312.01201\n\nData dependent DP\n- https://arxiv.org/pdf/1905.12813\n- https://papers.nips.cc/paper_files/paper/2018/hash/9a0ee0a9e7a42d2d69b8f86b3a0756b1-Abstract.html", "other_strengths_and_weaknesses": "Strengths:\n- The paper is well written, and the theory is easy to parse and grounded.\n- The problem is very interesting to work on, as it corresponds to reducing multiple sub-problems in privacy preserving machine learning to a single optimization problem\n- Using data dependent anisotropic gaussian noise for privacy preservation is intuitive, and in general having data dependent privacy is the way for the future.\n- The proposed approach seems to be more relevant for unlearning, obfuscation than privacy in general whose definition can vary a lot depending on the application.\n\nWeaknesses:\n- The proposed methods are similar to the existing methods in DP literature, so the novelty of the proposed appears to be low.\n- The baselines considered are weak, and thus it would be useful if it can be improved.", "other_comments_or_suggestions": "I will recommend the authors to spin the paper as an unlearning / obfuscation paper rather than differential trustworthiness because privacy in general is too broad, and there is a lot of work in relaxing DP which makes the novelty of the proposed approach low. \n\nAlso, there is no limitations section in the paper. One trivial limitation I can see is that the proposed method has no worse case bounds which can be of utmost importance in privacy. I will encourage the authors to add a section the limitations of the proposed method.", "questions_for_authors": "I think if the authors provide a detailed comparison against methods like per-instance DP, DP in public-private setting, PAC-privacy that will be great. The current method seems to be very correlated with existing methods, and thus its essential to make it clear what is the major contribution which is different from the past works.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lP0PnfEAzx", "reviewer": "Reviewer_9rXN", "summary": "This paper proposes a concept of (gaussian) data-specific indistinguishability (DSI), which relaxes Input-Independent Indistinguishability (or differential privacy in many sense) by enforcing constraints only for a set of pre-defined input pairs instead of globally.\n\nSimilar to what we already have for differential privacy, they derive the corresponding Gaussian mechanism, composition rules and group versions for DSI, which consequently allows using Gaussian noise in parameter optimizations (SGD) to ensure DSI of the entire scheme (just like DP-SGD for differential privacy in many ways). They denote this as DSI deep learning.\n\nExperimentally, they show example application of DSI deep learning in reducing memorization of fine-tuned language models and defending backdoor attacks in federated learning.\n\n---\n\n### update after rebuttal\nWhile DSI is an interesting notion different from existing ones, I am not yet convinced if/how DSI is more preferable in applications suggested by the authors. I can see how DSI can lead to better provable (robustness) bounds than differential privacy, but I think the bounds are still not meaningful practically in most cases as $e^\\epsilon$ can be large even for single-digit $\\epsilon$ and there are established methods with better provable robustness (e.g. DPA or bagging or Finite Aggregation for provable backdoor/poisoning defenses).\n\nI am ok with it being accepted but will not champion for it based on current materials provided, which still aligns with my initial recommendation of \"3: Weak accept (i.e., leaning towards accept, but could also be rejected)\".", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive assessment and valuable suggestions.\n\n**1. Differential Trustworthiness (DT) vs. Differential Privacy (DP)** \n\nWe fully agree with your insightful comment: “Data-Specific Indistinguishability (DSI) and Differential Privacy (DP) (or Input-Independent Indistinguishability (III)) **even under the same** parameters $(\\epsilon, \\delta)$ do **not** imply the same level of trust or privacy.” In Section 1.2 and our Response 1 to Reviewer 53pn, we explain why III is necessary for privacy—instance-based privacy statements or noise mechanisms themselves may already leak information.\n\nThe goal of this paper is **not** to relax existing privacy frameworks (e.g., DP) or improve privacy solutions. Instead, we focus on a broad class of trust concepts that are **not** primarily concerned with preventing information leakage but rather with **controlling data usage** (e.g., copyright protection), **mitigating influence** (e.g., backdoor defense), and **governing model behavior** (e.g., memorization mitigation).\n\nA key insight from our Differential Trustworthiness (DT) framework is that if we can bound the distinguishability between a target model and a set of safe reference models—such as a model trained on clean data (for backdoor defense) or a dataset excluding memorization-sensitive information—we can obtain a probabilistic guarantee that the target model meets objective trustworthiness requirements.\n\nMore importantly, when information leakage is not a concern, input independence is **not** necessary to derive DT guarantees. This explains why DP (which enforces III) is a **sufficient but not necessary** condition for DT. Our framework establishes an optimal noise mechanism to achieve DSI, significantly more efficient than **existing distinguishability control** methods in DP. But, in Section 1.2 including Footnote 2, we clarify DSI noise mechanisms **cannot** be used to address privacy concerns or provide DP-like guarantees, and we will emphasize this further in our revision.\n\n**2. Efficiency of Indistinguishability Control**\n\nThe goal of our experiments is indeed to compare the **efficiency of achieving indistinguishability** using DP's standard methods—**clipping (sensitivity control) and isotropic noise**—versus our DSI framework for black-box data processing. By targeting the same divergence bound (measured in Hockey-Stick divergence with $(\\epsilon, \\delta)$ parameters), we demonstrate the significant utility improvement enabled by our optimized noise. We will clarify and emphasize that we are **not** comparing the privacy guarantees ensured by DP-SGD in the revision.\n\n**3. Baseline and Additional Experiments Comparing Isotropic and DSI Noise**\n\nWe apologize for any misunderstanding regarding our notation. In all experiments, we include a baseline (see the column with $\\epsilon = \\infty$), which represents the original case **without** any perturbation.\n\nAdditionally, we provide **new experiments in Exp 3 in the attachment (https://anonymous.4open.science/r/4722-1FD5)**. Specifically, in a backdoor defense scenario (see Appendix F.3) where training data is collected from $m$ sources (one of which is malicious), we compare the trust-utility tradeoff achieved via DP-SGD (gradient clipping + isotropic noise) versus the DSI framework. \n\n- 3-1) Matching Utility Performance: For $m$ varying from $10$ to $80$, we first adjust the variance of isotropic noise to match the test accuracy obtained with the DSI framework under $(\\epsilon=4, \\delta=10^{-5})$ and $(\\epsilon=8, \\delta=10^{-5})$, respectively.  We then compare the adversarial success rate (ASR), showing to produce the **same empirical test accuracy**, DSI outperforms isotropic noise (leading to lower ASR rates) in defending against backdoor attacks in most cases.\n\n- 3-2) Evaluating Provable Indistinguishability: We also report the provable $(\\epsilon, \\delta)$ guarantees achievable by isotropic noise in this setup. As expected, given that the number of data sources $m$ is relatively small, the worst-case sensitivity $O(1/m)$ remains high. Even with $m=80$, the resulting bounds **$(\\epsilon=114, \\delta = 10^{-5})$** for low-frequency attacks [1] and **$(\\epsilon=179, \\delta = 10^{-5})$** for Blended attacks [2] are too weak to provide meaningful guarantees, to match the same performance from DSI framework with  $(\\epsilon=4, \\delta=10^{-5})$.\n \nThese results will be included in our revision, further supporting our claim that standard mechanisms in DP-SGD cannot produce usable divergence bounds in our experimental settings (especially for small dataset and scaling/high-dimensional models).\n\nFinally, we would greatly appreciate your feedback on whether we have adequately addressed your concerns. Please let us know if you have any additional questions.\n\n[1]. Zeng, Yi, et al. \"Rethinking the backdoor attacks' triggers: A frequency perspective.\" \n\n[2]. Chen, Xinyun, et al. \"Targeted backdoor attacks on deep learning systems using data poisoning.\"", "claims_and_evidence": "Most claims are well supported, except:\n1. In section 4 and in conclusion, it is claimed that \"DSI local-SGD outperforms DP-SGD in all cases\", \"our initial results demonstrate a significant improvement in the utility-trust trade-off compared to traditional, such as DP-based, methods\", which is not supported by existing evidence. While in table 1 it is showed that DSI-Local-SGD can offer higher accuracy than DP-SGD **assuming the same $\\epsilon$ and $\\delta$**, using the same $\\epsilon$ and $\\delta$ for differential privacy and DSI does not indicate the same level of trust/privacy/indistinguishability, rendering the arguments unsupported. Naturally, this issue also affect many other claims made in section 4 involving advantage of DSI (DSI-local SGD) over DP (DP-SGD).\n2. Section 5 experiments: \"for all the following experiments with (ε, δ) parameters considered DP-SGD always requires prohibitively large noise which fully destroys the performance.\" This is not well justified, due to the very same issue as the one above.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "While I did not check the proofs line by line, I skimmed through the main theoretical results and they made sense.", "experimental_designs_or_analyses": "The experimental designs for (reducing) memorization in LLM and (defending) backdoor attacks in federated learning are ok, with a flaw that no baseline is compared to, including but not limited to DP-SGD.\n\nThe primary issue in existing experiments across different parts is that the submission assumes, without good justifications, that different privacy and the proposed DPI should be compared by simply assuming the same epsilon and delta.", "relation_to_broader_scientific_literature": "I can see the proposed DSI framework is very closely related to differential privacy, with many tools/results most likely adapted or at least inspired from existing, well-established results for differentially private deep learning research.\n\nNotably, the suggested applications of DSI, i.e. reducing memorization and mitigating backdoor attacks, also overlap with applications previous suggested for differential privacy.", "essential_references_not_discussed": "Not that I am aware of.", "other_strengths_and_weaknesses": "[strength]\nThe idea of relaxing the global requirement for Input-Independent Indistinguishability/differential privacy is very natural, and the authors put in great efforts in providing a fairly comprehensive set of theoretical tools/results, which is impressive.\n\n[weakness]\nThe primary weakness is the limited investigation (regardless of theoretical/empirical) regarding the degree of trust/privacy/robustness provided by the proposed DSI notions, especially in comparison with differential privacy. Basically there is no clear indications regarding whether/why DSI is a more useful notion compared to differential privacy, for example.", "other_comments_or_suggestions": "minor:\nThe running title of the submission is not updated.", "questions_for_authors": "The primary question/concern I have is simply: What supports are there indicating DSI is promising/a better tool than e.g. differential privacy?\n\nAs there is no results aligning/comparing the effectiveness (i.e. the degree of trust/privacy/protection they provide) of DSI & differential privacy, this remains unclear and renders the assessment quite tricky.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work introduces Data‑Specific Indistinguishability, a framework for “differential trustworthiness” that relaxes the worst‑case guarantees of classical Differential Privacy (DP) into data‑dependent indistinguishability bounds. There are a number of key contributions of this work: (1) it formalizes DSI, derives an optimal anisotropic Gaussian mechanism, and proves composition and grouping theorems tailored to one‑way divergence against a set of “safe” reference models. (2) it shows how to integrate DSI noise into black‑box deep learning training without clipping bias, optimizing noise directions to tightly control trust‑related risks.(3) the paper demonstrates DSI’s efficiency/utility trade‑off on (a) memorization mitigation in LLM fine‑tuning, (b) backdoor defense in federated learning, and (c) copyright/contribution control in diffusion models, outperforming standard DP‑SGD at matching indistinguishability budgets. Overall the reviewers find the contributions interesting and solid, therefore I decide to accept the paper, but I would also encourage the authors to fully address the reviewers' additional requests.
Decision: Accept (poster)