# Privacy Auditing in Differential Private Machine Learning:
## 4 Privacy Auditing Schemes
### Privacy Audits
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Privacy Auditing Scheme** & **Privacy Attack** & **Auditing Methodology** \\ \hline \multirow{8}{*}{Membership inference auditing} & White-box membership inference auditing & Auditors analyze gradients, hidden layers, intermediate activations measuring how training data influences model behavior. \\ \cline{2-3}  & Black-box membership inference auditing & Auditors observe probability distributions and confidence scores by analyzing these outputs and assessing the likelihood that certain samples were part of the training data. \\ \cline{2-3}  & Shadow model membership auditing & Auditors use “shadow models” to mimic the behavior of the target model. \\ \cline{2-3}  & Label-only membership inference auditing & Auditor evaluates the privacy guarantee leveraging only output labels, training shadow models, generating a sparsity classifier, and quantifying two-step testing are and accuracy. \\ \cline{2-3}  & Single-training membership inference run auditing & Auditor leverage the ability to add or remove multiple training examples independently during the run. \\ \cline{2-3}  & Single-training membership inference run auditing & This approach focuses on estimating the lower bounds of the privacy parameters without the need for extensive training of the models. \\ \cline{2-3}  & Metric-based membership inference auditing & Auditor assesses privacy guarantees directly evaluating metrics and statistics derived from the model’s outputs (precision, recall, and \(\pi\)-score) on data points. \\ \cline{2-3}  & Data augmentation-based auditing & Auditor generates or augmented data samples similar to training, testing whether these samples reveal membership risk. \\ \cline{2-3}  & Influence-function analysis & Evaluate privacy by introduction malicious data. \\ \cline{2-3}  & Gradient manipulation in DP training & Auditor alters the training data using back-gradient optimization, gradient ascent poisoning, etc. \\ \cline{2-3}  & Empirical evaluation of privacy loss & Auditor conducts quantitative analyses of how the privacy budgets is affected. \\ \cline{2-3}  & Simulation of worst-case poisoning scenarios & Auditor constructs approximate upper bounds on the privacy loss. \\ \cline{2-3}  & Sensitivity analysis & Auditor quantifies how much private information is embedded in the model outputs. \\ \cline{2-3}  & Gradient and weight analyses & Auditor attempts to recruit input features or private data pairs from model outputs using gradient-based or optimization methods. \\ \cline{2-3}  & Empirical privacy loss & Auditor calculates the difference between theoretical and empirical privacy losses. \\ \cline{2-3}  & Embedding and reconstruction test & Auditor examines whether latent representations or embeddings could be reversed to obtain private data. \\ \cline{2-3}  & Model extraction auditing & Query analysis & Auditors simulate extraction attacks by extremely querying the model and analyzing how well they can replicate its outputs or decision boundaries. \\ \hline \multirow{2}{*}{**Property inference auditing**} & \multirow{2}{*}{Evaluating property sensitivity with model outputs.} & The auditor performs a test to infer whether certain properties can be derived from the model and whether the privacy parameters are sufficient to obscure such data properties. \\ \cline{2-3}  & & & \\ \end{tabular}
\end{table} 
Table 2: A list of privacy auditing schemes.