Package: anonymeter
Anonymeter: Unified Framework for Quantifying Privacy Risk in Synthetic DataAnonymeteris a unified statistical framework to jointly quantify different
types of privacy risks in synthetic tabular datasets.Anonymeteris equipped
with attack-based evaluations for theSingling Out,Linkability, andInferencerisks, which are the three key indicators of factual anonymization
according to theArticle 29 Working Party.Anonymeter has been positively reviewed by the technical experts from theCommission Nationale de l’Informatique et des Libertés (CNIL)which, in their words,“have not identified any reason suggesting that the proposed set of methods could not allow to effectively evaluate the extent to which the aforementioned three criteria are fulfilled or not in the context of production and use of synthetic datasets”. The CNIL also expressed the opinion that the results of Anonymeter (i.e. the three risk scores)should be used by the data controller to decide whether the residual risks of re-identification are acceptable or not, and whether the dataset could be considered anonymous.Hereyou can find the full letter with the CNIL opinion on Anonymeter.Anonymeterin a nutshelInAnonymetereach privacy risk is derived from a privacy attacker whose task is to use the synthetic dataset
to come up with a set ofguessesof the form:"there is only one person with attributes X, Y, and Z" (singling out)"records A and B belong to the same person" (linkability)"a person with attributes X and Y also have Z" (inference)Each evaluation consists of running three different attacks:the "main" privacy attack, in which the attacker uses the synthetic data to guess information on records in the original data.the "control" privacy attack, in which the attacker uses the synthetic data to guess information on records in the control dataset.the "baseline" attack, which models a naive attacker who ignores the synthetic data and guess randomly.Checking how many of these guesses are correct, the success rates of the different attacks are measured and used to
derive an estimate of the privacy risk. In particular, the "control attack" is used to separate what the attacker
learns from theutilityof the synthetic data, and what is instead indication of privacy leaks.
The "baseline attack" instead functions as a sanity check. The "main attack" attack should outperform random
guessing in order for the results to be trusted.For more details, a throughout
description of the framework and the attack algorithms can be found in the paperA Unified Framework for Quantifying Privacy Risk in Synthetic Data, accepted at the 23rd Privacy Enhancing Technologies Symposium (PETS 2023).Setup and installationAnonymeterrequires Python 3.8.x, 3.9.x or 3.10.x installed. The simplest way to installAnonymeteris fromPyPi. Simply runpip install anonymeterand you are good to go.Local installationTo installAnonymeterlocally, clone the repository:gitclonegit@github.com:statice/anonymeter.gitand install the dependencies:cdanonymeter# if you are not there alreadypipinstall.# Basic dependenciespipinstall".[notebooks]"# Dependencies to run example notebookspipinstall-e".[notebooks,dev]"# Development setupIf you experience issues with the installation, we recommend to installanonymeterin a new clean virtual environment.Getting startedCheck out the example notebook in thenotebooksfolder to start playing around
withanonymeter. To run this notebook you would needjupyterand some plotting libraries.
This should be installed as part of thenotebooksdependencies. If you haven't done so, please
install them by executing:pipinstallanonymeter[notebooks]if you are installing anonymeter fromPyPi, or:pipinstall".[notebooks]"if you have opted for a local installation.Basic usage patternFor each of the three privacy risks anonymeter provide anEvaluatorclass. The high-level classesSinglingOutEvaluator,LinkabilityEvaluator, andInferenceEvaluatorare the only thing that you need to import fromAnonymeter.Despite the different nature of the privacy risks they evaluate, these classes have the same interface and are used in the same way. To instantiate the evaluator you have to provide three dataframes: the original datasetoriwhich has been used to generate the synthetic data, the synthetic datasyn, and acontroldataset containing original records which have not been used to generate the synthetic data.Another parameter common to all evaluators is the number of target records to attack (n_attacks). A higher number will reduce the statistical uncertainties on the results, at the expense of a longer computation time.evaluator=*Evaluator(ori:pd.DataFrame,syn:pd.DataFrame,control:pd.DataFrame,n_attacks:int)Once instantiated the evaluation pipeline is executed when calling theevaluate, and the resulting estimate of the risk can be accessed using therisk()method.evaluator.evaluate()risk=evaluator.risk()Configuring loggingAnonymeteruses the standard Python logger namedanonymeter.
You can configure the logging level and the output destination
using the standard Python logging API (seeherefor more details).For example, to set the logging level toDEBUGyou can use the following snippet:importlogging# set the logging level to DEBUGlogging.getLogger("anonymeter").setLevel(logging.DEBUG)And if you want to log to a file, you can use the following snippet:importlogging# create a file handlerfile_handler=logging.FileHandler("anonymeter.log")# set the logging level for the file handlerfile_handler.setLevel(logging.DEBUG)# add the file handler to the loggerlogger=logging.getLogger("anonymeter")logger.addHandler(file_handler)logger.setLevel(logging.DEBUG)Cite this workIf you use anonymeter in your work, we would appreciate citations to the following paper:"A Unified Framework for Quantifying Privacy Risk in Synthetic Data", M. Giomiet al, PoPETS 2023.
Thisbibtexentry can be used to refer to the paper:@misc{anonymeter,
  doi = {https://doi.org/10.56553/popets-2023-0055},
  url = {https://petsymposium.org/popets/2023/popets-2023-0055.php},
  journal = {Proceedings of Privacy Enhancing Technologies Symposium},
  year = {2023},
  author = {Giomi, Matteo and Boenisch, Franziska and Wehmeyer, Christoph and Tasnádi, Borbála},
  title = {A Unified Framework for Quantifying Privacy Risk in Synthetic Data},
}