Package: exhibit
Exhibit: Command line tool to create anonymised demonstrator dataThe goal of Exhibit is to make it easier to generate anonymised data at scale in a controlled and reproducible way.Key features:Control all aspects of the anonymisation process: which columns to anonymise and to what degreeRapidly iterate on the anonymisation optionsSet categorical weights to create custom distributionsUse regular expressions to bulk-anonymise identifiersAdd columns derived from newly anonymised dataPreserve important relationships between your columns (paired, hierarchical, custom)Add outliers to any subset of the generated dataGenerate and manipulate missing data and timeseriesGenerate geo-spatial data using H3 hexesAugment your synthetic data with compiled machine learning models and custom functionsUse SQL to generate conditional values based on external tablesInstallation:To install using pip, enter the following command at a Bash or Windows command prompt:pip install exhibitAlternatively, download or clone the repository and runpip install .from the root folder.QuickstartExhibit has two principal modes of operation:fromdataproduces a detailed, user-editable.ymlspecificationfromspecwhich produces the anonymised dataset from the supplied specificationSee the-hlisting for the full list of optional command line parameters.The repository includes a few sample datasets and specifications.You can find them inexhibit/sample/_dataandexhibit/sample/_specTo create a demo dataset, run:exhibit fromspec exhibit/sample/_spec/inpatients_demo.yml -o demo.csvTo create a demo specification that equialises all probabilities and weights, run:exhibit fromdata exhibit/sample/_data/inpatients.csv -ew -o demo.ymlDatabaseExhibit is bundled with a SQLite3 database and a Python utility tool to interact with it. Alternatively, you can connect directly to/exhbit/db/exhibit.db. The database contains three sample aliasing datasets:mountains,birdsandpatientsdesigned to help you quickly alias original values without manually editing individual column values.mountainshas 15 mountain ranges and their top 10 peaks making it useful for aliasing hierarchical pairs, like NHS Boards and Hospitals.birdshas 150 pairs of common / scientific bird names. This can be useful for 1:1 paired columns.patientshas 360 made-up patient records with details such as gender, 5-year age band, date of birth and CHI number. Fields from this dataset can be selectively pulled in when linked data is required.dateshas dates ranging from 1900-01-01 to 2100-01-01 at a single day interval. This table is useful if you have a SQL statement in theanonymising_setthat picks dates based on a condition.The database is also used to store temporary data for columns where the number of unique values exceeds user threshold and thus not available for editing directly in theymlfile.Note that original, confidential data might be saved in theexhibit/db/exhibit.dbfile on your local machine. You can purge all temporary tables by calling--purgecommand from the included utility tool or by interfacing with the database directly.DisclaimerPlease note that the degree of anonymisation for each dataset produced by the tool will depend heavily on user choices in the specification. As such, there is no guarantee that confidential data will be suitably masked under all scenarios. If you intend to work with sensitive data, make sure to thoroughly evaluate the output before making it public.