Package: pure-ldp
Pure-LDPpure-LDP is a Python package that provides simple implementations of various state-of-the-art LDP algorithms (both Frequency Oracles and Heavy Hitters) with the main goal of providing a single, simple interface to benchmark and experiment with these algorithms.If pure-LDP is useful to you and has been used in your work in any way we would appreciate a reference to:Graham Cormode, Samuel Maddock, and Carsten Maple. Frequency Estimation under Local Differential Privacy. PVLDB, 14(11): 2046 - 2058, 2021InstallationUse the package managerpipto install.pipinstallpure-ldpTo upgrade to thelatest versionpipinstallpure-ldp--upgradePure-LDP Requires the following Python modulesxxhash
numpy
scipy
bitstring
bitarray
matplotlib
seaborn
statsmodels
sklearnOutlineThe package has implementations of all three main frequency oracles detailed in paper"Locally Differentially Private Protocols for Frequency Estimation"by Wang et al which are:(Optimal) Unary Encoding - Underpure_ldp.frequency_oracles.unary_encoding(Summation/Thresholding) Histogram encoding - Underpure_ldp.frequency_oracles.histogram_encoding(Optimal) Local Hashing - Underpure_ldp.frequency_oracles.local_hashingThe package also includes an implementation of the heavy hitter algorithm Prefix Extending Method (PEM) underpure_ldp.heavy_hitters.prefix_extendingOver time it has evolved to include many more implementations of other LDP frequency estimation algorithms:Apple'sCount Mean Sketch (CMS / HCMS)Algorithm - This is underpure_ldp.frequency_oracles.apple_cmsGoogle's RAPPOR i.e DE combined with Bloom filters underpure_ldp.frequency_oracles.rapporHadamard Response (HR)- This is underpure_ldp.frequency_oracles.hadamard_responsethe code implemented for this is simply a pure-LDP wrapper of the repohadamard_responseHadamard Mechanism (HM) underpure_ldp.frequency_oracles.hadamard_mechanismDirect Encoding (DE) / Generalised Randomised Response underpure_ldp.frequency_oracles.direct_encodingFast Local Hashing (FLH) a heuristic variant of OLH underpure_ldp.frequency_oracles.local_hashingGeneric private sketching protocols (SketchResponse) underpure_ldp.frequency_oracles.sketch_responseThe library also includes implementations of other Heavy Hitter (HH) algorithms:Apple's Sequence Fragment Puzzle (SFP) algorithm underpure_ldp.frequency_oracles.apple_sfpTreeHistogram (byBassily et al) underpure_ldp.frequency_oracles.treehistogramBasic Usageimportnumpyasnpfrompure_ldp.frequency_oracles.local_hashingimportLHClient,LHServer# Using Optimal Local Hashing (OLH)epsilon=3# Privacy budget of 3d=4# For simplicity, we use a dataset with 4 possible data itemsclient_olh=LHClient(epsilon=epsilon,d=d,use_olh=True)server_olh=LHServer(epsilon=epsilon,d=d,use_olh=True)# Test dataset, every user has a number between 1-4, 10,000 users totaldata=np.concatenate(([1]*4000,[2]*3000,[3]*2000,[4]*1000))foritemindata:# Simulate client-side privatisationpriv_data=client_olh.privatise(item)# Simulate server-side aggregationserver_olh.aggregate(priv_data)# Simulate server-side estimationprint(server_olh.estimate(1))# Should be approximately 4000 +- 200Seeexample.pyfor more examples.Simulation FrameworkThis is currently WIP but there is already significant code underpure_ldp.simulationsthat allow you to build experiments to compare various frequency oracles/heavy hitters under various conditions. Generic helpers to run experiments for FOs and HHs are included underpure_ldp.simulations.helpers. Seepure_ldp.simulations.paper_experiments.pyfor examplesTODOBetter documentation !AcknowledgementsSome OLH code is based on the implementation byTianhao Wang:repoThe Hadamard Response code is just a wrapper of the k2khadamard.py code in the repohadamard_responsebyZiteng SunContributingIf you feel like this package could be improved in any way, open an issue or make a pull request!LicenseMIT