Package: mia
A library for running membership inference attacks (MIA) against machine learning models. Check out
thedocumentation.These are attacks against privacy of the training data. In MIA, an attacker tries to guess whether a
given example was used during training of a target model or not, only by querying the model. See
more in the paper byShokri et al. Currently, you can use the
library to evaluate the robustness of your Keras or PyTorch models to MIA.Features:Implements the original shadow modelattackIs customizable, can use any scikit learn’sEstimator-like object as a shadow or attack modelIs tested with Keras and PyTorchGetting startedYou can install mia from PyPI:pipinstallmiaUsageShokri et al. attackSee thefull runnable example.  Read the details of the
attack in thepaper.Lettarget_model_fn()return the target model architecture as a scikit-like classifier. The
attack is white-box, meaning the attacker is assumed to know the architecture. LetNUM_CLASSESbe the number of classes of the classification problem.First, the attacker needs to train severalshadow models—that mimick the target model—
on different datasets sampled from the original data distribution. The following code snippet
initializes ashadow model bundle, and runs the training of the shadows. For each shadow model,2 * SHADOW_DATASET_SIZEexamples are sampled without replacement from the full attacker’s
dataset.  Half of them will be used for control, and the other half for training of the shadow model.frommia.estimatorsimportShadowModelBundlesmb=ShadowModelBundle(target_model_fn,shadow_dataset_size=SHADOW_DATASET_SIZE,num_models=NUM_MODELS,)X_shadow,y_shadow=smb.fit_transform(attacker_X_train,attacker_y_train)fit_transformreturnsattack dataX_shadow, y_shadow. Each row inX_shadowis a
concatenated vector consisting of the prediction vector of a shadow model for an example from the
original dataset, and the example’s class (one-hot encoded). Its shape is hence(2 * SHADOW_DATASET_SIZE, 2 * NUM_CLASSES). Each label iny_shadowis zero if a corresponding
example was “out” of the training dataset of the shadow model (control), or one, if it was “in” the
training.mia provides a class to train a bundle of attack models, one model per class.attack_model_fn()is supposed to return a scikit-like classifier that takes a vector of model predictions(NUM_CLASSES, ),
and returns whether an example with these predictions was in the training, or out.frommia.estimatorsimportAttackModelBundleamb=AttackModelBundle(attack_model_fn,num_classes=NUM_CLASSES)amb.fit(X_shadow,y_shadow)In place of theAttackModelBundleone can use any binary classifier that takes(2 * NUM_CLASSES, )-shape examples (as explained above, the first half of an input is the prediction
vector from a model, the second half is the true class of a corresponding example).To evaluate the attack, one must encode the data in the above-mentioned format. Lettarget_modelbe
the target model,data_inthe data (tupleX, y) that was used in the training of the target model, anddata_outthe data that was not used in the training.frommia.estimatorsimportprepare_attack_dataattack_test_data,real_membership_labels=prepare_attack_data(target_model,data_in,data_out)attack_guesses=amb.predict(attack_test_data)attack_accuracy=np.mean(attack_guesses==real_membership_labels)Citing@misc{mia,
  author       = {Bogdan Kulynych and
                  Mohammad Yaghini},
  title        = {{mia: A library for running membership inference
                   attacks against ML models}},
  month        = sep,
  year         = 2018,
  doi          = {10.5281/zenodo.1433744},
  url          = {https://doi.org/10.5281/zenodo.1433744}
}