# An Understanding of the Vulnerability of Datasets to Disparate Membership Inference Attacks
## 7 Summary and Future Efforts  
This study provides an in-depth look at the vulnerability of datasets themselves to disparate membership inference attacks--those that focus on attacking individual classes as opposed to the overall dataset--contributing an addition to the current literature that focuses on model vulnerability. This understanding was accomplished through the creation of a vulnerability-classification model based on over 100 datasets--including frequently cited datasets within the AI security literature. The vulnerability-classification dataset used to create this classification model consisted of 118 features and a set of victim, shadow, and attack model accuracies, all used to describe and understand the vulnerability of these datasets to disparate membership inference attacks. The resulting ensemble model, consisting of a logistic regression model, Naive Bayes model, and a random forest model, obtained a testing accuracy of 84.5% in classifying datasets as vulnerable or secure to these disparate membership inference attacks. Of the seven features used in the classification model, five were based on in-label observational distance measurements. This heavy reliance on observational distances within class regions is consistent with other findings in the literature, which state that minority and sparsely populated classes tend to increase vulnerability. In addition to the vulnerability-classification model, this study also provided an increased understanding of the vulnerability of datasets to these attacks. First, it was shown that the uniformity of the in-class region distribution is an important factor in dataset vulnerability. Those datasets with a less uniform distribution of in-class observational distances were proven to be more vulnerable to attack. Second, it was shown that an increased proportion of binary features can result in an increase in vulnerability. This finding was established through the width ratio after one-hot encoding and the proportion of binary features exclusive of categorical one-hot encoded features (both of which indicated an increase in vulnerability with the increase of either the width of the dataset after one-hot encoding) or the increase in the proportion of binary features, while wider datasets in general can contribute to an overabundance of information and, therefore, an adversarial advantage for membership inference. Of particular interest was the inclusion of the post-one-hot encoding aspect. One-hot encoding results in a binary feature indicating a categorical response and is a common preprocessing step for datasets. Given understandings of entropic contributions to membership inference vulnerability and previous findings of low entropy features causing an increase in vulnerability, we concluded that these binary features, due to their low inherent entropy, result in an increase in attack success. Non-binary features can take on an infinite number of values and, thus, provide more entropy-influenced security compared with two-state binary features. To further understand these vulnerabilities and to provide exploratory mitigation strategies, we investigated preliminary hardening strategies based on the vulnerabilities discovered in the vulnerability classification process. In particular, feature reduction methods were used to treat an overabundance of information and intelligent over- and undersampling methods were used to treat class-region sparsity and imbalances. The best-performing method proved to be a manifold theory-based feature reduction combined with a CTGAN-based oversampling strategy. This hardening process resulted in a reduction in disparate attack accuracy of 0.02 on average and an effectively insignificant change in the victim model performance. Using this method, 19% of class-based sub-datasets were made more secure, and only 1% were more vulnerable.