# An Understanding of the Vulnerability of Datasets to Disparate Membership Inference Attacks
## 6 Discussion of Hardening Exploration  
This section discusses the results associated with the hardening exploration efforts. While the impetus of the article focuses on the understanding of dataset vulnerability to disparate membership inference attack, an exploration into hardening techniques based on these discovered vulnerabilities can assist in this understanding while also offering an introduction to mitigation strategies for the dataset owner.  
These hardening explorations were developed based on the features found to contribute to vulnerability, which can be summarized into an over-abundance of information relating to the width-ratio feature, sparse and unevenly distributed class regions relating to the in-label distance features, and a lack of entropy within individual feature realization possibilities as found in the proportion of binary features and one-hot encoded categorical variables. This study focused on the former two vulnerabilities through feature reduction efforts based on correlation and manifold theory and through class-balance methods based on oversampling via CTGAN and undersampling using NearMiss methodologies. In addition, combinations of feature reduction and oversampling methods were also explored given their more promising results to evaluate if further hardening could be accomplished.  
Feature reduction based in manifold theory provided better results than using correlation thresholds. Given that manifold theory finds a lower dimensional representation of the information contained within the feature set, this reduction to a more base layer could provide a perturbation effect on the membership inference attack attempt. In addition, by condensing the feature set, this hardening method could have changed the uniformity of in-region observational distribution to be more uniform and thus provided protection in this manner.  
However, end-users may find hardening via manifold theory to be less appealing due to the increased difficulty in understanding and explanation of the results of the classification exercise given the decreased definition of what is contained within a particular feature of importance within their ultimate algorithm.  
Oversampling provided for a larger increase in the percent of secured datasets and a lower percent in the number of datasets, which increased in vulnerability as compared to undersampling, while both methods attempted to balance classes, oversampling increased the fortification of existing observations. Therefore, it is reasonable that this increase in supporting members--and thus entropy--would reduce the ability of an attack to discern if an observation was or was not a part of the original training dataset. However, the oversampling method did not provide the level of improvement seen in the manifold theory-based feature reduction method. Given that the oversampling technique utilized maintained the original distributions, there would still be a similar non-uniformity of the in-class region distribution of observations, therefore, leaving this vulnerability-contributing factor unresolved.  
However, the best results were found with a combination of the two best-performing methods of feature reduction and class balancing efforts--a manifold theory-based feature reduction with CTGAN oversampling. This provided an effectively insignificant change in the victim model performance while reducing the disparate attack accuracy by 0.02 on average. A total of 19% of class subsets were made more secure, and only 1% were more vulnerable. This elevated protection can be attributed to the perturbing effect and increase of uniformity of the in-class region observational distribution of the manifold-theory-based feature reduction as well as the increase of supporting observations and entropy of the oversampling method.