# On the Privacy-Utility Trade-Off in Differentially Private Hierarchical Text Classification
## 8 Discussion  
Large values for privacy parameter \(\epsilon\) are sufficient to completely mitigate MI attacks with moderate decrease in model utility. In our experiments, we enforced the HTC models to satisfy DP guarantees by clipping and perturbing the computed gradients during the training process. As expected, the experiments showed that enforcing DP in this way reduces the effectiveness of the performed MI attacks but also harms model utility. Figure 8 summarizes the trade-off between classification accuracy and MI _AUC_ for each dataset. As can be seen, for all examined datasets, it is possible to completely mitigate the MI attack while reducing classification _Acc_ by \(<\)20%. For BestBuy, the CNN model yields the best model utility for \(AUC=50\%\). In contrast, for Reuters and DBPedia, the Transformer yields the best model utility for \(AUC=50\%\). This may be explained by the size and average text length of the datasets, since for small datasets with short texts (e.g., BestBuy), the CNN model is well suited, while for larger datasets with longer texts (e.g., Reuters, DBPedia), the Transformer architecture is suited better. Figure 7: Attack model advantage over ratio of records per hierarchy for an overall number of \(\{0.25,0.5,0.75,1\}\times n\) training records. (**a**) Relation between the records/levels ratio and attack advantage for BestBuy; (**b**) Relation between the records/levels ratio and attack advantage for Reuters; (**c**) Relation between the records/levels ratio and attack advantage for DBPedia. **Overfitting is a key driver for MI attack performance. ** Our experiments support the understanding that overfitting leads to higher MI attack performance. We therefore extend the findings of previous work for non-textual datasets to HTC [7, 8]. To prevent overfitting, our analysis of drivers for MI attack performance suggests to gather as many training examples as possible and only predict as many hierarchy levels as needed. Adding HTC-specific features to the attack model did not increase MI attack performance, confirming the validity of our results also for stronger adversaries. **Similar DP privacy parameters do not imply a similar MI attack effectiveness. ** The experimental results show that the empirical MI risk for similar DP guarantees varies within each dataset but also within each model architecture. Therefore, we can again summarize that the MI attack effectiveness depends on the chosen model architecture and the dataset. Unfortunately, the results do not point to a model architecture that is strictly better suited to mitigate the MI attack. However, we recommend using a model with relatively few parameters such as the BoW model for smaller datasets, whereas for larger datasets, models with a high number of parameters such as the Transformer classifier yield a favorable trade-off. **The BoW model's utility was reduced least by the added DP noise. ** Across all datasets, we observed that the CNN and Transformer model's utility scores were impacted more heavily compared to the BoW model's utility for similar DP guarantees. On two of the three datasets, the Transformer model's utility is impacted even more severely than the CNN model's utility. This finding suggests that a higher number of weights in an ANN might correlate with a stronger impact of DP training on the ANN utility. Specifically, the number of ANN weights is lowest in the BoW model and highest in the Transformer model. This insight should be taken into account when a data scientist wants to train an ANN based on a given formal DP guarantee. **HTC ANNs exhibit a big gap between empirical and theoretical MI risk**. The obtained results support the conclusions by [PERSON] et al. [16], who found that there remains a big gap between what state-of-the-art MI attacks can infer and what is the maximum that can theoretically be inferred according to the bound presented by [PERSON] et al. [32].