# On the Privacy-Utility Trade-Off in Differentially Private Hierarchical Text Classification
## 9 Conclusions  
This work analyzed and compared the privacy-utility trade-off in DP HTC under a white-box MI adversary. Even without the use of DP, white-box MI attacks only posed a minor risk to the three HTC model architectures and reference dataset that we considered. In consequence, large privacy parameters were sufficient to fully mitigate the white-box  
Figure 8: Privacy-utility trade-off per HTC model on each dataset. Privacy and utility are represented by MI _AUC_ and classification _Acc_, respectively. An optimal classifier would exhibit 100% _Acc_ and no vulnerability to the MI adversary, expressed by 50% _AUC_. attack. We particularly observed Transformer-based HTC models to be rather resistant to MI attacks without using DP at all. However, the privacy-accuracy trade-off for full mitigation of the white-box MI attack is differing widely for all considered models and datasets. Our results suggest that the Transformer model is also favorable for large datasets with long texts when using DP, while the CNN model is favorable for smaller datasets with shorter texts. However, if hardware costs shall be minimized or the training examples shall be protected with a strong formal DP guarantee (i.e., small \(\epsilon\) value), the fastText-based BoW model is a good choice due to the high robustness against DP perturbation. Our experiments also confirm a large gap between the empirical membership advantage of the MI white-box attack and the theoretical DP membership advantage bound for HTC datasets and models. Conceptualization, D.W., D.B., F.A., J.P.-A. and T.S. ; Data curation, D.W., D.B. and F.A. ; Formal analysis, D.W., D.B. and F.A. ; Funding acquisition, D.B. ; Investigation, D.W., D.B. and F.A. ; Methodology, D.W., D.B., J.P.-A. and T.S. ; Project administration, F.A. ; Supervision, D.B., F.A., J.P.-A. and T.S. ; Validation, D.W., D.B. and F.A. ; Visualization, D.W., D.B. and F.A. ; Writing--original draft, D.W., D.B. and F.A. ; Writing--review & editing, D.W., D.B. and F.A. All authors have read and agreed to the published version of the manuscript. This work has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No. 825333 (MOSAICROWNN). The project that gave rise to these results received the support of a fellowship from \"la Caixa\" Foundation (ID 100010434) and from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 847648. The fellowship code is LCF/BQ/PR20/11770009. The work of [PERSON] has been supported through an [PERSON] Post-Doctoral Fellowship. This work was also supported by the Spanish Government under research project \"Enhancing Communication Protocols with Machine Learning while Protecting Sensitive Data (COMPROMISE)\" (PID2020-113795 RBC31/AEI/10.13039/501100011033). Not applicable. We would also like to thank the anonymous reviewers for their immensely helpful suggestions to improve the readability and contents of this paper. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results. * [PERSON] et al. (2013) [PERSON]; [PERSON]; [PERSON]; [PERSON] Multi-label learning with millions of labels: Recommending advertiser bid phrases for web pages. In Proceedings of the International Conference on World Wide Web, Rio de Janeiro, Brazil, 13-17 May 2013; pp. 13-24. * [PERSON] et al. (2016) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] DeepMeSH: Deep semantic representation for improving large-scale MeSH indexing. _Bioinformatics_**2016**, 32, 170-179. [CrossRef] [PubMed]
* [PERSON] et al. (2017) [PERSON]; [PERSON] [PERSON]; [PERSON]; [PERSON] Membership Inference Attacks against Machine Learning Models. In Proceedings of the IEEE Symposium on Security and Privacy, San Jose, CA, USA, 22-26 May 2017; pp. 3-18. * [PERSON] et al.