# Privacy and Utility of Private Synthetic Data for Medical Data Analyses
## 8 Conclusions and Future Work  
According to our experiments, the use of private synthetic data generators seems very promising. However, the evaluation shows that the original DP parameter \(\epsilon\) is not sufficient as the only privacy metric and the meaning for privacy and utility still remains an open question. This is especially true for use cases in which time series data are released. Certain features are still preserved by the generative models which can pose a privacy threat. In our example, this can be observed in the heart frequency curve, where a change in the heart rate throughout the day or certain patterns could potentially be linked to an individual's activities. We therefore argue that a use case-specific analysis is required when employing such technologies. For our real-world scenario, we did this by introducing the Tachycardia Utility (TU) and Tachycardia Privacy (TP) metrics, which measure utility and privacy while also considering the specifics of the data. This is illustrated by the fact that these metrics do not necessarily correlate with DP's typical \(\epsilon\) metric for the technologies used. A variety of experiments using these metrics were conducted with the three synthetic data generators MWEM, PATE-CTGAN, and DP-CTGAN. These experiments include an evaluation of the privacy and utility metrics TP and TU compared to the original notion of \(\epsilon\). DP-CTGAN seems to perform best, but there is no clear winner for our benchmarks. Due to its different approach, MWEM is not recommended for scenarios to those in this article. In general, synthetic data generators provide a useful way to generate more accurate and private data than a static data release which uses traditional PETs. Finally, it should be considered that the meaning \(\epsilon\) for those technologies is not necessarily the only indicator of privacy and that more use-case-specific metrics are recommended.  
It remains to be stated that more work in this area of research is needed. Further experiments could include a comparison with interactive DP technologies provided by frameworks such as Google DP ([[https://github.com/google/differential-privacy](https://github.com/google/differential-privacy)]([https://github.com/google/differential-privacy](https://github.com/google/differential-privacy)) (accessed on 5 September 2022)). Additionally, a broader and more diverse set of data generator technologies should be evaluated for the use case presented here. While this future work focuses on the specific use case, the research question should also be posed more generally. It should be tested how robust the metrics introduced herein are for other time series data or if more general definitions are needed. Furthermore, the privacy attacker presented herein is very specific and should be defined more universally. Another question is that of how models could look that seek to preserve certain features through constraints defined by medical background knowledge while simultaneously providing privacy protection. This includes the question of how strictly consistency, structural, and correlation constraints could be defined for medical data. Such a technology would be even more useful if it could work with categorical values so that it could, for example, create synthetic patients' data with various disease types. Ultimately, this could lead to the vision of a private synthetic patient generator which generates accurate but private data based on real-world patients.  
Conceptualization, A.A. and M.L.; methodology, A.A. and M.L.; software, M.L.; investigation, M.L. and A.A.; writing--original draft preparation, A.A. and M.L.; writing--review and editing, A.A., P.P., M.L. and E.K.; supervision, P.P., E.K. and J.B. All authors have read and agreed to the published version of the manuscript.  
This work was supported as a Fraunhofer Lighthouse Project and was conducted in close cooperation with the KASTEL project.