## 5 Privacy Evaluation Methods  
Two different types of metrics are frequently used to evaluate how well privacy-preserving methods perform: (1) privacy metrics to determine how much privacy a dataset has lost/gained, and (2) utility metrics to determine how useful the protected data are for data analysis. A comprehensive survey of more than 80 privacy metrics is presented in [37], where two categories for privacy evaluation are identified: adversary models and data sources. Methods that employ adversary models evaluate an adversary's skills and intentions (e.g., MITM), whereas those that examine the data sources investigate potential privacy leakage risks (e.g., Malware attacks). Both are key elements in establishing the privacy measure for a given scenario and numerous metrics are often used to accurately analyse the related degree of privacy loss. Privacy-preserving measures often lead to compromising utility metrics, which are used to quantify the usefulness of data being protected. This is seen in both applications (FL for ETD) in Section 4.1 and differential privacy for stress detection in Section 4.2. Information loss measures are established to quantify the similarity between the original data and protected data for general analysis purposes. Usually, the usefulness of protected data is determined by how well the statistical details of the original data are preserved. By comparing the accuracy of the evaluation carried out on the protected data with that of the original data, the data utility is evaluated to determine its suitability for particular analytical goals, such as machine learning and statistical analysis [38]. Differential privacy is a rigorous mathematical definition quantifying privacy leakage in which noise is added to the original signal to reduce the probability of privacy leakage in case the data are intercepted. As shown in Figure 5, a higher noise level improves data privacy at the cost of reduced utility. It can also be seen that the proposed differential privacy's multimodal approach (ALL) is closest to the ideal point as it delivers high-accuracy with an acceptable privacy budget. As highlighted in [39], differential privacy has been extensively investigated in various AI domains, including machine learning, deep learning, and multi-agent learning. The study highlights the crucial role of differential privacy in surmounting significant AI challenges, including enhancing privacy stability, fairness, and security. It facilitates composition and utility in AI systems while effectively mitigating risks like membership and attribute inference attacks. These aspects not only ensure robust privacy guarantees but also have a profound impact on the performance and reliability of AI models. One of the fundamental aspects of differential privacy is the trade-off between utility and privacy, especially given the challenges in quantifying utility loss. In differential privacy, the \(\epsilon\) parameter plays a key role in defining the level of privacy. It quantifies the privacy budget, with lower values indicating stronger privacy. Larger \(\epsilon\) values are chosen to ensure utility, but this often leads to a gap between theoretical privacy guarantees and actual privacy loss observed in practice. Prior studies suggest that while choosing a higher \(\epsilon\) value to maintain data utility might weaken privacy guarantees, the actual leakage under inference attacks remains low [40; 41]. The authors in [38] provided a thorough overview of privacy and utility metrics for privacy-preserving machine learning applications, ranging from broad to targeted data analysis, including several privacy-preserving FL techniques. The authors examined the types of attacks that can be performed against FL systems [6] to assess user data privacy. As evaluation criteria, the attacker's success rate and the amount of information demonstrated in the attacks were used.