## 5 Further Exploring Differential Privacy in the Training Dataset--Results  
In this work, we put effort into elaborating on the work presented in [15]; more precisely, we investigate how several parameters of the ML model affect the accuracy of the model when applying DP to the training dataset. In this respect, an extensive set of experiments was carried out, focusing on two different network architectures (\"dense\" and \"cnn\") with two different sets of input data that are well-known in the literature, namely, the MNIST dataset for handwritten digit recognition [60] and the CIFAR-10 dataset, which consists of color images classified into 10 classes, such as ships, cats, and dogs [61]. For each combination (i.e., architecture/dataset), we applied two different modes of operation of the machine learning algorithms (i.e., without the addition of noise and, for the DP, with the addition of noise) to accurately recognize the various characters with respect to the input dataset MNIST and to accurately recognize the various objects with respect to the input dataset CIFAR-10. Note that the \"dense\" architecture corresponds to a dense neural network, whilst the \"cnn\" architecture corresponds to a convolutional neural network. The ML algorithm used is the stochastic gradient descent algorithm. As in [15], we rely on the following definition of DP, which is based on the application-specific concept of adjacent databases. More precisely, each training dataset is a set of image-label pairs, and we say that two of these sets are adjacent if they differ in a single entry, that is, if one such pair is present in one set but not in the other [15]. Therefore, we have the following definition based on the variant introduced by [62]:  
**Definition 1**. : _A randomized mechanism M: D\(\rightarrow\)R, with input domain D and output range R, satisfies (\(\varepsilon\),\(\delta\))-differential privacy if for any two d, d' being adjacent in D and for any subset S\(\subseteq\)R, it holds:_  
\[\Pr[\mathrm{M(d)}\in\mathrm{S}]\leq\mathrm{e}^{\varepsilon}\ \Pr[\mathrm{M(d^{ \prime})}\in\mathrm{S}]+\delta\]  
_where \(\varepsilon\) is the so-called privacy-loss parameter at a differential change in data (the smaller the value is, the better privacy protection), and \(\delta\) reflects the probability of a privacy leak where the added noise (controlled by \(\varepsilon\)) does not provide sufficient protection. (If \(\delta\) is low, it is less probable that an individual's privacy is going to be compromised, but this may yield a high noise to be added, which, in turn, may affect the applicability of the data)._  
For our experiments, we provided as input to the ML algorithm (i) the images of the input dataset MNIST and (ii) the images of the input dataset CIFAR-10, and we trained the model using either the \"dense\" network architecture or the \"cnn\" network architecture, both for the classical gradient descent algorithm as well as for the private stochastic gradient descent algorithm (having added noise). The hyperparameters of the algorithms are described as follows:  
1. Batch_Size: It determines the number of training samples used to train the network in an iteration (i.e., before updating the model parameters);
2. Learning_Rate: This is an important hyperparameter of a neural network that controls how much to change the model in response to the estimated error each time the model weights are updated;
3. L2 Norm_Bound: This hyperparameter specifies the bound value for the clipping of the gradient descent algorithm (i.e., to not have too large weights while ensuring that the important components in the weight vector are larger than the other components) [15];
4. Sigma: This hyperparameter specifies the noise scale to be added to the stochastic gradient descent algorithm [15];
5. Use_Privacy: This hyperparameter specifies whether the so-called private stochastic gradient descent algorithm, which introduces noise in each iteration of the stochastic gradient descent algorithm, will be used or not;
6.