dataesr/scientific-paragraphs-categorization huggingface dataset:
Another difference between the two settings is reliability. In cross-silo, the organizations are always available to train, unlike user devices. FL can also be classified by data partition. We distinguish Horizontal FL, Vertical FL and Hybrid FL. In HFL, the datasets of the clients have the same features space. In VFL, the local datasets have the same individuals, but with different features. The hybrid setting is a combination between HFL and VFL. A typical federated training process is considered by the algorithm of FedAvg proposed by McMahan et al. [3]. It consists of five steps: The server selects a subset of clients according to some criteria. The selected client downloads the current model weights and a training program from the server. Each client locally computes an update to the model by executing the training program.
Differential Privacy Differential privacy is a widely used standard to guarantee privacy in data analysis. The main idea of DP is to consider a thought experiment in which we compare how an algorithm behaves on a dataset D 1 with the way it behaves on a hypothetical dataset D 2 , in which one person's record has been removed or added. These two datasets are considered "neighbors" in the dataset space. Hence, we say that an algorithm is differentially private if running the algorithm on two neighboring datasets yields roughly the same distribution of outcomes. In other words, differential privacy ensures that the outcomes of M are approximately the same whether or not the person i joins the dataset. Formally, DP is defined by  Dwork et al. in 2006 [28]  as follows.
Definition 1. A randomized function M gives ( , δ)-differential privacy if for all datasets D 1 and D 2 differing on at most one element, and all S ⊂ Range(M), Pr[M(D 1 ) ∈ S] ≤ e × Pr[M(D 2 ) ∈ S] + δ (1) In the Definition 1, is a non-negative real number that determines the level of privacy protection provided by the algorithm. A lower value of corresponds to a stronger privacy guarantee. The value of δ is a small positive real number that represents the probability of any failure of the definition. When δ is set to 0, it is referred to as pure differential privacy. Three major properties arise directly from this definition: composition, post-processing, and group privacy. These properties are the key to design powerful algorithm from basic mechanisms:.
• Post-processing: ensures that the privacy guarantees of a differential privacy mechanism remain unchanged even if the output is further processed or analyzed. • Group privacy: this definition can be extended to group privacy by considering two datasets differing on at most k records instead of 1 record. As stated in the definition, DP is a property of an algorithm M. There are several methods to achieve DP based on adding noise to the input data, the output data, or the intermediate result. The noise can be generated using different mechanisms, the wellknown ones are the Laplace mechanism, the Gaussian mechanism, and the exponential mechanism. Two main settings are discussed in the literature for differential privacy: the centralized DP (CDP) and the local DP (LDP). In CDP, the noise is added by a centralized server that collects first the data then applies the mechanism.
Privacy Attacks in FL While federated learning (FL) is generally regarded as a privacy-preserving technique in machine learning, recent studies have revealed a potential privacy concern (see Figure  1 ). This concern arises from the fact that, although FL avoids the need to share private client datasets during the learning process, the exchange of gradients in FL can inadvertently disclose sensitive information about the client's private data. This issue is particularly pronounced in FL due to the large number of participants involved and the inherent white-box setting of the FL framework. An insider may exploit the exchanged gradients to perform powerful attacks using passive strategy (one that doesn't influence the learning process) or an active approach (where they actively influence the learning process)such as conducting membership inference or launching a reconstruction attack.
Membership Inference Membership inference is a type of attack in machine learning that aims to figure out whether a target data point is used to train a certain ML model. More formally, given x as the target point, M as a trained model, and some external knowledge K, this attack can be defined by the following function A : x, M, K → {False, True} Here, this function returns True if the target x is in the training dataset and False otherwise. This attack can be made either in a black-box setting, where the attacker has only access to an API of the model M, or in a white-box setting, where the attacker has access to the whole model. As we can remark, the attack model A is a binary classifier, and it can be constructed using different ways. The first membership inference attack against machine learning models was presented by  Shokri et al.
The proposed attack was evaluated against neural networks using different datasets: CIFAR, Purchases, Locations, Texas hospital stays, MNIST, and UCI Adult. The authors confirm that their results show that models created using MLaaS can leak a lot of information about their training datasets. The assumptions considered by Shokri et al. are considered strong, which reduce the scope of the membership inference attack  [34]. This motivated Salem et al. in 2019  [34]  to propose relaxations of these assumptions. They showed that relaxing the number of shadow models to one shadow model and assuming that the shadow training data are distributed similarly to target training data produce performances that are similar to those provided by Shokri et al. [33]. Furthermore, considering an attack model that is independent of the training data distribution may also reveal some information, but this is not as efficient as in the previous work.
The attack model proposed by Nasr et al. consisted of feature extraction components and an encoder component. To extract features from the output of each layer, they employed a fully connected network, incorporating the one-hot encoding of the true label and the loss. For the gradients, a convolutional neural network was utilized. The output from this step was then fed into an FCN encoder, which provided the membership probability of the input. Through experimentation with various datasets such as CIFAR100, Texas100, and Purchase100, the authors demonstrated that even well-generalized models are highly susceptible to white-box membership inference attacks. Gu et al. in 2022  [38]  proposed a membership inference attack named CS-MIA, which utilizes prediction confidence series (PCS) in federated learning.
Class Representatives Inference The inference of class representatives tries to extract generic class representatives of the global data rather than the real data in the training datasets  [39]. This is similar to the concept of model inversion attack, proposed by Fredrikson et al. in 2015  [40]. In the special case where all class members are similar, the result of this attack is similar to the training data. For example, in a facial recognition model, each class corresponds to a single individual, and the output of this attack is similar to any image that represent this person. In the realm of federated learning, the utilization of Generative Adversarial Networks (GANs) allows the execution of such attacks. Hitaj et al. in 2017 [41]  designed such an attack in federated learning using GAN.
The attacker, acting as an honest-but-curious client in the federated learning topology, tries to influence the learning process. In their work  [41] , the attacker locally trains a GAN model capable of generating synthetic samples that look like the samples from the victim's data but that are supposed to belong into a different class. In this way, the victim will work harder to distinguish the fake class from his class and reveal more information about his dataset. The experiment results showed that this attack is effective and generate representative samples of training datasets. Furthermore, in 2018, Wang et al. [39]  expanded the scope of the attack to breach clientlevel privacy. While acknowledging the effectiveness of the GAN-based attack proposed by Hitaj et al. [41] , the authors observed that the adversarial influence of the client could alter the architecture of the shared model.
Moreover, they considered a powerful malicious client in their analysis. In order to overcome these limitations, Wang et al. introduced a more practical and inconspicuous attack on the federated learning model, known as mGAN-AI. In contrast to the previous attack, which was conducted by the client, the authors of mGAN-AI assumed the presence of a malicious server. They went a step further in breaching client-level privacy by utilizing a GAN with a multitask discriminator. This discriminator not only performed the task of a standard GAN, but also distinguished the real data distribution of the victim from that of other clients. Through experimental evaluations on datasets such as MNIST and AT&T, the researchers demonstrated that mGAN-AI could reconstruct samples close to the victim's training samples.
To accomplish this, shadow classifiers were trained on the same task and using similar datasets as the target classifier, but constructed to possess or lack the property P. The parameters of these shadow classifiers were then employed to train the meta-classifier. The work presented in  [42]  focuses on a centralized machine learning (ML) context using Support Vector Machines (SVM) and Hidden Markov Models (HMM). In contrast,  Melis et al. in 2018 [43]  were the first to explore the unintended feature leakage within collaborative machine learning and federated learning (FL). They demonstrated that the exchanged updates during the FL process can inadvertently disclose information about the participants' data. By exploiting this leakage, both passive and active property inference attacks can be executed to infer properties that are unrelated to the original task of the model.
Training Samples and Labels Inference Training samples and labels inference, also known as reconstruction attacks, aim to reconstruct the original dataset belonging to a client involved in the federated learning process. These attacks focus on recovering the training samples and their corresponding labels from the aggregated model. By exploiting the information present in the model's parameters or gradients, adversaries attempt to recreate the client's original dataset, potentially compromising the privacy and confidentiality of the client's data. Zhu et al. [4]  in 2019 demonstrated that it is possible to obtain the private training data from the publicly shared gradients. Their method, known as Deep Leakage from Gradient (DLG), utilizes an optimization algorithm to recover pixel-wise accurate information for images and token-wise matching for texts.
Related Works The combination of DP with FL is an active research area. The main challenge in this field is the trade-off between privacy and utility. This challenge is addressed using the different setting in DP, including Centralized DP (CDP), Local DP (LDP), and the shuffle model. Table  3  presents the different selected works, presenting the key ideas and the shortcoming. The use of CDP for federated learning was explored in 2017 by McMahan et al. [47]. They were the first to show that it is possible to train large recurrent language models with CDP. Their proposed algorithm is based on the FedAvg algorithm  [3]  and the moments' accountant technique  [48] , which provides tight composition guarantees for the repeated application of the Gaussian mechanism. The authors extended the FedAvg and FedSGD algorithms to provide differential privacy guarantees.
Related Works The first-level combination of FL and HE has been initiated by researchers. The main purpose of HE in the context of privacy preserving is to safeguard the leakage of gradients, thereby by enabling secure aggregation during the learning process. Table  4  presents the different selected works, presenting the key ideas and the shortcoming. [63]  introduced BatchCrypt, a solution that reduces encryption and communication overhead when applying HE in cross-silo FL. The authors proposed a batch encryption technique where clients encode a batch of quantized values of gradients to a long integer and encrypt it. The main challenge addressed in their work is finding a feasible batch encryption scheme that allows direct summation of ciphertexts without intermediate decryption. To achieve this, they proposed a novel encoding technique using quantization of gradients.
Zhang et al. in 2020 Fang and Qian in 2021  [64]  introduced a multi-party privacy-preserving machine learning framework called PFMLP (private federated multi-layer perceptron). This framework is based on partially homomorphic encryption and federated learning to protect privacy. The main objective is to mitigate membership inference attack. The authors proposed to counter such attack by hiding the shared gradients from the server using HE. In order to reduce the computational overhead of homomorphic encryption, they proposed to use an improved version of the Paillier scheme described by Jost et al. in 2015  [65]. Using this version, they speed up the training by 25-28% compared to the initial version of the Paillier scheme  [30]. The authors conducted experimentation on MNIST and fatigue datasets and demonstrated that PFMLP achieves the same accuracy as the standard MLP (multi-layer perceptron) without HE.
The work was compared to BatchCrypt  [63]  and the results show a speedup of up to 20 times while obtaining a similar accuracy to non-secure, uncompressed FedAvg. Shin et al. [68]  noticed that previous works do not protect the dataset size of each client. This information can inadvertently reveal sensitive data, such as the number of patients in the local hospital, rare diseases among the regions, etc. They considered a healthcare scenario and proposed a protocol for private federated averaging for the cross-silo setting using partial homomorphic encryption based on the Paillier scheme. In their protocol, each client interacts with a randomly selected neighbor to send the encrypted calculation result, instead of sending them to the server. The final result is then sent to the server for decryption.
Jin et al. in 2023 Use a doubly homomorphic secure aggregation by using homomorphic encryption and masking technique. Computational and communication overhead  [68]  2022 Additive Additionally to previous work, protect the dataset size by adding interactions between clients using homomorphic encryption. Computational and communication overhead  [69]  2023 Additive Encrypting only a part of the model instead of the whole model. They showed that encrypting just 10% of the model parameter using a robust selection mechanism is efficient to counter DLG attack. Need for theoretical analysis of the trade-offs among privacy guarantee, system overheads and model performance.
Related Works Each technique has its own advantages and drawbacks in the context of privacy and security in federated learning. However, by combining these two techniques, we can potentially mitigate the drawbacks of each and achieve more comprehensive privacy protection. Several works have tried to combine these two techniques; Table  5  presents the different selected works, presenting the key ideas and the shortcomings. Xu et al. in 2019  [70]  proposed HybridAlpha, an FL framework that combines additive homomorphic encryption with differential privacy. The goal is to limit inference attacks from a curious aggregator during the process of learning and when using the final model. The system consists of a third-party authority (TPA) that generates the keys and distributes them, as well as an Inference Prevention Module.
The module examines requests for private keys for specific vectors that may allow a specific curious aggregator to make an inferenceenabling inner product. Hence, after receiving public keys from the TPA, the client will use LDP to protect their model updates from the server and then encrypt them. The server will then accomplish the aggregation before decrypting the data. The experimental results show that HybridAlpha can reduce the training time by 68% and data transfer volume by 92% while having similar privacy guarantees or model performance compared to existing works that use SMC, DP, and HE. Wang et al. in 2020  [71]  proposed two protocols to improve the utility of the data while guaranteeing better privacy. They proposed to build their solution based on the shuffler model proposed in Prochlo  [72].
In their work, each client applies successive transformations to achieve DP (clipping, noising, and quantization) then encrypt the data using HE before sending them to the server. HE protects the data from the semi-honest server, which performs calculations in an encrypted domain, while DP protects the data from the malicious clients. The challenges raised in this work are the computation cost of HE and the noise generation in DP. To decrease the computation cost, the authors suggest to use fixed-point numbers with a limited number of bits instead of floating-point numbers. They propose a new probabilistic quantization operator called "Poisson quantization" to handle the noise generation in a distributed manner, preventing the server from sharing the noise with other clients. In order to prove the feasibility of this framework, the experimentation was conducted using the FEMNIST dataset, a largely used dataset in previous works on federated learning.
Discussion While federated learning (FL) is often recognized as a technique that inherently protects privacy, it can still fall prey to numerous privacy attacks, as discussed in Section 3. The process of exchanging gradient updates across participating nodes in FL might inadvertently lead to potential privacy leaks. These leaks can expose sensitive aspects of the client's private data even without directly sharing the actual training datasets. This vulnerability is amplified due to the large number of participants involved in FL and the transparency of the framework's operations, which could provide ample opportunity for adversaries to launch powerful attacks. In an effort to mitigate these vulnerabilities, our research highlights the potential of two techniques: differential privacy (DP) and homomorphic encryption (HE).
[4]. They summarized works dealing with the representation of the urban surface in GIS platforms, listing the calculated parameters, type of software (commercial or open source), type of spatial data used (vector or raster), and how the city (as a real or a hypothetical arrangement) and the results (scale of analysis) were represented. The main urban parameters investigated were surface cover and urban structure by area ratios (plan area ratio/plan area index, building coverage ratio), height and volume of buildings, urban density, and roughness variables (frontal area index, roughness length, zero-plane displacement height). In the present research, some of those morphological parameters (i. e. , mean building height, aspect ratio, sky view factor, and aspect ratio) are calculated, and results are compared in terms of similarities and differences among the investigated cities located in southern (Bari and Lecce), central (Naples and Rome), and northern (Milan) Italy.
Estimation of Morphological Parameters Using GIS The morphological analyses have been conducted for the selected cities, employing the Geographic Information System (GIS) software QGIS 3. 22. 1 (https://www. qgis. org/en/site, accessed on 14 December 2022). Hence, morphological and land use data have been collected in shapefiles and analyzed. In addition, for the city of Lecce, the Digital Elevation Model (DEM) has been employed to calculate building heights. Different institutional sources have been used for collecting the relevant data: Note that a region of interest has been identified for each city selected. The dimensions of the region of interest have been set depending on the selected city size. Therefore, a 10 km × 10 km area has been used for the cities of Lecce and Bari (referred to as "Lecce region" and "Bari region" hereinafter), while a 20 km × 20 km area has been used for the cities of Naples, Rome, and Milan (referred to as "Naples region", and so on).
Limitations and Future Morphological Works An advantage of the methodology presented in this paper is that it employs only two sets of input data (building heights and grid maps) based on the actual shape of buildings to calculate the morphological parameters of a given city or region. Hence, for its application, it is necessary to collect appropriate and accurate datasets of building geometry to apply zonal statistical functions and calculate morphological parameters. Looking at Figures  4  and 5  and Tables  2 3 4 , larger spatially-averaged values can be expected in the tables, especially for AR and λ p for the Naples, Rome, and Milan regions in CLC 1. First, it should be noted that results do not refer only to the city but to a region of 20 km × 20 km that includes, in some cases, nearby towns, which are less dense than the main city.
This may be due to the small grid cells (50 m × 50 m), which experience a larger number of values close to 0 (cell without buildings) and 1 (cell fully occupied by buildings) as confirmed by the high standard deviation. These preliminary results suggest that using a grid resolution finer than 50 m × 50 m will produce lower differences with respect to the 100 m × 100 m grid resolution and thus building a dataset of parameters averaged over cells of 100 m × 100 m may be an appropriate compromise to calculate parameters over large areas. These preliminary findings will be explored in future work, performed by conducting the grid sensitivity test to the other regions of the cities investigated. Finally, it is worth mentioning that there are several methods in the literature for calculating SVF  [31].
The morphological parameters are H, SVF, AR, λ p , impervious surface fraction, pervious surface fraction, and terrain roughness class  [15]. There are three different levels of LCZ maps. The World Urban Database and Access Portal Tools (WUDAPT, www. wudapt. org, accessed on 19 December 2022), which uses freely available Landsat imagery and Google Earth for creating the training areas (TAs), provides a procedure for generating the LCZ level 0 product. Level 0 represents the first level of information, while Levels 1 and 2 provide more detailed information  [40]. The European LCZ level 0 map from Demuzere et al. [41]  is available for download as a GeoTiff at the WUDAPT database. There are other approaches for generating LCZ level 0 products that can also lead to detailed Level 1 and 2 data.
Conclusions Morphological analyses have been conducted for five cities characterized by different climate, structure, and size, located in southern (Lecce and Bari), central (Naples and Rome), and northern (Milan) Italy. The calculation of four morphological parameters, namely mean building height, sky view factor, aspect ratio, and plan area index, has been performed using the GIS software QGIS. The methodology employs only two sets of data as input, i. e. , building heights and footprints, and is based on the creation of a vector grid covering the study area to compute zonal statistics thanks to the appropriate module of QGIS. This methodology can be replicable in other urban contexts. Results have been presented in maps for selected regions of interest, including the cities and the respective surrounding areas.