Federated Black-Box Adaptation for Semantic Segmentation

Authors: jay n paranjape
shameema  sikder
s  swaroop vedula
vishal m patel
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-12 05:52:59

Abstract: Federated Learning (FL) is a form of distributed learning that allows multiple institutions or clients to collaboratively learn a global model to solve a task. This allows the model to utilize the information from every institute while preserving data privacy. However, recent studies show that the promise of protecting the privacy of data is not upheld by existing methods and that it is possible to recreate the training data from the different institutions. This is done by utilizing gradients transferred between the clients and the global server during training or by knowing the model architecture at the client end. In this paper, we propose a federated learning framework for semantic segmentation without knowing the model architecture nor transferring gradients between the client and the server, thus enabling better privacy preservation. We propose \textit{BlackFed} - a black-box adaptation of neural networks that utilizes zero order optimization (ZOO) to update the client model weights and first order optimization (FOO) to update the server weights. We evaluate our approach on several computer vision and medical imaging datasets to demonstrate its effectiveness. To the best of our knowledge, this work is one of the first works in employing federated learning for segmentation, devoid of gradients or model information exchange. Code: https://github.com/JayParanjape/blackfed/tree/master

Main paper: 
 introduction: 
 With data-driven methods becoming immensely popular in Artificial Intelligence (AI) research and applications, there has been a surge in data collection and curation across the world. This has, in turn, led to the development of AI models that require substantial amounts of data to train. Federated Learning (FL) [34] was developed as a viable approach towards training such models by effectively harnessing the data collected at different centers across the world. Through FL, it becomes possible for multiple institutions to collaborate and build a joint model that learns from all of them, while reducing the burden of collecting more data individually. However, collaborations between different institutions present a non-trivial challenge due to disparity in data distributions as well as the imperative of safeguarding data privacy. Consequently, FL aims to jointly learn a shared model that performs well on data from all participating institutions without exchanging raw data. Various FL approaches have been proposed in the literature [1, 10, 12, 16, 25, 22, 35, 37, 41, 44, 55, 56, 30, 34, 29, 31] . Among Figure 1 : Comparison of our method against traditional FL methods. Existing FL methods are primarily "white-box" as they involve transfer of model weights [34] , or gradients [20] . In contrast, our method only utilizes forward passes to update the client and does not require sharing weights or gradients, making it a "black-box" model. these, FedAvg [34] was one of the pioneering FL methods, which proposes training local models at every center using local data and periodically averaging the local model weights to craft a global server model. Various subsequent works improve FedAvg by improving the local model updates [30, 31] or global updates [24, 29] .While FL was primarily proposed for classification tasks, it is also suited for other computer vision tasks such as segmentation. Generating annotations in segmentation entails creating pixel-level annotations per image, that are more tedious to label than classification tasks. Consequently, it is not always possible for a single institution to collect a large amount of data, underscoring the importance of collaborative efforts. A few approaches have been proposed in the literature for FL-based segmentation [36, 5, 18, 43, 53, 14] . However, all these methods for segmentation and classification, while not involving raw data transfer, employ techniques like model information transfer [34, 30, 36] or gradient transfer [24, 20] , as shown in Figure 1 . However, recent research has revealed that these techniques are vulnerable to attacks that can recreate training data from the participating centers, thus undermining the privacy preserving characteristic of FL [11, 54, 17, 57, 27, 26, 58, 15, 23] . These attacks employ methods like gradient inversion [54, 17, 57] or adapting model architecture and weights [11] . In this work, we propose a new approach, named BlackFed, for segmentation using FL that does not involve gradient transfer between the server and the client and at the same time, passes no knowledge about the client model architecture to the server, thereby avoiding the necessary conditions for these attacks, as shown in Figure 1 . This is done by formulating the FL learning problem as a distributed learning problem using split neural networks (split-nn) [20] and combining first order and zero order optimization techniques for training. Our contributions are as follows:1. We introduce BlackFed -a black-box algorithm that facilitates distributed learning for semantic segmentation without transferring model information or gradients between the client and the server. For this, we formulate the FL problem using split-nn and use first and zero order optimization for training the server and the clients, respectively. 2. We suggest a method to reduce the effect of catastrophic forgetting in BlackFed by retaining client-wise checkpoints in the server. 3. We evaluate the proposed approach on four segmentation datasets and show its effectiveness as a distributed learning method by showing improvements over individual training.tumor segmentation by utilizing data from multiple institutions. FedSM [38] attempts to mitigate the effect of non-iid nature of the data from different centers on the global server model. However, most of the approaches for medical segmentation focus on the problem of segmenting out the foreground from the background (one class problem). There are relatively fewer works in the literature for multi-class segmentation [14, 36, 5, 18, 43, 53] . FedSeg [36] deals with the class label inconsistency problem that may be present at the local clients. In other words, it builds a robust system that works well even when the clients have annotations for only a subset of the classes. FedDrive [14] , on the other hand, sets up various benchmarks for FL algorithms on multi-class datasets like Cityscapes [9] . However, all the existing algorithms require the global server to know the model architecture used in the clients and thus, these methods are vulnerable to recent attacks [11] . In our work, we develop an algorithm to perform multi-class segmentation which does not require gradients or client model sharing.Split Neural Networks: Split networks [20] were introduced as an alternative to FedAvg-like techniques which require model sharing. They offer an approach to perform collaborative learning by splitting a larger network into two segments. The latter segment of the network is shared across all centers and placed at the global server, while the former part is distributed such that each center possesses its own sub-network. During training, the clients perform a forward pass using their data and send the encoded features to the server, which further passes the features to the subsequent layers in the network. The server and client models are trained using backpropagation, where the gradients from the first layer of the server model are sent back to each of the clients. Split networks have mainly been used in the literature for the task of classification, mostly in the medical domain [20, 40, 47] . Our work marks one of the initial explorations of split learning in the context of semantic segmentation. Furthermore, split networks are shown to be more robust to reconstruction attacks which arise from sharing model information [48] . Nonetheless, they remain susceptible to gradient leakage attacks since there is gradient transfer between the server and the client. In this work, we formulate the problem of distributed semantic segmentation using split network and introduce a training algorithm that does not require gradient-sharing. 
black-box adaptation: 
 In the following section, we define the problem of semantic segmentation under the federated setting. Then, we describe our proposed formulation for the black-box setting and the algorithm.The setting of FL consists of N clients, denoted by C i , where i ∈ 1, 2, ..., N and a global server G. Each of the clients has a local dataset, consisting of images X i = x i j ∈ R H×W ×C ; j ∈ {1, 2, ..., n i } and their corresponding ground truth segmentation maps Y i = y i j ∈ R H×W ×Nc ; j ∈ {1, 2, ..., n i } . Here, x i j and y i j represent the j th image and its corresponding segmentation mask, while n i represents the number of data points in client i and N c represents the number of classes in the output. Note that the distributions of the input images vary among the clients. Each client uses its dataset to learn a function f i : R H×W ×C → R H×W ×Nc , which is parameterized by Θ i , that minimizes its local loss function as follows:EQUATIONwhere l denotes the loss per data point. The global server, on the other hand, tries to aggregate the knowledge of all the clients by fusing the parameters of the individual clients to optimize a joint loss function as follows:arg minEQUATIONA commonly used algorithm, called FedAvg [34] , proposes a linear combination of client parameters to optimize the loss function, defined in Eq. 2, as follows:EQUATIONHowever, this formulation requires the server to be aware of the exact model architecture or function of the client, which can results in data leakage [11] .Proposed Problem Formulation: In this work, we model the problem of distributed learning using a split neural network, which takes away the requirement of the server being aware of the client architecture. In this case, each client learns a function f i : R H×W ×C → R H ′ ×W ′ ×C ′ , which is parameterized by Θ i . Similarly, the global server learns a function g : R H ′ ×W ′ ×C ′ → R H×W ×Nc , which is parameterized by Φ. Thus, the forward pass for a given center is given as follows:EQUATIONwhere ŷi j denotes the predicted segmentation map. Hence, the objective function of a given client is as follows:EQUATIONAs in the existing FL literature, the goal of our approach is that after training, all clients should benefit from each other. Hence, during evaluation, given any client, we aim to perform well on data from other clients as well as its own data, thus showing good generalization. More specifically, we want to optimize any combination of data and client as follows:EQUATIONOne way to optimize this equation is by attending to every client in a round-robin fashion. This involves selecting a client, performing the forward pass, as defined in Eq. 4 and then performing backpropagation to update the server and client using the client loss function defined in Eq. 5.Performing this operation several times in a round-robin fashion enables the server to learn from all client sources, and hence, improves the overall performance. We term this method as "White-box Round-Robin FL", and is similar to the existing methods in the literature [24, 20] . However, recent works have shown that such a method which involves transfer of gradients between the server and client can be utilized to regenerate the training data, thus undermining the privacy preservation principle of FL [54, 23] . Hence, we add one more constraint -i.e. no gradient can flow back from the server to the client in Eq. 5.BlackFed Algorithm: To optimize the clients without using gradients, we utilize a ZOO method called Simultaneous Perturbation Stochastic Approximation with Gradient Correction (SPSA-GC) [39] which involves perturbing the weights of the client model slightly and approximating a two-sided gradient based on the change in the loss function due to the perturbations. However, this method was developed to perform black-box adaptation of pretrained foundation models and hence, expects the server model to be initialized with good weights, which is not the case in our formulation, making it non-trivial. To overcome this, we propose to iteratively use an alternating optimization technique, which factorizes the optimization problem in Eq. 5 into two optimization problems as follows:EQUATIONDuring training, we first select a client using the round-robin policy. Next, keeping the server weights fixed, we train the client weights using SPSA-GC for a few iterations. Next, we fix the client weights and use a first order optimizer (i.e. Adam-W) to optimize the server for a few iterations. This process is repeated multiple times. During inference for a client, we simply run the forward pass as described in Eq. 4 using the final weights of the client and server. The training process is described in Algorithm 1. We refer to this approach as BlackFed v1.Input: (i) N, number of clients (ii) c_e, number of epochs to train one client (iii) s_e, number of epochs to train server (iv) runs, number of complete round-robin runs n ≥ 0 Output: (i) Φ, server model weights (ii) Θ, array of client model weights BlackFed (N, c_e, s_e, runs) Begin: initialize Φ, Θ for r = 1,2,...,runs do for i = 1,2,...,N do for j = 1,2,...,c_e doΘ[i] ← SPSA-GC(Θ[i]) //Zero order optimization for client end for for j = 1,2,...,s_e do Φ ← Φ -η s ∂L i ∂Φ//First order optimization for server, where η s is the learning rate end for end for end for Return: Φ, Θ End Reducing the Effect of Catastrophic Forgetting: Since the model at the server is shared among all the clients and is updated in a round-robin fashion, it may happen that training with the data from a given client may cause it to unlearn patterns for the previous client. This phenomenon is often called catastrophic forgetting. This effect is observed in BlackFed v1 especially when the number of clients increases or if there is a significant change in the data distribution among clients. This causes the algorithm to perform well on certain clients and poorly on the rest of the clients. To reduce the effect of catastrophic forgetting, we propose a simple additional step during training with Algorithm 1. After updating the server weights for a given client during training, we store the updated weights of the server model in a hashmap indexed by the index of the client. During inference for a given client, we use the latest weights of the client model and the indexed weights of the server model to perform the forward pass. Note that the server state is loaded from the hashmap only during inference. During training, the server still benefits from the data from all clients and updates its weights as well as the hashmap. This approach is visualized in Figure 2 . We refer to this method as BlackFed v2.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] Justification: We have provided the complete mathematical formulation and the algorithm, but theoretical claims are not made. Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
experiments and results: 
 Datasets: For evaluating our method, we consider four publicly available datasets, namely (i) Cityscapes [9] (ii) CAMVID [4] , (iii) ISIC [21, 8, 45] and (iv) Polypgen [2] . Cityscapes and CAMVID are two road-view semantic segmentation datasets with 19 and 32 classes of interest respectively, collected from multiple cities. In the FL setup, we consider each of the cities as separate clients. While CAMVID has predefined train, test and validation splits, for Cityscapes, we divide the data from each client into training, validation and testing splits for that client in a 60:20:20 ratio. In this manner, we generate 18 clients for Cityscapes and 4 clients for CAMVID. Further details about the dataset size in each center is provided in the supplementary document. The ISIC dataset corresponds to a skin lesion segmentation challenge. We generate three clients for this dataset, which utilize the datasets from ISIC 2016 [21] , ISIC 2017 [8] and ISIC 2018 [45] , respectively. The data every year is collected from different centers and hence, has different distribution amongst centers. Finally, PolypGen is a colon polyp segmentation dataset which was collected from six different centers. The training, validation and testing splits for each of these were done in a 60:20:20 ratio for Polypgen whereas they were already provided for ISIC. We visualize the pixel-intensity histograms of the CAMVID and ISIC datasets to verify different data distributions amongst clients in Figure 3 .Experimental Setup: We use a two-layer convolutional network for modeling f in the client and DeepLabv3 [7] for modeling g in the server. The number of output channels from the client is kept Then (a) client performs a forward pass using its part of the network (b) Server performs a forward pass using its part of the network (c) With server weights fixed, client weights updated using ZOO (d) Keeping client weights fixed, server weights updated using FOO (e) The best server weights are stored in the hashmap corresponding to client index. During inference, the client performs a forward pass and calls the server with the output. Server queries the hashmap using the client index and gets its set of weights, using which the prediction is obtained. Note that there is no gradient transfer, thus making this a black-box setup. Results: Given the trained client models and the trained server model, we assess a given client's performance with test datasets from its own local data repository and present the mIoU in the "Local" column. This metric represents the local performance of the model on its own dataset. In addition, we assess each client on test data from other centers and note down the average mIoU scores in the Out-of-Distribution ("OOD") column. This metric serves as an estimate of the general performance of a given client post-training. We consider the latter more important as it can be considered to be the direct outcome of the FL paradigm. These results are presented in Table 1 for CAMVID and Cityscapes, and in Table 2 for ISIC and Polypgen. For both tables, in the first row, each client is trained on its own dataset, independently of others, indicating no collaboration. For this case, while the client performance on its own test set is high, its general performance suffers. The next two rows represent our method (BlackFed v1 and v2). Notably, BlackFed v2 generally exhibits better performance than v1 since it addresses the catastrophic forgetting that occurs during training. The following three rows represent the expected upper bounds to our performance. "Combined Training" represents the scenario where raw data can be freely shared and a single model is trained using the combined data from all clients. "White-box training" represents the case where gradients can flow freely between the server and the client. Thus, instead of the ZOO optimization, we use FOO to optimize the client part of the model. Finally, the last three rows represent the performance of traditional FL using FedAvg, and recent methods like FedPer and FedSeg, where model sharing is allowed. Here, all the clients follow the same model architecture as the server (DeepLabv3) and the server can aggregate weights from the client models. All three methods represent a relaxation over the constraints imposed in our approach, thus acting as an upper bound to the black-box performance.During training, the client and server models are trained for the same number of epochs per client. This produces a more uniform distribution of results across centers, in contrast with traditional FL methods like FedAvg, where the aggregation of weights is weighed by the client dataset size. We observe that for the Polypgen dataset, BlackFed v1 and v2 perform slightly better or on par with individual client performance for OOD case. For this case, there is little difference between the performance of v1 and v2. This behavior may be related to the data distribution of Polypgen and suggests that BlackFedv2 is not able to correctly avoid the catastrophic forgetting for centers C5 and C6. However, for rest of the scenarios, we see that v2 significantly outperforms v1 as well as the individual training cases on OOD mIoU metric. At the same time, the performance on local data does not suffer significantly as compared to the individual training. Moreover, BlackFed v2 performs on par with "Combined" and "White-box" Training. All results of BlackFed have a p-value less than 0.001, showing the statistical significance of our black-box approach. The visual comparisons for our method with the individual training is given in Fig. 4 . As can be seen in all four rows, individual training can lead to overfitting, which harms the general OOD performance. Using our method, we are able to improve the OOD results across all datasets without significant decrease in the local results.Additional Model Architectures While we evaluate a DeepLabv3-based server in our main experiments, we show the effectiveness of our method on additional segmentation architectures. More specifically, for the CAMVID dataset, we choose UNext [46] and SegFormer [51] as the server models and present average mIoU results across the test datasets from each client in Table 3 . As can be seen from the table, using our approach can improve the performance over individual training for all three models. As the model complexity increases from UNext to DeepLab to Segformer, we Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: We will be releasing the code, data splits and pretrained models after review. The algorithm for the approach is included in the paper. The experimental settings are also provided in the main paper. Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
ablation studies: 
 Analysis of the Order of Optimization: In the alternating optimization proposed in Eq. 7, we choose to first update the client followed by the server. This order is important since it allows us to store the correct server weights in the hashmap. If the server is trained before the client, we found that SPSA-GC often gives unstable results and reduces the metric on the validation set after an epoch. This is corrected only when its the turn of the same client again. Conversely, in the case where the client is updated first, the server adapts in a stable manner to the client weights since backpropagation is allowed in the server. Consequently, in the same epoch, we get a higher and more stable performance for each client, which can be saved in the hashmap for usage during inference. We demonstrate this empirically by comparing the performance of the two training strategies in Table 5 .Analysis of Computational Cost: For each of the three model architectures, namely DeepLabV3, UNext and Segformer, we calculate the floating point operations required in a single forward pass at the institution end. These are shown in Table 4 . If the clients were to train individual models with their local data, they would require running the entire forward pass on their local systems. This is also the case in existing FL approaches like FedAvg. In the proposed approach, as can be seen from Row 3 in the table, the client has a significantly reduced load, with the majority of computation being offloaded to the server. The server uses the respective architectures starting from the second layer, while all the clients use a two-layer convolution network, with the second convolutional layer being similar to the first layer of the respective architecture. The BlackFed algorithm takes in two hyperparameters, namely c_e and s_e, that determine the number of epochs for training the client and server, respectively. We conduct experiments for observing the effect of changing these parameters on the model performance. We use the CAMVID dataset with server architecture being DeepLabv3 and note down the average mIoU score in Table 6 for different values of c_e and s_e in {10, 20, 30, 40, 50}. Interestingly, we observe that there was no significant difference in performance by increasing the number of server epochs or client epochs. While increasing the server epochs can improve training, it also makes the server more specific to a given client, thus reducing average performance. On the other hand, more ZOO-based iterations marginally improve the model. In this context, ZOO primarily serves to guide the server towards a more robust minima, resilient to perturbations in client features. Consequently, a greater number of client epochs does not notably impact model performance.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: The code and data splits will be released on Github post review.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
conclusion: 
 In this work, we introduce BlackFed, an FL algorithm that enables distributed learning without transfer of gradients or model weights. This characteristic distinguishes our approach as a black-box model compared to existing FL methods, which can be considered as white-box approaches. Recent research on attacking FL methods require the knowledge of either the gradients or the model information, thus rendering BlackFed more resistant to such attacks since gradients or weights are not shared. BlackFed consists of a global server and multiple clients, each possessing their own data. The server is optimized using first order optimization while the client weights are updated using zero order optimization in a round-robin fashion. This introduces the effect of catastrophic forgetting in the network, for which we propose a simple hashmap-based approach. During training, we store the client weights per server so that they can be utilized during inference. With these modifications, our approach demonstrates superior results compared to non-collaborative training and matches the performance of white-box methods, despite being a black-box method itself. Extensive experimentation on the natural and medical domain datasets highlights the effectiveness of BlackFed. Through this endeavor, we aim to propel research towards the development of better privacy preserving federated learning systems. Potential directions for future research can include analysis of other policies for selecting client order during training, and its relation with the disparity in data distribution. One more interesting direction for future research would be the effect of adversarial attacks using generative models on this method. Since masks are shared between the client and server, it would be interesting to check if existing methods are able to recreate client data using them. A Centerwise Dataset InformationIn this section, we describe the number of data points in the training, validation and testing splits of each center of each dataset. The center represents the client in FL, where each center is in possession of data that cannot be shared directly with the other centers or to the public. CAMVID [4] has 4 centers, which are described in Table 7 . Cityscapes [9] has 18 centers, which are described in Table 8 . These centers represent the different cities from which the data is collected for these datasets. ISIC [8, 21, 45] has 3 centers and Polypgen [2] has 4 centers, which represent the different hospitals from which the data is collected. These results are present in Table 7 .Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: While the code will be released along with pretrained models, we have mentioned the experimental settings in the paper. We have also mentioned the data splits in the supplementary.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: Introduction discusses about our method's positive impacts. There are no apparent negative impacts. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: No risk posed by the work. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes]  
 
 Appendix: While the average mIoU is presented in the main paper, we also present the centerwise performance of DeepLabv3-based server for the Cityscapes dataset in Tables 9 and 10 . It can be seen from the tables that individual training performs poorly, the reason being that each of the centers has limited amount of data. In contrast, our method makes use of data from all clients to improve performance.Here, we also see that BlackFed v2 performs better than v1 in all cases, thus showing the effectiveness of the server hashmap in countering catastrophic forgetting. With our approach, the performance of the model reaches close to the white-box methods, but without sharing any gradients or model information. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 


Decision: Authors present a gradient-free federated learning approach in which knowledge of the client architecture nor the weights are not required by the server. The client and server networks are optimized differently. The main claim for the article is that this approach is more privacy preserving because the gradients and weights of clients remain at the client. However, representations and masks need to be transmitted to the server which may still cause some privacy issues. 

The main concerns for the rebuttal were on experimental analysis and the privacy concerns. In the experimental analysis, more comparisons and clarifications were required. For the latter, transmitting representations and masks seem to violate the privacy concerns. In other words, the proposed technique also cannot guarantee full privacy of the data. 

During the rebuttal authors successfully answered questions on experimental analysis. Questions on the privacy concerns remained open to the best of my understanding. Among four reviewers, three of them suggested to accept the article, two with BA and one with A. Only one reviewer suggested BR. However, I believe concerns this reviewer raised were addressed to some extent during the rebuttal. Unfortunately, the reviewer did not revise their score. 

Given the positive reviews, acknowledged technical novelty, successful rebuttal and the final ratings, I believe this article would be a good contribution to the conference.

Reviews: review: paper_summary: paper_summary: A common issue now with federated learning systems is that they are not completely private owing to gradients transferred between the clients and the global server during training or by knowing the model architecture at the client end. 

The paper proposed a workaround to this by removing the need for the passage of the gradients from client to server or knowing the model at the end. For the gradients, the authors propose zero-order optimization (ZOO) to update the client model weights and first-order optimization (FOO) to update the server weights. 

While achieving the above, they also perform reasonably well on most datasets.
strength_weakness: strength_weakness: 1. The case of polypgen (given the dataset size, medical background, and criteria of data collection) is where there is the most requirement/use of such a setup. And it underperforms over there. "This behavior may be related to the data distribution of Polypgen and suggests that BlackFedv2 is not able to correctly avoid the catastrophic forgetting for centers C5 and C6. ". --- I think this has less to do with catastrophic forgetting and more to do with the model just not able learn. The data points are very few across all the centers. Given how the latest weights are stored, the weights used are just poor as they are not able to learn appropriately given the tough case of low data centers. It may also be that it is overparameterised by the other centres. If it was the case of it being affected by not being able to solve catastrophic forgetting, this would have been reflected everywhere. Especially in C1 C2 of CAMVID.
questions: questions: 1. Why was the white-box training so subpar in ISIC? Further, while I am not well-versed in the literature, I imagine there are ways to tackle catastrophic forgetting in the white-box method as well. Would it be possible to do an ablation with that? Comparing V1 and white box, apart from ISIC, it beats V1 in all cases.

2. Given that CAMVID has a Significantly lesser number of classes and centers, why is the performance better/on-par (and more stable) in the Cityscape dataset for BlackFed but worse in the case of "individual"? Even considering the majority classes.
limitations: limitations: 1. "After updating the server weights for a given client during training, we store the updated weights of the server model in a hashmap".  - This step may increase significantly with scale, especially with multiple client updates. Lower floating point operations are indeed handy, but this is coming at the cost of high memory requirements.

2. Issues with performance in the most important setting.

Suggestion -

3. I think in Table 1 and 2, Bolding the highest value and underlining the highest value without gradients will be a more proper representation.
paper_summary: paper_summary: In this work, the authors introduce BlackFed, an FL algorithm that enables distributed learning without transfer of gradients or model weights. This characteristic distinguishes the approach as a black-box model compared to existing FL methods, which can be considered as white-box approaches. Recent research on attacking FL methods require the knowledge of either the gradients or the model information, thus rendering BlackFed more resistant to such attacks since gradients or weights are not shared. BlackFed consists of a global server and multiple clients, each possessing their own data. The server is using first order optimization while the client weights are updated using zero order optimization in a round-robin fashion. This introduces the effect of catastrophic forgetting in the network, for which the authors propose a simple hashmap-based approach. The white-box methods, despite being a black-box method itself. Extensive experimentation on the natural and medical domain datasets highlights the effectiveness of BlackFed.
strength_weakness: strength_weakness: The weakness is as follows
1. For Table 1, it's indeed the proposed methods work well. But there is not experimental result for deeplabv3 on a single machine. Deeplab v3 achieves 80.0 on Cityscape val dataset on https://paperswithcode.com/lib/detectron2/deeplabv3-1. The paper generates 18 clients for Cityscapes. What's the relationship between the Table 1 evaluation result and benchmark result on Cityscape? This may make the paper not convincing enough.
2. It's unclear why "As the model complexity increases from UNext to DeepLab to Segformer, we observe a decrease in individual training performance. ". 
3. The image size for Cityscape is  256 × 512. why using so small resolution? it needs to do more experiments on higher resolution.
4. What's the details for " Consequently, we start the DeepLabv3 network in the server from the second layer, which expects a 64-channel input. " ? Does it mean the deeplabv3 trained on server only unfreezed from the second layer? so how about the client?
questions: questions: No. Please see Weakness.
limitations: limitations: No. Please see Weakness.
paper_summary: paper_summary: This manuscript introduces a federated learning framework for semantic segmentation that neither requires knowledge of the model architecture nor involves transferring gradients, thereby preventing privacy leakage. BlackFed incorporates split neural networks and first/zero order optimization for training the server and clients, respectively.
strength_weakness: strength_weakness: (1)	In addressing privacy leakage caused by sharing weights and gradients in current federated learning methods, this manuscript proposes BlackFed, which iteratively updates client and server networks. However, this approach involves transmitting extracted features and labels between the client and server, leading to privacy leakage and security risks. For instance, some algorithms like stable diffusion [1] can reconstruct client images from these features and segmentation masks. Therefore, I believe BlackFed does not effectively protect privacy and fails to show superiority over methods based on network weights and gradients.

(2)	There are too few comparison methods. In the Related Work section, many federated learning methods for segmentation tasks are introduced; FedSeg and FedSM could be used for comparison.

(3)	BlackFed preprocesses client data using a two-layer convolutional neural network on clients, allowing different clients to extract global features. These preprocessed features are then sent to the server for segmentation. The client-side training follows the method proposed in [2], which limits the approach's innovation.

(4)	In the inference phase, the features extracted by the client network need to be uploaded to the server each time. Does this significantly increase testing time consumption?

(5)	FedPer [3] is another federated learning method using split neural networks. It is recommended to introduce it in the Related Work section and add comparison results in the experimental section.

[1] High-Resolution Image Synthesis with Latent Diffusion Models

[2] Blackvip: Black-box visual prompting for robust transfer learning.

[3] Federated Learning with Personalization Layers
questions: questions: (1) More comparison of Federated learning methods for segmentation should be performed;

(2) All experiments were conducted using DeepLabV3 as the server segmentation network. It is recommended to include experimental results on different segmentation architectures.
limitations: limitations: The motivation behind BlackFed is privacy protection. However, the feature maps extracted by clients and segmentation masks are also transmitted to the server, making them vulnerable to attacks that could reconstruct the images from different clients. Therefore, this method does not achieve the goal of privacy protection.
paper_summary: paper_summary: The authors proposed a black-box tuning method to address the problem of Semantic Segmentation in FL. Specifically, they propose to split the network into two parts and store them in server and clients separately. The server modules are optimized via first-order optimization while the client modules are optimized via black-box optimization. The proposed method achieves promising results on different benchmarks.
strength_weakness: strength_weakness: 1. From my understanding, the privacy problems still exist. Since the authors claimed that they are using a two-layer CNN at the client, which is relatively light-weight and compact compared to the network used at server, the attackers at server may still be possible to reproduce the training data using the intermediate output (the uploaded values from each client). The authors may need to add further discussion about this issue.
2. In the process 1b (Figure 2), how is the first-order gradient computed at the server? Are the ground-truth semantic maps uploaded to the central server? Or the network predictions will be downloaded to different clients and then evaluated? For the first solution, there will be privacy threatens. For the second one, the communication costs will be very large. The authors should clarify this in more details.
3. The authors introduce a new two-layer CNN at each client, could the network DeepLabv3 (the network at central server) directly be split in the clients and server?
4. It would be interesting to compare with other FL algorithms which also applies Black-box Tuning such as [1].

[1] Sun J, Xu Z, Yin H, et al. Fedbpt: Efficient federated black-box prompt tuning for large language models[J]. arXiv preprint arXiv:2310.01467, 2023.
questions: questions: Please refer to the questions in the Weaknesses section.
limitations: limitations: The authors did not address the limitations of the proposed method.

Comments: title: Reminder for the discussion
Rebuttal is online - please respond
comment: Dear reviewer,
Today is the last day for the discussion period. Please let us know of we were able to address all the concerns regarding the paper. If so, kindly consider raising the score.
Thanks for the author's comment. There is indeed no perfect federated method that can achieve perfect privacy protection. Since the main contribution of this paper lies in privacy protection, it would be beneficial to substantiate the privacy protection performance through experiments or mathematical analysis in the next version. Finally, I will increase my score to Broadline Accept.
Since we are approaching the end of the discussion period, this is a gentle reminder. Please let us know if you have additional questions so that we can address them in the next day during the discussion period.

We hope you would consider raising the score if we have addressed your concerns satisfactory

Thanks
We are happy that we could address most of your concerns. For the privacy protection concern, please note that we do not claim a perfectly attack-proof method in this work. In lines 42-52 of the main paper, we reference works that propose attacks using the existing FL frameworks. Hence, we propose a mechanism which would not satisfy the necessary conditions of gradient transfer / model transfer for these attacks. In the contributions section from lines 53-60, we clearly state that we propose a new framework for FL without gradient and model transfer, and do not claim that it would solve the privacy preserving problem completely.

However, we will be adding a discussion on this in the future work section that can encourage more research on attacks and defenses given the new framework. We believe that our work is an important step in the direction of FL algorithms which involve minimal transfer and it will encourage future research in this direction, diffusion-based attacks and defense being one such example.

We hope that you would consider raising the score if we address the present concern. 
Thanks
Thanks for the author's rebuttal. The majority of my concerns have been addressed. However, the explanation of BlackFed's privacy protection is not convincing to me, so I will keep my rating unchanged.
During the rebuttal, the authors have addressed most of my concerns. Therefore, I would increase my score to Broadline Accept.
I would chose not to change my decision (for better) based on the fact that - 1. Thinking of models in terms of 100MB is not fair. No production model is going to be that small. 

2. “ Hence, since the number of clients is more, on an average, the individual performance is lower.” - This is a fundamental  problem which again is not being addressed in any form.

The results overall are quite normal. However, given the novelty to form a gradient-free approach is why I proposed the accept and that is the only contributing factor.

Thank you for the detailed rebuttal.
Dear reviewer,

Thank you for the comments on our paper.

We have submitted the response to your comments and a PDF file. Please let us know if you have additional questions so that we can address them during the discussion period. We hope that you can consider raising the score after we address all the issues.

Thank you
Dear reviewer,

Thank you for the comments on our paper.

We have submitted the response to your comments and a PDF file. Please let us know if you have additional questions so that we can address them during the discussion period. We hope that you can consider raising the score after we address all the issues.

Thank you
Dear reviewer,

Thank you for the comments on our paper.

We have submitted the response to your comments and a PDF file. Please let us know if you have additional questions so that we can address them during the discussion period. We hope that you can consider raising the score after we address all the issues.

Thank you
Dear reviewer,

Thank you for the comments on our paper.

We have submitted the response to your comments and a PDF file. Please let us know if you have additional questions so that we can address them during the discussion period. We hope that you can consider raising the score after we address all the issues.

Thank you
Dear Reviewers, 

Authors carefully prepared their rebuttal - trying to address the concerns you have raised. Please check the rebuttals and join the discussion about the paper. 

Regards,