Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Federated learning based on client-side uploading of spiking neural networks (SNNs) is considered. Each client's SNN model parameters are obtained by converting locally updated regular NNs (called ANNs here). The server obtains and downloads the aggregated global ANN parameters from the SNN parameters uploaded from the clients. Experiments show performance improvements and enhanced robustness against attacks, compared to baselines.
strength_and_weaknesses: Strengths: timely topic (enhancing robustness of FL under privacy attacks) and interesting solution.

Weaknesses: I am confused with the very basic premise here. Each client uploads the updated SNN weights and the BN parameters. The server then converts them to the corresponding local ANN weights before aggregation. So if the eavesdropper can access to the uploaded parameters of a client, then she can reproduce the ANN model weights of that client (as the serve does it easily). So, where's the protection against privacy threats that the ANN-SNN conversion process is supposed to bring to the table?  
clarity,_quality,_novelty_and_reproducibility: I am stuck with the very basic premise of the paper as given above, but other than that the paper is clearly presented. The method is properly motivated, and the work is original given that my question can be resolved. 
summary_of_the_review: The paper deals with a timely topic (enhancing robustness of FL under privacy attacks) and appears to provides an interesting solution. However, I am confused with the very basic premise here. Each client uploads the updated SNN weights (converted from the updated ANN) along with the BN parameters. The server then converts them to the corresponding local ANN weights before aggregation and downloading. So if the eavesdropper can access the uploaded parameters of a client, then she can reproduce the ANN model weights of that client in a given iteration (as the serve does it easily). So, where's the protection against privacy threats that the ANN-SNN conversion process is supposed to bring to the table? My evaluation score would be a tentative one and I will reevaluate after/once my confusion gets cleared.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a privacy-preserving framework (SNFL) in federated learning by incorporating the conversion from artificial neural networks to spiking neural networks. SNFL is validated to robust to gradient inversion attack and backdoor attack and it guarantees privacy while improving accuracy.
strength_and_weaknesses: Strength: 
This paper is somewhat novel, as it provides a solution to break the privacy-utility tradeoff. 

Weakness: 
- 1 One major weakness is that the most relevant related works are not well discussed and compared, which weakens the novelty and solidity of this paper. Before this submission, there are existing works about spiking neural networks in federated learning and privacy-preserving spiking neural networks, such as
①	Venkatesha Y, Kim Y, Tassiulas L, et al. Federated learning with spiking neural networks[J]. IEEE Transactions on Signal Processing, 2021, 69: 6183-6194.
②	Yang H, Lam K Y, Xiao L, et al. Lead federated neuromorphic learning for wireless edge artificial intelligence[J]. Nature communications, 2022, 13(1): 1-12.
③	Kim Y, Venkatesha Y, Panda P. PrivateSNN: Privacy-Preserving Spiking Neural Networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022, 36(1): 1192-1200.

The authors did not compare the algorithms about directly incorporating SNN in FL and discuss the accuracy gains and privacy protection in this case. It is likely that the accuracy gains of SNFL are caused by the SNN itself, not the ANN-SNN conversion. Additionally, the author claims that the privacy guarantee of SNFL lies in “the server of SNFL keeps bias as a private key and does not share it with clients”. But in the vanilla FL with SNN, one can keep the bias of SNN as private keys in the server and clients, and it also can realize better privacy protection, so what is the difference and novelty of SNFL when compared with this case? 

Besides, the author did not compare the privacy-preserving SNN directly adopting in FL, and they did not discuss the existing works about privacy issues in SNN. As in “Kim Y, Venkatesha Y, Panda P. PrivateSNN: Privacy-Preserving Spiking Neural Networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022, 36(1): 1192-1200.”, they mentioned the privacy leakages in ANN-SNN conversion. Since the SNFL is just a simple adaptation of ANN-SNN conversion in FL, will these leakages happen in SNFL?

- 2 Another major concern of this paper is that the authors did not well support their claims in certain aspects. Specifically, the authors mentioned “SNFL introduces a lightweight overhead” in the abstract, but the computation overhead of the conversion is not discussed in the main paper. And I reckon the overhead is of particular importance, because the conversion process is conducted at the clients’ sides. While the clients’ devices are always resource-constrained, such a computation of conversion may cause extra burden to the clients. The authors should fully discuss the overhead and compare it with other privacy-preserving methods to show its superiority. By the way, I am not sure whether SNN is lighter than ANN in parameter space, and if SNN introduces more parameters, it will also cause extra communication overhead, which should to be claimed or discussed.

The main point of this paper is that SNFL can provide better privacy protection. However, I find the experimental part about privacy is weak and short. It seems that the author focused more on the accuracy rather than privacy in the experiments.

- 3 The authors need give more insights on their method. Such as, why SNFL works better when the number of clients is larger. And for the privacy parts, I reckon it is not direct for the readers to get why your framework can better protect privacy. More empirical findings, like visualization and case study, or more theoretical findings are needed.

- 4 Apart from the technical parts, the paper is somewhat poor in presentation. As I can see, there are many typos, including wrong spelling of words, incomplete formula symbols, wrong method names, mistakes in citations of tables, and so on. Also, some sentences are also confusing, and I am always lost in what the authors are trying to convey. For example, in the first paragraph of Section 4.2, it says “Considering that μc(e) is dependent on the data distribution rather than distribution free, we cannot use random dataset on the server side for the recovery. Our framework can take advantage of this algorithm to encrypt the gradient information, therefore improving the privacy in FL.”. I don’t see the logic behind the two sentences, and it is confusing why the authors mentioned the calibration dataset on the server. Did they use the calibration dataset on server or not? The authors should correct all these typos in the revision and present better in writing.


clarity,_quality,_novelty_and_reproducibility: This paper has some merits in certain aspects since the ANN-SNN conversion is novel in federated learning. However, the authors did not support their claims well and some extremely relevant related works are not well discussed and compared, which weakens the novelty and solidity of this paper. Apart from the technical parts, the paper is somewhat poor in presentation.
summary_of_the_review: Overall, this paper doesn’t meet the criteria of publication in ICLR. The major concern is that the authors did not support their claims well and some extremely relevant related works are not well discussed and compared, which weakens the novelty and solidity of this paper. I recommend to reject it and hope the authors can do better in the future version.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: A framework for private federated learning is proposed. The conversion from ANN to SNN includes the encryption property for privacy protection. The results show competitive performance for IID and non-IID cases.
strength_and_weaknesses: Strengths:
1. The tackled problem is relevant to the ICLR community.
2. The results look promising and competitive.

Weaknesses:
1. There are several typos throughout the text. Please revise and correct.
2. The proposed method combines multiple techniques together. Besides such a combination, each individual technique seems strongly influenced by previous works. It is recommended to discuss in more detail the key differences between the related works and the proposed method.
3. The table numbering in the text does not match the numbers in the table captions.
4. Please specify which dataset has been used for generating the results reported in Table 3 and Figure 3.
5. It would be useful to provide the source code for reviewers' inspection during the rebuttal.
clarity,_quality,_novelty_and_reproducibility: Clarity: 5/10

Quality: 5/10

Novelty: 4/10

Reproducibility: 5/10
summary_of_the_review: The idea of the paper is interesting but several concerns should be clarified.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a privacy-preserving framework (SNFL) in federated learning by incorporating the conversion from artificial neural networks to spiking neural networks. The reviewers appreciate the efforts of the authors. However, they found that most of the problems are not well addressed in the revision. The reviewer suggests to study deeper and do more convincing comparison with related work. The current version is not well discussed and compared, which weakens the novelty and solidity of this paper. Moreover, the proposed method combines multiple existing techniques. The novelty is quite limited, and the advantages and contributions are not clear. 

Please take the comments and suggestions from the reviewers in their detail reviews to improve the manuscript for the future submission. 

justification_for_why_not_higher_score: - The proposed method combines multiple existing techniques. The novelty is quite limited, and the advantages and contributions are not clear. 
- The current version is not well discussed and compared, which weakens the novelty and solidity of this paper.
Decision: Reject