$\lambda$-SecAgg: Partial Vector Freezing for Lightweight Secure Aggregation in Federated Learning

Authors: Siqing Zhang, Wei Sun, Yong Liao, Peng Yuan Zhou
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Secure aggregation, Federated learning

Abstract:
Secure aggregation of user update vectors (e.g. gradients) has become a critical issue in the field of federated learning. Many Secure Aggregation Protocols (SAPs) face exorbitant computation costs, severely constraining their applicability. Given the observation that a considerable portion of SAP's computation burden stems from processing each entry in the private vectors, we propose Partial Vector Freezing (PVF), a portable module for compressing computation costs without introducing additional communication overhead. $\lambda$-SecAgg, which integrates SAP with PVF, "freezes" a substantial portion of the private vector through specific transformations, requiring only $\frac{1}{\lambda}$ of the original vector to participate in SAP. Eventually, users can "thaw" the public sum of the "frozen entries" by the result of SAP. To avoid potential privacy leakage, we devise Disrupting Variables Element for PVF. We demonstrate that PVF can seamlessly integrate with various SAPs and it poses no threat to user privacy in the semi-honest and active adversary settings. We include $7$ baselines, encompassing $5$ distinct types of masking schemes, and explore the acceleration effects of PVF on these SAPs. Empirical investigations indicate that when $\lambda=100$, PVF yields up to $99.5\times$ speedup and up to $32.3\times$ communication reduction.

Decision: reject
Reviewer scores: 1, 3, 5, 5, 6, 1

Reviews and discussion:
<thread>
creation date:
1730366206682

thread_type:
official_review

<thread_object>
creator: reviewer_r5dg

<content>
summary:
The paper studies how to reduce the computational cost in privacy-preserving federated learning (FL) with secure aggregation (SecAgg). SecAgg is a primitive that improves the privacy-utility trade-off in FL as it hides individual model updates sent to the server. However, most efficient applications require to mask each parameter of the model with random noise, incurring into a large computational cost if models are big. 

The current work proposes a technique that reduces the computational cost by only performing SecAgg to a subset of model parameters, while still recovering the (claimed to be) private aggregation of the entire model.

strengths:
- Reducing the communication cost in privacy preserving ML an interesting topic.

- The presentation of the protocol is fairly clear.

weaknesses:
# Main Weaknesses 

The major weakness of the protocol is the **lack of any standard notion of security**. The protocol is based on the fact that revealing the undetermined system of linear equations $\breve{A}x = y$ where $\breve{A}$ and $y$ are public does not compromise the privacy of $x$. From a security point of view, letting the adversary gain the knowledge of $\breve{A}x$ is **completely unsafe**. A clear example is already given the detailed comments subsection below for certain choices of $\breve{A}$. 

Even if the paper proposes some defenses to avoid the most obvious threats in the choice of $\breve{A}$ (i.e. if the system of equations already directly exposes some coordinates of $x$), these defenses are only a minor improvement in the overall security. The protocol is in fact insecure for any $\breve{A}$. For example, consider that a party joins the aggregation protocol with a vector $x = (x_1, 0, \dots, 0)$ (i.e., a vector where $x_1$ is the only non-zero value). In this case, values of $\breve{A}x$ will always be multiples of $x_1$. Therefore, the claim of Theorem 1 does not hold: multiples of $x_1$ are *distinguishable* from uniformly random numbers, contrary to what is claimed in hybrid 1 ($H_1$) in the proof of Theorem 1 (Appendix D.1).  This renders the proof of Theorem 1 incorrect. 

The computational improvements of this protocol come from the insecure modification described above. This makes the protocol inapplicable. Moreover, the attempts to further "complicate" the linear equations by the presented enhancements also follow an unsafe methodology lacking proper proofs (see my detailed comments below). 

In addition to the above, the work ignores important lines of work in compression under privacy constraints (e.g. see [R1-R5] below) and differentially private-based aggregation (e.g., by the use of correlated noise [R6-R8]), directly related to the current contribution. 


# Detailed Comments 

- Page 2, Section 2: 
    - "Mask-based" approaches are an instantiation of "SMPC-based" approaches. 
    - "(i) improving the masking mechanism":  it is not clear what this means 
    - "Note that the security of FL remains an open issue": this is too broad and it is not clear what "security" means in this context
- Page 3: 
    -  Section 2: "However, their ability to prevent poisoning attacks is limited (Ma et al., 2023)": not sure how the reference is relevant here. Does  (Ma et al., 2023) provides evidence about this statement? 
    - Section 3,  "ultimately imposing significant computational burdens": "burdens" $\rightarrow$ "burden"; is this computational burden significant with respect to the computational cost of local training steps required by ML? 
- Page 4: 
    - Def 1: "... where $AK$ denotes the additional knowledge ..": so far no mention of "additional knowledge has been made", so it is not clear to what this refers. Also, it should be explicitly clarified that $rank(A, Ax)$ means the rank of the horizontal concatenation of $A$ and $Ax$. 
   - "... rendering it impossible to determine that specific confidential vector." this is an overly strong statement (at least if no additional context is given). Consider for example that $A$ equals the identity matrix. Indeed $\breve{A}x$ has infinite solutions (all possible values of the removed coordinate of $x$). However, almost all coordinates of $x$ will be revealed if $\breve{A}x$ is revealed. 
- Page 5, Sec 3.3: 
    -  "... even if some secagg entries are compromised": There is no motivation of the extra defense, explaining how these entries would be compromised. 
    - "complicating the relationships among entries and further enhancing privacy": This lacks a proof. Providing privacy by obscurity (i.e, providing an obfuscation technique without proving that indeed it reduces to hard problem for the adversary) is a bad practice in the field of security.


# References 

[R1] Bassily, Raef, and Adam Smith. “Local, Private, Efficient Protocols for Succinct Histograms.” In Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing, 127–35. STOC ’15. New York, NY, USA: Association for Computing Machinery, 2015. https://doi.org/10.1145/2746539.2746632.

[R2] Feldman, Vitaly, and Kunal Talwar. “Lossless Compression of Efficient Private Local Randomizers.” In Proceedings of the 38th International Conference on Machine Learning, 3208–19. PMLR, 2021. https://proceedings.mlr.press/v139/feldman21a.html.

[R3] Liu, Yanxiao, Wei-Ning Chen, Ayfer Özgür, and Cheuk Ting Li. “Universal Exact Compression of Differentially Private Mechanisms.” arXiv, May 28, 2024. https://doi.org/10.48550/arXiv.2405.20782.

[R4] Shah, Abhin, Wei-Ning Chen, Johannes Ballé, Peter Kairouz, and Lucas Theis. “Optimal Compression of Locally Differentially Private Mechanisms.” In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, 7680–7723. PMLR, 2022. https://proceedings.mlr.press/v151/shah22b.html.

[R5] Triastcyn, Aleksei, Matthias Reisser, and Christos Louizos. “DP-REC: Private & Communication-Efficient Federated Learning.” arXiv, December 7, 2021. https://doi.org/10.48550/arXiv.2111.05454.

[R6] Imtiaz, Hafiz, Jafar Mohammadi, and Anand D. Sarwate. “Distributed Differentially Private Computation of Functions with Correlated Noise.” arXiv, February 22, 2021. https://doi.org/10.48550/arXiv.1904.10059.

[R7] Kairouz, Peter, Brendan Mcmahan, Shuang Song, Om Thakkar, Abhradeep Thakurta, and Zheng Xu. “Practical and Private (Deep) Learning Without Sampling or Shuffling.” In Proceedings of the 38th International Conference on Machine Learning, 5213–25. PMLR, 2021. https://proceedings.mlr.press/v139/kairouz21b.html.

[R8] Sabater, César, Aurélien Bellet, and Jan Ramon. “An Accurate, Scalable and Verifiable Protocol for Federated Differentially Private Averaging.” Machine Learning 111, no. 11 (November 1, 2022): 4249–93. https://doi.org/10.1007/s10994-022-06267-9.

questions:
Could please you address the points raised in "Main Weaknesses" above? 

In addition to these questions: 
- Could you illustrate in more detail what which masking operations of the compared SecAgg protocols do you avoid by the use of your proposal? It seems that either performing a matrix-vector multiplication or masking operations does not eliminate the dependency of the dimension of a vector $x$ in the computation. 

- If we compare the computational cost of the SecAgg protocol and the computational cost of a local training step for a client, what proportion of the computation the SecAgg overhead represents? How does this change for different models?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730627105820

thread_type:
official_review

<thread_object>
creator: reviewer_9mlh

<content>
summary:
This paper devises a portable module named \lamba-SecAgg for secure aggregation in federated learning. The authors also propose an extension involving disrupting variables to enhance privacy. Through extensive experiments, they demonstrate the efficiency of the proposed method, achieving up to 99.5× speedup and up to 32.3× communication reduction.

strengths:
1.Theoretical proofs.
2.The experimental results demonstrate that PVF achieved 99.5 \times speedup and up to 32.3 \times communication reduction.

weaknesses:
1. Writing/technical issues:
(1) In the Introduction section, the author methioed that "the minimal noise added by DP is insufficient to thwart attacks", yet they also suggest considering "DP in the extension for enhanced privacy." Is there a deeper reason or gap that I might have overlooked?
(2) The introduction of "compression-based techniques" in Figure 3 and Section 2 feels somewhat abrupt, primarily due to the lack of clarity in the classification of existing solutions outlined in the Introduction section. I suggest providing a clearer explanation of the criteria used to categorize the methods into secure aggregation techniques and compression-based techniques. Additionally, providing an analysis of the limitations of existing methods would help readers better understand the motivation behind the development of the PVF.
(3) The definition of adversary in threat model is not very clear, particularly regarding key aspects such as adversary knowledge and adversary capabilities, which have not been adequately explained or defined.
(4) Figure 4 is too abstract to understand.
(5) In Section 3.3, while discussing secure aggregation, it is noted that the requirements for data accuracy are relatively high. However, the introduction of DP typically involves adding noise to the data. It would be beneficial to clarify how the accuracy of the data can be maintained after noise has been added, particularly in the context of the freezing and melting processes.

2. Experimental issues:
(1)The neural network architectures and datasets are not intruduced in the ‘Experimental settings’.
(2)The setting of (\lambda = 100) in some experiments requires further explanation.
(3)The experimental validation, although comprehensive, is limited to specific neural network architectures and datasets. Its generalisability to other models and types of data may require further examination.

questions:
Please refer to the weakness above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730641519873

thread_type:
official_review

<thread_object>
creator: reviewer_a2ue

<content>
summary:
The paper addresses the challenges of secure aggregation in federated learning, particularly the high computation costs associated with Secure Aggregation Protocols (SAPs). The paper introduces a novel approach called Partial Vector Freezing (PVF), designed to reduce computation without increasing communication overhead. In addition, the paper proposes the disrupting variable extension to PVF to support enhanced privacy. The extensive experiments show the effectiveness of the proposed proposal.

strengths:
1. The PVF significantly compresses the length of the vector involved in SAP.
2. The disrupting variables extension method improves privacy, without the computational overhead.
3. The authors conduct extensive experiments

weaknesses:
1. Lack of Novelty in the Proposed Solution:
While I appreciate the clarity and straightforwardness presented in your methodology, I am concerned about the apparent simplicity of the proposed solution. The approach, as described, seems to lack the level of innovation. Consider expanding on the theoretical background, comparing your method with others in detail, and emphasizing any novel insights or improvements that your solution offers.
2. Informality in Security Analysis:
The security analysis section of your paper appears to be somewhat informal and lacks the rigor typically required for a comprehensive evaluation of a proposed system or method. Security is a critical aspect in many research domains, and a thorough, formal analysis is essential to establish trustworthiness and robustness. I recommend conducting a more structured and detailed security analysis, possibly incorporating formal security proofs, case studies, or simulations to demonstrate the effectiveness of your security measures.

questions:
1. In practical applications, how should this value \lambda be determined?

2. Are there any fundamental differences between the aggregation method of k^i and that of y^i?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730688573491

thread_type:
official_review

<thread_object>
creator: reviewer_z5kj

<content>
summary:
This paper introduces a novel method called λ-SecAgg, which integrates a module named Partial Vector Freezing (PVF) into Secure Aggregation Protocols (SAPs) for federated learning. The main goal of this method is to reduce the computational overhead by “freezing” most of the entries in user update vectors, allowing only a fraction (1/λ) of the original vector to be processed through secure aggregation. The frozen entries can later be “thawed” to recover the full aggregated vector, ensuring that no information is lost in the final aggregation. Additionally, the paper proposes a Disrupting Variables Extension (DVE) that enhances privacy by adding noise to the frozen entries using Differential Privacy (DP). The authors perform extensive empirical evaluations across seven baselines, demonstrating that PVF can achieve up to 99.5× speedup and 32.3× communication reduction without compromising user privacy or security.

strengths:
Innovation: The concept of freezing and unfreezing vector entries in the context of secure aggregation is very novel. This approach effectively reduces the computational burden on SAP, which has been a significant bottleneck in real-world federated learning applications, especially for large-scale models such as Large Language Models (LLMs).

Comprehensive Evaluation: The authors evaluate their approach on seven different baselines covering various secure aggregation protocols (e.g., homomorphic encryption-based, SMPC-based, mask-based). The experimental results show substantial improvements in computation time and communication cost.

Privacy and Security: The paper proves the privacy guarantees of λ-SecAgg under semi-honest and active adversary models through security analyses. In addition, the authors introduce extensions such as DVE, which further strengthens the privacy guarantees.

weaknesses:
Clarity and readability: Although this paper presents a novel approach, some sections are dense and difficult to understand, especially the mathematical derivations and safety analyses. It is suggested that the authors could improve the readability of these sections by providing more intuitive explanations and breaking down the steps as much as possible. In addition, the readability of some diagrams and formulas (e.g., those in Sections 3 and 4) is too low, and it is suggested that the reader could improve them by simplifying them or providing more detailed explanations.

Impact of noise on accuracy: Although the paper claims that the impact of DVE (adding noise to DP) on accuracy is negligible, the experimental results on the loss of accuracy due to DP noise are not detailed enough. It is suggested that the authors can add relevant experiments for this part.

Scalability to Multiple Users: This paper focuses on performance improvements for single users and servers, but does not discuss scalability to multiple users. It is suggested that the authors validate the approach of this paper in the context of multiple simultaneous users, especially with respect to communication overheads and system latency.

questions:
This paper presents a novel and practical approach to reduce the computational overhead of secure aggregation in federated learning by proposing λ-SecAgg with partial vector freezing (PVF). The strengths of the method lie in its innovative design, theoretical rigor and comprehensive evaluation, showing significant performance improvements. However, there are areas that could benefit from further clarification, particularly in terms of readability, real-world evaluation, and the impact of noise on accuracy. It is recommended that the authors consider the above comments to further refine and optimize the paper.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730714404887

thread_type:
official_review

<thread_object>
creator: reviewer_zane

<content>
summary:
The authors present a new system to improve the computational overhead of secure aggregation using a new approach called Partial Vector Freezing. This approach reduces the number of entries processed in the secure aggregation protocol, by projecting chunks of the client input vector onto a different space, and only aggregating $1/\lambda$ of the entries of each chunk securely and sending the rest of the entries in the clear. The server aggregates the entries from all clients and recovers the original input vectors by projecting the inputs back to the original space. The paper further bolsters privacy through the Disrupting Variables Extension, which applies noise calibrated for Local Differential Privacy to frozen vectors. Experimental results demonstrate substantial computation improvements compared to state-of-the-art secure aggregation protocols.

strengths:
- The focus of privacy-preserving federated learning is a crucial topic
- Extensive evaluation that covers a wide range of existing secure aggregation protocols

weaknesses:
- The approach impacts the robust privacy guarantees traditionally upheld by state-of-the-art secure aggregation protocols. These protocols typically ensure that an adversary gains no additional information about the inputs of honest clients beyond what is inferred from the aggregated output. However, Partial Vector Freezing (PVF) significantly reduces this privacy. As pointed out by the authors, it is possible for the server to learn whether two clients have similar vector chunks. Although the authors propose a mitigation strategy through Local Differential Privacy to reduce the detection of exact matches, this measure does not fully mitigate the issue of input privacy. The noised client inputs may still leak partial information that allows the server to deduce similarities between inputs. Given this trade-off, the computational gains provided by PVF do not justify the notable privacy impact. For instance, in the context of PracAgg, the masking computation is relatively lightweight. It involves field operations and pseudorandom generator evaluations, typically implemented with efficient cryptographic functions like AES. Additionally, the more computationally intensive pairwise key agreements are independent of the vector size and remain necessary regardless of the implementation of PVF.
- Another concern is the soundness of the security proof presented in Theorem 1. Specifically, the claim that the protocol execution is indistinguishable from random simulation seems to be inaccurate. The distribution of Hybrid 1 is not indistinguishable from that of Hybrid 0, as the distribution of frozen vectors $y_i$ does not exhibit properties of uniformly sampled vectors. While the random vectors are sampled uniformly from $\mathbb{Z}_p$, the frozen vectors in the protocol are the actual inputs masked with centered Gaussian noise of bounded variance. This results in a non-uniform distribution over $\mathbb{Z}_p$ undermining the indistinguishability between the two hybrids. Furthermore, other parts of the security proof are incomplete.  For instance, in Hybrid 3, it is stated that the adversary-controlled clients $\mathcal{C}$ call the ideal functionality. However, in simulation-based proofs, it is typically the simulator, not the adversary, that has direct access to the ideal functionality. Clarifying this aspect would strengthen the proof’s rigor and ensure alignment with standard cryptographic practices.

questions:
1. The baseline runtime figures for the secure aggregation protocols presented in Figure 1 and Table 1 appear notably higher than those reported in related literature. For instance, in the case of PracAgg with a vector length of 100k elements, Figure 1 shows a client runtime of 14 seconds and a server runtime of 140 seconds. In contrast, the original paper by Bonawitz et al. (2017) reports significantly lower runtimes for similar conditions, with client runtimes around 300 milliseconds (Figure 6a) and server runtimes at most 5 seconds (Figure 7a). Could you clarify the reasons for this discrepancy in runtime comparisons?
2. Could you provide a more detailed analysis of the privacy impact of your scheme, particularly focusing on the amount of differentially private noise that would be sufficient to mitigate privacy risks effectively? A clearer discussion on how the noise level was determined and its implications on both privacy and utility would be valuable.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730725684617

thread_type:
official_review

<thread_object>
creator: reviewer_sayu

<content>
summary:
This paper introduces $\lambda$-SecAgg, a secure aggregation protocol for federated learning (FL) designed to reduce computational and communication overhead through Partial Vector Freezing (PVF). This paper claims that by freezing and processing only a fraction of the private vector entries, the method significantly reduces the burden on the server and participating devices while ensuring all vector entries are eventually aggregated. To further enhance privacy, the paper incorporates Disrupting Variables Extension (DVE). The authors empirically demonstrate substantial performance gains in terms of speedup and communication reduction across various secure aggregation protocols.

While the paper presents an interesting method to reduce the overhead in secure aggregation, the privacy analysis in Section 4.1 is fundamentally flawed. The authors underestimate the information leakage from $y^{i}$, which compromises the claimed privacy guarantees.

strengths:
1. This paper considers a timely and important problem in secure aggregation protocol to reduce computational and communication overhead.

weaknesses:
1. Most importantly, the privacy analysis in Section 4.1, which claims no privacy leakage from $y^{i}$, is flawed. Although the paper asserts that no specific element of the original vector $x$ can be deduced directly from $y^{i}$, this does not mean there is no privacy leakage. In fact, $y^{i}$ reveals significant information about $x$. For example, in the case where $\lambda = 2$ and $x$ has two elements, the server can infer $x_1$ in terms of $x_2$ from $y^{i} = a_{11}x_1 + a_{12}x_2$. While $x_1$ cannot be fully determined without $x_2$, the conditional probability of guessing $x_1$ correctly is now $1/p$ instead of $1/{p^2}$. This reduction in entropy, $H(x)$, shows that $y^{i}$ contains valuable information, thus reducing privacy. The authors should revise the privacy analysis and clarify the impact of knowing $y^{i}$ on the security of the original vectors.

questions:
Please see the comment in the weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732005113781

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response 1: A simple example of PVF

comment:
Here we provide a simple example of PVF to help the reviewers gain a clearer understanding:
Assume the model vectors for two users are $\mathbf{x^1}=(1,2,3,4,5,6,7,8,9)$ and $\mathbf{x^1}=(9,8,7,6,5,4,3,2,1)$, we set $\lambda=3$, meaning we divide $\mathbf{x}$ into $\frac{9}{\lambda}=3$ groups. Following PVF, we generate the public parameters:
$$
    \mathbf{A}=
\begin{pmatrix}
6 & 9 & 5 \\\\
8 & 4 & 1 \\\\
5 & 7 & 5 \\\\
\end{pmatrix},
$$
$$
    \mathbf{\check{A}}=
\begin{pmatrix}
6& 9 &5 \\\\
8& 4& 1 \\\\
\end{pmatrix},
$$ and 
$$\mathbf{\alpha}= (5,7, 5).$$
In the secure aggregation phase, user 1 obtains the freezing vectors:
$$
\begin{array}{lll}
\mathbf{y}_1^1=\mathbf{\check{A}} \mathbf{x}_1^1=(39,19), \\\\
\mathbf{y}_2^1=\mathbf{\check{A}} \mathbf{x}_2^1=(99,58), \\\\
\mathbf{y}_3^1=\mathbf{\check{A}} \mathbf{x}_3^1=(159,97),
\end{array}
$$
and the key vector:
$$\mathbf{k}^1 =(k_1^1,k_2^1,k_3^1)=(\mathbf{\alpha}\mathbf{x}_1^1,\mathbf{\alpha}\mathbf{x}_2^1,\mathbf{\alpha}\mathbf{x}_3^1)=(34,85,136).$$
User 2 obtains:
$$
\begin{array}{lll}
\mathbf{y}_1^2=\mathbf{\check{A}} \mathbf{x}_1^2=(161,111),\\\\
\mathbf{y}_2^2=\mathbf{\check{A}} \mathbf{x}_2^2=(101,72),\\\\
\mathbf{y}_3^2=\mathbf{\check{A}} \mathbf{x}_3^2=(41,33),
\end{array}
$$
and the key vector is:
$$\mathbf{k}^2 =(k_1^2,k_2^2,k_3^2)=(\mathbf{\alpha}\mathbf{x}_1^2,\mathbf{\alpha}\mathbf{x}_2^2,\mathbf{\alpha}\mathbf{x}_3^2)=(136,85,34).$$
The secure aggregation of the key vectors gives:$$\mathbf{k}^1+\mathbf{k}^2=(170,170,170).$$

For any group (taking the first group as an example), the server cannot deduce $\mathbf{x}^i$ from $\mathbf{y}^i$, but when given $\\sum\mathbf{k}$, the server can solve the following system of equations:
$$
\begin{array}{lll}
A _ {1,1} \sum \mathbf{x}_1+ A _  {1,2} \sum \mathbf{x}_2+A _ {1,3} \sum \mathbf{x}_3=39+161=200 \\\\
A _ {2,1} \sum \mathbf{x}_1+ A _ {2,2}\sum \mathbf{x}_2+A _ {2,3}\sum \mathbf{x}_3=19+111=130 \\\\
A _ {3,1} \sum \mathbf{x}_1+ A _ {3,2}\sum \mathbf{x}_2+A _ {3,3}\sum \mathbf{x}_3=34+136=170
\end{array}
$$
to obtain $\sum \mathbf{x}=(10,10,10)$. Similarly, the aggregated values for other groups can be obtained.

It is evident that Sec. 3.2 and Fig. 4 provide a detailed, general derivation of the above example. Although it may appear complex, each equation is essential for explaining the PVF computation process. We kindly ask the reviewers to read it with patience. If you have any questions, please do not hesitate to raise them.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732348724911

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response 2: Significant Progress made during Discussions

comment:
Dear reviewers, PCs and ACs:

We sincerely thank the reviewers for their constructive feedback. After multiple discussions with several reviewers, we summarize the significant progress made regarding our submission as follows:

1. **Privacy** (@Reviewer SAYu, @Reviewer zANE, @Reviewer r5Dg):  
   In Appendix D.3, we have refined the proof of Theorem 1, demonstrating that **PVF effectively preserves user privacy**. This point has been **acknowledged by Reviewer r5Dg during the discussion**. We provide the theoretical support indicating that PVF does not leak private elements or their relationships, grounded in the hardness of the Learning With Errors (LWE) decision problem. And we present a detailed security analysis of PVF relying on the Universal Composability (UC) framework. Please refer to page 18 of the latest PDF.

2. **Clarity** (@Reviewer z5KJ, @Reviewer a2uE):  
   In General Response 1, we included a concise example to provide a clearer explanation of PVF, which has **aided Reviewer SAYu and Reviewer r5Dg in better understanding PVF**.

If our responses have addressed your concerns, we kindly ask you to consider **raising your scores**. Should any questions remain unresolved or if we have overlooked any of your points, please do not hesitate to raise them.

Thank you very much.

Best regards,

Authors.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732373925363

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General response 3: Privacy Protection Overview

comment:
Dear reviewers, PCs and ACs:

The primary concerns raised by @Reviewer SAYu, @Reviewer zANE, and @Reviewer r5Dg revolve around the potential leakage of partial information about $\mathbf{x}^i$ through $\mathbf{y}^i$, leading to privacy compromise. Below, we briefly outline the privacy protection of our approach to substantiate the claim that "*PVF effectively preserves user privacy*" in General Response 2:
### **1. Basic version: privacy in the Main PVF method (Sec. 3.2)**
The privacy of $\mathbf{x}^i$ in the Main PVF is primarily safeguarded by **the hardness of determining a specific solution to an under-determined system of linear equations**. For instance, given the private vector $\mathbf{x} = (x_1, x_2, x_3) = (7, 0, 0)$ and the matrix $\mathbf{A}$ in General Response 1:
$$
    \mathbf{A}=
\begin{pmatrix}
6 & 9 & 5 \\\\
8 & 4 & 1 \\\\
5 & 7 & 5 \\\\
\end{pmatrix},
$$
From the server's perspective, it only has access to $\mathbf{y}$, which is expressed as:
$$
\begin{array}{lll}
6x_1+9x_2+5x_3=42, \\\\
8x_1+4x_2+x_3=56,
\end{array}
$$
The server cannot distinguish whether $\mathbf{x}$ is $(7, 0, 0)$ or $(18, -34, 48)$ or any other possible solution. While the server cannot obtain any individual element of $\mathbf{x}$, it still obtains certain linear relationships involving the private elements, as pointed out by the reviewers. For this reason, we introduced Disrupting Variables Extension (DVE) in Sec. 3.3 to provide enhanced privacy.
### **2. Enhanced version: Disrupting Variables Extension (Sec. 3.3)**
DVE ensures that **the server cannot obtain any information about $\mathbf{x}^i$ from $\mathbf{y}^i$**, relying on **the hardness of the Learning With Errors (LWE) decision problem**. As detailed in Appendix D.3 (Lemma 3: The hardness of the Learning With Errors decision problem):

*Given a finite field $\mathbb{F}_p$ and a discrete probability distribution $\mathcal{X}$ over $\mathbb{F}_p$. Let $\mathbf{s} \in \mathbb{F}_p^n$ be a secret vector, $\mathbf{A} \in \mathbb{F}_p^{m \times n}$ be a matrix that is chosen uniformly at random and $\mathbf{e} \in \mathbb{F}_p^m$ be the error vector that is sampled from $\mathcal{X}$. The Learning With Errors (LWE) (search) problem is to find $\mathbf{s}$, given the pair $(\mathbf{A}, \mathbf{b})$, where $\mathbf{b} = \mathbf{A} \mathbf{s} + \mathbf{e}$. And the LWE decision problem is to distinguish between two uniformly randomly generated pairs. When the size of $p$ is polynomial in $n$, the LWE decision problem is at least as hard as the LWE search problem.*

$\mathbf{x}^i$ is added with noise $\mathbf{e}$ through Eq. 10 and $\mathbf{y}=\check{\mathbf{A}}(\mathbf{x}+ \mathbf{e})=\check{\mathbf{A}}\mathbf{x}+ \mathbf{e'}$ ($\check{\mathbf{A}}$ is public). Therefore, given a uniformly random vector $\mathbf{w}^i$, Lemma 3 ensures that $(\check{\mathbf{A}}, \mathbf{y}^i)$ and $(\check{\mathbf{A}}, \mathbf{w}^i)$ are indistinguishable, which guarantees $\mathcal{S}$ does not obtain private information from honest users through frozen vectors.

### **3. Overall security analysis**
Finally, we performed a security analysis of the secure aggregation protocol integrated with PVF under the Universal Composability (UC) framework, i.e., the proof of Theorem 1.

We sincerely invite you to review our responses and hope they resonate with your insights. Your dedication and expertise in evaluating our work have been invaluable in enhancing its quality.

Best regards,

Authors.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733193245194

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General response 4: Looking Forward to Your Reply

comment:
Dear reviewers, PCs and ACs:

We are immensely grateful to the reviewers for their valuable feedback, as well as to the PCs and ACs for their coordination efforts. The discussions over the past three weeks have yielded numerous insightful suggestions, which have been instrumental in improving the quality of our work.
Following several discussions with multiple reviewers, we have now responded to all the questions and concerns raised. With **fewer than 12 hours** remaining until the conclusion of the discussion period, we sincerely invite you to participate in the dialogue. If you have any further questions or new insights, your input would be greatly valued. **If we have already addressed your concerns to your satisfaction, we kindly request that you consider raising our score.**

**In General Response 3**, we demonstrated that PVF effectively protect user privacy. Furthermore, through extensive discussions with Reviewer r5Dg (to whom we express our sincere respect and gratitude), we have thoroughly examined and provided detailed explanations regarding privacy protection (please refer to **Responses GR2-6 to GR2-14**). Since PVF significantly reduces the overhead of secure aggregation, we are confident that our work will contribute greatly to both community research and practical engineering applications.

Best regards,

Authors.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes $\lambda$-SecAgg, a secure aggregation protocol for federated learning (FL), to reduce computational and communication overhead using Partial Vector Freezing (PVF). However, most reviewers raised their concerns that PVF significantly reduces this privacy. Despite extensive discussion on these issues, reviewers have maintained their reviews and scores. Given these issues, I recommend rejection.