Paper: FedQV: Leveraging Quadratic Voting in Federated Learning (ICLR-2024)

Reviewer comment (Evaluation):
It might be worth to discuss what happens if advanced attackers optimize their malicious w to cause maximum damage while still keeping the similarity just below the detection threshold.

Author response:
Lastly, although due to page limitations, we have placed the details of the adaptive attack in the appendix C.3, this attack demonstrates the manipulation of both similarity scores and model updates as you suggested. The procedure involves:
1. The malicious client $i$ generates benign updates $w_{i}^{t}$ using unpoisoned data $D_{i}$ in round $t$ and calculates the corresponding similarity score.
2. Malicious clients (with counts of $m$) collectively normalize all the similarity scores. They optimize the similarity score using AGR-agnostic Min-Max to manipulate the scores of malicious clients, enhancing the likelihood that the resulting scores are accepted by the server.
3. Then the attackers optimize the local model poisoning by optimizing the following problem:
$$\max_{ \upsilon} \upsilon$$

$$\text{s.t.} {w^{t}}^{'}_{i \in m} = \text{FedQV}(w_1^{t},w_2^{t},\cdots,w_m^{t})$$

$${w^{t}}^{'}_{i \in m} = w^{t} - \upsilon \hat{d}$$

Here, $\hat{d}$ represents a column vector encompassing the estimated changing directions of all global model parameters. The variables $w_{i \in m}^{t}$ and ${w^{t}}^{'}_{i \in m}$ correspond to the local model $i$ before and after the attack, respectively. The parameter $\upsilon$ denotes the extent of the attack's impact on the model.

Reviewer comment (Experiments):
The paper assumes a very high number of malicious clients in the system, e.g., it discusses cases where "attackers manage to control the majority of votes, then via poisoning their tyranny will manifest itself as a degradation of the accuracy of the FL model used by the minority". In the experiments, between 10%-50% is assumed. However, Shejwalkar et al. (SP'22) have shown that in cross-device production systems the corruption is below 0.1%. It remains unclear how FedQV performs in these settings.

Author response:
**A3 (Percentage of corruption):** The reviewer’s point about the percentage of attackers is well-taken. The choice of this percentage is guided by the consideration of potential scenarios involving a larger number of attackers in which the damage to the global model can be very substantial. However, we completely acknowledge the importance of evaluating FedQV also under more realistic lower attack percentages. We have already conducted evaluations with low percentages such as 1%, 5%, and 10%, but placed these results in Appendix Table 6 due to the page limit. This supplementary result illustrates that FedQV continues to perform effectively even with lower attacker percentages. We will make a more clear note on this in the revised version.

Reviewer comment (Novelty):
The paper states that robust aggregations cannot be combined with secure aggregation. However, works like HyFL (arXiv:2302.09904) implement such aggregation schemes in secure computation.

Author response:
**A5 (HyFL)**: In our paper, we state that robust aggregations are difficult to combine with secure aggregation due to the heavy computation, which is also mentioned In HyFL. In the HyFL framework, a Hierarchical FL approach incorporates a lightweight variant of the Trimmed Mean defence designed for fighting against data-poisoning attacks. In this context, FedQV can be implemented atop the Trimmed Mean variant, serving as an additional layer of defence against data poisoning attacks.

Reviewer comment (Theory):
The threat model of the paper is not clear. A well defined threat model is crucial when studying defenses against active adversaries. In the current contribution, I do not see what are the adversaries' possible behaviors and what are they willing to risk to harm the model (e.g. are they able to collude?). Therefore, it is not clear in practice in which scenarios would the truthfulness and convergence properties hold. This makes it hard to evaluate the real impact of the contribution. For example, consider an adversary which controls many parties and each party sends a harmful update in a different round. The adversary is willing to sacrifice parties to harm the model. Would this affect the truthfulness property?

Author response:
The reviewer is correct that the threat model is crucial. In our paper, attackers can collude based on the type of attack. For instance, for example, the QV-adaptive attack in Section 5, where collusion is used to manipulate similarity scores. The results in Table 2 demonstrate that FedQV effectively defends against this collusion-based attack.

The truthfulness property is considered from each party's perspective, focusing on individual utility. This implies that malicious parties also aim to remain in the system while contributing to model harm. Sacrificing some colluders wouldn't yield better results than our considered scenario because our designed QV-adaptive attack avoids such sacrifices, ensuring more powerful harm without compromising colluders unnecessarily. However, even if the manipulated similarity scores are accepted, the inherent constraints of QV, as shown in Figure 1, limit the influence the attackers can exert through restricted aggregation weights, minimizing their harm to the global model. This aspect will be clarified in the revised version for better understanding.

Reviewer comment (Experiments):
The paper introduces quadratic voting (QV) as a defense to reduce the impact of malicious users that falsely claim to have large datasets. However, the paper does not show how this kind of attack can affect other current Federated Learning defenses in practice and how QV is an effective defense. Moreover, the paper does not evaluate the resilience against any attack in which the adversary falsely claims to have a large dataset. Therefore, I am not sure if the initial motivation of QV is present in practice.

Author response:
2a-  We acknowledge the risk of malicious users falsely claiming larger datasets and introduce Quadratic Voting (QV) as a defence, specifically evaluating its impact on FedAvg. Figure 1 illustrates how the attack works when a malicious party claims to possess a larger dataset size. The result shows that having or claiming to have a larger dataset does not succeed in harming the model under FedQV.
Furthermore, it is crucial to note that many current FL Byzantine-robust defences, such as Krum, trimmed-mean, and Rep, do not rely on the size of the dataset as the sole determinant for aggregation weights. Unlike FedAvg, where the attack in question could be effective, these defences utilize alternative aggregation mechanisms that are not susceptible to the same type of manipulation.

Reviewer comment (Novelty):
The paper introduces adaptive budgets as a reputation system. The original FedQV (Algorithm 1) already seems to penalize contributions. Therefore, the reputation protocol (Algorithm 2) appears a bit redundant. In the following sections of the paper, it is not clear when FedQV-Alg1 and FedQV-Alg2 are applied.

Author response:
2b- FedQV-Alg2's introduction is not redundant; it strategically assigns uneven budgets, rewarding benign peers and limiting malicious influence. This boosts performance by at least 26%, as shown in Fig. 2c compared to the baseline FedQV. 
In the sections preceding 5.6, we assess the outcomes using the naive FedQV. Subsequently, in Section 5.6, we conduct evaluations of FedQV with reputation (denoted as FedQV + REP) in Figure 2c. The revised version will offer additional clarity on this distinction.

Reviewer comment (Theory):
Meaning of the truthfulness property. Section 4.2 claims that any possible corrupted contribution diminishes the probability of harm as server penalties reduce the influence of corrupted parties. This gives the impression that being honest is better for local utility than performing an attack. However, this is not what we can see empirically in section 5.

Author response:
2d- It's important to clarify that local utility for clients is rooted in enhancing their influence on the model, aiming for more aggregation weights and increased participation in FL training rounds. From this perspective, being honest aligns with their goal. However, as attackers seek to harm the model, they may strategically attempt to maximize their harm while remaining within the system. This behaviour is evident in Section 5. We are happy to provide additional clarification in the revised version to ensure a comprehensive understanding.

Reviewer comment (Evaluation):
Evaluation of FedQV. The proposed protocol claims to be easy to integrate with other resilience measures because similarities are computed in the client side. As said, this is an interesting feature, but is particularly vulnerable to client-side manipulations of the similarity scores and dataset sizes. It is true that the protocol is evaluated against QV-adaptive, an attack that exploits this vulnerability. However, delegating the control of the defenses to possibly malicious clients is a high risk to take. Therefore, its evaluation of this risk requires a more comprehensive treatment. For example, an important aspect of the evaluation which I feel is missed is (as said in point 2a) the resilience against an attack in which the adversary falsely claims to have a large dataset.

Author response:
3- We acknowledge the potential vulnerability that the reviewer pointed out. Hence in our experimental evaluation, we have already considered a range of attack scenarios:
- Data poisoning attacks: Attackers submit the similarity score based on their poisoned updates.
- Model poisoning attacks: Attackers submit the true similarity score based on their clean updates and poison their model updates.
- Adaptive attack: Attackers manipulate the similarity score and submit poisoned updates.

Regarding the attack where adversaries falsely claim to have a large dataset, this attack can be successful against FedAvg (as mentioned in point 2a). But against FedQV it cannot succeed because, as illustrated in Figure 1, FedQV effectively excludes the malicious party from the aggregation process, thereby improving the accuracy of the resulting global model. This demonstrates FedQV's efficacy as a defence against this attack compared to FedAvg.

Reviewer comment (Experiments):
In the evaluation section, FedQV is only compared with FedAvg. There are many existing Byzantine-robust FL strategies that should be compared with [1] Attack-resistant federated learning with residual-based reweighting. Shuhao Fu, Chulin Xie, Bo Li, and Qifeng Chen. [2] Robust aggregation for federated learning. Krishna Pillutla, Sham M Kakade, and Zaid Harchaoui. [3] Byzantine stochastic gradient descent. Dan Alistarh, Zeyuan Allen-Zhu, and Jerry Li. [4] Learning from history for byzantine robust optimization. Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. [5] Byzantine-Robust Federated Learning with Optimal Rates and Privacy Guarantee. Banghua Zhu, Lun Wang, Qi Pang, Shuai Wang, Jiantao Jiao, Dawn Song, Michael I.Jordan. [6] Byzantine-resilient non-convex stochastic gradient descent. Zeyuan Allen-Zhu, Faeze Ebrahimian, Jerry Li, and Dan Alistarh. [7] Byzantine-robust learning on heterogeneous datasets via bucketing. Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. [8] Variance reduction is an antidote to byzantines: Better rates, weaker assumptions and communication compression as a cherry on the top. Eduard Gorbunov, Samuel Horváth, Peter Richtárik, and Gauthier Gidel. [9] Secure byzantine-robust distributed learning via clustering. Raj Kiriti Velicheti, Derek Xia, and Oluwasanmi Koyejo. [10] Byzantine-resilient high-dimensional sgd with local iterations on heterogeneous data. Deepesh Data and Suhas Diggavi.

Author response:
Thank you for your valuable feedback on our paper. Regarding the question raised, it's crucial to note that our objective is not to position FedQV in competition with existing Byzantine-robust FL strategies but rather to demonstrate that FedQV can act as a complementary approach to advanced strategies. Through the integration of FedQV with other state-of-the-art defence strategies, our aim is to establish a comprehensive defence framework capable of effectively tackling a broader spectrum of challenges within FL. As showcased in Sections 5. 6 and 5.7, our experiments highlight how FedQV can seamlessly join forces with other defence methods, such as  Muilt-Krum Blanchard et al. (2017), Trim-mean Yin et al.(2018) and Reputation Chu et al. (2022)., to achieve heightened performance collaboratively, rather than engaging in direct competition.

Reviewer comment (Evaluation):
Another limitation of FedQV is that its accuracy on the test set is worse than FedAvg, with a big gap for some datasets.

Author response:
**A3. Accuracy of FedQV**: In the comparison of accuracy between FedQV and FedAvg, as depicted in Fig 1(a), FedQV demonstrates comparable accuracy to FedAvg in the absence of attacks. However, under attack scenarios, as evidenced by the results presented in Table 1 and Fig 1(b), FedQV exhibits significantly improved accuracy compared to FedAvg.

Reviewer comment (Experiments):
In many cases, the reputation strategy was able to improve the robustness significantly. It would be interesting to compare FedAvg + Rep against FedQV + Rep to understand if the main improvements are due to quadratic voting or the reputation strategy.

Author response:
**A4. Reputation Scheme**: To assess the contribution of quadratic voting (QV) versus the reputation scheme, we compare FedQV + Rep with Rep in Section 5.7. The results indicate that incorporating FedQV enhances the robustness of the reputation scheme, underscoring the effectiveness of FedQV as a complementary element to existing FL defense algorithms.

Reviewer comment (Writing):
The paper needs to have clear definitions for the notation; for example, in the definition of L(w), the authors need to clearly define \mc{X}.

Author response:
**A5. Notation and Presentation Clarity**:
- In the definition of L(w), \mathcal{X} represents the Non-IID distribution where the dataset is drawn from, as described in Section 3.1.

- Regarding Figure 1: We appreciate the suggestion to enhance clarity. We will revise the plots, separating them for each dataset to improve understanding.

- Presentation Clarity: Due to page limitations, assumptions are currently in the appendix. In the revised version, we will present key assumptions in the main body of the paper to improve the clarity of notation in theorems and throughout the document.

Reviewer comment (Presentation):
Plots (a) and (b) in Figure 1 have too many details and datasets. It is suggested to separate the plots and have a different plot for each dataset to make it easier to understand the figures.

Author response:
**A5. Notation and Presentation Clarity**:
- In the definition of L(w), \mathcal{X} represents the Non-IID distribution where the dataset is drawn from, as described in Section 3.1.

- Regarding Figure 1: We appreciate the suggestion to enhance clarity. We will revise the plots, separating them for each dataset to improve understanding.

- Presentation Clarity: Due to page limitations, assumptions are currently in the appendix. In the revised version, we will present key assumptions in the main body of the paper to improve the clarity of notation in theorems and throughout the document.

Reviewer comment (Presentation):
The presentation could be improved in certain parts of the paper; for example, assumptions are only presented in the appendix, making the notation in Theorem 1 unclear without going to the appendix.

Author response:
**A5. Notation and Presentation Clarity**:
- In the definition of L(w), \mathcal{X} represents the Non-IID distribution where the dataset is drawn from, as described in Section 3.1.

- Regarding Figure 1: We appreciate the suggestion to enhance clarity. We will revise the plots, separating them for each dataset to improve understanding.

- Presentation Clarity: Due to page limitations, assumptions are currently in the appendix. In the revised version, we will present key assumptions in the main body of the paper to improve the clarity of notation in theorems and throughout the document.

Reviewer comment (Writing):
The section on truthfulness (section 4.2) lacks more intuitions and explanations; for example:  
1. Why would the user want to maximize v_i(a) - p_i(v_i,v)?  
2. What is v_i(a) in definition 4.4?

Author response:
**A6. Truthfulness** 
- The expression  v_i(a) - p_i(v_i,v_-i) represents the utility of party $i$, which means the gain from voting (v_i(a)) minus its cost (p_i(v_i,v_-i) ). This formulation ensures that telling the truth is more beneficial than lying(v_i(a) - p_i(v_i,v_-i) > v_i(a') - p_i(v'_i,v_-i).

-  v_i(a) in Definition 4.4 denotes the gain of party $i$ if the outcome of the voting is a. In the revised version, we will provide more intuition and explanation to enhance clarity.