Paper: Stress Testing Byzantine Robustness in Distributed Learning (ICLR-2024)

Reviewer comment (Theory):
Although the adversary model (P) looks reasonable, it seems oversimplified in section 3. Specifically, after the simplification (I) and (ii) in section 3, the adversary vector is restricted to be colinear with the average of the true gradients. Moreover, considering that in the FOE attack (Xie et al., 2020), the adversary vector is $-\epsilon$ times the average of the true gradients. Although I understand that the hyper-parameter $\epsilon$ in FOE is a pre-fixed constant, while $\lambda_t$ in JUMP is obtained by solving the optimization problem ($P_l$), the novelty of JUMP is limited due to the oversimplification. It seems that simplification (ii) is to simplify the optimization problem but greatly weakens the generalization of the proposed model. The reason why $a_t$ is restricted to be colinear with the average of true gradients is not well specified. I would greatly appreciate it if the authors could comment on this.

Author response:
We agree that the simplifications may seem to degrade the power of the adversary. In fact, we dedicate Section 4 to explaining why these simplifications $\textit{only marginally}$ degrade the strength of the attacker.
For example, regarding the collinearity constraint brought up by the reviewer, we show in Section 4.1 that removing this constraint does not improve the attack by much, since the main ingredient of the attack is to overshoot minima, i.e., going in the same direction as the average of honest gradient, which is allowed by our collinearity constraint.

Reviewer comment (Theory):
The proposed attack requires much more information than existing attacks. Other than the clients' local gradients (or momentum) that are required in ALIE (Baruch et al., 2019) and FOE (Xie et al., 2020), the proposed attack JUMP also assumes that the adversary has access to the global honest loss $\mathcal{L}_\mathcal{H}(\cdot)$ and the robust aggregation rule $F(\cdot)$ on the server. The extra assumption needs to be specified. Moreover, could the proposed method JUMP deal with the case where there is randomness in the robust aggregation $F(\cdot)$ and the aggregated result is not a deterministic value?

Author response:
We acknowledge that our attack is open-box and has full knowledge of the system and defense. This is currently stated clearly in sections 1.1 and 2.
The purpose of our paper is not to propose a realistic or black-box attack. Rather, we (i) formulate and study the strategy followed by the optimal attack (i.e., solving (P)) in sections 2 and 4, respectively, and (ii) propose a tractable attack which retains most of the strength of the worst-case attack by building on the same strategy, as explained in Section 4.
We recall that the aforementioned strategy consists in overshooting minima by amplifying the direction of the average honest gradient, at the right moment of training by being aware of the loss landscape.
Furthermore, based on our result, we provide researchers with a tool to stress test the strength of their defenses in the worst case, using one attack, since it is theoretically claimed by Byzantine-robust defenses (Allouah et al. 2023, Karimireddy et al. 2021, 2022, Farhadkhani et al. 2022) to be able to withstand any attack.
For example, even with the full knowledge of our attack, the SOTA defenses all perform well under low data heterogeneity, which actually confirms existing theory in the i.i.d. setting (Farhadkhani et al. 2022).

Besides, our attack does not need to know all the data of honest clients; it only needs to know the average of honest gradients (see Algorithm 1) and some proxy data of the same (global) distribution. Although this is not a concern in this paper, but in practice it can be done by eavesdropping and data reconstruction attacks (e.g., gradient inversion). 
We will emphasize this point further in the paper.

Finally, Shejwalkar \& Houmansadr (2021) analyze state-of-the-art attacks which possess the same level of knowledge as Jump, i.e., knowledge of the system, honest updates and global distribution. Our work belongs to the same line of research.

Reviewer comment (Experiments):
JUMP seems to have a much higher computation cost than existing attacks such as ALIE and FOE. However, there is neither a theoretical analysis of the time complexity nor the experimental results of the running time in the main text.

Author response:
We provide an analysis in Section A of the computational cost and also its trade-off with the power of the attack for various segment lengths.
Notably, the greedy version of Jump $(\tau=1)$ has a reasonable runtime, especially when considering that future benchmarks will only need to run Jump instead of several other attacks.
We could not include this discussion in the main body due to lack of space, and because we admittedly focus on a worst-case unbounded adversary.

Reviewer comment (Theory):
The JUMP optimization problem seems to assume realized gradient sampling which might be a bit strong in practice, it can be possibly formulated into an robust or expectation optimization problem without assuming noise is realized.

Author response:
We appreciate your advice on loosening our assumption by formulating our problem into a robust optimization problem without assuming noise is realized. 
We agree this could yield more realistic attacks, but we recall that our goal is to propose a worst-case attack of a truly Byzantine adversary, i.e., has full knowledge and controls all malicious workers.
Indeed, theoretical works in our line of research (Allouah et al. 2023, Karimireddy et al. 2021, 2022, Farhadkhani et al. 2022) claim to defend against such a worst-case adversary, yet we argue that they do not actually implement the latter in their validation experiments.

Reviewer comment (Reproducibility):
Could the authors clarify more on if the same gradient sampling noise is used in the solver and true algorithm update or different realizations are used?

Author response:
In the results shown in our paper, the attack uses the same noise realizations as those of the honest workers.
This is in line with our threat model, where the adversary has full knowledge, including which batches are used by honest workers.
Interestingly, even with this knowledge, the SOTA defenses all perform well under low data heterogeneity, which confirms existing theory in the i.i.d. setting (Farhadkhani et al. 2022).
We will include a discussion on this point in the paper.

Reviewer comment (Reproducibility):
It is not clear what is the solver used in the Jump attack.

Author response:
>I did not understand what is the solver used in the work and how it solved the Jump’s objective.

In Section 3.2, we discuss appropriate nonlinear solvers for our tasks. We recall  that the constraints of Problem $(P_\ell)$ are non-differentiable in general due to robust aggregation. This motivates the use of derivative-free non-linear solvers such as Powell’s (Powell, 1964), Nelder-Mead’s (Nelder \& Mead, 1965), and Differential Evolution (Storn \& Price, 1997). In fact, we have tested all the aforementioned methods on MNIST and our toy two-dimensional tasks. Both cases showed that Powell's is most effective while Differential Evolution can obtain better solutions but is much slower than Powell's, so we use Powell's in our main experiments. 
Powell's method, strictly Powell's conjugate direction method, minimizes the objective function by a bi-directional search along each search vector, successively. The bi-directional line search along each search vector can be done by Golden-section search, or Brent's method when search space is boundless. The algorithm iterates an arbitrary number of times until no significant improvement is made.
We will add a detailed description in the paper.

Reviewer comment (Novelty):
Shejwalkar & Houmansadr (2021) also use something similar to Jump’s malicious gradient in (4); e.g., in equation (6) of Shejwalkar & Houmansadr (2021) attack, if we substitute \eta_t = (1 - \lambda_t) while p_t = -\bar{g}^H_t we get the Jump attack. Given this, authors should discuss the distinction between the two attacks more explicitly.

Author response:
>Shejwalkar \& Houmansadr (2021) also use something similar to Jump’s malicious gradient in (4); e.g., in equation (6) of Shejwalkar & Houmansadr (2021) attack, if we substitute $\eta_t = (1 - \lambda_t)$ while $p_t = -\bar{g}^H_t$ we get the Jump attack. Given this I think authors should discuss the distinction between the two attacks more explicitly.

We have included in Section 4.2 a comparison with the attacks developed by Shejwalkar \& Houmansadr (2021).

There, among other factors, we explain that a key difference is that Jump allows the perturbation vector to be in the $\textit{same direction}$ as the average honest gradient, in order to overshoot minima. For Shejwalkar \& Houmansadr's (2021) attacks, because of the optimization scheme they use, the adversary can only try to deviate from the direction of the average honest gradient (the scale factor is positive). 

Furthermore, Shejwalkar \& Houmansadr (2021) focus on the i.i.d. setting, which is arguably not the norm in real-world scenarios, and do not extensively investigate the heterogeneous case.
In our work, we consider different levels of heterogeneity, using Dirichlet sampling.
In terms of threat model, the strongest one considered by Shejwalkar \& Houmansadr (2021) also assumes full knowledge. Yet, their attacks only leverage the knowledge of the aggregation and honest updates, while our attack also utilizes the global honest loss function.
We will also include this discussion in the paper.

Reviewer comment (Experiments):
Experimental evaluation is on very small settings: total number of clients is 12 which is very small for the FL settings that are vulnerable to model poisoning in practice [1]. This does not discount the utility of the work, but authors should be clear in terms of the goals of the work and where it is useful, especially positioning it with the conclusions of [1] so that readers will understand how to use this work.

Author response:
>Experimental evaluation is on very small settings: total number of clients is 12 which is very small for the FL settings that are vulnerable to model poisoning in practice [1]. This does not discount the utility of the work, but authors should be clear in terms of the goals of the work and where it is useful, especially positioning it with the conclusions of [1] so that readers will understand how to use this work.

We acknowledge that our experiments are not large-scale, since our work is directed towards researchers who need strong attacks to verify their theoretical worst-case Byzantine robustness claims. We will emphasize this point further in the paper.

Besides, regarding how challenging the datasets are, we use heterogeneous variants of MNIST and CIFAR-10, where the extreme heterogeneity scenario is challenging and is common in Byzantine-robustness works (Allouah et al. 2023, Karimireddy et al. 2022). In any case, note that the more challenging the task, the harder it is for the defender and not the attacker, which is why we focus on moderate heterogeneity, or low fraction of Byzantine workers.

Reviewer comment (Experiments):
The experiments performed are not fair: Jump attack uses the knowledge of the robust aggregation but the attacks it is compared with are aggregation-agnostic attacks; authors should provide a fair comparison, i.e., compare with aggregation-tailored attacks of Shejwalkar & Houmansadr (2021).

Author response:
>Next, I think the experiments performed are not fair: Jump attack uses the knowledge of the robust aggregation but the attacks it is compared with are aggregation-agnostic attacks; authors should provide a fair comparison, i.e., compare with aggregation-tailored attacks of Shejwalkar \& Houmansadr (2021).

We believe that it would be inadequate to compare Jump to aggregation-tailored attacks of Shejwalkar \& Houmansadr (2021), given that these attacks choose perturbation vectors depending on the defense, unlike Jump. 
Now, to automatically choose a perturbation vector for any aggregation rule, Shejwalkar \& Houmansadr (2021) propose to simulate a smaller FL system and evaluate aggregation-tailored attacks with all possible perturbation vectors, then pick the best.
However, it is unclear how this can be implemented outside the i.i.d. setting they were intended for, since reproducing the data heterogeneity of the original system is not straightforward.
We will include this discussion in the paper.

For completeness, $\textbf{we updated our paper with results}$ of our experiments with the aggregation-tailored attack of Shejwalkar \& Houmansadr (2021) that uses the inverse unit perturbation vector as an example (see Section 4.2). 
The main change in our results of Table 1 (Section 5) is that when $f=3$ using Krum with NNM, our attack becomes comparable to aggregation-tailored attack (``AGRT'' in Table 1). Otherwise, our conclusions still hold.

Reviewer comment (Reproducibility):
It is unclear what solver is used in the work and how it solved the Jump’s objective; this should be added to the main paper.

Author response:
>I did not understand what is the solver used in the work and how it solved the Jump’s objective.

In Section 3.2, we discuss appropriate nonlinear solvers for our tasks. We recall  that the constraints of Problem $(P_\ell)$ are non-differentiable in general due to robust aggregation. This motivates the use of derivative-free non-linear solvers such as Powell’s (Powell, 1964), Nelder-Mead’s (Nelder \& Mead, 1965), and Differential Evolution (Storn \& Price, 1997). In fact, we have tested all the aforementioned methods on MNIST and our toy two-dimensional tasks. Both cases showed that Powell's is most effective while Differential Evolution can obtain better solutions but is much slower than Powell's, so we use Powell's in our main experiments. 
Powell's method, strictly Powell's conjugate direction method, minimizes the objective function by a bi-directional search along each search vector, successively. The bi-directional line search along each search vector can be done by Golden-section search, or Brent's method when search space is boundless. The algorithm iterates an arbitrary number of times until no significant improvement is made.
We will add a detailed description in the paper.

Reviewer comment (Experiments):
Experimental setup is an important part of conveying the utility of the work. Authors should check [3] for details and consider more practical settings, e.g., challenging datasets, more clients, etc.

Author response:
>But IMO, it will be much more useful if you increase the total number of clients from 13/15/17 to 1000 for MNIST or CIFAR10, consider cross-device setting (along with cross-silo).

We thank the reviewer for their suggestion.
We agree that a larger scale experiment can be an interesting addition.
However, we chose to focus on FL settings that are reasonably easy for the defender, and where theoretical defenses have tight guarantees.
In contrast, cross-device settings, where client subsampling and local steps are used, have considerably weaker and fewer theoretical guarantees.

>The extreme heterogeneity you consider is dirichlet with $\alpha=0.1$ which is great, only if you have a very large number of clients. With small number of clients considered here, this distribution will not be extremely heterogeneous.

For the experiments we show, most clients have only one label in their dataset, which we believe is an extreme form of heterogeneity.

Reviewer comment (Writing):
The style and quality of the writing is to me a significant issue, as it makes it difficult to parse the authors' intent at times.

Author response:
We appreciate your response and detailed comments. 
We will carefully incorporate your writing advice in the paper.
We believe we address your concerns below and hope that you re-evaluate our paper in this light. 
We are happy to answer any further questions.

Reviewer comment (Experiments):
The concept of robust aggregation processes is to filter out likely adversaries. If all adversaries are equal, then detecting this cluster would likely be a trivial inclusion into robust aggregation. How would your work perform if you were unable to impose uniformity among the adversary update vectors?

Author response:
First, we agree that Byzantine gradients could be detected by having the aggregation eliminate identical inputs. Please note that it is standard in existing attacks and their implementation to have the Byzantine gradients be identical (Allouah et al. 2023, Karimireddy et al. 2021, 2022, Farhadkhani et al. 2022), since it is generally understood that adding small noise can make the malicious gradients nonidentical and still remain in a bad region of the space.
However, such a defense (eliminating dense clusters) does not have theoretical guarantees and can easily be fooled in the presence of data heterogeneity, e.g., Byzantine gradients can gather around points they want to remove, which would be devastating if there is considerable heterogeneity.
Also, in our threat model, we assume that one adversary controls several workers, or equivalently that Byzantine workers collude. This makes the problem clear, embodies the worst case, and is also standard in Byzantine Machine Learning works (Allouah et al. 2023, Karimireddy et al. 2021, 2022, Farhadkhani et al. 2022).

Reviewer comment (Theory):
How reasonable is the assumption regarding the model dimension? Obviously this is a worst-case assumption, but if the adversary only has access to the local information on the set of compromised agents, does this assumption hold in practice? If an attacker has enough information to know the updates of all agents (both compromised and uncompromised) then they've compromised the central update server, which you've assumed is secure. This seems like an inherent contradiction in the threat model, though I believe I have seen this in other byzantine works, so this may not be an issue specific to your work.

Author response:
We are unsure of what the reviewer meant by ``model dimension''. We assume that they refer to the threat model.

We acknowledge that our attack is open-box and has full knowledge of the system and defense. This is currently stated clearly in sections 1.1 and 2.
The purpose of our paper is not to propose a realistic or black-box attack. Rather, we (i) formulate and study the strategy followed by the optimal attack (i.e., solving (P)) in sections 2 and 4, respectively, and (ii) propose a tractable attack which retains most of the strength of the worst-case attack by building on the same strategy, as explained in Section 4.
We recall that the aforementioned strategy consists in overshooting minima by amplifying the direction of the average honest gradient, at the right moment of training by being aware of the loss landscape.
Furthermore, based on our result, we provide researchers with a tool to stress test the strength of their defenses in the worst case, using one attack, since it is theoretically claimed by Byzantine-robust defenses (Allouah et al. 2023, Karimireddy et al. 2021, 2022, Farhadkhani et al. 2022) to be able to withstand any attack.
For example, even with the full knowledge of our attack, the SOTA defenses all perform well under low data heterogeneity, which actually confirms existing theory in the i.i.d. setting (Farhadkhani et al. 2022).

Besides, our attack does not need to know all the data of honest clients; it only needs to know the average of honest gradients (see Algorithm 1) and some proxy data of the same (global) distribution. Although this is not a concern in this paper, but in practice it can be done by eavesdropping and data reconstruction attacks (e.g., gradient inversion). 
We will emphasize this point further in the paper.

Reviewer comment (Experiments):
You note on page 5 that this approach works for any variant of robust D-SGD - but how do these modifications influence the performance of your technique?

Author response:
This is a good question. 
First, note that the SOTA defense in Byzantine ML is robust D-SGD with local momentum with pre-aggregation (Allouah et al. 2023, Karimireddy et al. 2022), so our experimental results are shown for this defense.
We also test on other variants in Appendix C.2 with less or no local momentum, where we explain that our conclusions regarding the performance of Jump still hold.

Reviewer comment (Theory):
Could you explain the segment length in additional detail? At the moment problem P_l implies that the attacker only acts every $\tau$ steps - but this would appear to be contradicted by the loss subfigure from Figure 1.

Author response:
We explain the steps of the \textsc{Jump} attack in Section 3.2. Segment length $\tau$ is the size of each subproblem $P_\ell$. In other words, we split the whole training process of $T$ iterations into $\lceil \frac{T}{\tau} \rceil$ segments with length $\tau$. Each time we solve a subproblem $P_\ell$, we obtain $\lambda_{(\ell-1)\tau+1}, \ldots, \lambda_{\ell \tau}$ which can be applied to training iterations $(\ell-1)\tau+1, \ldots, \ell \tau$. Hence, after we solve all subproblems, we obtain $\lambda_{1}, \ldots, \lambda_{T}$ which are used to generate the Byzantine vectors for the whole training process.

Reviewer comment (Experiments):
How would your technique perform in the context of a) more agents, or b) when the number of agents is fixed, but the proportion of byzantine agents increased?

Author response:
To showcase the strength of our attack, we focus our experiments on small-scale tasks and a low fraction of Byzantine workers.
Indeed, the more complex the task, and the larger the Byzantine fraction, the harder it is for the defender and not the attacker.
This is why, given that we propose an attack, we focus on this harder setting for the attacker.
Finally, we recall that we already conducted a larger number of experiments, for different levels of heterogeneity, on different datasets, and Byzantine fractions.

Reviewer comment (Evaluation):
Why does Table 1 show results that are only positive for your technique, whereas the "full numerical results" from Appendix C2 reveal ranges of setups where your technique is outperformed? This feels as if the main body contents have been pruned to show your technique in the best possible light. Could you explain the choices of content included in Table 1 vs C2?

Author response:
We recall that the results in the appendices essentially convey the same message as in Section 5, where JUMP outperforms existing attacks across the vast majority of scenarios. 
It is worth mentioning that the results for $\alpha = 10$ (i.e., low data heterogeneity) are slightly different than those of Section 5 because of the low data heterogeneity. Indeed, in the latter regime, the considered defenses all have excellent accuracies across \textit{all} attacks, which is predicted by the theory in i.i.d. settings when using local momentum (Farhadkhani et al., 2022).
In addition, since real-world federated learning datasets are typically non-i.i.d., we chose the results on CIFAR-10 under moderate heterogeneity setting to show in the table of the main paper.

Reviewer comment (Writing):
In the abstract, "critical points" is undefined and unspecified, relying on a familiarity with the field that the reader may not have. I assume this refers to convergence points of the learning process, however this is not clear.

Author response:
``critical points'' are also called stationary/saddle points in non-convex optimization, and refer to model parameters where the gradient of the loss is zero.

Reviewer comment (Writing):
In the abstract, "even the greedy version" is used without context. Greedy in what context? What is the non-greedy version of JUMP?

Author response:
``greedy'' refers to the attack obtained by setting $\tau$, the segment length, to $1$ in Jump (see Section 3.2). When $\tau>1$, the attack plans malicious gradients for the next $\tau$ iterations, which makes it a non-greedy approach.

Reviewer comment (Evaluation):
In the abstract, while "accuracy damage" is not a familiar metric to me, the framing of a doubling being going from 66% to 50% vs 81% is presented in a way that makes the impact of this work impenetrable.

Author response:
``accuracy damage'' refers to the decrease in accuracy (i.e., accuracy with best existing attack accuracy minus accuracy with our attack) relative to the Byzantine-free baseline.
We will make these clear in the paper.

Reviewer comment (Theory):
The proposed JUMP attack assumes that Byzantine clients have access to all data of honest clients. This assumption is almost impossible to hold in real-world applications. I understand that the authors try to explore the worst-case behavior of Byzantine attacks. Given the unpractical assumption of the proposed attack, I still doubt whether these worst-case results make any sense.

Author response:
We acknowledge that our attack is open-box and has full knowledge of the system and defense. This is currently stated clearly in sections 1.1 and 2.
The purpose of our paper is not to propose a realistic or black-box attack. Rather, we (i) formulate and study the strategy followed by the optimal attack (i.e., solving (P)) in sections 2 and 4, respectively, and (ii) propose a tractable attack which retains most of the strength of the worst-case attack by building on the same strategy, as explained in Section 4.
We recall that the aforementioned strategy consists in overshooting minima by amplifying the direction of the average honest gradient, at the right moment of training by being aware of the loss landscape.

Furthermore, based on our result, we provide researchers with a tool to stress test the strength of their defenses in the worst case, using one attack, since it is theoretically claimed by Byzantine-robust defenses (Allouah et al. 2023, Karimireddy et al. 2021, 2022, Farhadkhani et al. 2022) to be able to withstand any attack.
For example, even with the full knowledge of our attack, the SOTA defenses all perform well under low data heterogeneity, which actually confirms existing theory in the i.i.d. setting (Farhadkhani et al. 2022).

Besides, our attack does not need to know all the data of honest clients; it only needs to know the average of honest gradients (see Algorithm 1) and some proxy data of the same (global) distribution. Although this is not a concern in this paper, but in practice it can be done by eavesdropping and data reconstruction attacks (e.g., gradient inversion).
We will emphasize this point further in the paper.

Reviewer comment (Theory):
JUMP needs to repeatedly compute the *gradients* of all honest clients, which is computationally expensive. The computation complexity also increases *linearly* with the number of honest clients and segment length.

Author response:
The greedy version of Jump ($\tau=1$, which is the one we recommend) does not need to recompute honest gradients. It only needs to know honest gradients' average of current round, which is common in existing attacks. 

We provide an analysis in Section A of the computational cost and also its trade-off with the power of the attack for various segment lengths.
Notably, the greedy version of Jump $(\tau=1)$ has a reasonable runtime, especially when considering that future benchmarks will only need to run Jump instead of several other attacks.
We could not include this discussion in the main body due to lack of space, and because we admittedly focus on a worst-case unbounded adversary.

Reviewer comment (Experiments):
Based on the two aforementioned reasons, though the proposed JUMP attack demonstrates high attack effectiveness, I doubt whether the improvement is meaningful given the unrealistic assumptions and restrictions: have access to data of all honest clients, have knowledge of Byzantine defenses, and high computation cost.

Author response:
We acknowledge that our attack is open-box and has full knowledge of the system and defense. This is currently stated clearly in sections 1.1 and 2.
The purpose of our paper is not to propose a realistic or black-box attack. Rather, we (i) formulate and study the strategy followed by the optimal attack (i.e., solving (P)) in sections 2 and 4, respectively, and (ii) propose a tractable attack which retains most of the strength of the worst-case attack by building on the same strategy, as explained in Section 4.
We recall that the aforementioned strategy consists in overshooting minima by amplifying the direction of the average honest gradient, at the right moment of training by being aware of the loss landscape.

Furthermore, based on our result, we provide researchers with a tool to stress test the strength of their defenses in the worst case, using one attack, since it is theoretically claimed by Byzantine-robust defenses (Allouah et al. 2023, Karimireddy et al. 2021, 2022, Farhadkhani et al. 2022) to be able to withstand any attack.
For example, even with the full knowledge of our attack, the SOTA defenses all perform well under low data heterogeneity, which actually confirms existing theory in the i.i.d. setting (Farhadkhani et al. 2022).

Besides, our attack does not need to know all the data of honest clients; it only needs to know the average of honest gradients (see Algorithm 1) and some proxy data of the same (global) distribution. Although this is not a concern in this paper, but in practice it can be done by eavesdropping and data reconstruction attacks (e.g., gradient inversion).
We will emphasize this point further in the paper.

The greedy version of Jump ($\tau=1$, which is the one we recommend) does not need to recompute honest gradients. It only needs to know honest gradients' average of current round, which is common in existing attacks. 

We provide an analysis in Section A of the computational cost and also its trade-off with the power of the attack for various segment lengths.
Notably, the greedy version of Jump $(\tau=1)$ has a reasonable runtime, especially when considering that future benchmarks will only need to run Jump instead of several other attacks.
We could not include this discussion in the main body due to lack of space, and because we admittedly focus on a worst-case unbounded adversary.

Reviewer comment (Novelty):
The Byzantine attack and defense problem in distributed learning systems is a well-studied problem. This paper's novelty and contributions to this area are limited.

Author response:
We acknowledge that the Byzantine attack and defense problem in distributed learning systems is a well-studied problem. However, given the lack of works investigating worst-case Byzantine attacks in the standard Byzantine ML problem, we believe that our contribution is valuable to the community without the need to explore other variants of the problem.

Reviewer comment (Novelty):
Some key references in this area are missing.

Author response:
We will add the reference mentioned by the reviewer in our paper.

Reviewer comment (Novelty):
The Byzantine attack and defense problem in distributed learning has been studied extensively in recent years and there are many variants of this problem (e.g., by allowing the server to have an auxiliary dataset to boost trust [R1]). This paper remains focused on the most basic and standard problem setting, hence the novelty of this paper is limited in this sense.

Author response:
We acknowledge that the Byzantine attack and defense problem in distributed learning systems is a well-studied problem. However, given the lack of works investigating worst-case Byzantine attacks in the standard Byzantine ML problem, we believe that our contribution is valuable to the community without the need to explore other variants of the problem.

Reviewer comment (Experiments):
This paper simplifies the problem by restricting the solution space to rank-1 vectors that are co-linear with the average of honest workers. Although this idea is interesting, this paper missed many ideas and methods developed in recent years, e.g., by treating the original attack problem as a bilevel optimization problem and employing more sophisticated bilevel optimization algorithms. It remains unclear how well the simplified strategy in this paper will perform when compared to these methods. Also, how about other simplified and restricted solution spaces, e.g., rank-2, rank-3, etc.?

Author response:
We recall that the approach leading to the simplified optimization problem of Jump is principled and does not necessitate exploring, e.g., rank-two solution spaces. Indeed, in Section 4, we show that the successive simplifications do not significantly degrade the power of the attack.

Reviewer comment (Experiments):
Although the authors conducted extensive numerical studies, most of the experiments were conducted on CNN with MNIST and CIFAR-10, which is now considered relatively easy datasets. Also, the authors only compared MinMax and MinSum in the adaptive attack category. The authors should conduct more experiments with other adaptive attack schemes.

Author response:
We use heterogeneous variants of MNIST and CIFAR-10, where the extreme heterogeneity scenario is challenging and is common in Byzantine-robustness works (Allouah et al. 2023, Karimireddy et al. 2022). In any case, note that the more challenging the task, the harder it is for the defender and not the attacker, which is why we focus on moderate heterogeneity, or low fraction of Byzantine workers.

Reviewer comment (Experiments):
The solution quality of the Jump method clearly depends on the nonlinear optimization solver. The authors only used Powell's method. This paper can benefit from testing and comparing more nonlinear solvers, and analyzing the impact of different nonlinear solvers on the proposed Jump method.

Author response:
In Section 3.2, we discuss appropriate nonlinear solvers for our tasks. We recall  that the constraints of Problem $(P_\ell)$ are in general non-differentiable due to robust aggregation. This motivates the use of derivative-free non-linear solvers such as Powell’s (Powell, 1964), Nelder-Mead’s (Nelder \& Mead, 1965), and Differential Evolution (Storn \& Price, 1997). In fact, we have tested all the aforementioned methods on MNIST and our two-dimensional tasks. Our experiments have shown that Powell's method solver is most effective while Differential Evolution can obtain better solutions but is much slower than Powell's, so we use Powell's in our main experiments. We will add a detailed description in the paper.