Paper: STIMULUS: Achieving Fast Convergence and Low Sample Complexity in Stochastic Multi-Objective Learning (ICLR-2024)

Reviewer comment (Theory):
The definition of Pareto optimality and Pareto stationarity does not seem to align with the metrics used in the convergence results. For example, while the authors claim the convergence to a Pareto stationary point by STIMULUS due to the result obtained in Theorem 1, it is unclear why the merit function used in this result can measure the Pareto stationarity of iterates.

Author response:
Thanks for your comments. The reviewer is correct that the definitions of Pareto optimality and Pareto stationarity in Defintion 1 and Defintion 2, respectively, are different from the convergence metric we use in Theorem 1, which follows from Definition 3. Here, we want to clarify their connections as follows:


 
 The definitions of Pareto optimality and Pareto stationarity in Definition 1 and Definition 2, respectively, are their original and most general definitions. However, these definitions are based on the notion of "existence," which is not convenient and amenable for stopping criteria in algorithm design. Thus, we need equivalent definitions of Pareto optimality and Pareto stationarity, particularly when the problems are endowed with additional structural properties (differentiable objectives in this paper), which are more amenable for checking Pareto optimality and Pareto stationarity. Such equivalent equivalent conditions are stated in Definition 3, which is used as the convergence metric in Theorem 1.




 


As an analogy, Definition 3 is similar to the conventional definition of optimality in general optimization, where the most basic optimality concept is not useful for algorithm design and analysis in practice. Thus, when the problem has special structural properties, e.g., convexity and constraint qualifications, the famous KKT condition, which is both necessary and sufficient for optimality in this case, will be used to check whether we have achieved an optimal solution.

Reviewer comment (Evaluation):
Due to the problem mentioned above, it is unclear whether the comparison for theoretical results provided in Table 1 is a fair one.

Author response:
Thank you for your comment regarding the fairness of our theoretical comparisons presented in Table 1. We understand your concern and would like to clarify the basis on which these comparisons are made, emphasizing the consistency of our metrics with those used in other referenced works.

**1. Consistency with MGD and SMGD Metrics:** In our comparison, we use the same the metrics as in MGD (Fliege et al., 2019) Theorem 3.1 and Theorem 4.1, and SMGD (Yang et al., 2022) Theorem 1, thus ensuring a fair comparison in terms of theoretical analysis.

**2. Alignment with MoCo and CR-MOGM Metrics:** Similarly, in MoCo (Fernando et al., 2023) Theorems 1-3 and CR-MOGM (Zhou et al., 2022b) Theorems 2 and 3, the metric $||\sum _ {s \in [S]} \lambda _ t^* \nabla F\left(\mathbf{x} _ t\right) ||^2$ is employed. In contrast, our metric is slightly different and we use 
$||\sum _ {s \in [S]}\lambda _ {t}^{s} \nabla F\left(\mathbf{x} _ t\right)||^2$, which is not only compatible with (Fernando et al., 2023) but also avoids the limitation of making some subtle technical assumptions as in (Fernando et al., 2023). Due to this reason, the proposed metric in Definition 3 is also a novelty of this paper. Specifically, our metric satisfies $||\sum _ {s \in [S]} \lambda _ {t}^* \nabla F\left(\mathbf{x} _ t\right)||^2 \leq ||\mathbf{d} _ t||^2 = ||\sum _ {s \in [S]} \lambda _ {t}^{s} \nabla F\left(\mathbf{x} _ t\right)||^2$, meaning that our metric is an even tighter convergence metric and implicitly showing the convergence of $\{ \lambda_t \} \rightarrow \lambda^*$. Thus, a convergence rate measured by our metric will also hold when measured by the metric in (Fernando et al., 2023), thus further substantiating the fairness of our comparison.

Reviewer comment (Theory):
In proof of Lemma 1, the authors use Lemma 1 of Feng et al. (2018), yet it is hard to see how result in Feng et al. (2018) can be used here, since the problem setting in Feng et al. (2018) is single objective optimization.

Author response:
Thank you for your observation regarding the application of Lemma 1 from Feng et al. (2018) in our proof of Lemma 1. We appreciate the opportunity to clarify this aspect of our methodology. Although the problem setting in [Feng et al. (2018)] is focused on single-objective optimization, the underlying principles of their Lemma 1 can be appropriately applied to our multi-objective optimization (MOO) context. This is because the term $\mathbb{E}_t[\|\nabla f_s(\mathbf{x}_t) - \mathbf{u}_t^s\|]$ in Lemma 1 only involes **one** objective function $f_s(\mathbf{x}$) can be treated independently as a single-objective problem.

Reviewer comment (Theory):
The choice of stepsize in Theorems is unclear. For example, how does one go from equation (13) to (14) (in proof provided in appendix) by the choice of step size $\eta \leq 1/2$ ?

Author response:
In Eq.(13), we have $f_s(\mathbf{x} _ {t+1}) \leq f _ s(\mathbf{x} _ t) + \frac{\eta}{2}  || \nabla f _ s(\mathbf{x} _ t) - \mathbf{u} _ t^s ||^2 - \eta \left( \frac{1}{2}- \frac{1}{2}L \eta \right) || \mathbf{d} _ t ||^2$. 
As discussed in our paper, by setting $\left( \frac{1}{2} - \frac{1}{2} \eta \right) \geq \frac{1}{4}$, that is, $\eta \leq  \frac{1}{2}$. This ensures that the term associated with $|| \mathbf{d} _ t ||^2$in the inequality is *negative*, which implies a descent in the objective value of $f_s(\cdot)$. Thus, we have $f _ s(\mathbf{x} _ {t+1}) \leq  f _ s(\mathbf{x} _ t) + \frac{\eta}{2}  || \nabla f _ s(\mathbf{x} _ t) - \mathbf{u} _ t^s ||^2 -\frac{\eta}{4} || \mathbf{d} _ t ||^2$ as shown in our Eq.(14). We we will include more detailed explanations and justifications for the choice of step size in the final version of our paper.

Reviewer comment (Writing):
Using index $s$ in equation (3) seems not necessary.

Author response:
Thanks for catching these issues. We have fixed them accordingly in the revision of this paper.

Reviewer comment (Theory):
Can the authors elaborate on why the inequality (9) (in proof of Lemma 1) hold, and how it relate to Lemma 1 in Fang et al. (2018) ?

Author response:
Thank you for your question regarding the derivation of inequality (9) in the proof of Lemma 1 and its relation to Lemma 1 in Fang et al. (2018). We would like to provide further details:


Similar to the result shown on Page 12 in Appendix,  Eqs. (A.1) and (A.2) in [Fang et al. (2018)], we have: 

$\mathbb{E} _ t \| \nabla f _ s(\mathbf{x} _ t) - \mathbf{u} _ t^s \|^2$

$= \mathbb{E} _ t \| \nabla f _ s(\mathbf{x} _ {t-1}) - \mathbf{u} _ {t-1}^s \|^2 + \mathbb{E} _ t \| \frac{1}{|\mathcal{A}|} \sum _ {j\in \mathcal{A}}\left( \nabla f _ {sj} (\mathbf{x} _ {t};\xi _ {sj}  ) - \nabla f _ {sj} (\mathbf{x} _ {t-1};\xi _ {sj} )  +\nabla f _ s(\mathbf{x} _ {t-1})-\nabla f _ s(\mathbf{x} _ {t}) \right) \|$

$\leq\mathbb{E} _ t \| \nabla f _ s(\mathbf{x} _ {\left(n _ t-1\right) q}) - \mathbf{u} _ {\left(n _ t-1\right) q}^s \|^2 +  L^2 \sum _ {i=\left(n _ t-1\right) q}^t\frac{1}{|\mathcal{A}|} \mathbb{E}\|\mathbf{x} _ {i+1}-\mathbf{x} _ {i}\|^2$.

The first equality stems from Proposition 1 in [Fang et al. (2018)], where the expectation of the gradient difference is expanded by adding and substracting identical terms. The second inequality leverages Eq. (2.3) from [Fang et al. (2018)] and bounding based on the Lipschitz continuity of the gradients.

We can draw a similar conclusion to that in Lemma 1 of [Fang et al. (2018)] because the term $\mathbb{E}_t[\|\nabla f_s(\mathbf{x}_t) - \mathbf{u}_t^s\|]$ in Lemma 1 only involes **one** objective function $f_s(\mathbf{x}$) can be treated independently as a single-objective problem.

Reviewer comment (Experiments):
The comparison with existing algorithms in Table 1 may not be fair because they are not focused on the same settings. The setting analyzed in this paper is the finite-sum setting which is more restrictive.

Author response:
**Our Response:** Thank you for your comments. However, it is inaccurate to state that "the setting analyzied in this paper is the finite-sum setting.". Thus, the comparison in Table 1 remains fair and justifiable for the following reasons:

**1. The comparisons between STIMULUS/STIMULUS-M and MGD are fair:** Both STIMULUS/STIMULUS-M involves computation of full gradients, which is typically feasible in the fnite-sum setting. To see this, note that the sample complexity results in MGD and STIMULUS/STIMULUS-M in Table 1 all depend on $n$, which is the size of the dataset.

**2. The comparisons between STIMULUS$^+$/STIMULUS-M$^+$ and SMGD/MOCO/CR-MOGM are fair:** All these methods, including our STIMULUS$^+$/STIMULUS-M$^+$ do **not** require full gradient evaluation. Thus, our proposed STIMULUS$^+$/STIMULUS-M$^+$ methods are also applicable to the same expectation minimization MOO problems (batch size is chosen as $|\mathcal{N} _ s| = \min \\{ c _ \gamma \sigma^2\gamma _ {t}^{-1}, c _ \epsilon \sigma^2 \epsilon^{-1} \\}$. Specifically, in STIMULUS$^+$/STIMULUS-M$^+$, we propose a **adaptive batch** approach, which allows STIMULUS$^+$/STIMULUS-M$^+$ to work with expectation minimization MOO problems as those studied by SMGD/MOCO/CR-MOGM.

Reviewer comment (Evaluation):
Quantitative results are too limited to understand the practical performance of the proposed method. Also, in addition to performance of each task, a widely used measure is $\Delta m $\% (e.g. in MOCO paper) to show the overall performance on all tasks.

Author response:
**Our Response:** Thank you for your comments regarding the quantitative results and the suggestions on using of measures like $\Delta m\%$ to assess the overall performance on all tasks. We would like to further clarify why we conduct experiments in the way we did. Note that the main theoretical results and findings in this paper is the demonstration of enhanced sample efficiency in our proposed VR-assisted MGDA-based algorithms for MOO problems. We emphasize the convergence rate and sample complexity as these are key indicators of the theoretical advancements we have made. Thus, our experimental setup and results are designed to validate these theoretical aspects. By showcasing that our algorithms not only converge but do so with greater convergence speed and sample efficiency (cf. Fig. 2, Fig. 3, Fig. 4, and Table 3), we provide substantial evidence of their practical utility and effectiveness.
    
Regarding the suggestion on comparing with other algorithms using the $\Delta m \%$ metric, we agree that this metric is useful to show the overall performance on all tasks, particularly useful in scenarios where individual task performance and trade-offs among tasks are crucial. While $\Delta m\%$ is a valuable measure, it is challenging for us to conduct comprehensive experiments using this metric in this rebuttal period due to time limitation. As mentioned earlier, our primary aim is to highlight the low sample complexity and fast convergence properties of our algorithms. Thus, we hope that our existing experiments still provide sufficient evidence to showcase the effectiveness of our proposed algorithms. We will defnitely conduct experiments using the $\Delta m\%$ metric in revisions of this work, and we appreciate the reviewer for this great suggestion.

Reviewer comment (Writing):
Some notations or definitions are not clear. See **Questions-2**.

Author response:
**Our Response:** Thank you for pointing out the inaccuracy in our description of the work [Fernando et al., 2022] in the overview of MOO algorithms. You are correct in stating that [Fernando et al., 2022] primarily employed momentum-based methods to mitigate bias in multi-objective optimization (MOO). Their approach indeed revolves around the use of these methods, which is a significant aspect of their contribution. The application of these methods to bi-level optimization problems is an additional facet of their work, rather than the core mechanism for bias mitigation as our original statement may have implied.

We acknowledge this oversight and will revise the section accordingly to accurately reflect the contributions of [Fernando et al., 2022]. Our revised text will clarify that their work focuses on using momentum-based methods for bias mitigation in MOO, and these methods are further applied within the context of bi-level optimization problems.

Reviewer comment (Writing):
In Section 2 - 2) overview of MOO algorithm, it is inaccurate to say that "recent work such as (Fernando et al., 2022) uses bi-level formulation to mitigate bias". In fact, (Fernando et al., 2022) uses momentum-based methods to mitigate bias, and apply to bi-level optimization problems.

Author response:
**Our Response:** Thank you for pointing out the inaccuracy in our description of the work [Fernando et al., 2022] in the overview of MOO algorithms. You are correct in stating that [Fernando et al., 2022] primarily employed momentum-based methods to mitigate bias in multi-objective optimization (MOO). Their approach indeed revolves around the use of these methods, which is a significant aspect of their contribution. The application of these methods to bi-level optimization problems is an additional facet of their work, rather than the core mechanism for bias mitigation as our original statement may have implied.

We acknowledge this oversight and will revise the section accordingly to accurately reflect the contributions of [Fernando et al., 2022]. Our revised text will clarify that their work focuses on using momentum-based methods for bias mitigation in MOO, and these methods are further applied within the context of bi-level optimization problems.

Reviewer comment (Writing):
Typos: Below Definition 2: "non-convex MOO probolems" -> "non-convex MOO problems". Below Theorem 1: "sample compleixty" -> "sample complexity".

Author response:
**Our Response:**  Thanks for catching these typos, which we have fixed accordingly. In addition to these typos, we have conducted a thorough review of the entire paper to ensure the quality of writing.

Reviewer comment (Novelty):
What is the benefit of the proposed algorithm compared to applying variance-reduced algorithms such as SVRG to linear scalarization in MOO? In other words, applying such algorithms can also achieve similar sample complexity or convergence rate to Pareto stationary points. Therefore, the benefit of using the proposed stochastic variant of MGD is unclear.

Author response:
**Our Response:** 

Thank you for your question. We recognize the effectiveness of linear scalarization as a straightforward and commonly used approach in MOO. However, our method, which employs a multi-gradient descent approach(MGDA), offers unique benefits that do not exist in linear scalarization:

1.  Benefits of MGDA: Unlike linear scalarization, which typically uses fixed or pre-defined weights for each objective, MGDA dynamically calculates the weights for each task based on the gradient information at each iteration. This dynamic weighting offers a much more flexible exploration on the Pareto front.

2. Benefits Over VR-techniques on solving MGDA-based MOO:
Variance reduction techniques can significantly improve the convergence rates and sample-complexities in MOO optimization problem. Faster convergence and smaller sample-complexities are particularly beneficial in large-scale problems or when dealing with complex objective functions. 

In conclusion, balancing the trade-offs between different objectives is a key challenge in MOO. VR techniques in MGDA-based methods help in achieving a more balanced and efficient exploration of the trade-offs between multiple objectives, leading to more satisfactory and well-rounded Pareto optimal solutions.

Reviewer comment (Writing):
Some notations are not defined clearly. In Definition 3, Theorem 2 and 4, what is $i$ in $\lambda_i^s$? Shouldn't it be $\lambda_t^s$? What is $\xi$ in Eq.(6)? In Eq.(6), are you missing a sum of all samples $\xi \in \mathcal{N}_ s$? In Definition 4, what is "incremental first-order oracle (IFO)"? I know it is a widely used concept in finite-sum problems, but it is better to provide a formal definition or at least some references for completeness. In addition, it could benefit to introduce finite-sum problems and IFO earlier to provide some context for readers. In Algorithm 1, line 5, it says "compute $\mathbf{u}_ t^s$ as in Eq.(4)", but Eq.(4) computes $\lambda_ t^s$, is this a typo?

Author response:
**Our response:** Thanks for your comments. 

1. Thanks for catching. In Theorem 2 and 4, our convergence metric is $\mathbb{E}||\sum _ {s\in [S]}\lambda _ t^s\nabla f _ s(\mathbf{x} _ t)||^2$ for non-convex MOO problems.

2. $\xi _ {sj}$ denote the $j$-th sample for task $s$. Eqs.(6) is $\mathbf{u} _ t^s = \frac{1}{|\mathcal{N} _ s|} \sum _ {j\in \mathcal{N} _ s}\nabla f _ s(\mathbf{x} _ {t};\xi _ {sj} ), \quad \forall s \in [S].$ We are grateful for your constructive feedback and assure you that similar issues will be addressed in the final version of our paper.

3. **Basic Concept of Incremental First-Order Oracle (IFO):** An incremental first-order oracle (IFO) in the context of optimization, particularly finite-sum problems, is a computational model that, given a point $\mathbf{x}$ and a sample index $j$, returns the gradient of the function with respect to $f(\mathbf{x};\xi _ j)$ for that particular sample. In the case of MOO, the IFO evaluates the $\nabla f _ {sj}(\mathbf{x}; \xi _ {sj})$ for task $s$. 
**Role in Optimization:** The IFO is a crucial concept in finite-sum optimization problems, where the objective function is expressed as the sum of several component functions. Each call to the IFO provides gradient information for a specific component, aiding in the iterative optimization process.
A comprehensive explanation of IFOs can be found in the work of [R4, R5]. This reference provides a formal definition and explores the application of IFOs in various optimization scenarios.
We appreciate your constructive feedback and will incorporate a formal definition and reference for the incremental first-order oracle (IFO), as well as provide necessary background on finite-sum problems early in the paper. 
[R4] Gao, Hongchang, and Heng Huang. "Can stochastic zeroth-order Frank-Wolfe method converge faster for non-convex problems?." International conference on machine learning. PMLR, 2020.
[R5] Reddi, Sashank J., et al. "Fast incremental method for smooth nonconvex optimization." 2016 IEEE 55th conference on decision and control (CDC). IEEE, 2016

4. Thanks for cathching this typo. In line 5, ALgorithm 1. We compute  the full gradient as follow: $\mathbf{u} _ t^s=\frac{1}{n}\sum _ {j=1}^n \nabla f _ {sj} (\mathbf{x} _ {t};\xi _ {sj}  ),\forall s \in [S]$. In light of this, we will conduct a careful review of our manuscript to ensure such typographical errors are corrected.

Reviewer comment (Theory):
Below Table 1, it mentions $\mathbf{x}^*$ is the Pareto-optimal point. However, there can be multiple Pareto-optimal points with different function values. This will result in the term $||\mathbf{x}_ 0 - \mathbf{x}_ *||$ not well defined in Theorem 2. Could you elaborate more on this?

Author response:
**Our Response:** In MOO, there are typically **multiple** Pareto-optimal points, each representing a different trade-off among the objectives. A Pareto-optimal point is one where no objective can be improved without worsening at least one other objective.
Different Function Values: Each Pareto-optimal point can have distinct function values for the different objectives. This diversity is a key feature of the Pareto front in MOO.

In our paper, $\mathbf{x} _ *$ represnts a **Pareto-stationary point, not a common minimizer for all $f _ s(\cdot)$**. In the strongly convex setting, a Pareto-stationary solution further implies Pareto-optimal solution [R4]. More specifically, a solution $\mathbf{x} _ *$ is considered Pareto-optimal if there is no other feasible solution that would **improve one objective without causing at least one other objective to worsen**. Essentially, a Pareto-optimal solution represents a point of equilibrium where no objective can be improved without compromising others.


Regarding Theorem 2, the term $||\mathbf{x} _ t - \mathbf{x}^*||^2$  measured the difference between the current point $\mathbf{x} _ t$ in the algorithm's iteration and a Pareto-stationary (and hence Pareto-optimal) point $\mathbf{x}^*$. 

While it's true that there could be multiple such Pareto-optimal points, each theorem or result in the context of MOO typically refers to convergence or behavior relative to one such point, chosen based on the pareto optimal point corresponding to the limit point $\lambda _ *$ of the sequence {$\lambda _ t$}.

Reviewer comment (Theory):
The measure $\sum_{s\in [S]} \lambda_t^s [f_s(x_t) - f_s(x_*)]$ has some issues because it can be negative. See more discussions in (Liu & Vincente 2021). You need to make additional assumptions to make this a valid convergence metric.

Author response:
**Our Response:** Definition 3 in our initial submission was indeed not stated clearly. We also note that our metric is slightly different from that in (Liu & Vincente 2021), where we replace $\lambda _ *$ by $\lambda _ k$. In the following, we first restate Definition 3 more clearly (we have also updated Definition 3 acccordingly in the revision):

**Definition 3** ($\epsilon$-Pareto stationarity). In MOO, a point $\mathbf{x} _ t$ is $\epsilon$-Pareto-stationary if for any $\epsilon>0$, there exists a set $\{ \lambda _ t^s>0, \forall s \in [S]: \sum _ {s=1}^{S} \lambda _ t^s =1\}$, such that the following conditions hold: 1) $\mathbb{E}||\sum _ {s\in [S]}\lambda _ t^s\nabla f _ s(\mathbf{x} _ t)||^2 \leq \epsilon$ for non-convex MOO problems; or 2) $\mathbb{E}[\sum _ {s \in [S]} \lambda _ t^s [ f _ s(\mathbf{x} _ t) - f _ s(\mathbf{x} _ *) ] ] \in [0,\epsilon]$ for strongly-convex MOO problems.

Next, regarding your question on the potential non-positivity of the expression $\mathbb{E}[\sum _ {s \in [S]} \lambda _ t^s \left[ f _ s(\mathbf{x} _ t) - f _ s(\mathbf{x} _ *) \right] ]<\epsilon$, in what follows, we will show that it is always possible to make $\mathbb{E}[\sum _ {s \in [S]} \lambda _ t^s [ f _ s(\mathbf{x} _ t) - f _ s(\mathbf{x} _ *) ] ]$ non-negative, hence Definition 3 is meaningful convergence metric (i.e., Definition 3 is well-defined). To see this, from strong convexity, we have $f _ s(\mathbf{x} _ t) \geq f _ s(\mathbf{x} _ *) + \nabla f _ s^{\top}(\mathbf{x} _ *)(\mathbf{x} _ t-\mathbf{x} _ *) +\frac{\mu}{2}||\mathbf{x} _ t-\mathbf{x} _ *||^2$ for some $\mu>0$. Thus, for any $\lambda _ t^s>0$, $\forall s \in [S]$ with $\sum _ {s=1}^{S} \lambda _ t^s =1$, we have

$\sum _ {s\in[S]} \lambda _ t^s[f _ s(\mathbf{x} _ t) - f _ s(\mathbf{x} _ *)]$

$\geq \sum _ {s \in [S]} \lambda _ t^s \nabla f _ s^{\top}(\mathbf{x} _ *)(\mathbf{x} _ t-\mathbf{x} _ *) + \frac{\mu}{2}||\mathbf{x} _ t-\mathbf{x} _ *||^2$

$= \sum _ {s \in [S]} \lambda _ t^s \nabla f _ s^{\top} \mathbf{d} + \frac{\mu}{2}||\mathbf{x} _ t-\mathbf{x} _ *||^2$,

where we define $\mathbf{d} \triangleq \mathbf{x} _ t-\mathbf{x} _ *$ in the last equality for convenience. Since $\mathbf{x} _ *$ is Pareto-stationary and all objective functions are strongly convex, it follows that $\mathbf{x} _ *$ is also Pareto-optimal. Thus, there must exist at least one $\tilde{s} \in [S]$ such that $\nabla f _ {\tilde{s}}^{\top}(\mathbf{x} _ *) \mathbf{d} > 0$.

Now, consider the following strategy for choosing $\lambda _ t^s$, $\forall s\in [S]$: For $\tilde{s}$, we choose a $\lambda _ t^{\tilde{s}}$-value that is close to 1. For all other $s \ne \tilde{s}$, we choose a small $\lambda _ t^s$-value that is close to 0. Then, by pushing the $\lambda _ t^{\tilde{s}}$-value toward 1 and other $\lambda _ t^s$-values towards 0 (whiling maintaining $\sum _ {s=1}^{S} \lambda _ t^s =1$), we can always make $\sum _ {s\in[S]} \lambda _ t^s[f _ s(\mathbf{x} _ t) - f _ s(\mathbf{x} _ *)]$ non-negative. 

Therefore, as long as an algorithm can find a $\lambda _ t^s$-convex combination such that $\sum _ {s\in[S]} \lambda _ t^s[f _ s(\mathbf{x} _ t) - f _ s(\mathbf{x} _ *)] \in [0, \epsilon]$, then it is a near Pareto-stationary point under the strongly convex setting.

Meanwhile, we also want to point out that, due to research on MOO is still in its infancy, there is no standard consensus definition of Pareto-stationarity that is universally adopted in the MOO literature. The condition in Definition 3 is proposed by us, which is also part of the novelty of this paper. Also, it is worth noting that several existing papers, including [R1, R2, R3], employed similar metrics, rendering our results directly comparable to theirs. Additionally, we would also like to note that by using an additional assumption similar to Assumption 5.6 in [R3], it is possible to adopt a strong Pareto-stationarity condition in the strongly convex case, where $\lambda _ t^s$ is replaced by an optimal convex combination at $\mathbf{x} _ *$ for all $t$.

[R1] Fliege, Jörg, A. Ismael F. Vaz, and Luís Nunes Vicente. "Complexity of gradient descent for multiobjective optimization." Optimization Methods and Software 34.5 (2019): 949-959

[R2] Yang, H., Liu, Z., Liu, J., Dong, C., & Momma, M. (2023). Federated Multi-Objective Learning. arXiv preprint arXiv:2310.09866.

[R3] Suyun Liu and Luis Nunes Vicente. The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learning. Annals of Operations Research,pp. 1–30, 2021

[R4] Hiroaki Mukai. Algorithms for multicriterion optimization, IEEE Transactions on Automatic Control, 25(2): 177-186, 1980.

Reviewer comment (Presentation):
The presentation of adaptive-batching versions is a bit ambiguous. I am not sure whether the adaptive batch is applied to the $q$-periodic full gradient or to each step. Adding more background knowledge on adaptive batch technique or a diagram for STIMULUS$^+$ would be helpful. In addition, it is unclear how to decide the batch size in experiments.

Author response:
Thank you for your comments regarding the presentation of the adaptive-batching versions in our algorithms, STIMULUS+/STIMULUS-M+. We would like to clarify our adaptive batch size approach:

1. **To periodic full gradient or to each step:** The adaptive batch size is applied to replace the *periodic full gradient*. Specifically, we modify the gradient estimators in Line 5 Algorithm 1 in the t-th iteration that satisfies $\mathrm{mod}(t, q) = 0$ as follows (i.e., every $q$ steps): $\mathbf{u} _ t^s = \frac{1}{|\mathcal{N} _ s|} \sum _ {j\in \mathcal{N} _ s}\nabla f _ s(\mathbf{x} _ {t};\xi _ {sj} )$, $\forall s \in [S],$ where $\mathcal{N} _ s$ represents an $\epsilon$-adaptive batch sampled from the dataset uniformly at random. In the revision, we will provide more background knowledge on the adaptive batch technique as you suggested.

 2. **How to decide the batch size:** We choose the batch size adaptive to $\epsilon$ as: $|\mathcal{N} _s| = \min \\{ c _ \gamma \sigma^2\gamma _{t}^{-1}, c _ \epsilon \sigma^2 \epsilon^{-1}, n \\}$, where we use $c _ \gamma \geq 8$, $c _ {\epsilon}\geq \eta$ for the non-convex case and use $c _ {\gamma}\geq \frac{8\mu}{\eta}, c _ {\epsilon}\geq \frac{\mu}{2}$ for the strongly-convex case. In our experiments, as shown in our paper, we choose constant $c _ {\gamma}=c _ {\epsilon}=c = 32$ and solution accuracy $\epsilon = 10^{−3}$.

Reviewer comment (Experiments):
Besides SMGD and MOCO, CR-MOGM (Zhou et al., 2022b) should also be considered in experiments as a SOTA method.

Author response:
Thank you for your suggestion to include CR-MOGM (Zhou et al., 2022b) in our experimental comparisons. We note that that CR-MOGM can be viewed as a momentum version of the SGD method tailored for solving MOO problems. Also, both MOCO and CR-MOGM utilize a similar approach in employing momentum-based SGD for MOO. 



However, there are key distinctions between these two methods. More specifically, CR-MOGM can be viewed as a special case of the more general MOCO framework. By contrast, MOCO not only tackles the general MOO problem formulation, but also considers MOO problems with special structures, such as regularization and bilevel structures. Given the facts that i) MOCO is more recent and state-of-the-art and ii) The similarities in foundational technique between MOCO and CR-MOGM provide a reasonable basis to anticipate comparable experimental outcomes. In future work, we aim to conduct additional experiments specifically focusing on CR-MOGM to further explore and validate this hypothesis.

Reviewer comment (Theory):
The theoretical analysis seems to be problematic. For example, Theorem 1 gives convergence rates on $\frac{1}{T}\sum_{t=0}^{T-1}\|d_t\|^2$. However, the terms $d_t$ are just common descent directions built based on stochastic gradients (which is similar to the stochastic gradient in SGD). According to Definition 3 and the paragraph above, the quantity of interest is $d=\lambda^\top\nabla F(\mathbf{x})$, where $F(\mathbf{x})$ are the true objective functions, instead of the stochastic functions randomly sampled in the optimization process. Therefore, Theorem 1 does not give convergence rates on the $\epsilon$-stationarity, and the convergence in terms of $\|d_t\|^2$ does not show the real behavior of the algorithm.

Author response:
**Our Response:** Thank you for your detailed observations regarding the theoretical analysis in our paper, particularly concerning the convergence rates and the interpretation of $||\mathbf{d} _ t||^2$in Theorem 1. Your insights have led us to reconsider and refine our approach:

In our initial submission, we derived the convergence rate for $||\mathbf{d} _ t||^2=||\sum _ {s \in [S]}\lambda _ {t}^{s} \mathbf{u} _ {t}^s||^2$, where $\mathbf{u} _ {t}^s$ represents the gradient-estimation-based moving direction with VR-adjustment, which can be seen in Line 4-11 in Algorithm 1.
 

We understand your concern that this might not fully capture the convergence behavior in terms of the true gradient. Based on your feedback, we have revised our approach to derive the convergence rate for $||\sum _ {s \in [S]}\lambda _ {t}^{s} \nabla F\left(\mathbf{x} _ t\right)||^2$, which more accurately reflects the convergence behavior in terms of the true objectives. We have updated our proofs to reflect this new metric, ensuring that our theoretical results align more closely with the actual behavior of the algorithm in converging to $\epsilon$-stationarity.

Reviewer comment (Theory):
From the proof of Theorem 1, one can get convergence rates of $\|d_t\|^2$ if only $q=|\mathcal{A}|$, even if $q$ is very small. In this case, one can choose very small $q$ to derive the same convergence rates for $\|d_t\|^2$, but with much less sample complexity.

Author response:
**Our Response:** Thank you for your detailed observations regarding the theoretical analysis in our paper, particularly concerning the convergence rates and the interpretation of $||\mathbf{d} _ t||^2$in Theorem 1. Your insights have led us to reconsider and refine our approach:

In our initial submission, we derived the convergence rate for $||\mathbf{d} _ t||^2=||\sum _ {s \in [S]}\lambda _ {t}^{s} \mathbf{u} _ {t}^s||^2$, where $\mathbf{u} _ {t}^s$ represents the gradient-estimation-based moving direction with VR-adjustment, which can be seen in Line 4-11 in Algorithm 1.
 

We understand your concern that this might not fully capture the convergence behavior in terms of the true gradient. Based on your feedback, we have revised our approach to derive the convergence rate for $||\sum _ {s \in [S]}\lambda _ {t}^{s} \nabla F\left(\mathbf{x} _ t\right)||^2$, which more accurately reflects the convergence behavior in terms of the true objectives. We have updated our proofs to reflect this new metric, ensuring that our theoretical results align more closely with the actual behavior of the algorithm in converging to $\epsilon$-stationarity.

Reviewer comment (Theory):
Definition 3 implicitly assumes that all $f_s$ should have the same minimizer $x_*$, which is a very strong assumption. In multi-objective optimization, it is very unlikely that we have the same minimizer for all tasks. Therefore, the convergence rates for strongly convex problems are restrictive.

Author response:
**Our Response:** Thanks for your comments. It appears that there is some misunderstanding on the notation $\mathbf{x} _ *$, which represnts a **Pareto-stationary point, not a common minimizer for all $f _ s(\cdot)$**. In the strongly convex setting, a Pareto-stationary solution further implies Pareto-optimal solution [R4]. More specifically, a solution $\mathbf{x} _ *$ is considered Pareto-optimal if there is no other feasible solution that would **improve one objective without causing at least one other objective to worsen**. Essentially, a Pareto-optimal solution represents a point of equilibrium where no objective can be improved without compromising others.

Also, we want to clarify Definition 3 in our initial submission by restating it as follows (with further elaborations):

**Definition 3** ($\epsilon$-Pareto stationarity). In MOO, a point $\mathbf{x} _ t$ is $\epsilon$-Pareto-stationary if for any $\epsilon>0$, there exists a set $\{ \lambda _ t^s>0, \forall s \in [S]: \sum _ {s=1}^{S} \lambda _ t^s =1\}$, such that the following conditions hold: 1) $\mathbb{E}||\sum _ {s\in [S]}\lambda _ t^s\nabla f _ s(\mathbf{x} _ t)||^2 \leq \epsilon$ for non-convex MOO problems; or 2) $\mathbb{E}[\sum _ {s \in [S]} \lambda _ t^s [ f _ s(\mathbf{x} _ t) - f _ s(\mathbf{x} _ *) ] \in [0,\epsilon]$ for strongly-convex MOO problems.

Reviewer comment (Theory):
Can we derive convergence rates in terms of $d_t=\lambda^\top \nabla F(\mathbf{x}_t)$? Indeed, the convergence of $\lambda^\top \nabla F(\mathbf{x}_t)$ reflects the convergence behavior of the algorithm.

Author response:
**Our Response:** Thank you for your detailed observations regarding the theoretical analysis in our paper, particularly concerning the convergence rates and the interpretation of $||\mathbf{d} _ t||^2$in Theorem 1. Your insights have led us to reconsider and refine our approach:

In our initial submission, we derived the convergence rate for $||\mathbf{d} _ t||^2=||\sum _ {s \in [S]}\lambda _ {t}^{s} \mathbf{u} _ {t}^s||^2$, where $\mathbf{u} _ {t}^s$ represents the gradient-estimation-based moving direction with VR-adjustment, which can be seen in Line 4-11 in Algorithm 1.
 

We understand your concern that this might not fully capture the convergence behavior in terms of the true gradient. Based on your feedback, we have revised our approach to derive the convergence rate for $||\sum _ {s \in [S]}\lambda _ {t}^{s} \nabla F\left(\mathbf{x} _ t\right)||^2$, which more accurately reflects the convergence behavior in terms of the true objectives. We have updated our proofs to reflect this new metric, ensuring that our theoretical results align more closely with the actual behavior of the algorithm in converging to $\epsilon$-stationarity.

Reviewer comment (Theory):
Can we relax the assumption in Definition 3 by letting the $t$-th task have a minimizer $\mathbf{x}_*^t$, i.e., each task has its own minimizer?

Author response:
**Our Response:** Thanks for your question. Again, it appears that this question is due to a misunderstanding of MOO. Yes, we **do allow** each task in MOO to have their own minimizer. However, the notion of minimizer is **irrelevant** in MOO. Exactly because each task could have its own minimizer, it is in general impossible to find a common solution that can minimize all objective functions simultaneously. Moreover, in MOO, the tasks (objectives) could even be **conflicting** (i.e., improving one objective could worsen some others). Thus, achiving a minimizer for one objective function could signifiantly worsen another objective function. As a result, a more suitable notion of "optimality" in MOO is the **Pareto-optimality**.

Specifically, the goal of MOO is to find a solution in the "Pareto front" (i.e., the set of all Pareto-optimal solutions). A solution $\mathbf{x} _ *$ is considered Pareto-optimal if it cannot improve any objective without worsening at least one other objective. Also, it is important to note that a Pareto-optimal solution may not be an minimizer of any objective function. Again, a Pareto-optimal solution represents a point of **equilibrium** where no objective can be improved without compromising others.

Reviewer comment (Theory):
In Corollary 2, if $\epsilon>\mu$, then $\log (\mu/\epsilon)<0$. In this case, it seems that the result would no longer hold.

Author response:
**Our Response:** Thanks for your comment. There is no error if $\epsilon > \mu$ in Corollary 2.
Recall that, in Corollary 2, where we show that the overall sample complexity of SITIMULUS/STIMULUS+ for solving strongly-convex MOO is $\mathcal{O}\left(n+ \sqrt{n} \ln ({\mu/\epsilon})\right)$. Depending on the relationships between $\mu$ and $\epsilon$, we have different interpretations as follows:

**1. The $\epsilon \leq \mu$ case:** In this case, $\log(\mu/\epsilon)\geq 0$. Note that $\epsilon>0$ denotes the Pareto-stationarity gap and $\mu>0$ denotes the strong convexity modulus. If $\epsilon$ is small in the sense that $\epsilon \leq \mu$, then it is natural that converging to within this $\epsilon$-Pareto-stationarity gap takes more training samples, which increases at a rate $\mathcal{O}(\ln(\mu/\epsilon))$ as $\epsilon \rightarrow 0$.

**2. The $\epsilon > \mu$ case:** In this case, if $\log(\mu/\epsilon)<0$. However,, this does **not** mean the result is invalid. Rather, it implies the following interpretation: As $\epsilon$ increases, the Pareto-stationarity convergence criterion becomes more "relaxed"  increases. In particular, when $\epsilon$ grows larger than $\mu$, indicating that the convergence criterion is so relaxed, then it takes fewer and fewer samples to reach the convergence criterion, the number of samples **shrinks** at a rate $\mathcal{O}(\ln(\mu/\epsilon))$ as $\epsilon$ continues to increase after $\epsilon > \mu$.

Reviewer comment (Theory):
Eq (2) has a missing summation over $\mathcal{A}$.

Author response:
**Our Response:** Thanks for catching these issue. The corrections are as follows:

1) Regarding Eq. (2): The correct expression should be: $\mathbf{u} _ t^s = \mathbf{u} _ {t-1}^s + \frac{1}{|\mathcal{A}|} \sum _ {j\in \mathcal{A}}\left( \nabla f _ {sj} (\mathbf{x} _ {t};\xi _ {sj}  ) - \nabla f _ {sj} (\mathbf{x} _ {t-1};\xi _ {sj} ) \right), \text{for all }s \in [S].$

Reviewer comment (Writing):
In line 4 of Algorithm 1, Eq (4) does not give a formula to compute $u_t^s$.

Author response:
**Our Response:** Thanks for catching these issue. The corrections are as follows:

3) Update for Line 5 in Algorithm 1: The computation of $\mathbf{u} _ t^s$ should be written as: $\mathbf{u} _ t^s=\frac{1}{n}\sum _ {j=1}^n \nabla f _ {sj} (\mathbf{x} _ {t};\xi _ {sj}  )$.

Reviewer comment (Theory):
Why does it hold that $\|\nabla f_s(x_t) - u_t^s\|^2 = \sum_{...} \|x_{i+1} - x_i\|^2 + \| \nabla f_s(x_{(n_t−1)q}) − u^s_{(n_t−1)q}\|^2$ in Eq.(23)?

Author response:
Thanks for your question. There appears to be some misunderstandings. We want to clarify that the equality relationship $\left|\nabla f_s\left(x_t\right)-u_t^s\right|^2=\sum_{\ldots}\left|x_{i+1}-x_i\right|^2+\left|\nabla f_s\left(x_{\left(n_t-1\right) q}\right)-u_{\left(n_t-1\right) q}^s\right|^2$ mentioned by the reviewer is *never* used in Eq. (23).

Rather, in the latest version of our paper as shown in Eq. (25), we have used the following *inequality relationship*: $\left||\nabla f _s\left(\mathbf{x} _t\right)-\mathbf{u} _t^s\right||^2 \leq \frac{L^2}{|\mathcal{A}|} \sum _{i=\left(n _t-1\right) q}^t\left||\mathbf{x} _{i+1}-\mathbf{x} _i\right||^2+||\nabla f_s\left(\mathbf{x} _{\left(n_t-1\right) q}\right)-\mathbf{u} _{\left(n_t-1\right) q}^s||^2$. This inequality is stated and proved in Lemma 1 (cf. Eq. (8)).

Reviewer comment (Writing):
In the Line-7, it should be "gradient" other than "graident".

Author response:
Thank you for pointing out the typo. We have carefully reviewed our paper and corrected all such typos.

Reviewer comment (Writing):
This paper considers the case that $f_s(x)$ are of the finite sum form. However, detailed description of finite sum form is lacked.

Author response:
Thank you for your feedback. We appreciate your observation, and we will address this issue by providing a more detailed description of the finite sum form of $f_s(x)$ in our paper Section 1. This will help clarify the context and enhance the understanding of our work.