From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: This paper proposes a novel approach to an understudied training setting, parallel continual learning, in which heterogeneous tasks are continually streamed during training. The paper proposes an approach for how to construct a weight update combining the gradients from different tasks. To this end, the authors introduce an asymmetric distance metric and attempt to find a weight update jointly minimizing the distance between the update and the task gradients with respect to this metric. They compare this approach against various PCL baselines. Unfortunately, the reviewers found shortcomings with the work and presentation. At a high level, the reviewers did not find the particular choice of asymmetric distance metric well motivated, and were not convinced by the performance gains. At a more detailed level, reviewer 6yMu was not clear that the approach used to find an approximate solution to the constrained optimization problem yielding the weight update rule was well structured, and their concerns weren’t sufficiently addressed.
justification_for_why_not_higher_score: There was a consensus among the reviewers that this paper did not meet the threshold for publication. Quoting from above: At a high level, the reviewers did not find the particular choice of asymmetric distance metric well motivated, and were not convinced by the performance gains. At a more detailed level, reviewer 6yMu was not clear that the approach used to find an approximate solution to the constrained optimization problem yielding the weight update rule was well structured, and their concerns weren’t sufficiently addressed.
justification_for_why_not_lower_score: N/A

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer 6yMu (11)  
comment: **A**: You are not correct. The inequality relationship is not equivalent to the two metrics being the same one. Updating with Euclidean distance means the optimization overlooks the asymmetric property in gradient updating. In contrast, our AGD considers that a large $\mathbf{d}$ and a small $\mathbf{d}$ may have a large difference in updating the model. Becasue $\mathbf{d}\_i$ is the linear combination of other gradients except for $\mathbf{g}\_i$, the asymmetric distance $\hat{D}(\mathbf{d}\_i, \mathbf{g}\_i)$ should reduce the asymmetric negative effects as much as possible. This goal can not be reached through Euclidean distance.
The performance comparisons of two metrics can be seen in our ablation study. We show the results using MaxDO and Eq. (6) (DualOptim).
The results show that using Euclidean distance w/ or w/o MaxDO has significantly worse performance than the proposed AGD metric because using these distance ignores the asymmetric feature of model update.

|Method|$A_T$|$F_T$|
|---|:---:|:---:|
|DualOptim (EuDist)|67.765 |23.103 |
|DualOptim (AGD)|68.866 |24.093|
|MaxDO (EuDist)|69.344 |24.268 |
|MaxDO (AGD)   |70.078  |24.907 |

From: ICLR.cc/2023/Conference/Paper166/Reviewer_6yMu
title: About the optimization problem
comment: I agree, but as far as I understand, in Eq. 7 and in the code you are solving each of the problems individually (that's why you use a matrix $W$), instead of solving the linearly scalarized problem $\sum_i D(d, g_i)$, so my reasoning should still be valid. For each optimization problem, you compute the $d_i$ that solves $\min D(d_i, g_i) = \min ||d_i - g_i||$. *Afterwards,* you compose the obtained weights from solving these $T$ problems and MGDA into a single vector $w$. Therefore, the asymmetry of the proposed distance does not play any role.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer 6yMu (10)  
comment: > Your setup is not regular projected gradient descent. In PGD, you need to project back to the feasible set each time you update your parameter, but you are doing 5 steps of SGD with a high learning rate before projecting back. If your variation for solving the problem is equivalent to PGD and enjoys the same properties is not obvious to me and it should be discussed, as this directly affects your results.

**A**: Thank you for your comment. In fact, we do project the solution onto the feasible set at each step in the multiple steps (e.g., 5 iterations used in this paper) because this can have fast convergence, as suggested in [1], which performs multiple gradient descent iterates, and then uses one project step. This is a standard projection operation, and the optimization is able to converge to a feasible solution as other iterative projection methods such as [1].

[1] Zhao Y B. Optimal k-thresholding algorithms for sparse optimization problems. SIAM Journal on Optimization, 2020, 30(1): 31-55.

> Assume you have a solution for the first part of the problem of Eq. 7 for a given i. Let me call that solution $w^\*$ (openreview breaks with double sub-indices) and the followed direction $d^\* = \sum\_j w^\*\_j g\_j$ with $j \neq i$. This means that $\hat{D}(d^\*, g\_i) \leq \hat{D}(d, g\_i)$ for any other direction $d$ from the feasible set.
> - Let me call $\epsilon^\* = || d^\* - g\_i ||$, $\epsilon = || d - g\_i|| $, and $C = ||g\_i||$.
> - Therefore $\hat{D}(d^, g\_i) \leq \hat{D}(d, g\_i)$ is equal to $\frac{\epsilon^\*}{C + \epsilon^\*} \leq \frac{\epsilon}{C + \epsilon}$, which equals $\epsilon^(C + \epsilon) \leq \epsilon(C + \epsilon^) \Leftrightarrow \epsilon^ C + \epsilon^\* \epsilon \leq \epsilon C + \epsilon\epsilon^\* \Leftrightarrow \epsilon^\* C \leq \epsilon C \Leftrightarrow \epsilon^\* \leq \epsilon$.
> - This means that all the solutions of the proposed asymmetric distance equal the solutions of the problem $\min\_d ||d - g\_i|| $.

**A**:Thank you for your comment. Your deduction is correct only for one single gradient, but is not applicable for multiple gradients. We point out two mistakes in your deduction:

1.  > This means that $\hat{D}(d^\*, g\_i) \leq \hat{D}(d, g\_i)$ for any other direction $d$ from the feasible set.

    This statement is only correct for a single gradient with the single objective $\hat{D}(d, g\_i)$. In MOO, because of the conflict among objectives, a feasible solution to multiple objectives may be not the optimal solution to one of them. MOO seeks a whole improvement for multiple objectives simultaneously.

2. > This means that all the solutions of the proposed asymmetric distance equal the solutions of the problem $\min\_d ||d - g\_i|| $.

    Your deduction is only correct for one single gradient. We follow your notations and we have $n (n>1)$ gradients $\\{\mathbf{g}\_1,\cdots,\mathbf{g}\_n\\}$. We need to verify that $\sum\_i \hat{D}(\mathbf{d}^\*, \mathbf{g}\_i)\le\sum\_i \hat{D}(\mathbf{d}, \mathbf{g}\_i)$ instead of any of them such as $\hat{D}(\mathbf{d}^\*, \mathbf{g}\_i)\le\hat{D}(\mathbf{d}, \mathbf{g}\_i)$. We take $n=2$ for example and illustrate the difference.

    - Let $\epsilon\_1^\* = || \mathbf{d}^\* - \mathbf{g}\_1 ||$, $\epsilon\_1 = || \mathbf{d} - \mathbf{g}\_1|| $, $C\_1 = ||\mathbf{g}\_1||$ and $\epsilon\_2^\* = || \mathbf{d}^\* - \mathbf{g}\_2 ||$, $\epsilon\_2 = || \mathbf{d} - \mathbf{g}\_2|| $, $C\_2 = ||\mathbf{g}\_2||$.
    - If $\hat{D}(\mathbf{d}^\*, \mathbf{g}\_1)+\hat{D}(\mathbf{d}^\*, \mathbf{g}\_2)\le\hat{D}(\mathbf{d}, \mathbf{g}\_1)+\hat{D}(\mathbf{d}, \mathbf{g}\_2)$, then $\cfrac{\epsilon\_1^\*}{C\_1 + \epsilon\_1^\*}+\cfrac{\epsilon\_2^\*}{C\_2 + \epsilon\_2^\*}\le\cfrac{\epsilon\_1}{C\_1 + \epsilon\_1}+\cfrac{\epsilon\_2}{C\_2 + \epsilon\_2}$. 
        - Assume $\epsilon^\*\_1+\epsilon^\*\_2\le\epsilon\_1+\epsilon\_2$ and $C\_1=C\_2=C>0$.
        - $\Rightarrow (\epsilon^\*\_1+\epsilon^\*\_2)(\epsilon\_1\epsilon\_2-C^2)+2\epsilon\_1\epsilon\_2C \ge (\epsilon\_1+\epsilon\_2)(\epsilon\_1^\*\epsilon\_2^\*-C^2)+2\epsilon\_1^\*\epsilon\_2^\*C$
        - If $ \epsilon\_1\epsilon\_2\ge  C^2 $, then $(\epsilon\_1+\epsilon\_2)(\epsilon\_1\epsilon\_2-C^2)+2\epsilon\_1\epsilon\_2C \ge (\epsilon\_1+\epsilon\_2)(\epsilon\_1^\*\epsilon\_2^\*-C^2)+2\epsilon\_1^\*\epsilon\_2^\*C$,
        - Thus, we have $(\epsilon\_1\epsilon\_2-\epsilon\_1^\*\epsilon\_2^\*)(\epsilon\_1+\epsilon\_2+2C)\ge 0\Rightarrow \epsilon\_1\epsilon\_2\ge\epsilon\_1^\*\epsilon\_2^\*$.
        - In short, under the assumption, we have $\epsilon^\*\_1+\epsilon^\*\_2\le\epsilon\_1+\epsilon\_2\Rightarrow \epsilon\_1^\*\epsilon\_2^\*\le\epsilon\_1\epsilon\_2$.
        - The deduction of the above step is not always correct. For example, $\epsilon\_1^\* = 3, \epsilon\_2^\*=4, \epsilon\_1=2, \epsilon\_2=5$, we have $\epsilon\_1^\*\epsilon\_2^\*=12>\epsilon\_1\epsilon\_2=10$.

From: ICLR.cc/2023/Conference/Paper166/Reviewer_6yMu
title: Projected optimization
comment: Dear authors,

Your setup is not regular projected gradient descent. In PGD, you need to project back to the feasible set *each time you update your parameter*, but you are doing 5 steps of SGD with a high learning rate before projecting back. If your variation for solving the problem is equivalent to PGD and enjoys the same properties is not obvious to me and it should be discussed, as this directly affects your results.

Also, I just realized of something while looking closely at the optimization problem. Please let me know if I am wrong.

Assume you have a solution for the first part of the problem of Eq. 7 for a given i. Let me call that solution $w^*$ (openreview breaks with double sub-indices) and the followed direction $d^* = \sum_j w^*_j g_j$ with $j \neq i$. This means that $\hat{D}(d^*, g_i) \leq \hat{D}(d, g_i)$ for any other direction $d$ from the feasible set.
- Let me call $\epsilon^* = || d^* - g_i ||$, $\epsilon = || d - g_i|| $, and $C = ||g_i||$.
- Therefore $\hat{D}(d^*, g_i) \leq \hat{D}(d, g_i)$ is equal to $\frac{\epsilon^*}{C + \epsilon^*} \leq \frac{\epsilon}{C + \epsilon}$, which equals $\epsilon^*(C + \epsilon) \leq \epsilon(C + \epsilon^*)  \Leftrightarrow \epsilon^* C + \epsilon^* \epsilon \leq \epsilon C + \epsilon\epsilon^*  \Leftrightarrow \epsilon^* C \leq \epsilon C   \Leftrightarrow \epsilon^* \leq \epsilon$.
- This means that all the solutions of the proposed asymmetric distance equal the solutions of the problem $\min_d ||d - g_i|| $.

If my reasoning is right, I am not sure whether the proposed metric adds something new wrt existing symmetric metrics.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer 6yMu (9) 
comment: Dear Reviewer, we have revised the main manuscript following your suggestions and uploaded the compared clean code.


> 1. The learning rate for SGD in Eq. 7 is not 50, but 5, as you multiply your loss by 0.1 in line 48 of maxdo.py.

**A**: This is a typo. Thank you for pointing it out. We have already revised the learning rate in the main manuscript.

> 2. How did you compare with other methods? They do not appear in the code as far as I can see, and only PCGrad was originally implemented in Tensorflow.

**A**: To address your concern, we have uploaded the codes of all the compared methods in this version. It takes some time to clean our code.


> 3. I see in line 57 of maxdo.py that you map the weights $\widetilde W$ to the simplex after optimizing with SGD, not before. Could you clarify why, and if this is equivalent to solving the proposed problem? I am familiar with tensorflow, so it might be my fault.

**A**: It is one of the standard approaches for solving constrained optimization problems that optimize without constraints followed by projecting to the feasible set. This kind of method first optimizes without constraint, where the solution may be outside the feasible set (constraints). Then we need to project the solution to a feasible set. The two steps are iterative for meeting the convergence condition. For example, in the task of nonnegative matrix factorization, [1] projects gradients in a bound-constrained optimization. Similar examples can be found in many applications such as low-norm matrix factorizations[2].

- [1] Lin C J. Projected gradient methods for nonnegative matrix factorization. Neural computation, 2007, 19(10): 2756-2779.
- [2] Jaggi M, Sulovsk M. A simple algorithm for nuclear norm regularized problems. In ICML. 2010: 471-478.

From: ICLR.cc/2023/Conference/Paper166/Reviewer_6yMu
title: Clarification on the optimization of Eq. 7
comment: Dear authors, thanks again for the detailed response and for sharing the code.

While I still have some follow-up doubts, I think I understand where are you coming from and what were your intentions/motivations with the design choices you've made. As a final note, I think all this discussion we've had should be made really clear in the main manuscript.

I have been checking your code to better understand the method, and I have three comments, and the last one I really want to get clarified:
1. The learning rate for SGD in Eq. 7 is not 50, but 5, as you multiply your loss by 0.1 in line 48 of maxdo.py.
2. How did you compare with other methods? They do not appear in the code as far as I can see, and only PCGrad was originally implemented in Tensorflow.
3. I see in line 57 of maxdo.py that you map the weights $\widetilde W$ to the simplex **after optimizing with SGD, not before.** Could you clarify why, and if this is equivalent to solving the proposed problem? I am not familiar with tensorflow, so it might be my fault.

I am concerned with the final item since, to my understanding, this is not equivalent to doing constrained optimization and can yield really wrong values. For example, say you have $g_1 = [1, 0]$, $g_2 = [-1, 0]$, $g_3 = [0, 0.1]$, and you try to solve the problem $\arg\min_{w_2, w_3} \widehat{D}(w_2 g_2 + w_3 g_3, g_1)$ as in Eq. 7. If you solve the unconstrained problem, the solution is $w_2 = -1$, $w_3 = 0$. Applying then a softmax yields $w_2 = 0.2689$ and $w_3 = 0.7311$, which is not optimum. Indeed, $w_2 = 0$, $w_3 = 1$ gives a smaller value than that.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer 6yMu (7)
comment: Dear reviewer, we answer your new questions as follows. We have also uploaded our source code in the supplementary material.

> 1. Which kind of dual problem is Eq. (6)? There are no contraints to be solved there, so it is clearly not a Lagrangian dual problem.

**A**: In fact, we misuse the word "dual", and we have removed the misleading word in the new version. What we want to present is we transform the problem with a large decision dimension into only searching in a probability simplex $\mathcal{W}=\\{\mathbf{w}:\sum\_i\mathbf{w}\_i=1 \text{  and  } \mathbf{w}\_i\ge 0\\}$. It is not practical to directly solve for $\mathcal{d}$ on each optimization step. 
This kind of method is named linear scalarization, which is the fastest variant for gradient-based learning, and it is a standard approach to solving MOO, e.g., [1][2][3]. 

- [1] Jean-Antoine Désidéri. Multiple-gradient descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique, 2012.
- [2] Liu X, Tong X, Liu Q. Profiling pareto front with multi-objective stein variational gradient descent. Advances in Neural Information Processing Systems, 2021, 34: 14721-14733.
- [3] Ruchte Michael, Josif Grabocka. Efficient multi-objective optimization for deep learning. arXiv preprint arXiv:2103.13392 (2021).

> 2. Provided the intuition of your metric: Why would you think that the worst case is $D(d^\*, g) = 1$ instead of $D(-g, g) = \frac{2}{3}$? In the former case there is at least some exploration and randomness.

**A**: Our metric depends on not only the angle difference but also the magnitude difference between gradients. We hold the opinion that when the angle is 180°, $\mathbf{d}=-\mathbf{g}$ is not the worst case in $D(\mathbf{d},\mathbf{g})$. This is because $-\mathbf{g}$ may make the optimization still under control and $-100\mathbf{g}$ may directly fail the optimization, that is, the solution more quickly escapes from a local minimum.

> 3. What I was trying to say in W2 R4 was that, even if you "decoupled" the parameters using a matrix $W$, this matrix is then collapsed into a vector $w$. Why should we expect $w\_i = \sum\_j W\_{ij}$ to preserve any of the meaning for a particular task? Don't we just shift the conflict to these weights instead? While you forced the problems to be solved independently, weights are combined and then dependent, given the observed weight values. Assume the case with two tasks: the matrixs will be 1 in the off-diagonal and assume the diagonal is really unbalanced (say, 0.95 vs. 0.05). Then the final weights will be 1.95 and 1.05 (or 0.65 and 0.35 if you forced them to the simplex). Why are these values semantically meaningful? Clearly the goal of MGDA is not achieved anymore.


**A**:  
- We sum up each column of the weight matrix $\mathbf{W}$. And the goal is in two aspects:
    - For each off-diagonal entry of the first column, their sum means the effect of the gradient $\mathbf{g}\_1$ reducing the maximum discrepancy from other gradients.
    - In Eq. (7), we introduce the MGDA, which obtains a weighted gradient that does not damage any tasks with optimization. On one hand, MGDA is used to reduce the possible negative effect in only SGD. On the other hand, SGD (w/ Maximum discrepancy) reduces the training failure of new tasks in MGDA. Detailedly, in MGDA, the weights of small gradients are always LARGER than the weights of large gradients (See the MGDA's performance in Tables 1, 2 and 3). In this case, the new task may fail its training, which is not unacceptable in continual learning. In our experiment, we find that MGDA has some performance improvement based on reducing Maximum Discrepancy, but the improvement is limited, because the training processes of different tasks are different, which means gradients have large magnitude differences. In Eq. (7), SGD (w/ Maximum discrepancy) plays a leading role against MGDA, and MGDA plays only a supplementary role.
- When there are only two tasks, each task has only one other task and the first term of Eq. (7) is meaningless. Thus, in your two-task example $\mathbf{W}=\left[ \begin{matrix} 0.95&1 \\\\ 	1 & 0.05 \end{matrix}\right]\Rightarrow[1.95, 1.05]$, the off-diagonal entry $1$ is meaningless in reducing maximum gradient discrepancy but is useful to tackle the problem that the training of new task is restrained. We prefer to encourage new task training rather than meet Pareto optimality strictly. Note that, the two-task situation is not common in rehearsal-based PCL, because the rehearsal data stream also serves as an independent task.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer 6yMu (8)
comment: > 4. Re W2 R5:
> - 4.1: My point here is that you are combining a closed-form solution with an iterative solution. 

**A**: In Eq. (7), $\mathbf{W}^\*$ means the optimal solution in theory, which is difficult to obtain in a non-convex problem. By combining the closed-form solution and the iterative solution, we can obtain an approximate solution, which is sufficient for solving practical problems.

> - 4.2: Results in Table 4 do not show that MGDA helps in any way, as they are pretty much identical (standard deviation is higher than the difference between them).

**A**: 
- See more details of MGDA in the above Q3. In our experiment, we find that MGDA has some performance improvement based on reducing Maximum Discrepancy, but the improvement is limited. This is because in most situations, using MaxDO is able to obtain good weights. MGDA is only used to reduce the possible negative effect in the MaxDO. 
- For fair comparisons, we report the average values of 27 different runs for each compared method in our experiments. We think a large mean with a small standard deviation is better than a small mean with a large standard deviation, because the former means better stability, which is important in continual learning.

> - 4.3: I meant learning rate of the network optimizer, not MGDA itself. I saw that you have updated that in the new manuscript though.

**A**: Thank you for your suggestion.

> 5. Do you consider sharing the code at some point?

**A**: We have uploaded our MaxDO full code as supplementary material.

From: ICLR.cc/2023/Conference/Paper166/Reviewer_6yMu
title: Follow-up questions
comment: Dear authors, I really appreciate the detailed response. While many of my questions were answered, and I see more clearly some of the intuitions you were trying to transmit, I still do have some follow-up questions after your reply:

1.  Which kind of dual problem is Eq. (6)? There are no contraints to be solved there, so it is clearly not a Lagrangian dual problem.
2. Provided the intuition of your metric: Why would you think that the worst case is $D(d^*, g) = 1$ instead of $D(-g, g) = \frac{2}{3}$? In the former case there is at least some exploration and randomness.
3. What I was trying to say in W2 R4 was that, even if you "decoupled" the parameters using a matrix $W$, this matrix is then collapsed into a vector $w$. Why should we expect $w_i = \sum_j w_{ij}$ to preserve any of the meaning for a _particular_ task? Don't we just shift the conflict to these weights instead? While you forced the problems to be solved independently, weights are combined and then dependent, given the observed weight values. Assume the case with two tasks: the matrixs will be 1 in the off-diagonal and assume the diagonal is really unbalanced (say, 0.95 vs. 0.05). Then the final weights will be 1.95 and 1.05 (or 0.65 and 0.35 if you forced them to the simplex). Why are these values semantically meaningful? Clearly the goal of MGDA is not achieved anymore. 
4. Re W2 R5:
- My point here is that you are combining a closed-form solution with an iterative solution.
- Results in Table 4 do not show that MGDA helps in any way, as they are pretty much identical (standard deviation is higher than the difference between them).
- I meant learning rate of the network optimizer, not MGDA itself. I saw that you have updated that in the new manuscript though.
5. Do you consider sharing the code at some point?

Thanks again.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer UhaX (1)
comment: Dear reviewer, we have revised the manuscript and reuploaded the revised version, where the contents in cyan are revised.

## Weaknesses:

> 1. Since there are many different kinds of asymmetric distances, the paper should compare the proposed AGD with other asymmetric distances in analysis and experiments. 


**A**: Thank you for your suggestion. Following previous work [1][2][3][4], we design the two following asymmetric distance examples:

- Example distance 1 (ExDist1): $\alpha \in [0,1]$ is a random weight.

$$
D(\mathbf{x},\mathbf{y}) = \\|\mathbf{x}-\alpha \mathbf{y}\\|
$$

- Example distance 2 (ExDist2): $\mathbf{a}$ and $\mathbf{b}$ are two random vectors $\mathbf{a}\_{i}\in[0,1], \mathbf{b}\_{i}\in[0,1]$.  

$$
D(\mathbf{x},\mathbf{y}) =\\|\mathbf{a} \odot \mathbf{x}-\mathbf{b} \odot \mathbf{y} \\|
$$

|Metric|$A\_T$|$F\_T$|
|---|:---:|:---:|
|ExDist1|68.625|23.626|
|ExDist2|68.930|23.756|
|AGD    |70.078|24.907|

All the results show that the proposed AGD metric consistently outperforms all the other metrics. This is because the other asymmetric distances are not designed for gradient descent. In contrast, our AGD metric is built by considering both gradient angle and magnitude ratio, which is used to find an optimal descent direction that guarantees the new task training and old task keeping. 

- [1] Mennucci A C G. On asymmetric distances. Analysis and Geometry in Metric Spaces, 2013.
- [2] Chen Y C, Zheng W S, Lai J H, et al. An asymmetric distance model for cross-view feature mapping in person reidentification.TCSVT. 2016.
- [3] Dong W, Charikar M, Li K. Asymmetric distance estimation with sketches for similarity search in high-dimensional spaces. Proceedings of the annual international ACM SIGIR conference on Research and development in information retrieval. 2008.
- [4] Gordo A, Perronnin F, Gong Y, et al. Asymmetric distances for binary embeddings. TPAMI 2013.


> 2. It should give the details for optimizing the first term in Eq. (7) as AGD is nonsmooth. Moreover, it needs to check the coefficients in Eq.(8) satisfy the condition $\sum\_i \mathbf{w}\_i =1$ or give some explanations that why the choice of coefficients is necessary.

**A**: To address your concern, we have made the method clearer in the revised version.
- *Nonsmooth*: AGD is nonsmooth because we consider the edge situation when $\mathbf{g}\_\text{A}=\mathbf{g}\_\text{B}=\mathbf{0}$ to meet the definition of asymmetric metric in Lemma 1. The edge situation is almost unreachable in neural network optimization. Even though, the edge situations in Eq. (4) have no effects on the training of the problem in Eq. (7). This is similar to the L1 norm in many neural networks, such as [1][2]. When all gradients are zero, we set the final weights to $\cfrac{1}{n}$, i.e, average weights.
- *Training details of the first-term in Eq. (7)*: $\widetilde{\mathbf{W}}$, optimized by Stochastic Gradient Descent (SGD), is used to make up the maximum gradient discrepancy for each row. The objectives of any two rows in $\widetilde{\mathbf{W}}$ are different and independent. For row $i$, to formulate the maximum discrepancy of gradient $\mathbf{g}\_i$, the objective is the combination of non-diagonal entries. The weighted other gradients should be with the smallest asymmetric distance to $\mathbf{g}\_i$.
- *Condition $\sum\_i \mathbf{w}\_i =1$*: Yes, our solution satisfies the condition $\sum\_i \mathbf{w}^*\_i =1$. The weight matrix $W$ contains two parts, the diagonal vector $\grave{\mathbf{w}}$ and off-diagonal matrix $\widetilde{\mathbf{W}}$. The two parts are independent. Therefore,
    - For the diagonal vector $\grave{{w}}$, we have $\sum\_{i\in\mathcal{T}}\grave{\mathbf{w}}\_i=1$ according to MGDA.
    - For the off-diagona $\widetilde{\mathbf{W}}$, we use softmax on each row, thus  $\sum\_{j\in\mathcal{T}}\widetilde{\mathbf{W}}\_{ij}=1$ for any row $i$.
    - To sum up, we have $\sum\_{i,j}{\mathbf{W}}\_{ij}=|\mathcal{T}|+1$. And in Eq (8), we divide the sum with $|\mathcal{T}|+1$ to guarantee the condition.
- *Necessity of the coefficients*: To show the necessity of the choice of coefficients, we evaluate the naive equal coefficients, and each value is set to $\cfrac{1}{n}$, that is, the results are the average of gradients. The compared results of the average weight methods and the proposed MaxDO method are shown as follows. It is clear that a good choice of coefficients is important to the control of the target gradient. 

    |Coefficients|$A\_T$|$F\_T$|
    |---|:---:|:---:|
    |average (1/n)  |68.682|24.645|
    |AGD            |70.078|24.907|

- [1] Han Z, Yang F, Huang J, et al. Multimodal Dynamics: Dynamical Fusion for Trustworthy Multimodal Classification. In CVPR, 2022.
- [2] Wen W, Wu C, Wang Y, et al. Learning structured sparsity in deep neural networks. In NeurIPS, 2016.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer UhaX (2)
comment: > 3. In experiments, the author should give the details of the classifier which should be simpler than the backbone network. Also, it could test the performance of different backbones. 

**A**: To address your concern, we have shown more experimental details and results in the revised manuscript (highlighted in cyan).
- (1) *More details for classifers*: In PCL, we set each task as an independent classification task. This means that for any new task, we need to build a new classifier for it, and each new classifier is a fully-connected layer with a softmax function. 
- (2) *Different backbones*: For the results in Tables 1, 2 and 3, we used Resnet-18 on both PS-CIFAR-100 and PS-ImageNet-Tiny. We also report more experimental results by using deeper neural networks, e.g., Resnet-34 and Resnet-50 on PS-CIFAR-100 to demonstrate the effectiveness of the proposed method as follows. With the two deeper backbones, our method consistently performs much better than other methods in all the settings.

    *Resnet-34*:
    |Coefficients|$A\_T$|$F\_T$|
    |---|:---:|:---:|
    |MGDA       |64.230|22.983|
    |GradNorm   |56.855|18.492|
    |DWA        |61.131|22.216|
    |GradDrop   |65.093|24.207|
    |PCGrad     |64.981|24.059|
    |CVWeight   |45.803|12.001|
    |RLW        |64.918|23.988|
    |AGD        |66.108|24.529|


    *Resnet-50*:

    |Coefficients|$A\_T$|$F\_T$|
    |---|:---:|:---:|
    |MGDA       |64.153 |22.912 |
    |GradNorm   |56.464 |18.158 |
    |DWA        |61.119 |22.269 |
    |GradDrop   |64.792 |24.552 |
    |PCGrad     |65.390 |24.396 |
    |CVWeight   |46.110 |12.126 |
    |RLW        |64.855 |23.887 |
    |AGD        |66.888 |25.044 |

> 4. Writing issue. In page 4, lemma 1 is the definition and $D(\mathbf{x},\mathbf{y})\geq 0$ rather than $D(\mathbf{x},\mathbf{y})\leq 0$, the definition also contains if $D(\mathbf{y},\mathbf{x}) = D(\mathbf{x},\mathbf{y})=0$ it implies $\mathbf{x}=\mathbf{y}$. See Andrea CG Mennucci. On asymmetric distances. Analysis and Geometry in Metric Spaces, 2013. 

**A**: Thank you for your suggestion, and we have revised the typos in the revised manuscript.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer 6yMu (1)
comment: Dear reviewer, we have revised the manuscript and reuploaded the revised version, where the contents in cyan are revised.

## Weaknesses:

> - W1. While catastrophic forgetting is important in continual learning, I am not sure why does it have so much space in the experimental part, as the proposed approach is to use an existing solution. 

**A**: Thanks for your suggestions. To address your concern, we explain the catastrophic forgetting as follows.
- To the best of our knowledge, we are the first work that studies the problem of Parallel Continual Learning (PCL). It is necessary to point out the two challenges of PCL including both training conflict and catastrophic forgetting. Moreover, the forgetting metric is important to continual learning, and our experimental results in this paper are provided as a benchmark for future PCL studies including solving catastrophic forgetting. 
- Different from traditional rehearsal, rehearsal memory is used to represent an independent task in PCL. Thus, the training conflict may worsen the forgetting of old tasks. That is, the new task produces large gradients and may mislead the replay of old tasks with small gradients. In our method, we consider reducing this gradient conflict and propose to measure the asymmetric gradient distance. Moreover, we propose to minimize the maximum discrepancy among multiple gradients. To show the effectiveness of MaxDO on rehearsal gradient, we evaluate the result that only leverages MaxDO on new tasks instead memory data stream (finished tasks). In this case, the final gradient is calculated by $\mathbf{d}=\frac{1}{2}\mathbf{g}\_\text{reherasal} + \frac{1}{2}\mathbf{g}\_\text{new}$, where $\mathbf{g}\_\text{new}$ is the solution gradient via MaxDO on only new tasks. The result shows that it is necessary to put the rehearsal gradient to the MaxDO. Otherwise, the model will get worse accuracy and weak forgetting.

    ||$A\_T\uparrow$|$F\_T\uparrow$|
    |---|:---:|:---:|
    |MaxDO (w/o rehearsal gradient)   |68.398 |23.676 |
    |MaxDO (w/ rehearsal gradient)    |70.078 |24.907 |

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer 6yMu (2)
comment: > - W2. The paper in general lacks intuition and justifications (more on that below).
> Regarding W2: The manuscript have a number of assumptions and loose steps that need to be clarified:
> - W2 R1. Problem (1) and (3) are really loosely connected. What is the optimal d* supposed to be? And how does solving Eq. (3), which is tied to a specific local optimization process, connect with solving Eq. (1)? 

**A**: To address your concerns, we give more discussions and make them clearer.
- Eq. (1) is the classic Multi-Objective Optimization (MOO) definition. At each optimization step, the goal of MOO is to optimize multiple objective functions simultaneously while updating only once.
- In our paper, the only updated gradient is denoted as $\mathbf{d}^*$, and Eq. (2) represents the update using $\mathbf{d}^*$. The subsequent question is how to obtain $\mathbf{d}^*$.
- The motivation of Eq. (3) is that for an objective function in MOO, its own gradient $\mathbf{g}$ is the most qualified update one (for both direction and magnitude). Thus, the optimal $\mathbf{d}^*$ should be with a small distance to $\mathbf{g}$, i.e., the new objective $\min D(\mathbf{d}^*, \mathbf{g})$ in Eq. (3). 
- However, in MOO, we need to consider the distance from $\mathbf{d}^*$ to multiple objective-specific gradients, yielding the new MOO problem Eq. (3).


> - W2 R2. What is the intuition behind the AGD metric? What does each of its argument represents semantically?

**A**: To address your concern, we have revised the description of AGD to make it clearer. 
- *Why do we propose AGD?*: As illustrated in the above answer, Eq. (3) denotes the new MOO problem to find an optimal $\mathbf{d}^*$ for the only update. The key question is how to define the distance in Eq. (3) to measure the gradient discrepancy. Popular choices of distances are symmetric, such as Euclidean distance. However, the model update is highly related to gradient magnitude and direction, which is asymmetric to model updating. Thus, we need to propose a feasible asymmetric distance in Eq (3).
- *Why is AGD designed in the form of Eq (4)?*: 
    - AGD considers both gradient directions and magnitude, which are the two key elements in the MOO problem. Directly optimizing with magnitude-aware distance values may lead to a solution close to large gradients. To mitigate the bias from the magnitude difference, we employ the magnitude ratio to represent the magnitude difference.
    - Considering the definition of asymmetric metric, we consider the edge situation when $\mathbf{g}\_\text{A}=\mathbf{g}\_\text{B}=\mathbf{0}$. The edge situation is almost unreachable in the optimization, which means that the solution is optimal for all conflict tasks.
- *The semantical meaning of the AGD features*: 
    - AGD is highly related to gradient directions and magnitude differences.
    - When $\\|\mathbf{g}\_\text{A}\\|\gg\\|\mathbf{g}\_\text{B}\\|$, AGD has no tolerance and the distance is near 1, which means that a much larger gradient may destroy the learning of tasks with small gradients.
    - When $\\|\mathbf{g}\_\text{A}\\|\ll\\|\mathbf{g}\_\text{B}\\|$, AGD has a tolerance $\frac{1}{2}$ even if $\langle{\mathbf{g}\_\text{A}},{\mathbf{g}\_\text{B}}\rangle<0$, which means the impact of ${\mathbf{g}\_\text{A}}$ on task B is small. Updating with a zero gradient will neither improve nor damage the performance.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer 6yMu (3)
comment: > - W2 R3. What does it mean to "have a tolerance of 1/2"? Why is that beneficial/desired?

**A**: To explain the reason why we set a tolerance of $\frac{1}{2}$ in AGD, we take two parallel tasks A and B as an example at any time.
- When $\\|\mathbf{g}\_\text{A}\\|\gg\\|\mathbf{g}\_\text{B}\\|$ and we update task B with $\mathbf{g}\_\text{A}$, the update produces huge impact on task B even if $\langle{\mathbf{g}}\_\text{A},\mathbf{g}\_\text{B}\rangle>0$. This is because a much larger gradient may escape from a local minimum. Thus, we define that the distance $\widehat{D}({\mathbf{g}}\_\text{A},\mathbf{g}\_\text{B})$ in this situation should be large (near $1$).
- When $\\|\mathbf{g}\_\text{A}\\|\ll\\|\mathbf{g}\_\text{B}\\|$, the update produces a mild impact on task B even if $\langle{\mathbf{g}}\_\text{A},\mathbf{g}\_\text{B}\rangle<0$. That means that updating with a zero gradient will neither improve nor damage the performance. Even though, we prefer positive influence rather than non-influence. Thus, we define that the distance $\widehat{D}({\mathbf{g}}\_\text{A},\mathbf{g}\_\text{B})$ in this situation is the mid-level between the best ($\widehat{D}=0$) and worst ($\widehat{D}=1$).
- Note that the tolerance can be controlled by adding a factor $\gamma>0$. Omitting the edge situation, we have 
$$
\widehat{D}({\mathbf{g}}\_\text{A},\mathbf{g}\_\text{B}) = \frac{\\|\mathbf{g}\_\text{A}-\mathbf{g}\_\text{B}\\|}{\gamma \\|\mathbf{g}\_\text{B}\\|+\\|\mathbf{g}\_\text{A}-\mathbf{g}\_\text{B}\\|}
$$
- The experiments on PS-CIFAR-100 with different tolerances are shown in the following table. The results show either larger or smaller tolerances compared to $\frac{1}{2}$ will get the performance drop. 

    |$\gamma$|Tolerance|$A\_T$|$F\_T$|
    |:---:|:---:|:---:|:---:|
    |0.2|$\frac{5}{6}$|69.283 |24.126 |
    |0.5|$\frac{2}{3}$|69.486  |24.520 |
    |1(ours)  |$\frac{1}{2}$|70.078  |24.907 |
    |2  |$\frac{1}{3}$|69.626 |24.344 |
    |3  |$\frac{1}{4}$|69.505 |24.479 |
    |4  |$\frac{1}{5}$|69.332  |24.600 |

> - W2 R4. Why is broadcasting the vector $\mathbf{w}$ into a matrix $\mathbf{W}$ justified? I guess it is to have independent parameters to solve - Eq. (6), but why can we then combine them together?

**A**: The word "broadcast" means that we use a matrix $\mathbf{W}\in\mathbb{R}^{|\mathcal{T}|\times|\mathcal{T}|}$ instead a vector $\mathbf{w}\in\mathbb{R}^{|\mathcal{T}|}$. To make the illustration clearer, we delete the misleading word. Yes, the matrix $\mathbf{W}$ has each row optimized independently. 
- *Why use a matrix instead of a vector variables?*: Each objective of the dual problem (Eq. (6)) will be highly affected by the minimum discrepancy, i.e., each gradient itself. For example, by minimizing objective $\widehat{D}(\sum\nolimits\_j\mathbf{w}\_j\mathbf{g}\_j,\mathbf{g}\_i$), weight $\mathbf{w}\_i$ is more like to be activated than others. Thus, multiple objectives will be compromised by multiple self-interference but fail to reduce the maximum discrepancy in the dual problem optimization.
- *How to optimize with a matrix variable?*: The matrix can be decomposed into two parts. The diagonal vector is optimized via MGDA. The objectives of any two rows are different. 
    - The off-diagonal matrix $\widetilde{\mathbf{W}}$: The objectives of any two rows in the $\widetilde{\mathbf{W}}$ are different. For row $i$, the objective is the combination of non-diagonal entries, 
    $$
    \widetilde{W}\_i^*=\mathop{\arg\min}\_\mathbf{w}\quad \left\\{\widehat{D}\left(\sum\nolimits\_{j\ne i}\mathbf{w}\_j\mathbf{g}\_j,\mathbf{g}\_i\right)\Big|\forall i \in \mathcal{T}\right\\}.
    $$
    - The diagonal vector $\grave{{w}}$ is obtained by the Multiple Gradient Descent Algorithm (MGDA), which is used to obtain a weighted gradient not damage any tasks with optimization. MGDA can be easily solved by quadratic programming. The objective of MGDA is
    $$
    \grave{{w}}^*= \arg\min\_{\mathbf{w}}\quad  \Big\Vert\sum\_i \mathbf{w}\_i \mathbf{g}\_i\Big\Vert^2,
        \text{s.t.}\quad \sum\_i \mathbf{w}\_i = 1\text{  and  }\mathbf{w}\_i\ge 0, \forall i .
    $$

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer 6yMu (4)
comment: > - W2 R5. Following R4, why can we combine these off-diagonal weights with MGDA? Even more, one of the steps is a local optimization process (SGD with Maximum Discrepancy), while the other is solved in one-step. Isn't there any harm in combing both approaches? And which learning rate do we set for the first step? is that important?

**A**: To address your concern, we have made the optimization process clearer in the revised manuscript.
- *Why can we combine MGDA and SGD?*: In our MaxDO, MGDA is used to obtain weighted gradients with min-norm optimization without compromising any task. The target parameters calculated by SGD and MGDA are non-overlapping, thus the two optimizations are independent. 
- *Any harm in combining?*: In our ablation studies (Table 4), we evaluate the combination of the two approaches. The results show the combination of the two approaches achieves better performance than any of them.
- Learning rate：
    - SGD with Max-discrepancy: 50. We search in grid for a feasible learning rate of SGD. A good learning rate is important for fast convergence to local minima.
    - MGDA: Following [1], we use quadratic programming without setting a learning rate.

[1] Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. In NeurIPS, 2018.

> - W3. Experimental results are marginally better. 

**A**: 
- *Implementation*: For fair comparison, we have 27 different settings for each dataset, and we report the average results for each compared method in our experiments. Since this is the first work studying PCL, all the results of previous MTL methods are produced by ourselves with the claimed setting in their papers. All the compared methods are based on the same backbone, and they have achieved good performance on their papers in recent years. 
- *Compared to other methods*: On the three datasets, our MaxDO consistently outperforms other methods in terms of the final accuracy. On PS-EMNIST and PS-CIFAR-100, MaxDO outperforms the other best method by about 1%. On the more difficult dataset PS-Imagenet-TINY, MaxDO outperforms the other best method by 4.4%. We can observe that our method is more powerful on complex datasets. Some famous continual learning methods achieve marginally better than their competitors. For example, A-GEM[1] outperforms the other best methods by 1.7% on Split CIFAR. On the task-incremental setting, DER++[2] outperforms the other best methods by 0.8% on CIFAR-10 and by 1.2% on Tiny-ImageNet with buffer size 200.
- *The first PCL benchmark*: To the best of our knowledge, this is the first work that studies the problem of parallel continual learning. The experiment results in this paper are provided as a benchmark to PCL for future studies including solving catastrophic forgetting. 

- [1] Chaudhry A, Ranzato M A, Rohrbach M, et al. Efficient lifelong learning with a-gem. ICLR, 2019.
- [2] Buzzega P, Boschini M, Porrello A, et al. Dark experience for general continual learning: a strong, simple baseline. NeurIPS, 2020.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer 6yMu (5)
comment: ## Clarity, Quality, Novelty And Reproducibility:

### Clarity and quality.
> The manuscript would benefit a lot from some polishing. There are quite a few grammatically wrong phrases (e.g. "the min-distance problem should be asymmetry"), and phrases which I unfortunately do not understand (e.g. "symmetric metrics mean the forward and backward influences are treated as symmetric") or too convoluted (e.g. "the asymmetric metric does not require the symmetric property that D(x, y) = D(y, x) in symmetric metric.").

**A**: Thanks for your detailed comments. In the revised version, we polish our writing (Highlighted with cyan) and modify the above typos as follows:
- "the min-distance problem should be asymmetry" --> "the min-distance problem should be asymmetric" 
- "symmetric metrics mean the forward and backward influences are treated as symmetric" --> "A symmetric metric D(x, y) means the forward influence (x to y) and backward influences (y to x) are treated as symmetric" 
- "the asymmetric metric does not require the symmetric property that D(x, y) = D(y, x) in symmetric metric." --> "the asymmetric metric does not require the symmetric property, i.e., D(x, y) = D(y, x)."

### Literature review.

> - Continual Learning. While mostly complete, I feel the paper miss some works such as "Orthogonal Gradient Descent for Continual Learning" which orthogonalized gradients in continual learning to avoid catastrophic forgetting. 

**A**: Thank you for your suggestion. We have completed our literature review and cited the missing related works such as OGD[1].

[1] Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. In AISTATS, 2020.

> - Multitask Learning. The literature review regarding MTL leaves quite some room for improvement:
Relevant works such as CAGrad, RotoGrad, and GradVac were omitted. 

**A**: Thank you for your suggestion. We have completed our literature review and cited the missing related works such as CAGrad[1], RotoGrad[2], and GradVac[3].

- [1] Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. In NeurIPS, 2021.
- [2] Adrián Javaloy and Isabel Valera. Rotograd: Gradient homogenization in multitask learning. arXiv preprint arXiv:2103.02631, 2021.
- [3] Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. arXiv preprint arXiv:2010.05874, 2020.

> - To introduce MTL, conflicting gradients works are cited, instead of the seminal work by Richard Caruana.

**A**: Thank you for your advice. We have corrected the introduction citation of MTL to [1].

[1] Rich Caruana. Multitask learning. Machine learning, 1997.

> - The statement "Most existing MTL methods formulate the problem into finding an optimal gradient" is completely wrong. Gradient conflict in MTL is a relative recent direction and there is extensive work on other directions such as clustering approaches and adaptive networks.

**A**: Thank you for your suggestion. We have corrected the mistake in the revised version: 

"Most existing MTL methods formulate the problem into finding an optimal gradient" 

--> 

"Traditional MTL solutions can be mainly grouped into feature-based and parameter-based approaches[1]. The feature-based approach focuses on learning common feature representations for multiple tasks[2,3]. The parameter-based approach uses model parameters in a task to help learn model parameters in other tasks, such as task clustering [4,5] and decomposition[6]. In recent years, some MTL methods formulate the problem into finding an optimal gradient"

- [1] Yu Zhang and Qiang Yang. A survey on multi-task learning. IEEE TKDE, 2021.
- [2] Andreas Maurer, Massi Pontil, and Bernardino Romera-Paredes. Sparse coding for multitask and transfer learning. In ICML, 2013.
- [3] Jie Wang and Jieping Ye. Safe screening for multi-task feature learning with multiple data matrices. In ICML, 2015.
- [4] Sebastian Thrun and Joseph O’Sullivan. Discovering structure in multiple learning tasks: The tc algorithm. In ICML, 1996.
- [5] Aviad Barzilai and Koby Crammer. Convex multi-task learning by clustering. In Artificial Intelligence and Statistics. PMLR, 2015.
- [6] Ali Jalali, Sujay Sanghavi, Chao Ruan, and Pradeep Ravikumar. A dirty model for multi-task learning. In NeurIPS, 2010.


> - MGDA applied to the backbone parameters is not introduced by Sener & Koltum. Their contribution was to apply MGDA (by Désidéri) to the feature produced by the backbone.


**A**: Thank you for your suggestion. We have corrected the citation of MGDA to [1].

[1] Jean-Antoine Désidéri. Multiple-gradient descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique, 2012.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer 6yMu (6)
comment: > - Similarly, saying that any of the methods described in 4.3 are state-of-the-art is wrong. As of today, there is no a clear state-of-the-art method.


**A**: Thank you for your suggestion. We have removed the SOTA word in Sec. 4.3.

### Reproducibility. 
> The experiments are not reproducible, as important details such as the learning rate for Eq. (7) are not provided. The provided code only includes a single file with an implementation of the proposed method, which has been stripped down from the working code, as there are undeclared variables (e.g. variable lr in line 52).

**A**: Please see the response in weakness. W2 R5.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer KbKv (1)
comment: 
Dear reviewer, we have revised the manuscript and reuploaded the revised version, where the contents in cyan are revised.

## Weaknesses: 
The paper contains some flaws in its presentation and analysis. I have the following questions.

> - The proposed artifact metric has a corollary shown in Eq.(5). The authors mention that when $\mathbf{g}\_A \ll \mathbf{g}\_b$, the metric has some tolerance 1/2. Is there some specific meaning for this 1/2 here? Why it has to be 1/2? The authors should provide more explanation about this. 

**A**: To explain the reason why we set a tolerance of $\frac{1}{2}$ in AGD, we take two parallel tasks A and B as an example at any time.
- When $\\|\mathbf{g}\_\text{A}\\|\gg\\|\mathbf{g}\_\text{B}\\|$ and we update task B with $\mathbf{g}\_\text{A}$, the update produces huge impact on task B even if $\langle{\mathbf{g}}\_\text{A},\mathbf{g}\_\text{B}\rangle>0$. This is because a much larger gradient may escape from a local minimum. Thus, we define that the distance $\widehat{D}({\mathbf{g}}\_\text{A},\mathbf{g}\_\text{B})$ in this situation should be large (near $1$).
- When $\\|\mathbf{g}\_\text{A}\\|\ll\\|\mathbf{g}\_\text{B}\\|$, the update produces a mild impact on task B even if $\langle{\mathbf{g}}\_\text{A},\mathbf{g}\_\text{B}\rangle<0$. That means that updating with a zero gradient will neither improve nor damage the performance. Even though, we prefer positive influence rather than non-influence. Thus, we define that the distance $\widehat{D}({\mathbf{g}}\_\text{A},\mathbf{g}\_\text{B})$ in this situation is the mid-level between the best ($\widehat{D}=0$) and worst ($\widehat{D}=1$).
- Note that the tolerance can be controlled by adding a factor $\gamma>0$. Omitting the edge situation, we have 
$$
\widehat{D}({\mathbf{g}}\_\text{A},\mathbf{g}\_\text{B}) = \frac{\\|\mathbf{g}\_\text{A}-\mathbf{g}\_\text{B}\\|}{\gamma \\|\mathbf{g}\_\text{B}\\|+\\|\mathbf{g}\_\text{A}-\mathbf{g}\_\text{B}\\|}
$$
- The experiments on PS-CIFAR-100 with different tolerances are shown in the following table. The results show either larger or smaller tolerances compared to $\frac{1}{2}$ will get the performance drop. 

    |$\gamma$|Tolerance|$A\_T$|$F\_T$|
    |:---:|:---:|:---:|:---:|
    |0.2|$\frac{5}{6}$|69.283 |24.126 |
    |0.5|$\frac{2}{3}$|69.486  |24.520 |
    |1(ours)  |$\frac{1}{2}$|70.078  |24.907 |
    |2  |$\frac{1}{3}$|69.626 |24.344 |
    |3  |$\frac{1}{4}$|69.505 |24.479 |
    |4  |$\frac{1}{5}$|69.332  |24.600 |

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer KbKv (2)
comment: > - How was Eq. (8) inferred? Some detail about Multiple Gradient Descent should be discussed before jumping to Eq. (8)

**A**: To address your concern, we have revised the manuscript to make the method clearer. 
- *Inferring of Eq (8)*: At the current time,
    - if $|\mathcal{T}|=1$, it means that we have only one task pass and there exists no training conflict. Thus, we use the task's own gradient to update, i.e., $\mathbf{d}^*=\mathbf{g}\_i$;
    - if $|\mathcal{T}|>1$, it means that we have at least two tasks and we need to solve the training conflict problem. In our work, we define to find the optimal weights $w^*$ and the final update gradient is obtained by $\mathbf{d}^*=\sum\_{i\in\mathcal{T}}\mathbf{w}\_i\mathbf{g}\_i$. To avoid the self-interference problem, we use a matrix $W$ instead of the weight vector $w$, and solve it by using the proposed MaxDO approach. The weight matrix $W$ contains two parts, the diagonal vector $\grave{\mathbf{w}}$ and off-diagonal matrix $\widetilde{\mathbf{W}}$. To satisfy the condition $\sum\_i \mathbf{w}^*\_i =1$: (1) For the diagonal $\grave{\mathbf{w}}$, we have $\sum\_{i\in\mathcal{T}}\grave{\mathbf{w}}\_i=1$ according to MGDA. (2) For the off-diagonal $\widetilde{\mathbf{W}}$, we use softmax on each row, thus  $\sum\_{j\in\mathcal{T}}\widetilde{\mathbf{W}}\_{ij}=1$ for any row $i$. Thus, we have $\sum\_{i,j}{\mathbf{W}}\_{ij}=|\mathcal{T}|+1$. And in Eq (8), we divide the sum with $|\mathcal{T}|+1$ to guarantee the condition.

- *More details about MGDA*: We have added more description on MGDA in the revised manuscript. More details of MGDA can be seen in the Appendix.
    
    MGDA: 

    An elegant solution to the MOO for Pareto optimality[1] is the Steepest Descent Method (SDM)[2], which aims to obtain an optimal descent direction ${d}^*$ that satisfies
    $$
    \mathbf{d}^*,\alpha^* = \arg\min\_{\mathbf{d},\alpha} \quad \alpha+\frac{1}{2}\left\Vert \mathbf{d}\right\Vert^2,
    \text{s.t.}\quad \mathbf{g}\_i^\top \mathbf{d} \le \alpha,\quad\forall i \in \mathcal{T},
    $$
    where the constraints let each task have non-conflict with gradient ${d}$.
    Considering the Lagrange multipliers and Karush–Kuhn–Tucker (KKT) condition, the dual problem solved by the Multi-Gradient Descent Algorithm (MGDA)[3] is
    $$
    \mathbf{w}^*= \arg\min\_{{w}}\quad  \Big\Vert\sum\_i \mathbf{w}\_i \mathbf{g}\_i\Big\Vert^2,
        \text{s.t.}\quad \sum\_i \mathbf{w}\_i = 1\text{  and  }\mathbf{w}\_i\ge 0, \forall i .
    $$

    The objective of MGDA is $0$ and the resulting point satisfies the KKT conditions, or the solution gives a Pareto descent direction that improves all tasks.

- [1] Buchanan J M. The relevance of Pareto optimality. Journal of conflict resolution, 1962, 6(4): 341-354.
- [2] Jörg Fliege and Benar Fux Svaiter. Steepest descent methods for multicriteria optimization. Mathematical methods of operations research, 2000.
- [3] Jean-Antoine Désidéri. Multiple-gradient descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique, 2012.



> - The authors should describe more details about the experiments in section 4. Are the results of previous MTL methods in Table 123 produced by yourself? The authors should provide more implementation details. The evaluation metrics and description of table 123 are also unclear.

**A**: To address your concern, we have shown more experiment details in the revised manuscript. 
- *More experimental details*: In our experiments, we take a 2-layer MLP as the backbone network for PS-EMNIST and a Resnet-18 for PS-CIFAR-100 and PS-ImageNet-Tiny. The learning rate is set to 0.003, 0.0004 and 0.0005 for PS-EMNIST, PS-CIFAR-100 and PS-ImageNet-Tiny. The SGD in MaxDO has a learning rate of 50. Each task is data stream, i.e., each data point passes only once. For each task, we set the batch size to 128 per step.
- *Method production*: Yes, since we are the first work studying PCL, all results of previous MTL methods are produced by ourselves with the claimed setting in their papers.
- *Evaluation metrics*: Following previous continual learning works, we leverage two metrics to evaluate PCL methods.
    - Final accuracy ($A\_T=\frac{1}{T} \sum\nolimits\_{t=1}^T a\_{\bar{e}}^t,$): This metric is evaluated on all seen tasks when the last task finishes its training.
    - Final Forgetting ($F\_T=\frac{1}{T} \sum\nolimits\_{t=1}^T a^t\_{\bar{e}}-a\_{e\_t}^t$):  This metric is the accuracy difference between the final accuracy and the accuracy when each task just finish their training.

> - I didn't find any explanation or analysis of Figure 2. How does figure 2 support this paper?

**A**: 
Please refer to the last paragraph in Sec. 3.2. In this paragraph, we compared our AGD metric with Euclidean and Cosine distances and showed why it is necessary to utilize our asymmetric distance in Problem (3).

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer KbKv (3)
comment: > - Compared with cosine or euclidean distance, the improvement shown in table 4 looks marginal. And table 5 also show some drawback to the training time of the proposed method.

**A**: 

- *Performace improvement*: 
    - *Implementation*: For fair comparison, we have 27 different settings for each dataset, and we report the average values for each compared method in our experiments.
    - *Compared to other distance*: All this distance is based on the proposed MaxDO method. The optimization helps to reduce the effect of self-interference and makes the model perform well. But using these distance ignore the asymmetric feature of model update. Compared with our AGD, they still have a performance gap. However, only the distance differences can hardly result in a large gap. To show this, we evaluate the method that directly optimizes Eq. (6) (DualOptim) with Euclidean and Cosine distances. The results show that using other distances without MaxDO has significantly worse performance than the proposed AGD metric.

        |Method|$A\_T$|$F\_T$|
        |---|:---:|:---:|
        |DualOptim (EuDist)|67.765 |23.103 |
        |DualOptim (CosDist)|59.185 |17.376|
        |DualOptim (AGD)|68.866 |24.093|
        |MaxDO (EuDist)|69.344 |24.268 |
        |MaxDO (CosDist)|69.227 |24.552 |
        |MaxDO (AGD)   |70.078  |24.907 |

- *Time comparison*: Because of the embedding SGD in MaxDO, our method has a little more time cost. Even though, we think the time cost is at the same level as the compared methods. Considering this is the first PCL work, how to obtain efficient PCL is a future and interesting research direction for PCL.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer NWXM (1)
comment: 
Dear reviewer, we have revised the manuscript and reuploaded the revised version, where the contents in cyan are revised.

## Weaknesses:

> - (a) The writeup can be improved: in particular, the explanation of the main approach in Sec. 3.3 is vague and confusing. 

**A**: Thank you for your suggestion. In our revised manuscript, we make the following revision on Sec. 3.3.
- We provided more details about the relationship between Eq. (6) and (7). We illustrate why we use the weight matrix instead of a weight vector.
- We explained more about the optimization of Problem (7) and presented more discussions about the calculation of Eq. (8). We relate Figure 3 and the solving of Problem (7) more compactly.
- We give more descriptions of the Multiple Gradient Descent Algorithm (MGDA).

> - (b) Here are some typos: 
>   - (b1) Asymmetry → asymmetric in page 2 in the second paragraph 
>   - (b2) Last sentence of the second paragraph in Page 2 “instead magnitude itself” → missing “of” after “instead”
>   - (b3) Page 3 paragraph 3 incremental→ increments
>   - (b4) In (1) and (3), the objective functions are sets, which is meaningless. 
>   - (b5) Page 4 “will has” → will have 
>   - (b6) Is D in Lemma 1 supposed to be non-negative or non-positive? 
>   - (b7) Page 6 “a off-diagonal matrix” → “an off-diagonal matrix”

**A**: Thank you for your kind suggestion. We have corrected these typo errors in the revised manuscript (highlighted in cyan). Responses to (b4):
- We believe Eq. (1) and (3) are meaningful and necessary.
    - *The role of Eq. (1)(3)*: At any training step in PCL, multiple tasks make up a Multi-Objective Optimization (MOO) problem. Eq. (1) reveals the dynamic MOO definition. At each optimization step, the goal of MOO is to optimize multiple objectives simultaneously while updating only once. In our paper, the only updated gradient is denoted as $\mathbf{d}^*$, and Eq (2) represents the update using $\mathbf{d}^*$. The question is how to obtain $\mathbf{d}^*$. The motivation of Eq. (3) is that for an objective in MOO, its own gradient $\mathbf{g}$ is the most qualified update direction. Thus, finding an optimal $\mathbf{d}^*$ with a small distance to $g$ becomes the new objective, i.e., $\min D(\mathbf{d}^*, \mathbf{g})$ in Eq (3). However, in MOO, we need to consider the distance from $\mathbf{d}^*$ to multiple objective-specific gradients, yielding the new MOO problem Eq (3).
    - We believe it is necessary to illustrate related preliminary before introducing the details of our proposed method.

From: ICLR.cc/2023/Conference/Paper166/Authors
title: Response to Reviewer NWXM (2)
comment: ## Clarity, Quality, Novelty And Reproducibility:
> The explanation of the main approach in Sec. 3.3 is vague and I has difficulty understanding how the optimization problems (6) and (7) are solved. Besides, further details on how the authors design timelines and tasks in the experiments are not discussed. In general, the experiments are not reproducible based on the current writeup.

A: 

- *How the optimization problems (6) and (7) are solved?*:
    - *Relations of Problems (6) and (7)*: We define to find the optimal weights $\mathbf{w}^*$ and the final update gradient is obtained by $\mathbf{d}^*=\sum\_{i\in\mathcal{T}}\mathbf{w}\_i\mathbf{g}\_i$ (Eq. (6)). To avoid the self-interference problem, we use a matrix $\mathbf{W}$ instead of the weight vector $w$, and solve with the proposed MaxDO approach (Eq. (7)).
    - *Solving Problem (7)*: The weight matrix $W$ contains two parts, the diagonal vector $\grave{\mathbf{w}}$ and off-diagonal matrix $\widetilde{\mathbf{W}}$. The two parts are independent.
        - The diagonal vector $\grave{\mathbf{w}}$ is obtained by the Multiple Gradient Descent Algorithm (MGDA), which is used to obtain a weighted gradient not damage any tasks with optimization. MGDA can be easily solved by quadratic programming. The objective of MGDA is
        $$
        \grave{\mathbf{w}}^*= \arg\min\_{\mathbf{w}}\quad  \Big\Vert\sum\_i w\_i {g}\_i\Big\Vert^2,
            \text{s.t.}\quad \sum\_i \mathbf{w}\_i = 1\text{  and  }\mathbf{w}\_i\ge 0, \forall i .
        $$
        - The off-diagonal matrix $\widetilde{\mathbf{W}}$: The objectives of any two rows in the $\widetilde{\mathbf{W}}$ are different. For row $i$, the objective is the combination of non-diagonal entries, 
        $$
        \widetilde{\mathbf{W}}\_i^*=\mathop{\arg\min}\_\mathbf{w}\quad \left\\{\widehat{D}\left(\sum\nolimits\_{j\ne i}\mathbf{w}\_j{g}\_j,\mathbf{g}\_i\right)\Big|\forall i \in \mathcal{T}\right\\}.
        $$
- *How the authors design timelines and tasks?*: To address your concern, we illustrate the construction in the Appendix. In our experiments, we transform 3 image recognition datasets for PCL.
For effective transformation, several requirements are needed: (1) Random label set for each task, in which the data stream length of each task can be different; (2) Random timeline for each label set, in which the debut of each task can be any time between the first access of the former and latter tasks. For simplicity, we omit all blank time that all data streams are unavailable.
    - Parallel Split EMNIST (PS-EMNIST): We split EMNIST (62 classes) into 5 tasks and randomly generate 3 label sets for each task and 3 timelines for each label set (say 9 different situations). The size of the label set for each task, i.e., the number of classes, is set to larger than 2 while no more than 15.
    - Parallel Split CIFAR-100 (PS-CIFAR-100): We split CIFAR-100 into 20 tasks and randomly generate 3 label sets for each task and 3 timelines for each label set. The size of the label set for each task is set to larger than 2 while no more than 15.
    - Parallel Split ImageNet-TINY (PS-ImageNet-TINY): We split it into 20 tasks w.r.t. random 3 label sets, and each label set has 3 randomly generated timelines. The size of the label set for each task is set to larger than 5 while no more than 20.

From: ICLR.cc/2023/Conference/Paper166/Reviewer_UhaX
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The paper proposes a new method to update the parameters of the backbone network in PCL. Specifically, it provides an asymmetric distance to aggregate directions from different activated tasks. Several experiments validate the advantages of the method.
strength_and_weaknesses: The paper gives a nice observation that the asymmetric distance is more useful for aggregating the direction induced by different tasks. But, the paper should address the following concerns:

1. Since there are many different kinds of asymmetric distances, the paper should compare the proposed AGD with other asymmetric distances in analysis and experiments. 

2. It should give the details for optimizing the first term in Eq. (7) as AGD is nonsmooth. Moreover, it needs to check the coefficients in Eq.(8) satisfy the condition $\sum_i w_i =1$ or give some explanations that why the choice of coefficients is necessary.

3. In experiments, the author should give the details of the classifier which should be simpler than the backbone network. Also, it could test the performance of different backbones.

5. Writing issue. In page 4, lemma 1 is the definition and $D(x.y)\geq 0$ rather than $D(x,y)\leq 0$, the definition also contains if $D(y,x) = D(x,y)=0$ it implies $x=y$. See Andrea CG Mennucci.  On asymmetric distances.  Analysis and  Geometry in  Metric Spaces, 2013.

-------------------------- After rebuttal --------------------------------------------------

The authors have addressed most of my concerns and thus I increased my score as designing the proper metric is an important topic in PCL and the authors give a nice observation and method. Finally, I hope the authors can give some comparisons with some known asymmetric distances such as divergence.

-------------------------- After discussion --------------------------------------------------
It is worth mentioning that reviewer 6yMu has proposed some critical questions related to the inconsistency between the implementation and optimization models. The author needs more work and effort to clarify it. So, I return to my initial score.
clarity,_quality,_novelty_and_reproducibility: The paper is well-written and easy to read.
summary_of_the_review: The paper should address the aforementioned concerns for clarifying their significance.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper166/Reviewer_6yMu
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper studies parallel continual learning (PCL), a variant of continual learning in which tasks are processed asynchronously. That is, their availability during training is stochastic. Specifically, this paper tackles the problem of catastrophic forgetting and training conflict in PCL. Regarding catastrophic forgetting, the authors simply adopt existing methods that use replay-buffers. For the training conflict, the authors argue that they are produced by the interaction between task gradients during training, and propose a gradient-reweighting approach, MaxDO, which minimizes the maximum asymmetric discrepancy among multiply gradients. The proposed method is compared with MTL approaches in three different image recognition tasks.
strength_and_weaknesses: **Strenghts:**
- S1. The problem setting is interesting and understudied, and it provides a new environment where gradient conflict can occur.
- S2. The idea of having an asymmetric measure of discrepancy between gradients is really appealing and interesting.

**Weaknesses:**
- W1. While catastrophic forgetting is important in continual learning, I am not sure why does it have so much space in the experimental part, as the proposed approach is to use an existing solution.
- W2. The paper in general lacks intuition and justifications (more on that below).
- W3. Experimental results are marginally better.

Regarding W2: The manuscript have a number of assumptions and loose steps that need to be clarified:
- R1. Problem (1) and (3) are really loosely connected. What is the optimal d* supposed to be? And how does solving Eq. (3), which is tied to a specific local optimization process, connect with solving Eq. (1)?
- R2. What is the intuition behind the AGD metric? What does each of its argument represents semantically?
- R3. What does it mean to "have a tolerance of 1/2"? Why is that beneficial/desired?
- R4. Why is broadcasting the vector $w$ into a matrix $W$ justified? I guess it is to have independent parameters to solve Eq. (6), but why can we then combine them together?
- R5. Following R4, why can we combine these off-diagonal weights with MGDA? Even more, one of the steps is a local optimization process (SGD with Maximum Discrepancy), while the other is solved in one-step. Isn't there any harm in combing both approaches? And which learning rate do we set for the first step? is that important?
clarity,_quality,_novelty_and_reproducibility: **Clarity and quality.** The manuscript would benefit a lot from some polishing. There are quite a few grammatically wrong phrases (e.g. "the min-distance problem should be asymmetry"), and phrases which I unfortunately do not understand (e.g. "symmetric metrics mean the forward and backward influences are treated as symmetric") or too convoluted (e.g. "the asymmetric metric does not require the symmetric property that D(x, y) = D(y, x) in symmetric metric.").

**Novelty.** The application and most of the proposed method is novel. Yet it should be emphasized that it relies in existing methods for catastrophic forgetting and on MGDA for gradient balancing.

**Literature review.**
- Continual Learning. While mostly complete, I feel the paper miss some works such as "[Orthogonal Gradient Descent for Continual Learning](https://arxiv.org/abs/1910.07104)" which orthogonalized gradients in continual learning to avoid catastrophic forgetting.
- Multitask Learning. The literature review regarding MTL leaves quite some room for improvement:
  - Relevant works such as [CAGrad](https://arxiv.org/abs/2110.14048), [RotoGrad](https://arxiv.org/abs/2103.02631), and [GradVac](https://arxiv.org/abs/2010.05874) were omitted.
  - To introduce MTL, conflicting gradients works are cited, instead of the seminal work by [Richard Caruana](https://link.springer.com/article/10.1023/A:1007379606734).
  - The statement "Most existing MTL methods formulate the problem into finding an optimal gradient" is completely wrong. Gradient conflict in MTL is a relative recent direction and there is extensive work on other directions such as clustering approaches and adaptive networks.
  - MGDA applied to the backbone parameters is not introduced by Sener & Koltum. Their contribution was to apply MGDA (by [Désidéri](https://linkinghub.elsevier.com/retrieve/pii/S1631073X12000738)) to the feature produced by the backbone.
  - Similarly, saying that any of the methods described in 4.3 are state-of-the-art is wrong. As of today, there is no a clear state-of-the-art method.

**Reproducibility.** The experiments are not reproducible, as important details such as the learning rate for Eq. (7) are not provided. The provided code only includes a single file with an implementation of the proposed method, which has been stripped down from the working code, as there are undeclared variables (e.g. variable `lr` in line 52).
summary_of_the_review: The application of conflicting gradient solutions to PCL is interesting and intriguing, and the paper contains nice ideas such as using asymmetric metrics to compute the conflict between gradients. However, results are only marginally better than existing approaches, writing needs to be improved, and there exist a number of flaws and unjustified steps which I consider needs to be revised and improved before acceptance.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper166/Reviewer_KbKv
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper proposes an asymmetric metric to measure the discrepancy (or conflict) of gradients in different tasks from Parallel Continual Learning or multitask learning tasks. The authors also propose an optimization strategy named MaxDO that optimizes the dual problem of PCL with the Multiple Gradient Descent Algorithm.
strength_and_weaknesses: Strength:

Evaluating the gradients in multitask learning and parallel continual learning is an important and interesting topic but lacks attention in the community. The metric proposed in this paper contains some novelty, and the motivation behind the idea looks reasonable. It includes some potential impact that the authors point out that the metric that measures the effect of gradients should be asymmetric since the conflicts from one gradient to another in different tasks are different. They also compare the symmetric metric in the same scenario to support their claim. 

Weaknesses:
The paper contains some flaws in its presentation and analysis. I have the following questions.

1) The proposed artifact metric has a corollary shown in Eq.(5). The authors mention that when g_a << g_b, the metric has some tolerance 1/2. Is there some specific meaning for this 1/2 here? Why it has to be 1/2? The authors should provide more explanation about this. 

2) How was Eq. (8) inferred? Some detail about Multiple Gradient Descent should be discussed before jumping to Eq. (8)

3) The authors should describe more details about the experiments in section 4. Are the results of previous MTL methods in Table 123 produced by yourself? The authors should provide more implementation details. The evaluation metrics and description of table 123 are also unclear. 

4) I didn't find any explanation or analysis of Figure 2. How does figure 2 support this paper?

5) Compared with cosine or euclidean distance, the improvement shown in table 4 looks marginal. And table 5 also show some drawback to the training time of the proposed method.


clarity,_quality,_novelty_and_reproducibility: The representation of this paper is somehow clear. The idea is novel, and the motivation is reasonable. 

The paper has limited reproducibility due to the uncleared descriptions. The authors should provide more details about their experiments to improve reproducibility.
summary_of_the_review: Generally, I like the motivation and ideas the authors discussed in this paper. But I am hesitant to score because of the above flaws and concerns. I hope the authors will solve them in the rebuttal phase. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
details_of_ethics_concerns: Nothing
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper166/Reviewer_NWXM
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The authors study the Parallel Continual Learning (PCL) problem, where multiple data streams and corresponding tasks start and end during a timeline. The goal is to train a backbone model with a shared parameter to learn all the tasks. The main difficulty here is that gradients corresponding to different tasks may be in conflict with each other. To address this issue, the authors introduce a novel asymmetric distance measure the discrepancy among the gradients. Then they formulate an optimization problem to find the new direction using the asymmetric distance. They conduct experiments across three datasets and show that their model generally outperforms baselines in terms of (a) having better accuracy across tasks (b) and less forgetting across tasks. 
strength_and_weaknesses: Strengths:

(a) The authors study an interesting problem with broad applications. 
(b) The introduced distance and approach is noteworthy. 
(c) The experimental results are strong and show the better performance of the proposed method.

Weaknesses:

(a) The writeup can be improved: in particular, the explanation of the main approach in Sec. 3.3 is vague and confusing. 
(b) Here are some typos:
   (b1) Asymmetry → asymmetric in page 2 in the second paragraph
   (b2) Last sentence of the second paragraph in Page 2  “instead magnitude itself” → missing “of” after “instead”
   (b3) Page 3 paragraph 3 incremental→ increments   
   (b4) In (1) and (3), the objective functions are sets, which is meaningless. 
   (b5) Page 4 “will has” → will have 
   (b6) Is D in Lemma 1 supposed to be non-negative or non-positive?
   (b7) Page 6 “a off-diagonal matrix” → “an off-diagonal matrix”  



clarity,_quality,_novelty_and_reproducibility: The explanation of the main approach in Sec. 3.3 is vague and I has difficulty understanding how the optimization problems (6) and (7) are solved. Besides, further details on how the authors design timelines and tasks in the experiments are not discussed. In general, the experiments are not reproducible based on the current writeup. 
summary_of_the_review: The authors provide solid contributions and numerical results for an interesting application, but the writeup needs to be improved so that the reader can better understand the paper.  
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold