Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes MT-SGD for multi-objective optimization problems, which leverages the idea of SVGD to generate diverse particles w.r.t. the joint high-likelihood region for all distributions. Experiments on both synthetic dataset and real dataset (Multi-Fashion+Multi-MNIST and CelebA) demonstrate the effectiveness of MT-SGD compared to MGDA, Pareto MTL and MOO-SVGD.
strengths_and_weaknesses: Strength:
+ Writing is good and the paper is easy to follow
+ Finding diverse particles on the Pareto front is a well-motivated topic.
+ The proposed method is novel and inspiring

Weakness:
- Lack of convergence analysis of MT-SGD. Theorem 1 & 2 proves that MT-SGD reduces every KL divergence simultaneously, but it is unclear how the variational distribution $q$ or the particle set $\{\theta_i\}_{i=1}^M$ will be when it converged.
- The equation between line 166 and line 167 seems weird. It should be $p(\theta|D) \propto p(D|\theta) p(\theta)$. Besides, how the prior $p(\theta)$ is chosen in experiments?
questions: None
limitations: None
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposed a variant of SVGD for multi-task learning. In particular, instead of driving particles towards the high-density region of a single distribution like SVGD, the proposed MT-SGD drives the particles towards the joint region of all densities. The main novelty of this paper is to propose a weighted combination of the driving directions from each SVGD such that the combined direction is guaranteed to decrease all KL divergences between the current distribution and each target distribution. Additionally, like the connection of SVGD to SGD, the author showed the connection of MT-SGD to MOO. Empirically, the proposed MT-SGD achieves better results compared to baselines consistently. 
strengths_and_weaknesses: **Strength**:
This paper adopts the framework of MOO and extends to SVGD. The basic idea is very similar to MOO, the novelty comes from the incorporation of SVGD. The objective for searching $w^*$ can also be viewed as an adaptation of the MOO objective. In summary, the proposed approach is novel to some extent, but as mentioned by the author, this is not a completely novel framework. In terms of significance, the proposed framework demonstrates clear advantages compared to baselines and seems to be a good contribution to Bayesian multi-tasking learning. 

**Weakness**:
I have checked the proofs in the appendix, which seem to be correct. Thus, my main criticism is its presentation. I will elaborate more in the question section. 
questions: **Clarity**:
1. From my understanding, the main novelty is the objective in lemma 1. Is there an intuition on why $w^TUw$ is a reasonable objective?. It seems that this objective is very similar to the one used in MOO (Eq.3). But instead replacing the derivative of $L$ with the functional derivative of $D_{KL}$. Is this true? In that case, personally, I think it would be better if MT-SGD is introduced from the MOO point of view. In this way, it is also easier for the reader to understand intuition. Also, can MT-SGD be derived based on the Pareto stationary KKT condition?

2. In Eq.1, what does the comma mean? I understand this is a direct adaptation of MOO, but can you explain this minimization, since the typical minimization is targeted at a single value, but here, we have a list of $D_{KL}$. 

3. In line 49, do you mean empirical distributions?

4. In line 107, is the optimal transformation $T$ for the current step?

5. In line 112, should it be $O(\epsilon^2)$?

6. Just curious, can the objective $w^TUw$ in lemma 1 be translated to an optimization problem of the weighted combination of $D_{KL}$? 

7. In line 175, I suggest using a different subscript for $w$, since $t$ is already reserved for the number of SVGD steps in algorithm 1. 
8. For the captions of Figure 4, it should be red curve. 

**SVGD-related**:
1. One of my concerns about using SVGD is its mode collapse problem, which has been extensively reported [1]. This means without proper tuning of the repulsive force, the particles tend to collapse to a single point in high dimensions. Have you noticed this behaviour? If not, why?
2. In line 143, the author mentioned when $\sigma \rightarrow \infty$, $U$ reduces to the inner product of the score function. But it seems that the update of MT-SGD, in this case, does not recover the MOO? Since the kernel matrix in SVGD is now a matrix with value 1, which takes the gradient of other particles into consideration.


[1]: Gong, Wenbo, Yingzhen Li, and José Miguel Hernández-Lobato. "Sliced kernelized Stein discrepancy." arXiv preprint arXiv:2006.16531 (2020).

limitations: This work does not have any potential negative societal impact. In terms of the potential limitation, the author argues that the proposed MT-SGD tries to find the Pareto common, whereas the other baselines try to find the Pareto front. I am curious under what scenario Pareto front is preferred compared to Pareto common?
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this work, the authors introduce Stochastic Multiple Target Sampling Gradient Descent (MT-SGD), an extension of Stein variational gradient descent (SVGD) that tries to approximately sample from multiple distributions *simultaneously*. The authors then demonstrate that MT-SGD is well suited for Bayesian multi-task learning.
strengths_and_weaknesses: # Strengths

MT-SGD is a natural extension of SVGD that is theoretically motivated. Moreover, the authors were also able to demonstrate for RBF kernels, as the scale goes to infinity, MT-SGD reduces to multiple objectivation optimization which is amazing! Moreover, the computational complexity of the algorithm seems to be on the order of SVGD, with the main difference being solving the quadratic programming problem for finding the optimal combination of the gradients. While I'm not familiar with the SVGD and multi-task learning literature, this approach seems both highly novel and theoretically grounded. The experiments section is also compelling as well.

# Weaknesses

Firstly, I'm a little confused about the Multi-task learning formulation. Given the data, one would want to compute the following posterior distribution
$$ p(\alpha, \beta^1, \ldots, \beta^K \mid \mathbb{D})  \propto p(\alpha) \prod_{j=1}^K p(\beta^j) \prod_{i=1}^N p(y_{ij} \mid x_i, \alpha, \beta^j) $$
Conditioned on the non-shared parts, $\beta^1, \ldots, \beta^K$, the posterior for the shared parts is
$$ p(\alpha \mid \mathbb{D}, \beta^1, \ldots, \beta^J) \propto p(\alpha) \prod_{j=1}^K \prod_{i=1}^N p(y_{ij} \mid x_i, \alpha, \beta^j)  $$
where we can see that this conditional posterior already attempts to find an $\alpha$ that is good for all tasks simulatensouly. Moreover, just regular SVGD can be used for this. I don't see the advantage, nor the intuition, of formulating the problem as $K$ individual posteriors. I think it would strengthen the approach if they authors compared SVGD on the posterior I have outlined.

Next, while the experiments section is great, it is **very** concerning that there are no error bars for all the experiments in section 4.2, especially given how close the accuracies are for each of these tasks. Thus, I think it is imperative the authors run the experiments over a number of realizations to make me more confident in their results.

Please let me know if I have misunderstood anything! I will also be happy to raise my score if the authors get back to me about my concerns.
questions: 1. In section 2.2.1 in the appendix, there is an interesting phenomenon in Figure 2 where increasing the number of particles, $M$, seems to **deteriorate** the performance substantially.  A prior, I don't see why this should be the case. I would appreciate it if the authors could clear this up.
2. In  section on Multi-task learning, the authors should state that they are using a non-informative prior for the parameters of the model. 
limitations: While on the checklist the authors stated that they talk about the limitations in the appendix, I don't see it anywhere. To me, the two biggest limitations are the scaling with the number of tasks, $K$, and the number of particles, $M$ (which is inherent to SVGD). I think the authors should mention it in the work, though it doesn't detract from the novelty of the work.
review_title: null

Meta-review: metareview: The paper presents a particle-based method to approximate multiple target distributions simultaneously. The proposed particle-updating dynamics is shown to decrease the KL to every target (makes a Pareto improvement), and the resulting particles prefer the intersection of all targets (Pareto common) which differs it from a related method (which prefers Pareto front). Although the technical framework is not completely novel (follows MGDA (MOO)), reviewers agree that the proposed method for multi-distribution approximation is inspiring and the paper has well implemented the idea.

Nevertheless, there still remains a few imprecise statements that authors need to address upon acceptance.

1. Precise meeding of Eqs. (1,2). It is impossible that a single $q$ simultaneously minimizes each individual KL in general. Reviewer yFUe mentioned this but the reply did not clearly define this. The notation/formulation needs clarification even if it follows previous work.

2. Equation below Line 114 is only true if $\phi$ is in RKHS.

3. Some statements on MOO-SVGD might be improper. It seems conflicting that MOO-SVGD “updates the particles individually and independently”, while also “employs a repulsive term”, which is an interation among particles. More precise description is expected on (MOO-SVGD) “encourages the particle diversity without any theoretical-guaranteed principle to control the repulsive term”: MOO-SVGD is not originally intended for multi-distribution approximation, and it also provides a stationary distribution characterization.

confidence: Less certain
Decision: Accept