Many-Objective Multi-Solution Transport
Authors: Ziyue Li
Tian Li
Virginia Smith
Jeff Bilmes
Tianyi Zhou
Venue: ICLR 2025 Poster (2025)
Keywords: Multi-Objective Optimization
Mixture of Experts
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Optimizing the performance of many objectives (instantiated by tasks or clients) jointly with a few Pareto stationary solutions (models) is critical in machine learning. However, previous multi-objective optimization methods often focus on a few objectives and cannot scale to many objectives that outnumber the solutions, leading to either subpar performance or ignored objectives. We introduce ''Many-objective multi-solution Transport (MosT)'', a framework that finds multiple diverse solutions in the Pareto front of many objectives. Our insight is to seek multiple solutions, each performing as a domain expert and focusing on a specific subset of objectives while collectively covering all of them. MosT formulates the problem as a bi-level optimization of weighted objectives for each solution, where the weights are defined by an optimal transport between objectives and solutions. Our algorithm ensures convergence to Pareto stationary solutions for complementary subsets of objectives. On a range of applications in federated learning, multi-task learning, and mixture-of-prompt learning for LLMs, MosT distinctly outperforms strong baselines, delivering high-quality, diverse solutions that profile the entire Pareto frontier, thus ensuring balanced trade-offs across many objectives.

Published as a conference paper at ICLR 2025
MANY-OBJECTIVE MULTI-SOLUTION TRANSPORT
Ziyue Li! Tian Li²
Virginia Smith Jeff Bilmes+
Tianyi Zhou'* 1*
University Maryland Universiiyy of Chicago *Canngic Mellon University
University of Washington, Seattle (ziyueli, tianyi @umd. edu, Litian@uchicago.edi smithv@ cm/. edu,, bilmes@uwi edu Project: https: : / //gggtttuu com commtianyiiie Seayy--llb/aaaa MbMMoTT
ABSTRACT
Optimizing the performance of many objectives (instantiated by tasks or clients) jointly with few Pareto stationary solutions (models) is critical in machine learn- ing. However, previous multi-objective optimization methods often focus on a few objectives and cannot scale to many objectives that outnumber the solutions, leading to either subpar performance or ignored objectives. We introduce "Many- objective multi-solution Transport (MosT)", framework that finds multiple diverse solutions in the Pareto front of many objectives. Our insight is to seek multiple solutions, each performing as a domain expert and focusing on specific subset of objectives while collectively covering all of them. MosT formulates the problem as bi-level optimization of weighted objectives for each solution, where the weights are defined by an optimal transport between objectives and solutions. Our algorithm ensures convergence to Pareto stationary solutions for complementary subsets of objectives. On a range of applications in federated learning, multi-task learning, and mixture-of-prompl learning for LLMs, MosT distinctly outperforms strong baselines, delivering high-quality, diverse solutions that profile the entire Pareto frontier, thus ensuring balanced trade-offs across many objectives.
INTRODUCTION
MGDA
GDA
Fedfwy MosT (ours)
The underlying goal of many machine learning problems is simultaneously optimize multiple objectives. Usu- ally, no single solution (or model) is optimal for all objec- tives at once. Multi-objective optimization (MOO) aims to find a solution on the Pareto frontier where no objec- tive can be improved without degrading others. One ap- proach is to optimize a linear combination of all objec- tives, which may collapse to solutions for a small subset of objectives. Another method, multi multi-gradies gradient descent algorithm (MGDA) (Désidéri, 2012) finds a common de- scent direction each iteration to update the model so that no objective degrades. However, the trade-off's pro- vided by MGDA solutions are not fully controllable even with recent advances like reference vectors to guide the search space among all the objectives (Mahapatra & Rajan, 2020), especially when the Pareto frontier is unknown and complicated (e.g., non-smooth or discontinuous).
Linearization
FedProx
30
29
27
25
24
11
19
13
17
15
Figure 1: Accuracies of different meth- ods outputting 5 solutions serving 30 ob- jectives (clients) in federated learning. MosT results in better coverage of all the objectives than the other baselines.
Moreover, MOO approaches typically focus on two or three objectives and hardly scale to many objectives, as shown in Figure 1. As objectives increase, it is less plausible that they will reach an agreement on a single solution. Instead of balancing all of them, it is more appealing to find multiple diverse yet complementary solutions on the Pareto frontier each focusing
"Corresponding author.

---

Published as a conference paper at ICLR 2025
on local domain of objectives. This problem of finding Pareto solutions (or training m models) for n objectives can be understood as a multi-solution extension of MOO or a mixture of experts (MoE) (Jacobs et al., 1991) for multiple objectives. However, (e.g., those based on reference vectors or a uniform exploration of the Pareto frontier) can be computationally prohibitive and thus practically infeasible. It is also challenging to pre-determine the search regions of a few representative solutions profiling the entire Pareto frontier. The setting of n > m emerging in a variety of machine learning problems that involve many (i.e., big-n) users, domains, or evaluation criteria, each associated with different training objective, but the total available data or computation can only support the training of < n models.
increases, existing methods
In this paper, we ask: can we develop a colution-objectivv matching mechanism to guide the ex- ploration of m solutions on the high-dimensional (n D m) Pareto frontier? For example, some objectives share similar structures; so optimizing them with the same model can bring common im- provement, while optimizing separate models for objectives with mutual conflicts can effectively avoid poor performance and the tug-of-war among them. Hence, matching between models and objectives after every model update step is able to explore the correlation among n objectives along the optimiza- tion trajectory. Specifically, we capture the matching relations with a weight matrix I R R where I. reweighs the n objectives that model j E [m]] aims to optimize. For model j. such a reweighted single-model multi-objective optimization problem steers towards a domain expert focusing on locally consistent subset of objectives. With complementary (i.e., every objective being equally cov- ered) and balanced (i.e., no model dominating on most objectives) I'ss to adjust the descent directions per iteration, we aim to find m diverse yet complementary solutions that lie in the Pareto front.
More concretely, we model the optimization of I' as an optimal transport (OT) between the m models and n objectives, with two marginal constraints I'mm a' and P 1, = 3 in OT, which allow us to control the ratio of n objectives assigned to each model and the ratio of m solutions optimized for each objective. For example, with uniform OT weights to focus on different subsets of cesecives-otherwiine constraint could leave some objectives uncovered. Similarly, with uniform B (1/m)lm m. the training loads for the m models tend to be balanced so no model would dominate the others on a majority of the objectives (Figure 1).
(1/n)1n, the m < n models are enforced by
violate the I1mm = (1/n)
We propose an efficient algorithm for the above "Many-objective multi-solution Transport (MosT)" problem, which is bi-level optimization between model parameters and -oouutive-oolution matching parameterized by r. Specifically, the upper-level is a r-rrweighted MGDA problem for the m models. The lower-level is classic OT problem optimizing T with marginal constraints. We further introduce a regularization term on top of I to promote diversity of the optimal transport. Our algorithm converges to stationary points in the non-convex case, and converges Pareto stationary solutions in the strongly-convex case under additional stability assumptions. We additionally extend MosT to handle the n < m case by augmenting the n objectives with random linear combinations of them ("Mos'-EE"). In practice, we introduce curriculum for better model specialization by gradually varying the marginals a and in MosT-focusing more on 'models selecting objectives' at the earlier stage and later on transits to 'objectives selecting the best models'.
We apply MosT to various n >> m machine learning applications, spanning federated learning (McMa- han et al., 2017), multi-task learning (Lin et al., 2019), and mixture-of prompt learning (Qin & Eisner, 2021). Though the focus of this work is to address the challenges associated with scaling MOO to a large number of objectives, we also apply MosT to the n < scenarios, including fairness-accuracy trade-offs and other classic MOO problems (Appendix A.3). In all applications (Section 6), MosT finds diverse high-quality solutions on the Pareto front, consistently outperforming various strong baselines in terms of average accuracy and other popular metrics on the quality of multiple solutions, without extra computation cost.
mfxxure-o-pprompt
2 RELATED WORK
Single-Solution MOO. The classic goal of MOO is to find some solution lying on the Pareto front of multiple objectives (Désidéri, 2012: Roy et al., 2023; Halff'mann et al., 2022; Miettinen, 1999). One approach is to solve linearized aggregation (i.e.. weighted averuge) of all objectives. However,
'1, m denotes the m-dimensional all-ones vector and I' computes the row-wise sum of entries in r'.
2

---

Published as a conference paper at ICLR 2025
linearization, despite covering broad objective weights, may produce solutions in small Pareto front area (Boyd & Vandenberghe, 2004). The multiple-gradien descent algorithm (MGDA) (Désidéri, 2012) is widely used due to its capability to handle complicated Pareto fronts and compatibility with gradient-hased optimization. In this work, we use MGDA-style optimization methods in our algorithm, and compare with linearization- (Section 6), showing that our approach can find more diverse Pareto stationary solutions.
objectives (with different weights) empirically
Multi-Solution MOO. One line of work that aims to discover diverse solutions across the entire Pareto front builds upon MGDA and guides the search process via constraints of preference vectors (Lin al., 2019; Mahapatra & Rajan, 2020) or constraints of other objectives (Zafar et al., 2017). These methods do not generalize well to the setting where there are many objectives (constraints) or the model dimension is large, since the number of preference vectors to explore the whole Pareto front may depend exponentially on these factors in the worst scenario (Emmerich & Deutz, 2018). Even in the setting where there are only a few (e.g., two or three) objectives, diversity of the preference vectors in the action space (during exploration) may not translate to diversity in the solution space. We showcase the superiority of MosT relative to these works in Section 6, Some works that balance Pareto optimality and solution diversity cannot guarantee the final solutions are the Pareto front (Liu et al., 2021). For gradient-free methods, evolution strategies or Bayesian optimization (Coello, 2006; Sindhya al., 2012) has been explored to find multiple (as opposed to one) Pareto stationary solutions. However, they are usually not efficient when solving practical MOO problems in machine learning due the lack of gradient information (Liu al., 2021; Momma et al., 2022); hence, we do not compare with those methods.
Applications in Machine Learning. We demonstrate MosT's effectiveness on applications including cross-device federated learning (McMahan et al., 2017). There is extensive prior research on personalized federated learning (e.g., Smith et al., 2017: Ghosh et al., 2020; Wu et al., 2022), i.e., outputting multiple related models instead of one, to serve all clients. Our approach can be viewed as a personalization objective here. Note that our goal is not to achieve the highest average accuracy for federated learning, but rather, explicitly balance multiple objectives and guarantee that all output solutions are Pareto stationary. We also explore multi-task lcarning (Lin et al., 2019) and mixture- of-prompt learning (Qin & Eisner, 2021) on standard benchmarks where each objective is task or a training instance. For the n < m case, following the setup in prior MOO works (e.g., Zitzler et al., 2000; Mahapatra & Rajan, 2020), apply MosT to toy problem and address (algorithmic) fairness/utility trade-offs (two objectives).
3 MOST: MANY-OBJECTIVE MULTTI-OLLUTION TRANSPORT
In this section, we introduce MosT objective and the alternating optimization algorithm to solve it.
Let L.(-) (i e [n]) denote the empirical loss function of the i-th objective. When the number of objectives n is much larger than the number of solutions m, it is possible that the learnt solutions (for example, simply by running MGDA for m times with different randomness) cannot cover representative regions on the Pareto front. To address this, we use an assignment matrix I E x
with constraints I'mm a and r 1n 3 on top of the losses to enforce a balanced matching between objectives and solutions. We could additionally add a regularizer R(T)) to encourage more diverse assignment. Our many-objective multi-solution transport (MosT) objective is as follows. Find (fr.) such that every 9, (j E (m]) is the Pareto solution of m weighted objectives
ming, (L',gL((0)) (0z), ... LnnnLn (0,)), where T E arg minreee Eiejn) Eje(m) Te (1), + RT()), a (T TEER. T Raxm r1m = a, P 1 = 3).
(2)
(3)
E E", 19 ATh are two tunable vectors on and m-dimensiioaal probability simplexes, re- spectively. We encourage nondegeneracy of solutions by setting these vectors to follow uniform distributions, i.c.. a 1,/n, 3 1m/m. As discussed in Section 1, the constraint set Z prevents the undesired outcome where all objectives are matched with a subset of solutions or all solutions optimize subset of objectives. To explicitly encourage the diversity among the columns in I, we define a regularization term R(-)) as
R(T) Exe(ai maxye[m] Teg.
(4)
3

---

Published as a conference paper at ICLR 2025
Hence, only the maximum entry maxje[[] P.j in each row of I contributes to R(T). Under the marginal constraints on I' in Eq 3, minimizing R(T) leads to maxje(m) Fi,j 1/2 and zeros for the rest entries in each row-i, resulting in zero cosine similarity between different columns of Т. Moreover, if n > and (mod) column, securing an equal and disjoint partition of the n objectives to the m solutions, which indicates the diversity of objectives used to train the m solutions. Eq. 2 has weight 7 to balance R(T) and the transport cost. In the unregularized case when T = 0 (Eq. 2), the resulting I' would be a sparse matrix (Proposition 1) (Liu al., 2022a; Brualdi, 2006). Although enforcing marginal constraints without R(T) already leads to improvements over the baselines (Figure 9), empirically, we also showcase that leveraging this extra regularizer further benefits the diverse trade-offs among all objectives (Appendix A.5.3).
O, it has exactly "/m nonzero entries (with value 1/x) per
3.1 ALGORIHTMM FOR MOsT
At a high level, the bi-level optimization problem described above can be decoupled into two sub- problems (over I' and f1:m) when fixing one variable and optimizing the other. At each outer iteration, we first solve exactly by running an off-the-shelf OT solver (e.g.. IPOT (Xie et al., 2020)). Then we optimize by running a reweighted version of MGDA with min-norm solver (Désidéri, 2012). The exact algorithm summarized in Algorithm
Algorithm 1
Multi-Solution Transport
I: Input: objectives (L)()))=1+ a A", 3 A K
2: Initialize: solutions 81.m
3: for E {1.... T do
4: I't <- solution of Eq. 5 by an optimal transport (OT) solver given s
5: j E (1, ... m do
6:
for k E (1,... K] do
7:
dj + Eq. 8, where X is achieved by min-norm solver for Eq. 7 given I and 0z.
8:
0g - 02 + ndj
9:
end for
d' + dz: 03 8,
11: end for
end for Return 1:m
13:
From Algorithm 1, in each iteration, we first optimize I with Of.m fixed, i.e., finding the optimal transport matching) between the n objectives and the models by solving the following optimal transport problem with existing algorithms (Xie et al., 2020):
minren Σje[m] EiE[n]
(5)
Given the maximum entry per row in I, Eq. reduces to a new optimal transport transport problem with an augmented loss L,(-) + RT), solvable by an off-the-shelf optimal transport solver.
Fixing the optimal r, we then optimize reweighted version of MGDA across objectives (0,), n.j T., (0z)) for each solution 0j. j E [m]. To find Pareto stationary solutions, similar as MGDA, we aim find the common-descent directions d1amm for 01.m to ensure that all objective values do not increase at each iteration. This reduces to solving m MGDA-type MOO problems (more details in Appendix A. in parallel, aiming to find d, for every solution j E [m]
mina, max d. TT,, Vo, L,(6,) + Id,11
(6)
which rescales each objective's 's gradient Vo, (0,) by Ts.j. For simplicity, we will use (L, to denote V L,(,)) in the remaining of the paper. The dual of Eq. 6 is min-norm problem over variable A n as follows:
minxEA"
Eiejai
Vj E [m].
which can be solved by existing Frank- Wolfe algorithms (Fujishige, 1980). Given the optimal dual solution A from Eq. 7, the primal solution of d, (j [m]) to Eq. 6 can be derived by the following convex combination of the T-weighted gradients:
dø Eielo ATr VL.(O)).
(8)
4

---

Published as a conference paper at ICLR 2025
To understand the benefits of Eq. 7, let us consider the vanilla MGDA minxea" [Eeeta] (VV in which À may be biased towards the objective with a small gradi- ent norm (i.e., a well-optimized objective). In MosT, however, OT in Eq. 5 tends to result in large It, for a small (,()), , thus moving small gradient away from the origin in Eq. 7 (i.e., preventing a well-optimized objective from dominating d,).
method:
The MGDA direction d, guarantees that every objective with non-zero F. will be improved or remain the same after updating 8-- After obtaining dj. update the model parameters 0g by moving along this direction (Line in Algorithm 1). Optionally, we can also run such gradient descent steps for K steps in practice under the same r. In our convergence analysis (Section 4), we allow for K > and assume full batch setting with (0,) evaluated on all the local data of problem i, for all j's. Empirically, report our experiment results based on mini-batch gradients in Section
3.2 EXTENSION TO FEW-OBJECTIVE (n m) CASES
The MosT formulation discussed before is mainly motivated by the challenges of having many objectives in MOO. When >> m, the diversity of the models can be achieved by enforeing the two marginal constraints in the optimal transport problem. However, the diversity cannot be fully guaranteed when & m. For example, when n by even applying uniform distributions for and / (i.e., the strongest constraints for diversity), a trivial but feasible solution of I mm /m 0m/2: 1m/2 can collapse the m models to duplicates of only two different models, i.e. one minimizing the first objective while the other minimizing the second. To address this problem, we create (n' - n) > m dense interpolations of the n objectives (n' >> m) by sampling (n' n) groups of convex combination weights Wn+mmm on the simplex, i.e., E A drawn from Dirichlet distribution. Then each auxiliary objective L-(-) can be defined as
(0) & EiE(n) (i,),((),, yi=n+ 1, n''.
Thereby, we increase the number of objectives to n' >> m and MosT can be applied to achieving diverse models for optimizing the n' interpolations of the original objectives. This strategy can be explained as maximizing the coverage of the m models over the dense samples of the Pareto front regions using n' random reference vectors. We report the results of this technique with applications on a toy problem and fairnes//utility trade-off's in Appendix A.3,
3.3 A PRACTICAL SOLUTION-SPECIALIZA
CURRICULUM
In scenarios with diverse objectives, each corresponding to a distinct domain or unique dataset, a practical demand arises: optimizing multiple models and turning them into mixture of specialized experts (i.e., models). This allows each input sample to select the best expert(s) for inference. Such "objective selecting expert/models" or "objective choice routing" strategy corresponds to removing r1, = 3 from the constraint set in Eq. 3. But it may lead to training imbalance among the m models, c.g., model is chosen by most objectives while other models get nearly zero optimization. As training proceeds, the winning model(s) trained by more objectives tend to be chosen even more frequently; hence joint optimization of models can collapse training one single model.
To address this challenge, we propose a curriculum of varying the marginal constraints that progres- sively changes and 8 for different training stages. Initially, we mainly focus on enforcing uniform marginal distribution ,3 so every model receives sufficient training from multiple objectives. By relaxing the other marginal constraint over n objectives be slightly non-uniform, the m models have more freedom to choose the objectives they perform the best on (i.e., "model selecting objective" or "model choice routing") and we allow for slight imbalance among objectives.' 2 In later stages, the curriculum instead focuses more on enforcing a uniform a so every objective is covered by sufficient models. The marginal constraint 3 can be relaxed in this stage since models approach convergence. Empirical results in Appendix A.5.2 demonstrate the effectiveness of our curriculum strategy.
4 PROPERTIES OF MOST
In this section, we discuss how MosT encourages diverse solutions and provide convergence guaran- tees of Algorithm 1. Here we define solution diversity via the diversity of T, stated below.
2I seseeeeee objectives can be more difficult and it preferable learn them later when models improve.
5

---

Published as a conference paper at ICLR 2025
Definition 1 (Diverse Solutions), We informally say that set of solutions (0, keem)) more diverse if EE E cos(T", .f. ..) is small with some feasible I,, where cos( y Flll
This definition captures the goal diversifying solution specialization, ensuring at least one suitable model j (corresponding to arg max (i))) for every objective i, as losses are reweighted by r. We note that fixing the losses { (0,) (1,()))ieenn) je(m) when solving for I' without any regularization (i.e.. 7=0 in Eq. 5) the objective inherently results in sparse solutions, as stated in the proposition below. Proposition (Sparsity of T Brualdi (2006).. Any I that solves the optimal transport problem Eq. with -=0 has at most n +
non-zero entries.
Intuitively, sparse I's satisfying the marginal constraint Ω would prevent the scenarios where only subset of objectives are well-optimized. In Appendix A.5.3, we empirically verify that even without R(T), the sparse transport gives diverse solutions that balance all objectives. Also, setting R(T) to be the negative of our diversity measure (Definition 1), further encourage diversity as it is obvious to show that
Eie(n) max
Ti EiE(n) maxje[m] Ti,((0)
where (T) is defined as the optimal solution of Eq. 5 with a regularization constant r. We investigate training dynamics and diverse assignments between objectives and models Section 5.
4.1 CONVERGENCE
In this part, we analyze the convergence of MosT in Algorithm for both strongly-convex and non-convex functions. The alternate minimization scheme poses additional challenges to our analysis compared with prior convergence results in MOO (Fliege et al., 2019).
Theorem (Convex and Non-Convex). Assume each objective (,) is v-smooth. Given marginal
distribution constraints a E A and / e A"", under learning rate 7) = 2 after running Algorithm for T outer iterations with full batch multi-gradient descent, have that
1 EIETT EjE(m 35ld511 < (
T
We defer the full proofs to Appendix A.2.2. The main step involves leveraging the property of the common descent direction d, obtained from Eq. 8, i... for any E [rz] and [mz]. we have
VL d3
Our convergence rate is the same as that of nor- mal gradient descent for non-convex and smooth problems under a fixed learning rate. Note that our result uses full gradients of each objective for the min-norm problem (Eq. 6). In practice, we use K >>>>>>>>> to run multiple iterations to update the model parameters for each objective locally, and use stochastic mini-batch gradients in Eq. 6.
75.0
77.5
9.6
mmm 73
Tecentage
22.5
Parcentage of zero-valued entries (%)
70.0
Ki divergence on between consecutive iterations
67.5
50
100
150
200
250
300
350
400
loss
average test
oracle
Strongly-convex cases. (,) (i E [n]) is smooth and additionally strongly convex with respect to E Rd, we show that our proposed algorithm (using full gradients) can converge to Pareto stationary points with re- spect subset of matching objectives under a more restricted assumption on the stability of objective-model matching. Please see Theorem 2 in the appendix for complete statement and de- tailed proof.
When the losses
Training
Figure 2: (a) Left y-axis: Percent of zero-valued entries within I. Right y-axis: symmetric KL divergence between I' in successive iterations. quickly converges to sparsity. (b) Test loss av- eraged over all solutions vs. test loss of the best- performing solution for each objective (oracle). As training proceeds, average loss rises while or- acle loss continues to decrease, indicating trend of solution specialization and diversification.
ASSIGNMENT DYNAMICS DURING TRAINING
In this section, we empirically examine the optimal assignment matrix I during training, highlighting its advantages objective-solution matching and its impact solution specialization.

---

Published as a conference paper at ICLR 2025
MGDA
5.0
0.5
(orighter indicates greater clivese)
Unsarization
0.00
validation
0:5
3.0
Pair so attiion diiverity
80
2.0
Fed/vg
00
0
Accurase
60
00
FedProx
0.55
accuracy
40
Linearization FedAvg
MGDA
FedMGDA
0.5
Linearzzation
0.6
FedAvg
FedPrex
0.5
FedMGDA
FedProx
MosT
0
FedMGDA
MoST
MosT
100
200
300
400
100
200
300
400
20%
40%
60%
80%
training iteration
training iteration
group with worst performance (percentage)
Figure 3: (a) Training loss and test accuracy curves of each method. MosT demonstrates faster convergence with higher accuracy. (b) Diversity of solutions during training: each block column visualizes KL divergence of a pair solutions (brighter indicates larger value). MosT produces more diverse solutions. (c) Fairness: Accuracy of the worst 20%, 40%, 60%, and 80% client groups. Diversity leads to better tail performance among all the objectives.
Fast convergence r. We observe rapid convergence of by tracking its evolution with Kullback Leibler (KL) divergence between successive iterations (Figure 2(a)), detailed in Appendix A.4.
Sparsity of As training proceeds, becomes more sparse, with nearly 75.00% zero entries (Figure 2(a)). This implies that I prompts each solution focus only subset of objectives.
Sparsity promotes specialization. Figure 2(b) shows the mean test loss averaged across all solutions evaluated at all the objectives, and the loss when selecting the best-performing solution for each objective (denoted as oracle). Initially, the mean loss increases and then decreases, while the oracle loss consistently decreases. These trends indicate that using all the solutions the objectives is suboptimal. while abjective-specifi solutions can focus subsets of objectives, ultimately contributing to a more complementary and effective overall solution set.
all
MOST APPLICATIONS
In this section, we apply MosT to various MI. applications where n > m. Though we not specifically focus on the (less challenging) settings of m > MosT can naturally generate diverse solutions for those problems as well. We defer the readers to Appendix A.3, where we show superior performance of MosT relative to existing methods for the > case.
6.1 EXPERIMENTAL SETUP
This section outlines the experimental setup. We describe the specific setup for each application in their respective sections, and provide more details such hyperparameter values in Appendix A.4,
For all the applications, we least compare with the following baselines to generate m solutions.
Linearization-base MOO (abbr. Linearization below) where we optimize convex combination all objectives with m randomly-sampled sets of weights. Minimizing simple average all the losses (empirical risk minimization over objectives) is specific instance of using uniform weights. Running MDGA (Désidéri, 2012) independently for m times with different random seeds.
Evaluation metrics. We task-specific evaluation metrics, such as average accuracy (or tail accuracy) across all objectives or hypervolume (Zitzler & Thiele, 1999). We do not evaluate on hypevolume for many-objective scenarios due to its computational complexity and infeasibility in high-dimensional settings (i.c., many objectives). Each run is repeated three times using different seeds.
7

---

Published as a conference paper at ICLR 2025
Table 1: Federated Learning: Mean accuracy across clients (mean and std across 3 runs) on federated learning datasets. MosT outperforms the strong baselines.
Dataset
MGDA
Linearization
FedAvg
FedProx
FedMGDA+
MosT w/o R(TT))
MosT
Syn (0.0, 0.0)
77.22 77.22+0.41
75.91
75.71
75.60+0.42
75. 25.26.11.21
83,09
84.25
Syn (0.5.
87.09
87. 17.187
86,26-0.61
86.13 13..39
85.21
89.07 20:63
89.99
Syn (1.0, 1.0)
90.5210.13
89.87 10.51
88. 18.72
87.58 11.36
87.1611.0
10.00
92.21 10.08
FEMNIST
78.86
72.6220.05
72.47=0.19
72.45
20.08
80,94
81.16.0.08
6.2
FEDERATED LEARNING
One important scenario where > m is the cross-device federated learning application, where we jointly learn m models over a heterogeneous network of remote devices. The devices generate local data following non- identical distributions: hence we view the finite sum of empirical losses on each d;
device as one objective, i.c., (0) == J Ex= (0) where v, is the number of local samples on device i [ni), and le denotes the individual local loss sample MosT seamlessly integrates with the decentralized setting of federated learning by computing client-specific local updates (01.m) and aggregating them update the global model by r. Moreover, since clients are diverse, it is expected that the solution diversity benefits of MosT will contribute significantly to the final performance.
We conduct experiments synthetic data and Federated Extended MNIST (FEMNIST) (Cohen et al., 2017; Caldas et al., 2018) where the number of objectives n = 30 and n. = 206, respectively. We experiment on three synthetic datasets, denoted as Syn (P1.P2): P2). with different P1 and P2 controlling heterogeneity of local models and data, as detailed in Appendix A.4. We compare MosT with baselines described in Section 6.1 and state-of-the-art federated learning algorithms, including FedAvg (McMahan et al., 2017), FedProx (Li et al., 2020), and FedMGDA+ (Hu et al., 2022). We run each algorithm m times with different random initializations. It is worth noting that during evaluation, for all methods, we let each device pick a best model out of m solutions based on the validation set, and compute its performance. We then report the average and the quantile accuracy across all devices. As the results shown in Table 1, MosT outperforms the baselines by a large margin on all datasets. Furthermore, we have the following observations.
MosT results in significant convergence improvements. Figure 3(a) compares the training loss and test accuracy curves of different algorithms. Notably, MosT outperforms baselines with a lower training loss (faster convergence) and higher test accuracy (better generalization to unseen data).
MosT maintains diversity during training, We analyze how the diversity of solutions evolves for different algorithms. Diversity is quantified using the KL divergence between predictions of any pair of solutions generated by each algorithm. Figure 3(b) shows that initially, all algorithms exhibit high diversity due to randomized initialization. However, baselines witness notable decrease in solution diversity during training. In contrast, MosT maintains high diversity throughout the training process.
MosT promotes fairness in FL. MosT's improved diversity is anticipated to benefit clients typically overlooked by other algorithms, thus enhancing fairesss. To validate, we calculate the accuracy of the worst 20%, 40%, 60%, and 80% clients for each algorithm. As depicted in Figure 3(c). MosT outperforms the baselines by larger margin for clients with worse performance, which demonstrates that the diversity of MosT effectively promotes fairness in FL.
Furthermore, our study reveals that MosT strategically assigns diverse solutions for inference, preventing the collapse phenomenon in MOO (detailed in Appendix A.5.1).
6.3 MUL.TI-TASK LEARNING
MosT seamlessly extends its capabilities to multi-task learning by treating each task as an individual objective. Our experiments explore two real-world datasets, Office-Caltech10) (Saenko et al., 2010; Griffin et al., 2007) and DomainNet (Peng et al., 2019) with n = 4 and 6 objectives, respectively. We have = 3 solutions for Office-Caltech10 and m 4 for NetainNet. We conduct a thorough comparison with various sfate-of saiee--f-teeeaaae the-art multi-task learning approaches: MGDA, Linearization- based MOO, EPO which is based on user preference vectors (Mahapatra & Rajan, 2020), COSMOS (Ruchte & Grabocka, 2021), and TAG which identifies task grouping for MOO (Fifty et al., 2021). Similar to federated learning datasets, we select the best-performing solutions over the validation set for
8

---

Published as a conference paper at ICLR 2025
inference and report the accuracy averaged across all tasks. The results shown in Table 2, which demonstrates MosT's superiority over existing approaches.
Table 2: Multi-task Learning: Average accuracy across all tasks (mean and std across 3 runs) on Office-Caltech 10 and NomainNet.
Dataset
MGDA
Linearization
EPO
COSMOS
TAG
MosT
Office-10 10
80.74 80.74
61.2 61.2640.07
61 61.05+1.09
63.83+1.01 63.78.0.34
82.98 49,38 31.05
DomainNet
65.81
57.15
58.55
67.65
6.4 MIXTURE-OF-PReS PROMPT LEARNING
Another application is prompt learning for language models (Qin & Eisner, 2021), where solutions need to generalize well with diverse instances. address this, MosT trains m soft prompts to handle each training instance distinct objective, with being the total number of instances. We define the objective function as (0) == (,))), where 4 represents the loss on instance This allows MosT to tailor its learning process to each specific sample, adapting to diverse linguistic nuances. We experiment on three datasets from the SuperGL.UF benchmark (Wang et al., 2019). For all datasets, sample 128 training instances, while generating 3 soft prompts.
Note that MosT prioritizes training complementary solutions for multiple objectives, rather than their assignment. Simple ways, like selecting the best-performing solution over the validation set, are promising for scenarios like federated learning and multi-task learning. However, treating each instance an objective poses challenges in solution selection during inference. To address this, we train a dispatcher for each algorithm to learn correlations between prompts and instances (implementation details in Appendix A.4), selecting the highest correlated one for inference. We report average accuracy across all tasks, in Table 3. Results show Mos' exhibits significantly better ability to generalize to unseen instances compared to baselines.
6.5
ABLATION STUDIES AND COMPARISON WITH OTHER BASELINES
We conduct extensive ablation studies to validate the design of MosT, examining the necessity of OT (Appendix 1, 1) and MGDA (Appendix A.5.4), along with specific designs like solution- specialization curriculum (Appendix A.5.2) and diversity encouragement (Appendix A.5.3). These studies offer insights into the effectiveness of components of MosT:
Necessity of OT and MGDA. Comparing OT-generated and randomly generated weights reveals the necessity of OT for achieving balanced objective-solution matching. Additionally, MGDA consistently outperforms its linearization-bin based alternative, highlighting its effectiveness in parameter updates for weighted multi-objective optimization.
Effectiveness of curriculum. Our proposed scheduling of the marginal constraints (Section 3.3) boosts performance by over 2.00% and enhances training stability.
Benefits of diversity-enccuuragement regularization. It enhances performance and fairness.
Preventing collapse with OT. Comparison of three strategies introduced in Section 3.3, MosT, "objective selecting model", and "model selecting objective", reveals the collapse phenomenon in MOO, where limited solutions dominate all objectives. In contrast, MosT using OT with a two-way matching approach achieves more balanced distribution of objectives among models during training.
6.6 RUNTIME COMPARISONS
We enhance the computational efficiency of our algorithm employing several simple practices: I) detecting carly convergence of OT, 2) initializing transport plans using prior computations, 3) running MGDA subset of model parameters like the final layer's bias term in neural network, and 4) effectively reducing objectives in MGDA through the inherent sparsity regularized OT.
Table 4 presents the end-to-end runtime (in seconds) of different approaches federated learning datasets with n = 30 and n = 206. We see that the running time of MosT is comparable to that of baseline methods. Notably, the time needed for OT across datasets negligible due the of

---

Published as a conference paper at ICLR 2025
Table 3: Mixture-of-Prompl Learning: Test accuracy on three datasets of SuperGLUE benchmark (mean and std across 3 runs).
Task
MGDA
Linearization
Mos'I'
BoolQ 62.79
61.30+0.00
67.03_0.44
MultiRC
60.86
58.79
63.78 -0.15
WiC 40.80
57.16
62.38 =0.31
existing packages and the fast convergence of I (shown in Section 5), accounting for less than 1% of the total time. See Appendix A.6 for complete analysis of computation time in other applications.
Table 4: Runtime comparison (sec) on federated learning datasets.
Dataset
MGDA
Linearization
FedAvg
FedMGDA+
MosT
Syn (0.0)
219.86
225.90
222.22
516.25
217.59
Syn (0.5)
208.82
208.27
205.90
495.62
201.70
Syn (1.0)
269.44
268.33
270.65
557.58
260.50
FEMNIST
3522.83
3135.76
3147.62
5000.00
3368.63
7 CONCLUSION
In this paper, we propose "many-objective multi-solution transport (MosT)", aiming to find m Pareto solutions (models) that achieve diverse trade-offs among n optimization objectives. We investigate a challenging case of >> m, where existing methods often struggle with exploring a high-dimensional Pareto frontier. formulate MosT as a bi-level optimization multiple weighted objectives, where weights guide the exploration and are determined by optimal transport (OT) matching objectives and solutions. Our algorithm theoretically converges to m Pareto solutions by alternating between optimizing the weighted objectives and OT. Mos'T extends achieve diverse solutions for < m. We apply MosT to various machine learning problems, training m models to serve n users, domains, or criteria. Empirically, MosT outperforms other strong baselines in tasks like federated learning. multi-task learning, miixuree-of-prome fairness-accuracy trade-offs, and other MOO benchmarks.
ACKNOWLEDGMENTS
We would like to thank Shengjie Wang, Xin Yang, and Yuanyuan Yang for their insightful discussions in the carlier-stage exploration of this project's ''s initial idea. We appreciate the reviewers and area chairs for their constructive comments and suggestions.
REFERENCES
Arthur Asuncion and David Newman. Uci machine learning repository, 2007.
Stephen P Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
Richard A Brualdi. Combinatorial matrix classes, volume 13. Cambridge University Press, 2006.
Sebastian Caldas, Peter Wu, Tian Li, Jakub Konečný, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv preprint 1812iv:1112.01000 2018.
Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint 11014:1905.10044, 2019.
CA Coello Coello. Evolutionary multi-objective optimization: a historical view of the field. IEEE computational intelligence magazine, 1(1):28-36 2006.
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik. Emnist: an extension of mnist to handwritten letters. arXiv preprint 100.0333,,, 2017.
10

---

Published as a conference paper at ICLR 2025
US Supreme Court. Griggs duke power co, 1971.
Jean-Antoine Désidéri. Multiple-gradien descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique, 350(5):313-311 2012.
Michael TM Emmerich and André H Deutz. A tutorial on multiobjective optimization: fundamentals and evolutionary methods. Natural computing, 17:585-609, 2018.
Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Efficiently identifying task groupings for multi-task learning. Advances in Neural Information Processing Systems, 34: 27503-27516, 2021.
Jörg Fliege, A Ismael Vaz, and Luís Nunes Vicente. Complexity of gradient descent for multiobjec- tive optimization. Optimization Methods and Software, 34(5):949-955 2019.
Satoru Fujishige. Lexicographically optimal base of a polymatroid with respect to a weight vector. Mathematics of Operations Research, 5(2):186-196, 1980.
Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An efficient framework for clustered federated learning. In Advances in Neural Information Processing Systems, 2020.
Gregory Griffin, Alex Holub, and Pietro Perona. Caltech-256 object category dataset. 2007.
Pascal Halffmman,, Luca E Schäfer, Kerstin Dächert, Kathrin Klamroth, and Stefan Ruzika. Exact algorithms for multiobjective linear optimization problems with integer variables: A state of the art survey. Journal of Multi-Criterii Decision Analysis, 29(5-)):34-3633 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Zeou Hu, Kiarash Shaloudegi, Guojun Zhang, and Yaoliang Yu. Federated learning meets multi- objective optimization. IEEF Transactions on Network Science and Engineering, (4l):2039-2551 2022.
Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mixtures of local experts. Neural computation, 3(1):79-87, 1991.
Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. Looking beyond the surface: A challenge set for reading comprehension over multiple sentences. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, Volume (Long Papers). pp. 252-262, 2018.
Kuang- Huei Lee, Xi Chen, Gang Hua, Houdong Hu, and Xiaodong He. Stacked cross attention for image-text matching. In Proceedings of the European conference on computer vision (ECCV), pp. 201-216, 2018.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429-450, 2020.
Xiaoxiao Li, Meirui Jiang, Xiaofci Zhang, Michael Kamp, and Qi Dou. Fedbn: Federated learning on non-iid features via local batch normalization. arXiv preprint arXiv:2102.07623, 2021.
Xi Lin, Hui-Ling Zhen, Zhenhua Li, Qing-Fu Zhang, and Sam Kwong. Pareto multi-task learning. Advances in neural information processing systems, 32, 2019.
Tianlin Liu, Joan Puigcerver, and Mathieu Blondel. preprint arXiv:2209.15: 15466, 2022a.
optimal transport. arXiv
Xingchao Liu, Xin Tong, and Qiang Liu, Profiling pareto front with multi-objecttive stein variational gradient descent. Advances in Neural Information Processing Systems,
2021.

---

Published as a conference paper at ICLR 2025
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11976-11986, 2022b.
Debabrata Mahapatra and Vaibhav Rajan. Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization. In International Conference on Machine Learning, pp. 6397-6607. PMLR, 2020.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas. Communication-effici learning of deep networks from decentralized data. In International Conference on Artificial Intelligence and Statistics, 2017.
Kaisa Miettinen. Nonlinear multiobjectivve optimization, volume 12. Springer Science & Business Media, 1999.
Michinari Momma, Chaosheng Dong, and Jia Liu. A multi-objectivv/muliitaas learning framework induced by pareto stationarity. International Conference on Machine Learning, pp. 15895-15900 PMLR, 2022.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1406-1415, 2019.
Mohammad Taher Pilehvar and Jose Camacho-Collados. Wic: the word-in-context dataset for evaluating context-sensitive meaning representations. arXiv preprint arXiv:/808.0913 2018.
Guanghui Qin and Jason Eisner. Learning how to ask: Querying Ims with mixtures of soft prompts.
arXiv preprint arXiv:2104.06550 2021.
Abhishek Roy, Geelon So, and Yi-An Ma. Optimization on pareto sets: On theory of multi-objective optimization. arXiv preprint arXIv:2308.02145: 2023.
Michael Ruchte and Josif Grabocka. Scalable pareto front approximation for deep multi-objective learning. 2021 IEEE international conference on data mining (ICDM), pp. 1300-1311 IEEE, 2021.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In Computer Vision-ECCC 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part IV 11, pp. 213-226, Springer, 2010.
Karthik Sindhya, Kaisa Miettinen, and Kalyanmoy Deb. A hybrid framework for evolutionary multi-objective optimization. IEEE Transactions on Evolutionary Computation, 17(4):95-511, 2012.
Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet Talwalkar. Federated multi-task learning. In Advances in Neural Information Processing Systems, 2017.
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in neural information processing systems, 32, 2019.
Shanshan Wu, Tian Li, Zachary Charles, Yu Xiao, Ziyu Liu, Zheng Xu, and Virginia Smith. Mot- ley: Benchmarking heterogeneity and personalization in federated learning. arXiv preprint arXiv:2206.09222, 2022.
Yujia Xie, Xiangfeng Wang, Ruijia Wang, and Hongyuan Zha. A fast proximal point method for computing exact wasserstein distance. In Uncertainty in artificial intelligence, pp. 433-453. PMLR, 2020.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Faimeess constraints: Mechanisms for fair classification. In Artificial intelligence and statistics, pp. 962-970. PMLR, 2017.
12

---

Published as a conference paper at ICLR 2025
Eckart Zitzler and Lothar Thiele. Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE transactions on Evolutionary Computation, 3(4):257-271, 1999.
Eckart Zitzler, Kalyanmoy Deb, and Lothar Thiele. Comparison of multiobjective evolutionary algorithms: Empirical results. Evolutionary computation, 8(2):173-195, 2000.
A APPENDIX
A.1
BACKGROUND ON MGDA IN MULTI-OBJECTIN OPTIMIZATION
We first describe some background on the multi-gradient descent algorithm to solve multi-objective optimization.
Let L(0) E R be defined as
L( (.,,)))..III..(()))) TR.
(10)
The goal for the multi-objective optimization (minimization) problem is to find Pareto optimal solutions with respect to all objectives [., (n]. One line of method each iteration, to find common descent direction d for all objectives. Given the current model 0 we would like to find descent step to minimize each objective value. For the single-objective case, the direction is (V). For objectives, objective is to solve for d:
min
VL. d [d] FE(M)
(11)
and then apply d 0 rpd. If the optimal objective value of Eq. 11 is negative, then there exists a descent direction d' such that all objective values will be decreased. If 0 is Pareto stationary, then
and the optimal objective value is O. This formulation is equivalent
min b s.t.
ddl
(12)
2
Formally, have the following lemma.
Lemma (Good Descent Direction Désidéri (2012) ). Let d, b be the solutions of Eq. 12, then
1. If 0 is Pareto stationary, then d = 0 and b = 0.
2. If 0 is not Pareto stationary, then
< - |d] < <0,
(13)
VI,(0) d b, i [m].
(14)
Lemma (A Rescaled Version of Lemma 1). Let dy E Rd be the solution of Eq. 6 and Pa, some non-negative scalar, then
1. If 03 Pareto stationary, then d,
2. If 0g not Pareto stationary, then
-1 T,, S - 1d, i [m].
(15)
A.2 CONVERGENCE PROOFS
A.2.1 STRONGLY-CONVER CASES
MosT learns a matching between objectives and solutions represented by its non-zero entries. There- fore, show that proposed algorithm (using full gradients) converge to Pareto stationary points with respect to a subset of matching objectives in strongly-convex cases. We first make common assumption.
13

---

Published as a conference paper at ICLR 2025
Assumption 1. Each objective (,) (i E [n]) is 1-smoooh and j--strongly convex w.L. 0 E R
We introduce another assumption on objective-mode matching, as follows.
Assumption 2. There exists (s < oo) such that after iterations, all the non-zero entries in 1 remain non-zero, which are lower bounded by small constant c.
This assumption can be interpreted as that the assignment of solutions for each objective become stable during training. From Figure 2, we empirically observe that after certain iterations, the learnt will have the same non-zero patterns.
Theorem 2 Cerongly-Convexyy Let Assumption 1 and hold. Given marginal distribution constraints A" and B E A under fixed learning rate 1. after running Algorithm 1 for T outer iterations with full multi-gradient descent, we have for each solution j E [m]].
|10T** 071 s (1 unc) TK 110; 87 !|?
(16)
where 07 is Pareto stationary solution across objectives with non-zero 1M
Proof.
First, let us assume K = 1. At each iteration i, we have
df = Σ A A (O,) E [m]
(17)
for some (Ne A which is the solution of Eq. 7. First note that for every j [m]. 0, will converge. If df 0, then 8, has converged Pareto stationary point. Otherwise, for every objective Li we have monotonically decrease for the sequence: (,(0)+)) - L,, s Therefore, every solution will converge. We denote 8 as one of the Pareto stationary solutions of Eq. that solution 03 converges to. By definition and the properties of MDGA, we know that for every solution, every objective value will non-increasing throughout optimization. Hence, for every i E [n] and j E [m]. it holds that
- -
2 0.
(18)
For every i [n], the v-smoothness and y-convexity L; lead to
L,(05t)) L, - ndy))
(19)
s 1,())) (BL T d' |ndd' 1|2
(20)
2
< L,(8)) (0, T (0) 07)
P
100
-031 + d2 + V Indd (21)
2
By moving L,(0)) to the left-hand side and multiplying both sides by A,T (. we have
Σ
Ari (L.(0)+))- I.(0)))
(22)
KE/M)
xr VL.(0)) (03- 07 ndd)
kE[M]
(c(m)
*Ein]
(23)
As 1 2 0, then
Σ
Ar AT H 100 0711 < μe 100 0711.
(24)
15/m]
14

---

Published as a conference paper at ICLR 2025
Due to the Hölder inequality, we have Eve(n) A.Fi, $ WllalT..gllo == Bg S 1. Hence, we have
A (Lx(0)tt-- -x((0))
kE|n]
< X, (0) +7+ +ny)
Me
2 100 07]12
1/3 ||ndf
(25)
iE(m]
+ v/3 lnnd511
(0) - 0J) - nlld)11
8
0]|l²
2
2
s d' 00) (1 (1-
00 07 1| (taking 7 S +
||d
(26)
2
S d' d03 03) 2 1dd]11
"
μee
10]
0] || (using B3 11
2
(2nd (0) 07) + 1nnd
μe
2 100 0j1
=
21j
1
(2(0)
8)) (00 07) 10 03113)
2 00 0]
2n
1
(20
10 2110/12 2 05) 03 + 110 1 + 110g 10112-200++00 205+203)
μeE
110/ 071
=
21
2
1
(10))
112 2(0) 0) 10911) pC |10 1
=
21
1
(110 0311² 110 0311²) μe 00; 0
21
= 2m
(10) - 07 1|2 110 -0]11)
pC
0 13
(27)
2
Since during optimization, we guarantee that every objective value will be non-increasing each iteration, we have (0(+++ - (,,) 2 O. So the left-hand side of Eq. 22 is non-negative. Hence,
1 21) (10) 07 12 100+ 0713)
μi 100
071 > 0,
(28)
110++1 071² (1 mme)1103 031
(29)
which gives us linear convergence.
When > 1, at each outer iteration, fixing 17 we are running multiple updates on the model parameters. In this case, still have
| - -011 <
10 071
(30)
where ||0t,+t1|| denote the model parameters at the t-th outer iteration and l + 1-th inner iteration (Line 6 of Algorithm 1). Hence,
100 - -0 S (1 (un7) (amn))(105 110; 0712
(31)
holds.
A.2.2 NON-CONVEX AND SMOOTH CASES
For simplicity, we first consider the case where K 1. From Lemma 2, we know that at each iteration 1,
r (L.0) T S - |dl1 Vi E [n).
(32)
15

---

Published as a conference paper at ICLR 2025
Assuming y-smooth of each L;, we have Ti (Lx(0)+))) - (.())))
= ; (Ix(0)) + +nd) - I,(0)))
(33)
S Ln (0) dz wiis ||nd3]12
(34)
S 1 12 + pn² r T. ldd
(35)
2
S 2 Driاlldhi 11 + mm
2 
lygldfl (Tig 1)
(36)
711 - un)
(37)
2
Sum over all models j E [m],
The (L.(0)))) L.(0))) < 17(1 vt)
Σ
(38)
jE(mJ
jE[m)
The above result is for a single objective Li(.). Now let's consider the weighted sum of all the n objectives between two steps with different I's, i.e., I' and [e+1, By the optimality of p441
Σ
Σ
r++1 i.
T E [m]
Σ
max
Dt-1 1J
S
Σ jE(m)
r (o - T Σ
jE(m] i.''
max r
jE(m]
iE(m) jE(m!
(E(m)
iE(m)
We then have
Σ
Σ
(. - [ (0,))
(39)
ke|m| jE[m]
<
Σ
Σ
Th. (L:(0)4)) - Ie(03)) +T Σ max
JE(m)
p++1
i.
T
max n Σ
jE(m) 1J
(40)
jc(m]
xE|n
(E(m]
171 (mn)
S
r Elll 12
+
max r
max r (apply Eq. 38) JElm]
(41)
2
.E(m]
i.j
jE(m] iE(m)
iE(m]
<E(m)
1711 - mn)
B
max jE m
Σ
max D Ti. JE(m)
(42)
2
jE(m]
sE|m]
Here B, is the j-th dimension of 3 E A m in Eq. 3. Hence,
2
Σ
Bylld11
<
Σ (Fi5L,(0)) - - 1.(0)t1))
(43)
11(1 vn)
jE(m]
(E(m)jE[m)
+
7(1 - -mn)
2
イ Σ
jElml
max r++1
is
T Σ
max jE(ml
in
(44)
iE|n]
sE(n]
Applying telescope sum for E [T] on both sides gives
ΣΣ JE(m)
B 12 <
2 Σ Σ
(T)) -1T
1T+ L.(05+11)
(45)
11 vn)
(E(M)
JE(m)
max
T+ 15
Σ max Th
C.
(46)
11
vn)
JELm
jElm!
vE(x))
<e(m
Then we get the following bound on the average gradient norm:
>Σ C T JE
T
(47)
If we take vnj = 1. then C' = =O()). This gives us a O (%) rate in terms of gradient norms for non-convex cases under fixed learning rate.
16

---

Published as a conference paper at ICLR 2025
For the case where K
1, we have
Σ
Σ (TY)
1,(05t1) - , L,((
(48)
sE|n| jelml
s 7711 vn)
B
(49)
2
je(m
KE
< 7/( vn)
Bg
||
(50)
2
JE(m
where & denotes the index for inner updates on model parameters fixing 1 Similarly, we have the result
E C T EE [E JELmJ
1
(51)
T
A.3 EXPERIMENTS WITH n & m
Evaluation Metries. For applications with few objectives (small ni), we use the hypervolume measure, a widely used metric for evaluating the quality of MOO solutions and is a proxy of diversity (Zitzler Thiele, 1999). Hypervolume is feasible compute when the number of objectives is small. Given a solution set S C R" and set of reference points = [ri...., the Hypervolume of S measures the region weakly dominated by S and bounded above by r: H(S) A (ly E RR 3p S: S y and y S <r)), where A(-) denotes the Lebesgue measure. To ensure fair calculation of it, the reference points are kept consistent across all algorithms for each dataset. These reference points are determined either by following the settings of previous studies or by setting them as the upper bounds of the objective values from all algorithms to be compared.
C R",
Hyperparameters. For the extended version of MosT (denoted as MosT-E) described in Section 3.2, we introduce additional hyperparameters a1...., Onn and n' to handle the extension of existing objectives. The parameter a; represents the positive shape parameter of the Dirichlet distribution, used to generate diverse objective weights, and n' represents the number of extended objectives.
A.3.1 TOY PROBLEMS
We demonstrate the effectiveness of MosT on a toy ZDT problem set. is a popular MOO benchmark containing two objectives (n = 2) with oracle Pareto fronts (Zitzler et al., 2000). Specifically, we use ZDT-1, ZDT-2, and ZDT3, which are problems with 30 variables and exhibit convex, concave, and disconnected Pareto-optimal fronts, respectively. We compare MosT with two baselines described in Section 6.1, and two additional methods Exact Pareto Optimization (EPO) based on different preference vectors (Mahapatra & Rajan, 2020), and SVGD based on stein variational gradient descent (Liu et al., 2021).
Table MosT achieves higher Hypervolumes than the baselines on the ZDT bi-objective problem.
MGDA
Lincarization
SVGD
EPO
MosT
ZDT-1
4.02
5.72+0.01
5.54
4.40+0.01
5,87+0.00
ZDT-2
4.63
0.94
6.65
6.65
6.65 +0.00
6.88
ZDT-3
4.53=0.83
6.27+0.02
5.77
4.53
6.39+0.03
We report the Hypervolumes of each method in Table 5 and visualize the obtained solutions alongside the entire Pareto-optimal fronts in Figure for more intuitive comparison. The results in Table 5 demonstrate that MosT achieves higher Hypervolumes, indicating its superior ability to generate more diverse solution sets that cover larger areas. Further analysis of the Pareto fronts reveals the following observations: 1) EPO and SVGD prioritize reducing one loss, potentially resulting in biased trade-offs, with SVGD lacking guaranteed convergence to Pareto-optimal solutions; 2) MGDA produces diverse solutions but fails to cover the entire Pareto-optimal fronts; 3) Lincarization-ben based MOO is competitive baseline with high Hypervolumes, but its solutions do not provide satisfactory
17

---

Published as a conference paper at ICLR 2025
1.0
ZDT-1
1.0
ZDT-2
1.0
-1.96.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0 0.0
0.5
1.0
0.0
0.5
1.0
MGDA
Linearization
SVGD
EPO
MOST
Figure 4: Solutions derived by different methods (blue scatters) on the ZDT bi-objective task, with the oracle Pareto-optimal fronts for the objectives shown in red scatters.
diverse trade-off's, as evident from the Pareto fronts; 4) In contrast, MosT generates evenly-distributed solutions across the Pareto fronts.
Investigation for EPO and SVGD. Despite adhering to the official implementation of EPO, it fails to meet performance expectations due to its extensive requirement for preference vector sampling. Increasing the number of solutions (m) generated by EPO yields noticeable enhancements, particularly in ZDT-3 and, to lesser extent, in ZDT-1, as shown in Table 6. However, even with a larger m, EPO consistently lags behind MosT, which achieves well-distributed solutions across Pareto fronts without relying on extensive sampling.
worth noting that we focus on scenarios where computational constraints limit us to training only models while needing to address numerous objectives algorithms that afford better control over the number of generated solutions to achieve promising trade-off.
(n >> m.) Therefore, we prioritize
Table EPO's performance improves with larger m, yet still falls short compared to MosT, which achieves superior results with fewer m.
EPO (m = 5)
EPO (m = 100)
MosT (m = 5)
ZDT-1
4.40
4.46
5.87 10.00
ZDT-2
6.6540.00
6.65+0.00
6.88 60.800.00
ZDT-3
4.53
6.0940.00
6.39 10.03
Additionally, we conduct more experiments on SVGD with much larger number of solutions m (i.e., intentionally being unfair to our method). We observe that (a) using larger m can improve its performance, but (b) even with 100, SVGD still cannot outperform MosT (ours) with 5.
Table 7: SVGD's performance improves with larger m, yet still falls short compared MosT, which achieves superior results with fewer m.
SVGD (m = 5)
SVGD (m 100)
MosT (m
ZDT-1
5.54+0.12
5.73+0.03
5.87+0.00
ZDT-2
6.65+0.00
6.66+0.00
6.88+0.00
ZDT-3
5.77+0.15
6.01 +0.02
6.39 10.03
A.3.2 FAIRNESS-ACCURAN TRADE-OFFS (n < m)
In this section, we apply MosT to explore various trade-offs between accuracy and algorithmic fairness (i.e., statistical independence between predictions and sensitive attributes). Thus, the number
18

---

Published as a conference paper at ICLR 2025
of objectives is 2. However, in this scenario with limited number of objectives, using optimal transport to match solutions and objectives may produce feasible but trivial solutions as explained in Section 3.2. Hence, adapt the extension of MosT named MosT-E (introduced in Section 3.2).
As discussed in Section 2, prior works that address fairness-accuracy trade-offs can be limited due to the difficulty of setting constraints before training (Zafar al., 2017), or the mismatch between diverse exploration space and diverse solutions (Mahapatra & Rajan, 2020). MosT-E differs by sampling a wide range of preference vectors to encompass various trade-off' comprehensively and using optimal transport to automatically generate solutions that maximize coverage for all preference vectors. We quantify the fairness objective using disparate impact (Court, 1971), and optimize using its convex approximation (Zafar et al., 2017). We experiment on synthetic dataset (Zafar al., 2017) and real German credit dataset (Asuncion & Newman, 2007). We compare Mos' and MosT-E with MGDA, linearization-batt based MOO (which can be viewed as soft version of Zafar et al. (2017), and EPO (Mahapatra & Rajan, 2020), and select the best parameters for each method based on the highest Hypervolume validation set.
Synthetic
Solution
Solution
Solution
0.00
Solution
Solution
0.30
0.60
0.80
0.30
0.60
0.80
0
0.50
0.80
0.50
0.60
0.80
0.50
0.60
0.80
German
0.55
MGDA
Linearization
EPO
MosT
MosT-E
Figure 5: Hypervolumes (colored areas) formed by five solutions for classification loss (objective 1, x-axis) and fairness (objective 2, y-axis) on synthetic and German datasets.
Table 8: Hypervolumes (x 100) on 5 solutions with different fairness-accuracy trade-offs. MosT-E
achieves the highest Hypervolume coverage (fairness, accuracy) objectives.
MGDA
Linearization
EPO
MosT
MosT-E
Synthetic 3.28
6.7040.04
2.44
3.7840.04
7.65
German
5.14_0.06
4.9940.05
4.56
4.64
5.27 5.2710.06
MosT-E generates more diverse trade-off's Table shows that MosT-E achieves the highest Hypervolumes, suggesting superior quality of the solution set it generates. Furthermore, Figure demonstrates that MosT-E generates solutions that are not only more diverse but also more evenly distributed across the Pareto fronts.
MosT-E effectively addresses the problem of MosT under & m. When < m. MosT may assign models separately to dominate individual objectives, resulting in solutions without sufficient diversity. The solutions generated by MosT shown in Figure 5, align with our idea by predominantly prioritizing either low classification loss or low disparate impact. This limitation is effectively overcome by MosT-E, with diversely combining existing few objectives as new objectives.
A.4 EXPERIMENTAL DETAILS
We will detail the models and hyperparameters used for each dataset. All algorithms follow the same setup, including the mein-validation-tess. split, number of training epochs, and tunable learning rates.
Hyperparameters for baselines. addition to the standard setup, we ine-tune hyperparame- ters specific each baseline model, aligning with their original configurations. For instance, adjust the typerparameter responsible for scaling the proximal term in FedProx according to the recommendations provided in (Li al., 2020).
19

---

Published as a conference paper at ICLR 2025
Description of KL divergence used. We employ KL divergence to assess differences in I across iterations and solution diversity. Specifically, we utilize symmetric KL divergence, defined as (KL(PIIQ) where Q and P represent probabilities in the distribution of P and Q, respec- tively.
A.4.1
EXPERIMENTAL SETUP FOR TOY PROBLEMS
ZDT bi-objective problems (Zitzler al., 2000). It contains class of benchmark problems commonly used to evaluate optimization algorithms, particularly those designed for multi-objective optimization. These problems involve optimizing two conflicting objectives simultancously. We specifically employ ZDT-1, ZDT-2, and ZDT-3 to evaluate the performance of algorithms. We use multinomial logistic regression, maintaining a consistent learning rate of 0.005 throughout training. This configuration aligns with the established setup presented in (Liu et al., 2021). We run 1,000 epochs for the datasets.
A.4.2 EXPERIMENTAL. SETUP FOR FEDERATED LEARNING
Synthetic data (Li al., 2020). This synthetic dataset is specifically designed to provide controlled complexities and diverse scenarios for assessing the performance of algorithms. The synthetic data generation process relies on two hyperparameters, P1 and p2, which shape the dataset's characteristics. Pi controls the heterogeneity among local models used to generate labels on each device. While P2 governs the differences in data distribution among devices. Larger P1 or p2 introduces more heterogeneity. For the generated dataset, we conduct our experiments using a srain-validation-ees split ratio of 6:2:2. use multinomial logistic regression as the model and run 400 epochs total. The learning rates are swept from (0.005, 0.01, 0.05, 0.1) without decaying throughout the training process.
FEMNIST (Cohen et al., 2017; Caldas et al., 2018) 1). In addition to the synthetic datasets, also conduct experiments on the Federated Extended MNIST (FEMNIST) dataset, widely used real- world dataset in federated learning research (Li et al., 2020), using multinomial logistic regression. It comprises handwritten digit images from multiple users, encompassing 62 classes, including digits (0-9) and uppercase and lowercase letters (A-Z, a-z). The data is distributed across 206 clients, with each client holding a subset of the digit classes. This distribution simulates a real-world federated learning scenario, prioritizing data privacy and distribution concerns. We employ a convolutional neural network featuring two convolutional layers with ReLU activation, followed by max-pooling. Additionally, a fully connected layer maps the flattened features 62 output classes. We run 400 epochs for training. Learning rates are swept from {0.08, 0.1}.
A.4.3 EXPERIMENTAL SETUP FOR FAIRNESS- TRADE-OFF
In the context of fairness-accuracy trade-off, we experiment on two datasets, the synthetic dataset and the German dataset, introduced below. We employ multinomial logistic regression our model, conducting 20 epochs of training and sweeping learning rates from {0.08, 0.1}. We use the enhanced MosT-E for the German dataset. MosT-E extends the existing n objectives n'' by interpolating them with weights drawn from a Dirichlet distribution. We set all shape parameters, .i,..., an to the same value within the range 10.1, 0.5, 1.0). The number of extended objectives is chosen from [10, 15, 20). In practice, we observe that extending the original 2 objectives to 10 yields results similar to those obtained with 20 objectives.
Synthetic dataset (Zafar et al., 2017). The synthetic dataset contains 2,000 binary classification instances generated randomly as specified in (Zafar et al., 2017). Binary labels for classification are generated using a uniform distribution. It features 2-dimensional nonsensitive features generated from two distinct Gaussian distributions, and I-dimensional sensitive feature generated using a Bernoulli distribution.
UCI German credit risk dataset (Asuncion & Newman, 2007). This dataset comprises 1,000
entries, each characterized by 20 categorical and symbolic attributes. These attributes serve to classify individuals as either good or bad credit risks. Gender considered as the sensitive attribute this context.
20

---

Published as a conference paper at ICLR 2025
A.4.4 EXPERIMENTAL SETUP FOR MULTI-TASK LEARNING
In the realm of multi-task learning, assess the efficacy of MosT using the Office-Caltech 10 and DomainNet datasets. The number of objectives varies across the datasets: = 4 for Office- Caltech10 and n 6 for DomainNet. We initialize the models with pre-trained weights for both datasets, leveraging ConvNeXt-tiny (Liu al., 2022b) for DomainNet.
ResNet-18 (He et al., 2016) for Office-Caltech10 and
It is worth noting that Pareto Multi-task Learning (PMTL) (Lin et al., 2019) is notable method, but its exclusion from our comparison is due to concerns regarding computational efficiency, particularly when applied to large-scale real-world datasets.
Office-Caltech 10 dataset (Saenko et al., 2010; Griffin et al., 2007). The Office-Caltechl dataset comprises images from four distinct data sources: Office31((Saenle et al., 2010) (three data sources) and Caltech- (Griffin al., 2007) (one data source). These sources capture images using different camera devices or in various real environments with diverse backgrounds, representing different objectives.
DomainNet dataset (Peng et al., 2019). The DomainNet dataset includes natural images sourced from six distinct data sources: Clipart, Infograph, Painting, Quickdraw, Real, and Sketch. This dataset characterized by its diversity, covering wide range of object categories. For experiments, focus on a sub-dataset composed the top ten most common object categories from the extensive pool of 345 categories within NemainNet, following (Li et al., 2021).
A.4.5 EXPERIMENTAL SETUP FOR PROMPT LEARNING
We explore prompt learning across three datasets from the SuperGLUE benchmark: BoolQ (Clark et al., 2019), MultiRC (Khashabi et al., 2018 (), and WiC (Pilehvar & Camacho-Collados 2018). In our approach, each instance represents a distinct objective. This framework allows us delve into prompt learning using limited set of training instances while aiming for generalization to unseen test instances. We randomly sample 128 instances from the training dataset and evenly partition the original validation dataset to form both the validation and test datasets. Our training involves a soft prompt approach based on the T5-base model, with the base model parameters kept frozen. Parameter setup follows (Qin & Eisner, 2021).
For prompt learning, where each instance considered an objective, the absence of client groups or task types, as seen in federated learning multi-task learning, prevents from evaluating solution performance over the validation set and then selecting the best solution for inference. To address this, we train simple dispatcher to learn the correlation between instances and solutions (prompts), predicting the optimal solution for a given instance. Specifically, we train cross-attention on the hidden embedding of soft prompts and instances, with architecture following (Lee al., 2018) (Section 3.1). These hidden embeddings are generated from a fixed encoder of the T5-base.
A.S ABLATION STUDY ON MOST DESIGN
MGDA
Linear- ization
FedAvg
FedProx
FedMGDA+ MosT w/o R(T)
Syn (0.0, 0.0)
77.22+u.41 75.91 75.60 75.71 75.26 83.09
Syn (0.5, 0.5)
87.09+0 +0.09 87.18+0.27 86.26+0.61
83.13 0.39 85.21+1.12 89.07
Syn (1.O, 1.0)
10.12 89.87 10.51 88. 18.1210.75 87.58 1.36 87.16 91.7010.02
MosT (O)
MosT
MosT
MosT
w-
MosT (L)
MosT
(M)
(M, soft)
w/o CL
MGDA
76.65
67.62
74.66
81.97
76.80-0.79
82.62+0.a3
84.25 +0.51
86.94.0.0
78.9842.04
80. 1..
88. 19 0..4
86. 1.19
88.85 40.34
89.99 40.52
90.42 L0.22 10.22
75.20
73.20
91.25 10.55 10.51 89.32 10.76
91.2611.444
92.21 10.08
21

---

Published as a conference paper at ICLR 2025
To generate diverse and complementary solutions for multiple objectives, MosT first finds balanced matching between objectives and solutions by OT along with elaborated learning strategies and then locates the descent direction common to re-weighted objectives using MGDA. In this section, conduct comprehensive ablation studies to verify the MosT design. Specifically, for OT, we verify the necessity of it (Appendix A.5.1) and its specific designs, including solution-specializatie (Appendix A.5.2) and sparsity encouragement imposed by LI regularization (Appendix A.5.3). Additionally, we evaluate the necessity of MGDA in Appendix A.5.4. These ablation studies contribute to thorough understanding of the effectiveness of each component within the overall MosT framework. Experiments are carried out on three synthetic federated learning datasets, with results shown in Table A.5 and Figure 6.
curriculum
5.0
100
validation loss
3.0
80
2.0
1.0
Accuracy
MGDA
Linearization
FedAvg
FedProx
40
test accuracy
0.7
MGDA
FedMGDA
MosT (O)
Linea rization
MosT w/o
FedAvg
0.6
FedProx
FedMGDA
0.5
MosT (O)
MosT w/o
MosT w/o
0.4
MosT
250
400
training iteration
group with the worst performance (percentage)
Figure 6: Including MosT with series of ablation study results, (a) displays training loss and test accuracy curves; (b) shows the accuracy the worst 20%, 40%, 60% and 80% client groups. We omitted MosT (M) from panel for clarity, it exhibits inferior performance compared to other methods.
A.5.1 ABLATION STUDY FOR OT
OT-Generated v.s. Randomly Generated Weight Assignments. We compare the weight assign- ments generated OT with the randomly generated weight assignments. This can verify the impact of the choice of objective weighting method in MGDA the overall performance of MosT. In other words, we compare MosT with executing MGDA m times by using randomly generated weights to reweight objectives, which is denoted as w-MGDA Experimental results reveal that OT-generated weights work significantly better than random weights. This illustrates the necessity of using OT to find balanced match between solutions and objectives.
Different Matching Strategies. We also conduct ablation studies on the effectiveness of the optimal transport matching (Eq. 3) by comparing three strategies introduced in Section 3.3: 1) the original MosT objective, which utilizes optimal transport; 2) "objective selecting model", which selects the best expert/model for each objective (i.e., removing the I' 1, B constraint); and 3) "model selecting objective", which selects the best objective for each model (i.e., removing the I',,, = a constraint). These variants are denoted MosT (O) and MosT (M), resp., with results shown
22

---

Published as a conference paper at ICLR 2025
in Table A.5 and Figure 6. We also try "soft' version of MosT (M), utilizing normalized loss objectives when training each model, denoted as MosT (M. soft).
To ensure fair comparison, initialize comparisons with the same model weights. Throughout the training process, we track the assignment of objectives to each model, i.c., for every model, identifying the objectives with the smallest validation loss. We visualize the percentages of the selected objectives for each model over time in Figure on the Syn (0.0, 0.0) dataset. In the case of "objective selecting model" (middle), we observe that of the models progressively dominate all the objectives. Similarly, "model selecting objective" shows the early dominance of one model. While its soft version, MosT (M, soft), exhibits some improvement over MosT (M), it still lags behind other methods because of the imbalanced training over the objectives. These findings emphasize the necessity of achieving balanced model-objective trade-off. These observations confirm the presence of the collapse phenomenon in MOO, where limited solutions dominate all objectives. On the contrary, MosT using optimal transport involving two-way matching shows a more balanced distribution of objectives among the models throughout training.
0.5
022
200
400
400
200 selecting
200
200
400
optimal transport (OT) objective selecting
OT
Figure 7: The percentage of assigned objectives for each model under three matching strategies and two variants of MosTT Each color band represents model, with the y-axis indicating the corresponding percentage. We that MosT (leftmost) learns 5 diverse models that serve the 30 objectives a balanced manner.
A.5.2 ABLATION STUDY FOR OT DESIGN CURRICULUM LEARNING
We evaluate the impact of currieulum learning (from Section 3.3) optimizing multiple models using MosT.
Curriculum setup for MosT. As introduced in Section 3.3, proposed curriculum strategy involves adjusting marginal distributions a and 33 over different training stages to balance the freedom of "model selecting objective and 'objective selecting model'. In the initial stages, prioritize a uniform distribution for 3 to ensure exposure to multiple objectives. As training progresses, we transition a to a uniform distribution, covering all objectives, while relaxing This transition is achieved by a hyperparameter that gradually decreases from I to O. Though this hyperparameter gradually approaches zero, the transition direction differs: shifts 3 from uniform performance- oriented and, conversely, shifts ar in the opposite direction.
We compare standard MosT with a variant using uniform marginal distributions for a and 3 through- out training, denoted as Mos7' w/o CL. We hypothesize that curriculum learning enhances overall performance and training stability. We conduct experiments on three synthetic federated learning datasets. The results in Table A.5 show that using curriculum learning significantly improves the performance of MosT, proving its effectiveness. Notably, even without curriculum learning, MosT outperforms other algorithms. Furthermore, Figure 6 illustrates the training loss and test accuracy curves, highlighting the stability difference between the two approaches during training. Curriculum learning leads increased stability and better convergence towards better solutions.
A.5.3 ABLATION STUDY FOR OT DESIGN DIVERSITY ENCOURAGEMENT
As detailed in Section 4 and supported by empirical evidence in Section 5, encouraging sparse and balanced alignment between objectives and solutions leads to solution specialization on objectives. MosT goes step further by employing diversity regularized optimal transport to promote diversity.
Enhancing Diversity through Sparse Transport and Regularization. Before imposing diversity regularization to further enforce diversity, aim verify whether MosT without diversity regular-
23

---

Published as a conference paper at ICLR 2025
izatiin generate diverse solutions. We track solution diversity throughout the training process using KL divergence, as explained in Section 6.2, with results depicted in Figure 9. Our empirical findings indicate that even without R(T), sparse transport yields diverse solutions that balance all objectives. However, by setting R(T) to be the negative of our diversity measure (Definition 1), further encourage diversity. In Figure across objectives, assessed through normalized model accuracies. Notably, solutions trained with MosT exhibit tendency specialize in specific objectives, underscoring their heightened diversity compared to baseline approaches. And these diverse solutions then jointly perform better.
depict the distribution of model specialization
Performance Impact of Diversity Encouragement. Building on the motivation outlined in Sec- tion 5, this section focuses on showcasing the performance benefits resulting from diversity encour- agement. To assess the impact of diversity regularization, we conduct comparative analysis between scenarios with and without it. Table A.5 and Figure 6 highlight that MosT with diversity regularization not only enhances performance but also contributes to the fairness of federated learning.
A.5.4 ABLATION STUDY FOR MGDA
MGDA v.S. Linearization in Weighted Multi-Ohjective Optimization. We compare MosT that uses MGDA against the variant that updates model parameters based on the optimal transport solution weights. We aim to understand how effectively these two methods determine gradient updates for weighted multi-objective optimization. In this variant of MosT, denoted as MosT (L), instead of seeking the Pareto solution of m weighted objectives (as indicated in Eq. 1), we compute 8 as 0j -> Oj. where 03 represents the parameter 0, trained on data from the i-th objective. The experimental results showcase the consistent superiority of MosT over both MGDA and MosT (L) across three synthetic federated learning scenarios. proves the effectiveness of employing MGDA for parameter updates.
A.6 COMPARATIVE RUNTIME ANALYSIS
We assess the runtime of algorithms on the same platform, providing analyses for various applications: federated learning (Table 9), multi-task learning (Table 10), mixture-of-prompe learning (Table 11), ZDT datasets (Table 12), and fairness-accuracy trade-off (Table 13). Additionally, we include the computation time required for OT and MGDA within MosT.
Our results indicate that MosT exhibits comparable running times to baselines, despite MosT involving the computation of OT and MGDA, both of which only account for negligible time. However, for MosT-E, which explicitly extends the number of objectives, it will require more time than baselines.
Additionally, we provide a detailed decomposition on the complexity all the components below.
At each round, we alternate between the optimization of I and solving a reweighted MGDA problem. Utilizing IPOT, the complexity for optimizing I (Equation 5) roughly ((m)) [1]. Optimizing 81.m using reweighted MGDA (Equations 7 and 8) requires gradient access for each solution 0; (j E [m]), which takes up (mm) time total. Hence, The overall complexity of our algorithm per iteration is (mm). When
(our mainly focused setting), e.g., m Ollog n), the above complexity can reduce to On log n). Note that though the OT step and MGDA step both scale similarly, their actual running time is drastically different as MGDA requires gradient computation.
Table 9: Runtime (sec) comparisons for all methods on federated learning datasets, performed single Nvidia RTX A5500 platform.
MGDA
Linearization
FedAvg
FedProx
FedMGDA
MosT
MosT-OT
MosT-MGDA
Syn (0.0, 0.0)
219.86
225,90
222.22
281.69
516.25
217.59
1.00
0.44
Syn (0.5, 0.5)
208.82
208.27
205.90
258.19
495.62
201.70
0.92
0.23
Syn (L.O, 1.0)
269.44
268.33
270.65
333.74
557.58
260.50
0.98
0.35
FEMNIST
3522.83
3135.76
3147.62
3539.85
5000.00
3368.63
0.94
45.35
24

---

Published as a conference paper at ICLR 2025
(a) MosT (ours)
(b) MGDA
15 26
Solutions
Solutions
17 18
15
Objectives
Objectivs
(c) Linearization
(d) FedAvg
30
Solutions
Solutions
20
18 Objectives
Objectivs
(e) FedProx
(f) FedMGDA
Solutoo
Solutions
Objectives
Objecties
Figure 8: Proportion of 5 solutions (A/B/C/D/E), trained by MosT (a) and baselines (b)-(f), special- ized on 30 objectives (labeled as numbers). Notably, in (a), wider ribbons indicate that MosT-traines solutions address each objective with more specialization. In contrast, baseline solutions exhibit similar specializations across objectives. This highlights the enhanced solution diversity MosT, key factor contributing to its overall performance shown in Figure 1.
25

---

Published as a conference paper at ICLR 2025
MGDA
025
Incanzation
0.00
025
0.00
FedProx
0.25
FedMGDA
0.00
0.25
0.00
diversity ropul otiratio
MosT
0.00
0.5
0.00
100
300
400
training iteration
Figure 9: KT. divergence between pairwise solution predictions. Baselines show decreasing diver- sity over training iterations, whereas both MosT variants maintain diversity. MosT with diversity regularization fosters diversity.
Table 10: Runtime (sec) comparisons for all methods on multi-task learning datasets, performed on single Nvidia RTX A5000 platform.
MGDA
Lincarization
MosT
MosT-OT
MosT-MGDA
Office-Caltech0
465.73
669.88
775.24
37T.06
0.54
8.08
DomainNet
294.23
341.27
347.23
226.9
0.08
15.43
Table 11: Runtime (sec) comparisons for all methods on prompt learning datasets, performed on single Nvidia RTX A,000 platform.
MGDA
Linearization
MosT-OT
MosT-MGDA
BoolQ
2287.03
2170.97
1568 47
0.29
0.02
MulicC
2108.14
1820.73
1892.47
0.37
0.09
WIC
1286.30
1187.56
1254.51
0.37
0.05
Table 12: Runtime (sec) comparisons for all methods on ZDT datasets, performed on a single Nvidia RTX A4000 platform.
MGDA
Lincarization
SVGD
EPO
MosT
MosT-OT
MosT-MGDA
ZDT-
123.77
13.03
9.26
34.36
38.05
1.07
2.19
ZDT2
124.02
13.32
9.29
34.54
37.32
0.77
1.43
ZDT-3
140.85
14.98
10.08
37.75
40.82
0.87
1.99
Table 13: Runtime (sec) comparisons for all methods on fairness trade-off datasets, per- formed on single Nvidia RTX A4000 platform.
MGDA
Linearization
EPO
MosT-E
MosT-OT
MosT-MGDA
Synthetic
888.59
43.32
79.09
1159.17
0.15
0.08
German
454.64
23.98
43.10
595.34
0.14
0.03
26


Reviews:
[{"review_id": "lkirXw8Xah", "reviewer": "Reviewer_o2xh", "summary": "This paper proposes a multi-objective multi-solution transport (MosT) framework to identify solutions that achieve diverse trade-offs across multiple objectives. The proposed method aims to generate solutions over the Pareto set of a multi-objective optimization (MOO) problem to maximize diversity, particularly in cases with a large number of objectives.", "questions": "See my questions in the weakness part.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This work is well-organized and easy to follow. The idea is presented well.\n2. The proposed method can achieve good performance on different synthetic and application problems.", "weaknesses": "1. When we just focus on the optimization problem Eq. (1-3), it is a multi-objective bi-level optimization problem. This problem is not new; many recent works have focused on designing gradient-based methods to solve it, as seen in [1] and [2]. Clearly, solving $\\Gamma$ should rely on the current $\\theta$ according to Eq. (2), so $\\Gamma$ in the upper-level is a function of $\\theta$, i..e. $\\Gamma(\\theta)$. So, when we try to calculate the gradient of the objective function in the upper-level w.r.t. $\\theta$, there exists a hypergradient term $\\frac{d\\Gamma^*(\\theta)}{d\\theta}$. Can we just ignore the connection between $\\Gamma$ and $\\theta$ in the constraint part and directly write $\\Gamma$ instead of $\\Gamma(\\theta)$ in the upper level? Can the author provide some related work to support their approach? \n\n2. When the number of objectives is very large, I do not think MGDA-based methods can work well. The MGDA-based method needs to calculate the gradient of each objective in each iteration. This is not affordable when the model and the number of objectives are both large. I believe [3] can better solve such a problem, but the author does not discuss this work.\n\n3. MGDA is a quite old baseline. I believe there are many MGDA-based methods, but the author does not discuss those works listed in [4].\n\n4. MGDA and Linearization cannot reach the concave part in the Pareto front [4]. The author should discuss solution diversity with [4].\n\n5. Why does TAG fail in the MTL setting? In my view, the TAG just groups the tasks and trains these groups individually. It should not fail in such a setting. I quite doubt the authors' results.\n\n6. Does $\\Delta$ represent simplex in this paper? If so, the authors should use $\\Delta^{n-1}$ and $\\Delta^{m-1}$ in this paper, since $n-$simplex is a subset of $\\mathbb{R}^{n+1}$. The author can check the corresponding definition in Wikipedia: https://en.wikipedia.org/wiki/Simplex\n\n7. The article seems to have some compilation errors, e.g. $\\tau=0$ in Line 280 and different style dash in Line 74 and 86.\n\n8. Does $|| a ||$ represent the $l_2$ norm in this paper? I do not find where the author defines it since the authors also use $|| a ||_2$ in Eq. (6).\n\n9. I think the authors did not read their paper carefully. I see the expression \"Eq. equation *\" dozens of times in this paper (e.g., Line 196, 206), which I believe is an expression error.\n\n10. Eq. (45-46) $\\to$ Eq. (47) seems need $L_i$ to be bounded functions. The authors do not make this assumption.\n\n11. Lines 190-215 are just MGDA. I think those expressions can be placed in the preliminary part.\n\n---\n\n[1] Ye et al. Multi-Objective Meta Learning, AIJ, 2024.\n\n[2] Ye et al. A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimization, ECAI, 2024.\n\n[3] Yu et al. Enhancing Meta Learning via Multi-Objective Soft Improvement Functions, ICLR 2023.\n\n[4] Lin et al. Smooth Tchebycheff Scalarization for Multi-Objective Optimization, ICML, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kgiTIHwoU5", "reviewer": "Reviewer_z8sS", "summary": "The paper proposes MoST, an approach based on optimal transport, that finds multiple Pareto solutions with many (can outnumber the solutions) objectives.", "questions": "The convergence results seem to be regarding Pareto stationary only (which is fine). For convex and strongly-convex, is it possible to argue anything about Pareto optimality?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "It is interesting to see how optimal transport can be applied in MOO, to find diverse solutions. And I find the application natural and sound.\n\nThis paper addresses a particular aspect in MOO where number of objectives can be much larger than the number of solutions, and still find 'diverse' solutions. To the best of my knowledge, this aspect is not previously addressed enough in modern MOO (e.g. gradient-based methods that can handle DNN training) context.\n\nExperiments are quite comprehensive.", "weaknesses": "I don't find noticeable weakness in this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dNRB66Yfbc", "reviewer": "Reviewer_oKXp", "summary": "This work proposes a novel Multi-objective multi-solution Transport (MosT) method to find a small set of diverse Pareto solutions for problems with a large number of objectives. MosT is formulated as a bi-level optimization problem, where the upper-level problem is to determine different objective weights for each solution via optimal transport (OT) based on their current performance, while the lower-level problem is to find a Pareto solution for each subproblem with weighted objectives. MosT can also be generalized to tackle multi-objective optimization problems with few objectives via random objective interpolation. This work also provides theoretical analysis to prove MosT can find a set of Pareto solutions via solving the bi-level optimization problem. Experiments show MosT can achieve promising performance on different synthetic and application problems.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "+ This work is well-organized and easy to follow.\n\n+ The *small solution set for a large number of objectives* setting is important for many real-world applications. This work is a timely contribution to an interesting yet under-explored research direction.\n\n+ The proposed MosT method can achieve promising performance on different synthetic and application problems.", "weaknesses": "I was a reviewer of an earlier version of this work, and I have read the current submission in detail. I have the following major concerns about the proposed MosT method. \n\n**1. Motivation**\n\nThe key motivation of this work is to propose \"a framework that finds multiple diverse solutions in the Pareto front of many objectives\" and \"ensures convergence to Pareto stationary solutions for complementary subsets of objectives\". However, the terms diversity and complementary subsets are not clearly motivated and discussed in this work. The informal definition of diversity in Section 4 is helpful but not enough. How does the diversity connect to the complementary subsets of solutions found by MosT? Can we clearly define the \"complementary subset\" for MosT? Will the optimal solution set of MosT optimize some concrete diversity metrics?\n\n**2. Balanced Weight Matrix**\n\nFor the proposed optimal transport based solution-objective matching method, it is not very clear why we prefer a balanced weight matrix for the objectives (i.e., \"no model dominating on most objectives\"). For a real-life application, if most of the objectives do share similar or even nearly identical structures, it is expected that one single solution can solve them well at the same time. \n\n**3. Theoretical Analysis**\n\nIn MosT, the MGDAs can reach any Pareto optimal solution of each weighted problem, which is exactly the same Pareto optimal set of the original problem. In other words, the current theoretical guarantee of MosT is not stronger than the original MGDA. The reason why MosT can be ensured to find a set of diverse Pareto solutions does not have proper theoretical support.\n\n**4. Related Work**\n\nIn most experiments, the final metric for comparison is the mean of average performance across all objectives. Why not directly optimize this metric using a specially designed method like the sum-of-minimum optimization method [1]? It seems that the method in [1] can efficiently match each objective-solution pair via a clustering-like approach. What are the pros and cons of MosT compared with this method?  \n\n[1] Efficient Algorithms for Sum-of-Minimum Optimization, ICML 2024.\n\n**5. Runtime Analysis**\n\nFor the runtime comparison in Table 4, it is counter-intuitive to see linear scalarization require more runtime than MGDA. What is the computational overhead of linear scalarization over MGDA? \n\n**6. Experiment**\n\nComparison with the related sum-of-minimum optimization method [1] is needed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VnLuAeQSJD", "reviewer": "Reviewer_RKfr", "summary": "This paper introduces a framework called \"Many-Objective Multi-Solution Transport\" (MosT) aimed at efficiently optimizing numerous objectives across a limited set of models. Traditional multi-objective optimization methods struggle to balance many objectives simultaneously, often focusing on only a few, which can lead to neglected objectives or poor performance. MosT overcomes this limitation by assigning each model to a complementary subset of objectives, ensuring diverse solutions across the Pareto frontier. It does this by using a bi-level optimization process: (1) an optimal transport (OT) method matches objectives to solutions balancing the performance trade-offs (2) a multi-objective optimization (MOO) per solution to optimize assigned objectives using MGDA. The approach demonstrates success across applications such as federated learning, multi-task learning, and prompt learning in language models, outperforming simple baseline methods in terms of both accuracy and diversity.", "questions": "* The paper claims \"We do not evaluate on hypevolume for many-objective scenarios due to its computational complexity and infeasibility in high-dimensional settings (i.e., many objectives)\". Why? What's the time complexity of computing hypervolume of  M solutions over N objectives?\n\n* In the chosen evaluation metric of average accuracy, what's the average over? Is it over all M solutions and all N objectives (average over MxN items)? \n\n* In 3.2, the paper discusses the case of having more solutions than objective (M>N). What's the goal in here? We can clearly form \"experts\" i.e. each solution optimizes a single objective. Is our goal to capture M different trade-offs among N objectives even though M>>N?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- **Novelty for MOO:** The paper introduces a bi-level optimization method based on a very natural idea. Combining MGDA and OT is a elegant idea.\n\n- **Specialized Model Expertise:** It's a keen observation to see that each model naturally becomes an \"expert\" for a specific subset of objectives as a result of sparse OT solution.\n\n- **Applicable to multiple applications:** MOST demonstrates success across federated learning, multi-task learning, and mixture-of-prompt learning, highlighting the framework’s adaptability.", "weaknesses": "- **Missing stronger baselines**: The paper compared MOST against trivial benchmarks. If the goal is to compare methods based on their diversity and the coverage of Pareto Front, nontrivial benchmarks like [1] (and benchmarks therein) must be also taken into consideration. \n\n- **Evaluation Metrics**: Hypervolume [2] has been the established metric for evaluating and measuring coverage in Pareto Front for MOO methods. The reviewer does not see average accuracy across all objectives in all models as a good metric. \n\n\n[1] Ren, Yinuo, et al. \"Multi-objective optimization via Wasserstein-Fisher-Rao gradient flow.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2024.\n\n[2] https://gist.github.com/silviutofan92/df50cce796f226c57e958a6fc744b9fb", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iiGCLpf7id", "reviewer": "Reviewer_GmMP", "summary": "In this paper, the authors propose a novel method called many-objective multi-solution transport (MosT) for finding $ m $ multiple solutions for $n$ many-objective optimization. The framework is primarily focused on the case where $ n \\gg m $, but the paper suggests it is also suitable for cases when $ n \\ll m $. The problem of finding the aforementioned (few good) multiple solutions for many objectives is formulated as a bilevel optimization problem. In this setup, the lower-level problem is to find the optimal transport (OT) between a given set of objectives and solutions under certain constraints, while the upper-level problem is to determine the solution parameters by solving a multi-objective optimization (MOO) problem, weighted by the solutions of the lower-level OT problem. The paper provides convergence guarantees for MosT under non-convex, convex, and strongly convex settings. Additionally, it presents empirical results for the performance of the MosT method by applying it in federated learning (FL), multi-task learning (MTL), and mixture-of-prompt learning applications. The results suggest that MosT can be efficiently implemented in real-world applications and can outperform existing baselines for finding multiple solutions for many objectives.", "questions": "* If the goal is to find a set of solutions that can work on all objectives,  in $ n\\ll m$ case, why MosT is superior to optimizing separate parameters for each $n$ objective, to obtain $m=n$ solutions? \n* What is the rationale behind computing the MGDA weights just by considering the final layer bias weights?\n* In the FL setting, why client diversity is desirable, and why MosT does not seem to achieve consensus throughout the training process ( please refer to point 3 in Weaknesses for more details )?\n* Can the authors provide some qualitative/quantitative comparison between the proposed method and prior work [1] ?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The problem discussed in the paper is an interesting one, and seems to not have been given much attention in prior related work\n* The paper provide theoretical guarantees and empirical validation of the efficacy of the proposed method.\n* The flow of the paper towards the build up of the proposed method is fairly clear and easy to follow.", "weaknesses": "* In Figure 1, it is better to indicate the fact that the performance is based on 5 different solutions, and also discuss how you choose a particular solution (out of 5 solutions) for a particular objective in the text when you refer to Figure 1.\n\n* It is unclear why MosT can be useful for $n \\ll m$ case. Intuitively, shouldn’t separately optimizing each objective with separate parameters give you the smallest best possible solution set in this setting?\n\n* Figure 3 shows MosT achieves maximum diversity among clients, compared to other baselines. However, it is unclear why the diversity of parameters among clients in a distributed optimization setting is desirable since usually, the goal is to train a single global model using knowledge from distributed data among the clients. Also, it is unclear why the client models are diverse throughout the training process. Specifically, aren’t the client models aggregated and redistributed periodically, as done in FL traditionally?\n\n* The rationale behind computing the MGDA weights just by considering the final layer bias weights seems not clearly justified in the text.\n\n* Some closely related recent work [1] seems to be missing from discussion and comparison.\n\n**Minor comments**\n\n* In lines 349-351 “Initially, the mean loss increases and then decreases..” seems to be opposite to the observation from Figure 2?\n\n[1] Few for Many: Tchebycheff Set Scalarization for Many-Objective Optimization. Arxiv (2024)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YVCldB2A8I", "reviewer": "Reviewer_ew6B", "summary": "This paper presents Many-objective multi-solution Transport (MosT), a framework for optimizing many objectives simultaneously by finding diverse solutions on the Pareto front. Unlike traditional methods that struggle with numerous objectives, MosT assigns each solution to specialize in a subset of objectives, collectively covering all. Using bi-level optimization with optimal transport, it ensures convergence to Pareto stationary solutions. MosT outperforms baselines in applications like federated learning, multi-task learning, and LLM prompt learning, providing balanced trade-offs across many objectives.", "questions": "1) Since only one cited paper is from 2023-2024, question is that whether this reflects a small lag in the field’s development or if the paper is not keeping up with recent advancements. \n\n2) When comparing to works like \"Personalized Federated Learning under Mixture of Distributions\" and \"Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning,\" could you emphasize the key differences and advantages of your method in solving N objectives with M models?\n\n3) If appropriate, I suggest including papers such as \"Personalized Federated Learning under Mixture of Distributions\" and \"Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning\" in both the related works and experiments sections. If inclusion isn't suitable, please provide an explanation for this choice (also fine if reasonable).", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1) The paper presents a solid motivation based on the provided background (though further elaboration might be helpful, see weaknesses for details).\n2) The experiments show strong performance when compared to the baselines.\n3) The study includes both experimental results and theoretical analysis, providing a comprehensive evaluation.", "weaknesses": "1) The methods and baselines referenced in the related works and experiments seem somewhat outdated, especially considering the paper targets ICLR 2025. Only one cited paper is from 2023-2024, raising concerns about whether this reflects a small lag in the field’s development or if the paper is not keeping up with recent advancements. \n2) In both multi-task learning and federated learning, several methods have been proposed to address the challenge of using M models for N objectives. For example, in federated learning, \"Personalized Federated Learning under Mixture of Distributions\" uses EM algorithms, while in multi-task learning, \"Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning\" (HarmoDT) employs bi-level mask learning. These approaches could have been considered in the context of this work.\n3) The ablation studies shown in Section 6.5  are insufficient and lack in-depth analysis. For instance, in the conclusion regarding the \"Benefits of diversity-encouragement regularization,\" the explanation (line 475) is limited to stating that it \"enhances performance and fairness\" without providing detailed evidence or elaboration. More rigorous ablations would strengthen the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)