ICML 2025

Aligned Multi Objective Optimization
Yonathan Efroni 1 Ben Kretzu 2 Daniel R. Jiang 1 Jalaj Bhandari 1 Zheqing Zhu 1 Karen Ullrich 1
Abstract
To date, the multi-objective optimization litera-
ture has mainly focused on conflicting objectives,
studying the Pareto front, or requiring users to
balance tradeoffs. Yet, in machine learning prac-
tice, there are many scenarios where such con-
flict does not take place. Recent findings from
multi-task learning, reinforcement learning, and
LLMs training show that diverse related tasks can
enhance performance across objectives simultane-
ously. Despite this evidence, such phenomenon
has not been examined from an optimization per-
spective. This leads to a lack of generic gradient-
based methods that can scale to scenarios with a
large number of related objectives. To address this
gap, we introduce the Aligned Multi-Objective
Optimization framework, propose new algorithms
for this setting, and provide theoretical guarantees
of their superior performance compared to naive
approaches.
1. Introduction
In many real-world optimization problems, we have access
to multi-dimensional feedback rather than a single scalar ob-
jective. The multi-objective optimization (MOO) literature
has largely focused on the setting where these objectives
conflict with each other, which necessitates the Pareto domi-
nance notion of optimality. A closely related area of study is
multi-task learning (Caruana, 1997; Teh et al., 2017; Sener
and Koltun, 2018; Yu et al., 2020; Lin et al., 2021; Liu et al.,
2021; Navon et al., 2022; Zhou et al., 2022; Lin and Zhang,
2023; Liu et al., 2023a; Chen et al., 2023; Achituve et al.,
2024; He et al., 2024; Liu and Vicente, 2024), where mul-
tiple tasks are learned jointly, typically with both shared
and task-specific parameters. The hope is that the model
can perform better on individual tasks by sharing common
information across tasks. Indeed, the phenomenon of im-
*Equal contribution 1Meta AI 2Technion. Corresponde...