From Gradient Volume to Shapley Fairness: Towards Fair Multi-Task Learning
Authors: Xiao Wang
Yuying Han
Dazi Li
Fei Zhang
Min Tang
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-task Learning
Shapley Value
Fair Optimization
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We introduce SVFair, a fairness-driven  method using Shapley values to quantify and mitigate gradient conflicts in multi-task learning, achieving state-of-the-art performance through the novel Volume Determinant (VolDet and VolDetPro) metric.

Abstract: Multi-task learning often suffers from gradient conflicts, leading to unfair optimization and degraded overall performance. To address this, we present SVFair, a Shapley value-based framework for fair gradient aggregation. We propose two scalable geometric conflict metrics: VolDet, a gram determinant volume metric, and VolDetPro, its sign-aware extension distinguishing antagonistic gradients. By integrating these metrics into Shapley value computation, SVFair quantifies each task’s deviation from the overall gradient and rebalances updates toward fairness. In parallel, our Shapley value computation admits controllable complexity. Extensive experiments show that SVFair achieves state-of-the-art results across diverse supervised and reinforcement learning benchmarks, and further improves existing methods when integrated as a fairness-enhancing module.

Published as a conference paper at ICLR 2026
FROM GRADIENT VOLUME TO SHAPLEY FAIRNESS: TOWARDS FAIR MULTI-TASK LEARNING
Xiao Wang', Yuying Han', Dazi Li', Fei Zhang?, Min Tang?
"Beijing University of Chemical Technology National Innovation Institute of Defense Technology
{w_xiac, han-yuying, 1idz 1idz] edu. cn fe (00ang1008s.l sina. com, mtangon 63 com
ABSTRACT
Multi-task learning often suffers from gradient conflicts, leading to task-level un- fair optimization and degraded overall performance. To address this, we present SVFair, Shapley value-based framework for fair gradient aggregation that ex- plicitly targets task-level fairness under such conflicts. Unlike heuristic scalariza- tion or pairwise conflict penalties. SVFair combines geometric view of gradient interaction with cooperative-gane game view of fair contribution. We propose two scalable geometric conflict metrics: VolDet, gram determinant volume metric, and VolDetPro, its sign-aware extension distinguishing antagonistic gradients. By integrating these metrics into Shapley value computation, SVFair quantifies each task's deviation from the overall gradient and rebalances updates toward fairness. parallel, our Shapley value computation admits controllable complexity. Exten- sive experiments show that SVFair achieves state-of- offthe-ar art results across diverse supervised and reinforcement learning benchmarks, and further improves existing methods when integrated fairness-enhaccing module.
INTRODUCTION
Multi-task learning (MTL) (Caruana 1997 Zhang & Yang, 2021; 2018) enhances learning effi- ciency and generalization by jointly training model on multiple tasks. By leveraging shared rep- resentations, MTL can lead to better performance compared to training separate models for each task. Consequently, MTL has been widely applied in various fields like natural language process-
ing (Chen et al. [2024] Radford et al. 2019), computer vision (Misra et al. 2016 Liu et al. 2019 He et al. [020 Vandenhende et al. 2020), reinforcement learning (Oh al. 2017 Yu al. (0200) and natural sciences (Tang et al. 2023 Yang 2024 Eyuboglu et al. 2021). )) However, sharing a single model inevitably couples the optimization dynamics of all tasks: gradients from different objectives can conflict with each other, so that some tasks consistently dominate the update direction while others repeatedly compromised. This phenomenon, often referred to task-level optimization unfairness (Kendall et al., [2018] Lin et al. 2024) [Sener & Koltun 2018), not only degrades the worst-task performance but can also undermine the reliability of MTL in safety- or mission-critical applications (Ban & Ji 2024 Qin et al.) (025b). To address this phenomenon, existing methods
have adopted loss-based (Kendall et al. 2018 Liu al., 2019 2023; [Shnn et al. 2024 Xiao al.)
2025) and gradient-based (Sener & Koltun| 2018 Liu et al. 2021a Navon et al.
2022 Ban & Ji,
2024
Qin et al., 2025b: Zhang et al. 2025) methods.
Although some gradient-based methods (Sener & Koltun 2018 Navon et al. porate fairness, most still focus on final performance and fail to explicitly quantify how individual task gradients conflict with the overall gradient (Ban & Ji 2024 Qin et al. (0255). This leads to unfair allocation of optimization resources and limits overall performance improvement. While some studies use cosine similarity (Yu et al.) 2020a Liu et al. 2021b) inner products (Liu al.) 2021a) to measure task conflicts, these metrics are restricted to pairwise analysis and fail to capture the global structure of gradient conflicts across the entire task set. In particular, they do not provide
2022) attempt to incor-
"Corresponding author.

---

Published as a conference paper at ICLR 2026
coalition-level view of how subset of tasks collectively interacts with the others, making it diffi- cult to reason about which tasks contribute to, or suffer from. gradient interference at the level where fairness should be defined. This limitation becomes particularly severe in large-scale scenarios, such as CelebA (Liu et al. 2013) with 40 tasks, in which scalability and efficiency are critical challenges.
These observations motivate to seek framework that jointly (i) models gradient conflict from geometric perspective at the level of arbitrary task coalitions, and (ii) links this coalition utility to a principled fair-division rule for distributing gradient updates. To systematically address this limita- tion, we propose two scalable conffict metrics centered on task subset gradient differences. Building upon this metric, we construct a fairness metric centered on the task set as a whole, effectively quantifying gradient deviation, thereby enhancing optimization balance across tasks and improving overall performance. Our main contributions are as follows:
We introduce Shapley value- fairness measurement into MTL. We are the first to apply the Shapley value (Shapley 1953 Wang et al. 2022a Chen et al. 2023) to the MTL optimization framework, enabling precise quantification of individual task gradient deviations from the overall gradient, thereby alleviating inter-task conflicts and enhancing fairness.
We propose VolDet and VolDetPro, efficient geometric gradient conflict metrics. VolDet measures confiic as the volume of the parallelotope spanned by task gradients, computed from the determinant of their Gram matrix. VolDetPro preserves this intuition but is sign-aware, adding a lightweight penalty only when pairwise similarities are negative to fix the volume-only blind spot. Both metries remain efficient and scalable, enabling single-pass Shapley value computation, reducing cost, and supporting large-scale scale MTL.
We establish the SVFair gradient aggregation framework with Shapley values. SVFair lever- ages the Shapley values computed in a single pass to guide gradient aggregation. Both theoretical analysis and empirical results demonstrate convergence to Pareto stationary points, accompanied by significantly improved overall performance.
Extensive experiments validate the effectiveness and compatibility of SVFair. SVFair con- sistently surpasses state-of-the-ard methods across diverse supervised and reinforcement learning benchmarks. Moreover, the Shapley value can be seamlessly integrated into existing MTL algo- rithms, further enhancing overall performance.
RELATED WORK
Multi-task learning. Existing MTL optimization methods can be broadly categorized into loss-
based (Liu et al., 2019: Dai et al. 2023 and gradient-based (Sener & Koltun 2018 Chen et al.
2018 Yu et al. [020aa] Liu al. 2021a) methods. Loss-based (Liu et al., [023]] Shen et al.] 2024
Xiao et al. 2025) approaches adjust task weights at the loss level through heuristic strategies or
uncertainty modeling. Although computationally efficient, they often suffer from performance bot-
tlenecks in complex task scenarios. In contrast, gradient-based (Chen et al. 2020 Navon et al. 2022
Ban & Ji, 2024 Qin et al. [025b] Zhou et al.] 2025 Qin et al., 2025a) methods leverage task gradi- information, employing techniques such Pareto optimization (Sener & Koltun 2018 Ban & /Zhan et al. 2025), gradient projection (Yu et al. 2020a)). or conffict avoidance (Liu et al., 202 significantly enhance performance. Notably, these two paradigms are not mutually exclu-
Jil 2024
sive. Recent studies (Qin et al.) (025b) have begun to integrate both strategies, and even incorporate higher-order information (e.g., model performance) to better coordinate MTL optimization.
Shapley value. The Shapley value (Shapley 1953), a foundational allocation principle from coop- erative game theory, has been increasingly applied machine learning (Rozemberezki et al.) 2022) for quantifying the marginal contribution of individual components to the overall system. It has been widely used to evaluate feature importance in model interpretability (Wang et al. 2002a Chen et al.)
2023 Kumar et al. 2020; Tang et al. 2023), data point utility in data valuation (Ghorbani & Zou
2019 Jia et al., 2019), client contribution in federated learning (Fan et al., 2022 Rafi et al. 2024).
and policy value in multi-agent reinforcement learning (Li et al. 2024 Wang et al. 2022b While
some studies (Wang et al. 2022a Li et al. 2024) have explored incorporating Shapley value into MTL, deep integration between the two remains absent. Moreover, the high computational com- plexity of Shapley value has hindered its practical adoption. In this work, we establish a principled connection between Shapley value and MTL optimization.
2

---

Published as a conference paper at ICLR 2026
Gram matrix. The Gram matrix has been widely used in machine learning. As an inner prod- uet matrix in kernel methods, it enables nonlinear mappings (Pennington & Worah 2017). reveals structural patterns in attention mechanisms (Teo & Nguyen 2024), supports alignment kemels for time series (Cuturi 2011), detects aut-of-distribution samples (Sastry & Oorel 2020). accelerates kernel-based computations (Achlioptas al.) (00T)) enhances the expressive power of graph neural
networks (Morris et al. 2019), facilitates multi-modal alignment (Cicchetti et al.) 2025), models
diversity (Kulesza et al. 2012), and promotes policy diversity (Perez-Nieves et al.
202 in rein-
forcement learning via determinantal point processes (Kulesza et 2012).
3 PRELIMINARIES
Multi-task Learning. In MTL, there are N tasks, each with loss function 4()) for E (1....,N) and the overall loss L = (4,,... (x). The gradient of task i Ji = Vl,, and the gradient matrix is = [9t.. 9v]. To jointly optimize all tasks, a common gradient-based approach is aggregate gradients into an update direction := Σi= ag 9i, followed by the up- date 0++1 0₂ 7) d,, where a: (ai,...,..] T R denotes the weights and r denotes the step size. Using first-order Taylor expansion, the change in loss for task can be approximated as (6(+1) L (e) -1 -g.. d, which provides guide for designing the update direction. Recent
methods (Ban & Ji, 2024 Qin et al. (0255) for determining the aggregation direction d primarily
optimize the utility 9, d:
FairGrad arg max
(g d)²-7
PIVRG: arg min
1
Σ
s.t. g. T. 0, (1)
1-9
N g
Here, 7 E
wi Σ exp(Am,/7) the higher-order information weight, Am,
N-exp(Am./r)
the performance drop (see Eq 4 for more details), and r is the temperature parameter.
Pareto Optimality. For two points 01.02 Rm 02 is said to dominate 02 if (,)) S (02) for all E (1,..., N} and (01) + L (02). A point d" E TRm is Pareto optimal there does not exist any other point dominates it. The Pareto front is the set of all Pareto optimal points. Furthermore, point is Pareto stationary if convex combination of the task gradients at that point equals zero; this first-order necessary condition for Pareto optimality.
Shapley Value, For a cooperative game with N participants, the Shapley value quantifies the marginal contribution of each participant to the total utility. Let v(S) denote the utility of a sub- set S S (1,2,..., The Shapley value o; for participant is defined as:
| - |S| 1)! (c(SU v(S)).
Φi = E SCN
N
We set the e(S) in the Shapley value to a gradient conflict metric that is a proper set function and faithfully reflects antagonism across tasks, so the induced Shapley value weights steer updates toward balanced Pareto improvement.
4 SVFAIR: A SCALABLE SHAPLEY VALUE FRAMEWORK FOR FAIR MTL
4.1 EVOLVING NEEDS FOR GRADIENT CONFLICT METRICS
Existing MTL methods have made initial progress in fairness optimization but largely ignore the sys- tematic quantification of individual task gradient deviations from the overall gradient, thus limiting conflict mitigation while hindering overall performance improvement. Fortunately, Shapley value, general method for quantifying individual contributions, provides new perspective for addressing this issue. By integrating Shapley value, we quantify each task gradient's deviation from the overall gradient, providing more precise gradient conflict signal to replace w; in Eq So it is essential to design a suitable utility function that quantifies the gradient conflict of task subsets.
However, existing conflict metrics, such cosine similarity (Yu et al. 2020a Liu et al. 2021bb or inner product (Liu et al., 2021a) only analyze pairwise relationships between tasks, without cap- turing the collaborative and conflicting structure among the entire task set. As the number of tasks
3

---

Published as a conference paper at ICLR 2026
N increases, such local metrics face severe challenges in scalability and efficiency, making them unsuitable as utility function v(S).
Cosine Similarity: cos (gi. (j) 94 9;/ 11 9: 11 ll, Inner Product 9 9j:
An alternative is to define the utility function v(S) as the performance drop Am (Ban & Ji, 2024
Qin et al. 2025b Xiao al., 2025) which is commonly used in MTL, quantifying the impact of task
subsets on overall performance:
Am% = Σ k=1 (-1) (Mm.k - Mo,x)/Mb,e x 100,
(4)
where Ma, and Mm, denote the scores of the baseline and the compared method the k-th eval- uation metric, respectively, and 5₂- = indicates that a higher value the k-th metric is preferable. However, employing Am as v(S) requires a complete MTL model training for each subset S. re- sulting in an exponential computational complexity O(C-2") where C' denotes the cost of a single training run (See the analysis in Appendix [.4). Therefore, it is imperative design new efficient and scalable conflict metric that enables the practical application of Shapley value in MTL.
4.2 TOWARDS SCALABLE GRADIENT CONFLICT METRICS WITH PARALLELOTOPE VOLUME
Maa Mai Mas Mas Mas Mag Max Maz iMa Mas Man
â à=[0i>
Bar Bar dar As] (Mn My Maa"
May May Migg
M
Maz (Mg Max Mas Adga Adga sMiga Miaa Map
Ma.2.
Mi Maz Mas
Mal Max Max
12
A
B
2
z
Tormplined Cradient: 0g
Normalised Gradients 94
Notmalized Ghudient:
Nom finnt
Normalized Gradient: By.
The Orthigonnll
The Ortllggaal
Nordrnlliedd (Bradien::
Vector to 6
Vector to de h
X
Normalize Grudient: d2
Normalized Gradient: 0g
Normallsed Gradion:: d2
The less the gradients conflict
The greater the gradients conflict
The greater the gradients conflict
The smaller the volume
The larger the volume The larger the des (M,
The larger the volume
The smaller the det ( (,,)
The larerr the dot (2,,,
Figure 1: Overview of the VolDet metric. Up: Construction of the Gram matrix . M from the normalized gradient matrix G. and extraction of an example submatrix MM for 5= 2, via simple indexing. Down: 3D geometric interpretation of partial parallelotope volume: (A) when task gradients are nearly aligned, the spanned volume and (eM(Ms) are small; (B) when gradients diverge, both the volume and det(Ms, grow; (C) given the same orthogonal vector h, B and have identical volume and dsM. yet C's obtuse (negative) alignments indicate stronger conflict.
To systematically quantify inter-task gradient conflict, we propose a gradient conflict metric as the squared volume of the parallelotope spanned by normalized task gradients. Geometrically, this volume reflects directional alignment: it approaches zero as gradients align and the shape collapses (Fig. I Down-A), and increases as directions diverge (Fig. Down-B)
Definition In MTL, the Gram matrix M E NNNN is defined as M GTG where the normal- ized gradient matrix is G = 191,...,gN and g, - gg:lllgi), According to the volume determinant identity, the squared volume of the parallelotope formed by these normalized gradients satisfies: Vol? (0I,...,gN) (M) = det(MM. The proof can be found in Appendix B.
Definition 2 VolDet Metric ( Determinntt Metric), For any task subset S C (1...,,N) the squared volume of the corresponding subspace, as the utility function v(S) of Shapley value, can be computed by extracting the submatrix Mg E S) from the global Gram matrix M:
v(S) = V det(Ms).
(5)
4

---

Published as a conference paper at ICLR 2026
Instead. it directly captures Remarkably, Definition 2 avoids the limitations of Am in Section 4.1
the relationshss among gradients and characterizes the overall distribution of gradient direc- tions. The VolDet metric naturally satisfies the set function form required by the Shapley value (See the Appendix [8.2 for more details). It does not depend on specific training outcomes or loss values but is solely determined by gradient geometry, which provides stronger generalization and fairness
interpretability. However, VolDet is sign-agnostic. Since det (Mg) = det (GgGs) = det (Gs)
is invariant to any column sign flip 9. + - Hence Fig. Down-B and Fig. Down-C yield the same det (Ms) when the third vector contributes the same orthogonal component, even though Fig. Down-C contains obtuse (negative-cosine) pairs and thus stronger conflict.
To address the sign insensitivity of pure volumes while preserving the efficiency and Shapley-ready form of VolDet, we introduce a sign-aware refinement named VolDetPro. VolDetPro resolves this by augmenting VolDet with lightweight penalty driven only by negative pairwise similarities in the Gram subspace, which activates in antagonistic regimes and remains zero otherwise.
Definition 3 VolDetPro. For any task subset S S {1,.... N). let Ms R|six(S be the Gram submatrix, Define the index set of antagonistic (negative-similart pairs on the strict upper triangle [()) = (i. :1:0)=1s40 < |S], (Ms)is < =0} Then VolDetPro can be defined as followed:
I (()
+ +S|
E Eoseeris) (Ms).s)
(S) det (Ms)
(6)
|S|
Here I (S) denotes the index set of antagonistic (negative-similarity) pairs on the strict upper tri- angle of Ms. The term E(NEEI()) (Ms), aggregates the conflict strength by accumulating
the magnitudes of negative pairwise similarities, while the factor [I[( encodes the conflict coverage (how many antagonistic pairs) with sublinear growth to avoid inflation as |S] increases. VolDetPro equals VolDet when I (S) = 0 and otherwise increases smoothly with both the number and magnitude of antagonistic pairs, distinguishing configurations that share the same volume but exhibit stronger antagonism. More VolDetPro explanations are analyzed in Appendix B.3
Most importantly, VolDet and VolDetPro scale efficiently via simple indexing on single Gram ma- trix. After one round of gradient sampling and Gram construction, the v(S) for all subsets S is ob- tained without any per-subset retraining. Replacing the Am with the VolDet reduces Shapley value computation to one training pass while preserving conflict sensitivity. Computational complexity is analyzed in Appendix B.4, and a comparison conflict metrics is provided in Appendix [.5
4.3 SOLVE THE AGGREGATED GRADIENT WITH SHAPLEY VALUE FAIRNESS
To apply the VolDet (Eq 5) or VolDetPro (Eq 6 metrics, we introduce Shapley- Shapley-Value-e Value-based gra-
dient aggregation method.
calculate the Shapley value Φ; for each task i, which quantifies the extent to which the task gradient 9; deviates from the overall task gradient G. Intuitively, higher Φ; indicates that task i is more misaligned with others and should be assigned a higher influence. To achieve this, we incorporate Φi into the optimization framework of Eq and solve the following optimization problem aggregate gradients:
Shapley Value Fairness:
Based on Eq 2 and Eq (or Eq 6).
Σ - wi/(g? d), s.t. g d 0, Vi,
N
arg min d
1/
exp(os/7) where Wi D exp(
We refer to this Shapley-value-guune optimization framework as SVFair, which introduces a fairness-aware high-order information to the overall descent direction.
Solve the Aggregated Gradient d: To solve the 7 we constrain on a sphere centered at the origin with radius e. The optimal d' can be solved with the Kurush-Kuhn-1 Tucker (KKT) conditions: (I) The optimal solution lies exactly on the boundary of the Euclidean ball, i.e., ||('] || = <. (2) All the inner product constraints g. d > 0 are strictly satisfied.
(3) Introducing Lagrange multiplier À to get the optimality condition: (d') = -xd" e. (4) Finally, the stationarity condition simplifies to Eq.
eNw;
t=1 X(g d/2
=
(8)
5

---

Published as a conference paper at ICLR 2026
Following previous works (Ban & Ji 2024 and (Qin et al. 2025b), similarly assume that the gradients of tasks are linearly independent otherwise it would imply reaching a Pareto stationary point and d can be represented as linear combination of task gradients: d = Σi= 04gi For simplicity, let constant item e. 1, we then obtain ag = w// (g. d)?, and thus (G' Ga)² = w/a.; where [w'1..... WN] T E R element-wise. Algorithm T is the procedure of SVFair. See the Appendix D for proof details.
[a1..... aN] IR and the square operation
As mentioned earlier, we assume that the gradients of different tasks are linearly independent except at a Pareto station- ary point. Formally, we adopt the fol- lowing assumption, as also used in pior works (Navon et al. [2022 Ban 2024
Algorithm 1 SVFair for MTL
1: Input: model parameters fu. temperature paramete
т, learning rate {mt}
2: for ( (= 1 to 7 do
3: Compute normalized gradient matrix ((,,)
Qin et al. 2025b).
4: Compute Gram matrix M = G(O))) ((,))
5: Compute Shapley values {dx via Eq 2 and utility
Assumption For the ouiput sequence
{0,) generated by the proposed method,
function (Eq. VolDet or Eq. 6 VolDetPro)
we make the following assumptions: 1) Each task loss function 1,(0) is differ-
6:
Solve Eq. 7 obtain gradient weights at.
7:
Compute the aggregated gradient dN G(O,)
entiable and L-smooth. 2) If 0g is not Pareto stationary point, the gradients {gs (0.)) are linearly independent. 3) The update direction de satisfies ||dell ge g:(O.) de 2 m.
8:
Update parameters 0,41 0g 7/ de
9: end for
S and there exists constant m such that
Theorem With the Assumption 7 there exists subsequence {0,, } of the output sequence {0:) that converges to a Pareto stationary point 0", See the Appendix E. for more details.
EXPERIMENTS
We evaluate SVFair on variety of MTL benchmarks under toy example, supervised learning and reinforcement learning settings to show its effectiveness. Throughout, SVFair (VolDe) refers SV- Fair using VolDet as the Shapley utility function v(5), and SVFair SVFain (oolDetrr)) refers to SVFair using VolDetPro as the v(S). Furthermore, we compare our proposed SVFair method with the following methods in experiments: Single-task learning (STL), DWA (Liu al., 2019), UW (Kendall
et al.
2018), MGDA (Sener & Koltun, 2018), RLW (Lin et al., 2021). PCGrad (Yu et al. 2020a),
GradDrop (Chen al. 2020). CAGrad (Liu al. 202 la). IMTL-G (Liu al.) (02 Nash-
MTL
(Navon et al. (2022. FAMO (Liu et al.) 2023), FairGrad (Ban & Jil (0024) and PIVRG (Qin
al.
2023b). All experiments follow the settings of prior works (Navon et al.
2022 Liu et al.
2023
Ban & Ji 2024
Qin et al. 2025b). Our code is available in the supplementary material.
5.1
ToY EXAMPLE
We adopt the 2-task toy example introduced in (Ban & Jil 2024). where the objectives of the tasks, denoted as Li and L2. More details are provided in Appendix F.I We select 5 starting points and illustrate the optimization trajectories with different MTL methods
Fig. 2
Fringed
tm-50.00
-10
-15
-35
32-35-10-3 -5
-0-5-11 3
3.0
30-1-10-3 15
20-33 5.5
-6-15-LD--36 E5C)
20-15-13--50 ESC)
Figure 2: Comparison of MTL approaches challenging synthetic two-task benchmark. We visualize optimization trajectories w.r.t. objective values (Li, C2). Starting points are indicated by black dots, and the Pareto front (see Section 3 represented by thick gray lines.
Methods such as MGDA and Nash-MTL aim to find Pareto stationary point, and their optimization typically halts such a point is reached. As result, when the gradient of weak task is small,

---

Published as a conference paper at ICLR 2026
Table 1: Results on NYU-v2 (3-task) and Cityscapes (2-task) datasets. Each experiment repeated 3 times with different random seeds and the average is reported.
NYU-v2
Cityscapes
Method
MR
Am%
MR
Am% +
RLW
13.33
7.78
11.00
24.37
DWA
10.89
3.57
9.00
21.43
UW
10.78
4.06
9.25
5.88
MGDA
8.78
1.38
12.25
44.14
PCGrad
11.22
3.97
9.12
18.21
GradDrop
10.56
3.58
8.25
23.67
CAGrad
8.22
0.19
7.88
11.58
IMTL-G
7,44
-0.60
6,50
11,04
Nash-MTL
5.11
-4.05
4.25
6.72
FAMO
6.67
-4.10
8.75
8.13
FairGrad
4.78
-4.66
3.25
5.12
PIVRG
3.56
-6.50
2.50
-0,45
SVFair(VolDerl
1.44
-8.81
6.88
-2.08
SVFair (ooDDetro))
2.22
-8.29
6.12
-2.40
these methods struggle to further improve its performance, leading to incomplete coverage of the Pareto front. FairGrad continues to move along the front beyond local stationary points and provides compensation for weaker tasks, but it exhibits noticeable bias depending on the initialization and fails ensure uniform coverage. front, allowing all trajectories to consistently reach more complete and balanced solutions. More experimental details and results are provided in Appendix
contrast, SV-Fair enables smooth traversal along the Pareto
5.2 SUPERVISED LEARNING
evaluate the performance of SVFair different supervised learning benchmarks, with experi- mental details and results provided in the Appendix F.2
Evaluation Metrics Following prior works (Ban & 2024, Qin al.) 2025b), we consider two overall metrics to evaluate MTL performance of the MTL method: (1) Am%o (defined in Eq
the average per-task performance drop compared to the STL baseline (2) Mean Rank (MR), the average rank of each method across tasks (lower is better; MR means ranking first in all tasks).
NYU-v2 (Silberman et al., 2012) contains 1,449 densely annotated images focused on indoor scene understanding. It supports pixel-wise classification task (semantic segmentation) and two pixel- wise regression tasks (depth estimation and surface normal prediction). The result in Table shoos that SVFair achieves the best MR and overall performance drop the NYU-V2 benchmark. As can be seen from Table I0 in Appendix F.2. an interesting observation is that previous methods tend to focus their improvements on Segmentation and Depth, while achieving little no gains on the Surface Normal task-ofter often even underperforming compared to STL STL-indicating a clear task imbalance. In contrast, both PIVRG and SVFair consistently outperform STL across all evaluation metrics. More importantly, SVFair outperforms PIVRG with near-optimal metric scores, achieving an almost perfect average rank and the best performance drop with a Am. of -8.81.
Cityscapes (Cordts et al. 2016 contains 5,000 urban street view images, designed for two tasks: semantic segmentation and depth estimation. The result Table [ shows that SVFair also achieves the best overall performance drop the Cityscapes benchmark. As can be seen from Table []] in Appendix F.2 prior methods typically show strong improvements on Segmentation but struggle on the more challenging Depth task. SVFair places greater focus Depth, effectively mitigating the negative impact of gradient conflicts. As a result, it not only achieves the best Depth metrics but also leads to overall performance improvement over STL, indicated negative Am%.
Office-31 (Saenko et al. 2010) contains 4, images spanning three domains (Amazon, DSLR, and Webcam), designed for image classification. The result in Table 2 shows that Vair also achieves

---

Published as a conference paper at ICLR 2026
Table Results on Office-31 1 (3-task) and CelebA (40-task) datasets. Each experiment repeated 3 times with different random seeds and the average is reported.
Office-31
CelebA
Method
Amazon f
DSLR
Webcam 1
MR 4
Am% 4
MR Am% 4
STL
86.61
95.63
96.85
RLW
83.82
96.99
96.85
7.33
0.60
5.75
1.46
DWA
83.87
96.99
96.48
8.00
0.71
7.10
3.20
UW
83.82
97.27
96.67
7.17
0.56
6.00
3.23
MGDA
85.47
95.90
97.03
6.33
0.28
11.12
14.85
PCGrad
83.59
96.99
96.85
8.17
0.69
6.92
3.17
GradDrop
84.33
96.99
96.30
8.17
0.59
8.50
3.29
CAGrad
83.65
95.63
96.85
9.33
1.14
6.72
2.48
IMTL-G
83.41
96.72
96.48
10.50
0.98
4.90
0.84
Nash-MTL
85.01
97.54
97.41
3.83
-0,24
5.32
2.84
FairGrad
82.11
97.54
97.96
6.17
0.68
5,78
0.37
SVFair(VolDee)
85.81
99.45
98.33
1.67
-1.53
4.15
-0.63
VVllir(VolDetrro)
86.04
99.45
98.33
1.33
-1.62
5.72
0.11
best overall performance drop on the Office-31 benchmark and attains the best accuracy across all three domains.
CelebA (Liu al., 2015) a large-scale facial attribute dataset containing over 200,000 images, each annotated with 40 attributes. It can therefore be formulated MTL problem with 40 image- level classification tasks. The result Table 2 shows that SVFair also achieves the best overall performance drop the CelebA benchmark.
5.3 REINFORCEMENT LEARNING
We evaluate SVFair on the MT10, which includes 10 robotic tasks from MetaWorld environment YYu
Table Results on MT10 benchmark. Av- erage over 10 random seeds.
et al. 2020b). The goal is to learn a policy that gener- alizes across diverse tasks. Following prior work, we adopt Soft Actor- Ciiot-Critic (SAC) (Haarnoja et al. 2018) as the base algorithm and train the model for million steps with batch size of 1280. We compare SVFair with Multi-task SAC (MTI. SAC) (Yu et al.) (020b), CAGrad (Liu al.] 2021a), Nash-MTL (Navon et al.
Method
success rate (mean + stderr)
STL
0.90 + 0.03
MTL SAC
SAC
0,49 + 0.07
CAGrad
0.83 + 0.05
2022). FAMO (Liu et al. (023). FairGrad (Ban &
Nash-MTL
0,91 = 0.03
Ji 2024) and PIVRG (Qin et 2025b). The result
FAMO
0.83 + 0.05
is shown in Table 3 We evaluate each method ev- ery 10,000 steps and report the best average success rate achieved across 10 random seeds during the en- tire training process. The results indicate that SVFair achieves the best performance the MT10.
FairGrad
0.84 0.07
PIVRG
0.96 + 0.02
SVFair(VolDDe)
0.97 + 0.03 0.97 + 0.02
SVFair(VolDettrro)
5.4 PRACTICAL IMPLEMENTATION AND INTEGRATION OF SHAPLEY VALUE
Choice of Hyperparameter
Table 4: Am different 7 on three benchmarks. SViii uses VolDet for v(S); see Appendix for VolDetPro.
The Eq.
SVFair relies on temperature parameter т. To eval- uate the effect of r on the mean rank and overall performance drop, we conduct a small-scale hyperparameter sweep with {0.5, 1.0, 2.0, 5.0) the Cityscapes, Office-31 and CelebA benchmarks. The result in Table 4
Method
Cityscapes
Office-31
CelebA
SVFair (r 0.5)
-1.27
-1.49
1.20
SVFair (r = =1.0)
0.10
-1.30
0.22
SVFair -2.0)
-0.78
-1.44
0.23
SVFair (T ==550 =5.0)
-2.16
-0.97
-0.10
8

---

Published as a conference paper at ICLR 2026
shows that the has different effects across benchmarks. On Cityscapes and CelebA, larger (5.0) yields the greatest performance improvement, whereas on Office-31 only the smaller 7 (0.5) achieves slight positive gain. We recommend using a larger when Shapley values differ strongly and conflicts are sharp, and a smaller r when tasks are more balanced; a detailed justification is provided in Appendix H..T
Compute Shapley Value. For a small number of tasks, we directly compute the exact Shapley values. To further improve efficiency in large-task regimes (e.g., CelebA), we adopt Monte Carlo (MC) subset sampling (Rozemberczki et al., 2022): we randomly draw K subsets S to estimate Shapley values, reducing the enumeration cost. Table[]shows that SVFair's single-epoch runtime is comparable to the baselines; on CelebA, even when evaluating many subsets, the one-time training cost remains dominant and the overall overhead is acceptable (the sample size K this work 1000). Further experimental details and the MC procedure are provided in Appendix G
Table 5: Single-epoch runtime (seconds) across benchmarks.
CelebA (K=lk)
CelcbA (K=2k)
CelebA (K=3k)
CelebA (K=5k)
Method
NYU-v2
CityScapes
Office-31
MTI0
FairGrad
205.6
92.4
37.5
10.2
810
810
810
810
SVFair
209.0
94.8
38.1
10.3
870
936
980
1086
Reinforcement Learning. Following the implementation in FairGrad (Ban & Ji 2024), we employ the SGD optimizer to solve the (G Ga)² = w/a within the reinforcement learning setting. Specif- ically, we set the learning rate to 0.1, momentum to 0.5, and optimize over 20 epochs to effectively control computational costs. In addition, since MT10 involves 10 tasks, we also adopt the previously mentioned Monte Carlo sampling method to accelerate the computation of Shapley values.
Integrate Shapley Value. Previous MTL methods often overlook global gradient conflict informa- tion, leading to bad overall performance. We propose to integrate the Shapley Value-based global gradient conflict information W/ (From Eq. 7 into these methods enhance fairness in optimiza- tion. Specifically, we apply this strategy to a range of bass- MTL methods, including RLW (Lin al. 202T). DWA (Liu et al., 2019), and UW (Kendall et al.) 2018). The loss is adjusted using the weight wi, resulting in the modified loss L' = w ( L. The result in 6 that incorporating global gradient conflict information effectively improves the performance of these methods.
Table 6:
Results of integrating Shapley value into existing methods the NYU-v2 (3-task) dataset. Each experiment is repeated times with different random seeds and the average is reported.
Segmentation
Depth
Surface Normal
Method
Am% 4
Angle
Within (
mloU
PAcc f
AErr 4
REir
Mean
Median
11.25
22.5
30
RLW
37.17
63.77
0.5759
0.2410
28.27
24.18
22.26
47.05
60.62
7.78
SV(VolDeO-RLL RLW
39.06
64.56
0.5551
0.2221
29.05
24.91
21.65
45.53
$8.68
SYLV
39.14
64.85
0.56
0.22
29.04
24.99
21.53
45.37
58.59
7.71
DWA
39.11
65.31
0.5510
0.2285
27.61
23.18
24.17
50.18
62.39
3.57
SV(VolDer-DDWA
41.59
66.37
0.5482
0.2286
27.32
22.60
24.00
50.00
63.15
2.17
SYVV
41.88
66.48
0.54
0.22
27.48
22.85
23.75
49.31
62.66
2.20
UW
36.87
63.17
0.5446
0.2260
27.04
22.61
23.54
49.05
63.65
4.05
SV(VolDet-UU
40.79
65.6
0.5416
0.2238
26.58
21.65
25.75
51.69
64.79
0.09
SY(VolDetrro--Uw
39.67
65.39
0.53
0.22
26.17
20.90
26.67
53.20
66.08
-1.50
5.5 VOLDET VS VOLDET
VolDetPro extends VolDet by penalizing both the number and the magnitude of negative-cosine pairs, making the metric more sensitive to obtuse interactions. When the Proportion Negative Cosine Pairs (PNCP) in the normalized gradient matrix & high, VolDetPro amplifies true con- flict signals, shifts the Shapley weights away from opposing tasks, alleviates the tug-of-war war, and improves the weakest task; therefore tends to outperform VolDet. When PNCP low, the addi- tional penalty may overcorrect and suppress beneficial diversity, VolDet is usually more stable.

---

Published as a conference paper at ICLR 2026
As shown in Fig. training-stagee stage statistics indicate that PNCP on Office-31 stays above 0.25 for most of the training process, whereas PNCP on CelebA and Cityscapes remains below 0.25, which is consistent with the observed results. We thus conclude that VolDetPPro is preferable when PNCS is high, whereas VolDet is preferable when PNCS low. More details are provided in Appendix [
Proportion Nogative Cosine Pairs
NY--V2(T-2.0))
0.30
CitSSapes
Office3 (1-5.0)
CelchA (T-1.0)
0.26
0.22
0.18
35.3%
50,0%
66.7%
83.3%
100,0%
Training Pregress (%6
Figure 3: Proportion of Negative Cosine Pairs (PNCS) during training.
6 CONCLUSION
We presented SVFair, a Shapley-value framework that measures and mitigates gradient conflicts in multi-task learning through subset-level geometric utility. At its VolDet, the finement that penalizes negative cosine pairs to disambiguate obtuse interactions with identical vol- umes. Plugging these utilities into a single-pass Shapley computation yields fairness-aware weights that guide a provably convergent aggregation toward Pareto-stationary solutions. Empirically, SV- Fair achieves state-of-the-an results across supervised and reinforcement learning benchmarks, and further improves existing MTL methods when used as a drop-in fairness module. In practice, ex- act Shapley values feasible for few tasks, while Monte-Carlo subset sampling keeps large-task regimes efficient without additional training runs. We also find VolDetPro preferable when negative cosine pairs prevalent, whereas VolDet remains robust when conflicts are mostly acute.
scalable metries: volume of normalized task gradients, and VolDetPro, sign-aware re-
STATEMENT
Ethics Statement This paper raises no ethical concerns. Our study does not involve human subjects, sensitive personal data, or experiments that could potentially lead to ethical issues. All experiments are conducted on publicly available datasets, and the research process fully adheres to the ICLR Code Ethics.
Reproducibility Statement To ensure reproducibility, we provide the complete source code in the supplementary materials. addition, all theoretical results and detailed derivations are included in the appendix. Together, these resources enable work to be independently verified and fully reproduced by the community.
REFERENCES
Dimitris Achlioptas, Frank MeSherry, and Bernhard Schölkopf. Sampling techniques for kernel methods. Advances in neural information processing systems, 14, 2001.
Hao Ban and Kaiyi Ji. Fair resource allocation in multi-task learning. In Proceedings of the 41st International Conference Machine Learning, pp. 2715-2731, 2024.
Rafael Bischof and Michael A Kraus. Multi-objective loss balancing for physics-informed deep learning. Computer Methods in Applied Mechanics and Engineering, 439:117914, 2025.
Rich Caruana. Multitask learning. Mach. Learn., 28( July 1997. ISSN 0885-6125. doi:
10.1023/A:1007779666777 URI.httss:
10

---

Published as a conference paper at ICLR 2026
Hugh Chen, lan C Covert, Scott M Lundberg, and Su-In Lee. Algorithms to estimate shapley value feature attributions. Nature Machine Intelligence, 5(6):590-601, 2023.
Shijie Chen, Yu Zhang, and Qiang Yang. Multi-task learning in natural language processing: An overview. ACM Computing Surveys, 56(12):1-32) 2024.
Zhao Chen, Vijay Badrinarayanan, Chen- Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In International conference on machine learning, pp. 794-803 PMLR, 2018
Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick sign: Optimizing deep multitask models with gradient sign dropout. Advances in Neural Information Processing Systems, 33:2039-20551 2020.
Giordano Cicchetti, Eleonora Grassucci, Luigi Sigillo, and Danilo Comminiello. Gramian multi- modal representation learning and alignment. In In Proceedings of the 13th International Confer- on Learning Representations, 2025.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schicle. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213-3223, 2016.
Marco Cuturi. Fast global alignment kernels. In Proceedings of the 28th international conference on machine learning (ICML-11), pp. 929-936, 2011.
Yanqi Dai, Nanyi Fei, and Zhiwu Lu. Improvable gap balancing for multi-task learning. In Uncer- tainty in Artificial Intelligence, pp. 496-506. PMLR, 2023.
Sabri Eyuboglu, Geoffrey Angus, Bhavik N Patel, Anuj Pareek, Guido Davidzon, Jin Long, Jared Dunmmon, and Matthew P Lungren. Multi-task weak supervision enables anatomically-resol abnormality detection in whole-body fdg-pet/cs Nature communications, 12((): 2021.
Zhenan Fan, Huang Fang. Zirui Zhou, Jian Pei, Michael Friedlander, and Yong Zhang. Fair and efficient contribution valuation for vertical federated learning, arXiv preprint arXiv:2201.0266 026658, 2022.
Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In International conference on machine learning, pp. 2242-22222222222222 22511. PMLR, 2019.
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with stochastic actor. In International confer- ence on machine learning, pp. 1861-1870. Pmir, 2018.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 9729-9738, 2020.
Ruoxi Jia, David Dao. Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Zhang, Dawn Song, and Costas J Spanos. Towards efficient data valuation based on the shapley value. In The 22nd International Conference Artificial Intelligence and Statistics, pp- 1167-1176. PMLR, 2019.
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7482-7491, 2.
Diederik Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiw:1412.6980, 2014.
Alex Kulesza, Ben Taskar, et al. Determinantal point processes for machine learning. Foundations and Trendy in Machine Learning, 122-3)) 2012.

---

Published as a conference paper at ICLR 2026
I Elizabeth Kumar, Suresh Venkatasubramaniia. Carlos Scheidegger, and Sorelle Friedler. Prob- lems with sapple--vlue-bire explanations as feature importance measures. In International conference on machine learning, pp. 5491-5500. PMLR, 2020.
Chao Li, Shaokang Dong, Shangdong Yang, Yujing Hu, Tianyu Ding, Wenbin Li, and Yang Gao. Multi-task multi-agent reinforcement learning with interaction and task representations. IEEE Transactions on Neural Networks and Learning Systems, 2024.
Baijiong Lin, Feiyang Ye, Yu Zhang, and Ivor Tsang. Reasonable effectiveness of random weight- ing: A litmus test for multi-task learning, arXiv preprint arXiv:2111. 2021.
Xi Lin, Hui-Ling Zhen, Zhenhua Li, Qing-Fu Zhang, and Sam Kwong. Pareto multi-task learning. Advances in neural information processing systems, 32, 2019.
Xi Lin, Xiaoyuan Zhang, Zhiyuan Yang, Fei Liu, Zhenkun Wang, and Qingfu Zhang. Smooth tchebycheff scalarization for multi-objective optimization. In Proceedings of the 4 Interna- tional Conference on Machine Learning, pp. 30479-30509, 2024.
Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. Advances in Neural Information Processing Systems, 34:18878-18890, 2021a.
Bo Liu, Yihao Feng, Peter Stone, and Qiang Liu. Famo: Fast adaptive multitask optimization. Advances in Neural Information Processing Systems, 36:57226-57243, 2023.
Dehao Liu and Yan Wang. A dual-dimer method for training physics-constrainee neural networks with minimax architecture. Neural Networks, 136: 12-125, 2021.
Liyang Liu, Yi Li, Zhanghui Kuang, Jing-Hao Xue, Yimin Chen, Wenming Yang. Qingmin Liao, and Wayne Zhang. Towards impartial multi-task learning. In International Conference on Learning Representations, 2021b.
Qiang Liu, Mengyu Chu, and Nils Thuerey. Config: Towards conflict-free training of physics in- formed neural networks. arXiv preprint arXiv:2408. /1 104, 2024.
Shikun Liu, Edward Johns, and Andrew J Davison. End-to-end multi-task learning with attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1871-1880 2019.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Tomasz P Michalak, Karthik V Aadithya, Piotr L Szezepanski, Balaraman Ravindran, and Nicholas Jennings. Efficient computation of the shapley value for game-theoretic network centrality. Journal of Artificial Intelligence Research, 46:607-650, 2013.
Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert. Cross-stitch networks for multi-task learning. In Proceedings the nition, pp. 3994 4903-4003, 4003, 2016.
the IEEE conference on computer vision and pattern recog-
Rory Mitchell, Joshua Cooper, Eibe Frank, and Geoffrey Holmes. Sampling permutations for shap- ley value estimation. Journal of Machine Learning Research, 23(43): 2022.
Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 4602-4609, 2019.
Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Multi-task learning as a hargaining game. In International Conference on Machine Learning. pp. 16428-16446. PMLR, 2022.
Junhyuk Oh, Satinder Singh, Honglak Lee, and Pushmeet Kohli. Zero-shot task generalization with muulii-tak deep reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML.' pp. 2661-2670 JML..org, 2017.
12

---

Published as a conference paper at ICLR 2026
Jeffrey Pennington and Pratik Worah. Nonlinear random matrix theory for deep learning. Advances in neural information processing systems, 30, 2017.
Nicolas Perez-Nieves, Yaodong Yang, Oliver Slumbers, David H Mguni, Ying Wen, and Jun Wang. Modelling behavioural diversity for learning in open-ended games. In International conference on machine learning. pp. 8514-8524. PMLR, 2021.
Xiaohan Qin, Xiaoxing Wang, and Junchi Yan. Towards consistent multi-task learning: Unlocking the potential of task-specific parameters. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10067-10076, 2025a.
Xiaohan Qin, Xiaoxing Wang. and Junchi Yan. Revisiting fairness in multitask learning: A performance-driven approach for variance reduction. Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 20492-20501, 2025b.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Taki Hasan Rafi, Faiza Anan Noor, Tahmid Hussain, and Dong-Kyu Chae. Fairness and privacy preserving in federated learning: A survey. Information Fusion, 105:102198, 2024.
Benedek Rozemberczki, Lauren Watson, Péter Bayer, Hao-Tsung Yang. Olivér Kiss, Sebastian Nils- son, and Rik Sarkar. The shapley value machine learning. In The 31 International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence, pp- 5572-5579. International Joint Conferences on Artificial Intelligence Organization, 2022.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In Computer vision-ECCC 2010: 11th European conference on computer vision, Heraklion, Crete, Greece, September 5-11, 2010, proceedings, part iV 11, pp. 213-226. Springer, 2010.
Chandramouli Shama Sastry and Sageev Oore. Detecting oft-o--distribution examples with gram matrices. In International Conference on Machine Learning, pp. 8491-8501. PMLR, 2020. PMLR,
Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. Advances in neural information processing systems, 31, 2018.
LS Shapley. A value for n-person games. Contributions to the Theory of Games, (28):307, 1953.
Jiayi Shen, Qi Wang, Zehao Xiao, Nanne Van Noord, and Marcel Worring. Go4align: Group optimization for multi-task alignment. Advances in Neural Information Processing Systems, 37: 111382-111403 2024.
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and sup- port inference from rgbd images. In Computer Vision ECCCnEECCVV-C 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part 12, pp. 746-760. Springer, 2012.
Xin Tang, Jiawei Zhang, Yichun He, Xinhe Zhang, Zuwan Lin, Sebastian Partarricu, Emma Bou Hanna, Zhaolin Ren, Hao Shen, Yuhong Yang, et al. Explainable multi-task learning for multi- modality biological data analysis. Nature communications, 14(1):2546, 2023.
Rachel SY Teo and Tan Nguyen. Unveiling the hidden structure of self-attention via kernel principal component analysis. Advances in Neural Information Processing Systems, 37:101993-101477, 2024.
Simon Vandenhende, Stamatios Georgoulis, and Luc Van Gool. Mti-net: Multi-scale task interaction networks for multi-tase learning. Computer Vision - ECCV 2020: 16th European Conference, Glasgow, UK. August 23-28, 2020, Proceedings, Part IV. pp. 527-543, Berlin, Heidelberg, 2020. doi: 10. 1007/977 3-030-58544-- 31. URL https:
Springer- Verlag. ISBN 978-3-030-58547-L1
//doi.org/10.
www.org//..... 1007
13

---

Published as a conference paper at ICLR 2026
Dujuan Wang, Xinwei Wang, Sutong Wang, and Yunqiang Yin. Explainable multitask shapley explanation networks for real-time polyp diagnosis videos. IEEE Transactions on Industrial Informatics, 19(6):7780-7789, 2022a.
Jianhong Wang, Yuan Zhang. Yunjie Gu, and Tae-Kyun Kim. Shaq: Incorporating shapley value theory into multi-agent q-learning. Advances in Neural Information Processing Systems, 35: 5941-5954, 2022b.
Peiyao Xiao, Chaosheng Dong. Shaofeng Zou, and Kaiyi Ji. Scalable bilevel loss balancing for multi-task learning. arXiv preprint arXiv:2502.08885, 2025.
Jiancheng Yang. Multi-task learning for medical foundation models. Nature Computational Science, 4(7):473-474, 2024.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in neural information processing systems, 33: 5824-5836, 2020a.
Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: benchmark and evaluation for multi-task and meta reinforcement learning. In Conference robot learning, pp. 1094-1100, PMLR, 2020b.
Qi Zhang, Peiyao Xiao, Shaofeng Zou, and Kaiyi Ji. Mgda converges under generalized smoothness, provably. 2025.
Yu Zhang and Qiang Yang. An overview of multi-task learning. National Science Review, 5(1):
30 43, 2018.
Yu Zhang and Qiang Yang. A survey on multi-task learning. IEEE transactions on knowledge and data engineering, 34(12):5586-5609, 2021.
Zhipeng Zhou, Ziqiao Meng. Pengcheng Wu, Peilin Zhao, and Chunyan Miao. Continual opti- mization with symmetry teleportation for multi-task learning. arXiv preprint arXiv:2503.04046. 2025.
14

---

Published as a conference paper at ICLR 2026
APPENDIX CONTENTS
Use of LLMS
17
The Full Details VolDrt and VolDetPro Metric (i.e. Section 4.2)
17
B.1
Proof of det(M) (g1.....
17
B.2
Multilinearity of VolDet Metric and Compatibility with Subset v(S) of Shapley Value
18
B33
Vollet VS VolDetPro
19
B.4
Computational Complexity Analysis.
19
B.5
Comparison of Common MTL Conflict Metrics
21
Example: Shapley Value Computation
24
Normalize gradients
24
Gram matrix M
24
Shapley Value formula
24
C.4 Contributions for on
25
Summary of dn
25
Softmax normalization
26
D Proof of FViri Optimization Problem
27
SVFair Analysis
30
E.1
SVFair Convergence Analysis
30
................................
VolDetthased Fairness of SVFair
33
E.2.1
Feasible sets and regularity
................................
33
E.2.2
VolDet faireess functional
33
E.2.3
Unconstrained structure of >, wi/
................................
34
1.2.4
SVFair as unique VolDet-fair direction
34
E.2.5
FairGrad under VolDet fairness
35
Experiments Details
36
Experiments Details For Toy Example (Details For Section 5.1)
36
F2
Benchmark Descriptions and Training Details
41
Experiments Result Details (Details For Section 5.2
42
F4
Experiments Details In PINN: Burgers Equation
43
Details about the MC Shapley estimation
44
Details about the effectiveness, convergence, and cost-periormance trade-off of MC Shapley estimation
44
G.2
Monte Carlo Estimation of Shapley Values for VolDet
46
Details For Choice of (Details For Section 5.4)
48
15

---

Published as a conference paper at ICLR 2026
H.1
Theoretical perspective to guide the choice of
48
H.2 Softmax as the optimal solution of entropy- regullaized linear utility.
48
H.3
Experiments about the Choice of
50
Experiments Details For VolDet VS VolDetPro (Details For Section 5.5)
The link between Shapley value weighting and MTL optimization
52
K
Positioning of SVFair in the Literature from Geometric and Game- Thenn-Theertiic Perspec-
tives
K. Geometric Perspective
53
Cooperative Game Theoretic Perspective
53
16

---

Published as a conference paper at ICLR 2026
A USE OF LLMS
We made limited use of large language models (LLMs) during manuscript preparation, specifically to:
polish writing and check grammar,
2. assist with consistency and coherence checks of the text's logical flow.
All usage was conducted under author supervision. LLMs did not contribute to research design, experiment execution, result analysis, the generation of substantive conclusions. The authors solely responsible for all technical content, data, and conclusions.
B THE FULL DETAILS OF VOLDRT AND VOLDETPRO METRIC (I.E. SECTION 4.2)
B. PROOF OF det(M) Vol? ............ 9N)
Definition 1 MTL, the Gram matrix M E NxN defined as M = where the nor-
malized gradient matrix is G = :.. 99.] RaxN and 9: = 9s/ [|::ll According to the volume determinant identity, the squared volume of the parallelotope formed by these normalized gradients satisfies:
Vol? ,f....., 9N) = Vol((M) = det(M).
We analyze the identity det = Vol Vol (ā...... _N) under the three cases of d > N, = N, and d < N.
Proof:
Case 1: d d>> N
Linearly independent case: If the column vectors {9. are linearly independent, then rank(G) N,, and M is positive definite. Performing QR decomposition on & gives:
G = QR, Q RdNN QTQ = IN. Re NNN upper triangular and invertible.
Then
2
M= R R. det(M) (det R)²
R
The entries Ra correspond the norms of the orthogonal vectors in the Gram-Schmidt process, so:
N
N
Vol 9N) uul
Rai
which implies
det(M) = Vol? (ol............
Linearly dependent case: If the vectors linearly dependent, then rank(G) N and del = 0. The parallelepiped degenerates and its volume zero, so the identity still holds.
Case 2: d d= N
Here G E NNN is square matrix.
If & is invertible, then:
det(M) d ( G) (det G)
Since the volume of the parallelepiped spanned by the vectors in I. is Vol det G), have:
det(MM Vol?
17

---

Published as a conference paper at ICLR 2026
If G is singular, then det(.M) and Vol 0, and the identity holds trivially.
Case 3: N
In this case, at most d vectors can be linearly independent, ra)k(G) < d < N, and thus del(M) The N-dimensional parallelepiped Rd collapses, yielding zero volume:
Vol(@1......WN .Nv) = =0, = Vol = 0 = det(M).
Conclusion
In all cases, have:
det(M) = Vol 9N)
B.2 MULTILINEARITY OF VOLDET METRIC AND COMPATIBILITY WITH SUBSET v(S) OF SHAPLEY VALUE
Proposition 1 The VolDet metric v(S) det det(Ms)is multilinear with respect the input task gradients, and hence naturally satisfies the set function requirements needed for Shapley value com- putation.
Definition 4 Let V = -R- be vector space. The determinant
det V.... R (v1....,UN) - det(v)...... uNl
N times
is called multilinear if for any fixed position i E (1.....N} and for any vectors Ui E V and sealars a, B E R. the following holds:
dets. .., U-- a01; Bws, Us41,...]
det .., 8- US Ui+1,
det ... U 1n Wi; Ui+1s.....
Definition 5 Given task subset S C Then the Gram submatrix defined as
let {ûi hes denote the normalized task gradients.
Ms = Ms=GāGG E SS
where Gs 19ilies [dd The k-th column of Ms
C = [(r,, 9:. ) [S]
where = (i.,..., .
Definition 6 Suppose that for some i S (corresponding to column k), we have:
J au Bw, with u. E Rd. oz, R.
Then the k-th column becomes:
C [0r. om + BuO', aC% + 3C
where C% [( w)], and similarly for CV.
Proof of Proposition 1
By applying the Definitions
5 6 of the determinant function the k-th column:
dct(Cr,.... Ct-1, aC 3.%% Ck+1...., Css
det CI...., Ck-1, C ..... + 3 det Cki......k--1.. Ck C....
That is, let:
vSS) det(Mss
Hence, for the subset utility with 9, au :w:
e(S) a v (S) u) + B v(S) w).
This precisely aligns with the Shapley value's 's requirement that the subset utility function be linearly decomposable with respect to individual task gradient, thereby demonstrating that the utility function constructed by VolDet inherently satisfies the formal condition of marginal contribution additivity.
18

---

Published as a conference paper at ICLR 2026
B.3 VOLDET VS VOLDETPRO
Volume base term det(Ms). The determinant of the Gram submatrix det(Ms) measures the squared volume spanned by the normalized task gradients in subset S, capturing directional di- versity and independence. A small determinant indicates that gradients nearly collinear, leading to stronger risks of mutual interference, whereas a larger determinant reflects more diverse descent directions. By retaining this base term, VolDetPro can still faithfully reflect structural correlation conflicts even without antagonistic pairs. This choice also ensures consistency with the classical volume-based metric, allowing for intuitive interpretation and comparison. In partieular, when (S) = 2, dss serves as the sole contribution, providing a natural degenerate case.
Conflict strength term ((d))er(s) (Ms).s This term accumulates only the negative cosine simi-
larities in the strict upper triangle, directly quantifying the overall magnitude of antagonism. When- ever some task pairs form obtuse angles, these entries become strongly negative and thus signifi- cantly increase the score, reflecting the inherent optimization difficulty. Unlike the volume term, this component can discriminate between cases with the same des( but stronger antagonism, thereby making VolDetPro more sensitive to genuine conflict. When negative pairs exist, the term naturally vanishes, avoiding spurious contributions. Hence, the measure jointly covers both "crowding" due to correlation and "collision" due antagonism.
Coverage and scaling factor [ The square-root dependence on the number of antagonistic pairs [T()) provides sublinear growth, preventing uncontrolled escalation when the task number increases. The denominator |S] normalizes the scale, ensuring comparability across subsets of different sizes and suppressing combinatorial explosion. This factor balances the contri- bution of coverage (more pairs involved) and per-pair intensity, without being dominated by either extreme. As the number of antagonistic pairs grows, the amplification factor increases moderately, reflecting the intuition that broader coverage implies more substantial conflict. Overall, this design guarantees stability under varying task scales and sampling noise.
Natural degeneration and computational feasibility. When I(S) = Ø, VolDetPro reduces ex- actly to VolDet, i.e., v(S) det(Ms), maintaining interpretability consistent with the volume metric. This avoids generating artificial penalties during stages with no antagonism, such as early training or near convergence. The formulation only requires constructing the normalized gradient Gram matrix once, after which all subset values v(S) can be indexed directly. No extra retraining or complex optimization needed, resulting in low computational and memory overhead. This prop- erty also facilitates integration with Chapley-value-bare based aggregation, since both rely on the same Gram structure.
Sensitivity to both quantity and intensity of antagonism. Unlike metrics that consider only the number or the strength of conflicts, VolDetPro couples these two aspects its second term. The more negative pairs exist and the closer each is to 11, the larger the cumulative penalty becomes, signaling more severe optimization difficulties. This enables meaningful comparison between sce- narios such "few strong conflicts" versus "many mild conflicts." When volumes are similar, the configuration with stronger or denser antagonism always yields higher uS)). Therefore, VolDetPro provides stronger discriminative power during model selection and training.
Resolving misjudgment in obtuse-collinear scenarios. Classical volume can approach zero when two gradients nearly opposite, mistakenly suggesting "low conflict." VolDetPro corrects this by assigning a large penalty through its second term, which is consistent with the intuition that opposing gradients induce severe conflict. This eliminates the paradox that 0 and 180 0 yield the same determinant but represent very different levels of antagonism. For multi-task optimization, this correction exposes hard-to-align task pairs earlier in training. Hence, the monotonicity of the measure is strengthened around critical boundary cases.
B.4 COMPUTATIONAL COMPLEXITY ANALYSIS.
In many cooperative game scenarios, computing the Shapley value has been proven to be NP-Hard, and even #P-Complete (Michalak et al. 2013). Therefore, we will analyze the computational com- plexity in Definition
19

---

Published as a conference paper at ICLR 2026
We provide detailed comparison of the computational complexity analysis for four different methods, aiming to highlight the efficiency advantages of VolDet under large-scale tasks (Such as CclebA):
1. Am% (Exhaustive Enumeration) This baseline method uses the performance drop metric Am% to estimate the marginal contribution of each task subset. It requires training a separate MTL model for every one of the 2 possible task subsets, leading to an exponential computational complexity:
O 22,,
where C' denotes the cost of single MTL model training. While feasible for small N, this approach becomes computationally prohibitive as the number of tasks grows.
2. Am% Monte Carlo Sampling To mitigate the exponential cost, Monte Carlo strategy can be employed, where only < 2 subsets are randomly sampled for training and evaluation. This reduces the overall complexity to:
OKK-C)
However, since each subset still requires a full training run, the method remains expensive in prac- tice, especially for deep networks with high C. The procedure of Monte Carlo sampling (Rozem- berezki et al. 2022) is summarized in Algorithm
3. VolDet Metric (Exhaustive Enumeration) This approach eliminates the need for retraining by computing gradient-basee cooperative utility after a single full training with cost (C)). It includes gradient normalization for N task gradients of dimension d with complexity ((Ne), construction of the Gram matrix from normalized gradients with complexity O(N2d), and determinant computation for all 2 subsets, each costing (NN)) resulting in O(2 NN*) in total. The overall complexity is:
O + Nd N2d 2 No).
This is acceptable for small N, where matrix computations are negligible compared to the training cost. If the number of tasks N is small, then:
OCC + Nd N N*) O(C).
4. VolDet Metric + Monte Carlo Sampling To enable scalability in large-task settings such as CelebA with 40 tasks, we apply Monte Carlo sampling to the VolDet framework. After a single full training pass with cost O(C). we perform gradient normalization for N tasks at O(Nd) and construct the Gram matrix at (N2 We then randomly sample K subsets and compute the determinant for each, costing O(K- N2) in total. The overall complexity is thus:
O(C Nd + N KN se O(C),
It is important to note that in real-world MTL scenarios, model parameters often reach tens of millions, making the training cost (CC) (ranging from hours to days) significantly larger than the determinant computation cost O(N3), i,e, O(N3) < O(C). This further highlights the efficiency advantage of our approach.
5. VolDetPro Metric (Exhaustive Enumeration) Compared with VolDet, VolDetPro adds lightweight penalty computed from the strict upper-triangular negative entries of the Gram sub- matrix Ms. After one full training with cost OCC), we normalize N task gradients at O(Nd) and build the Gram matrix at (() For each of the 2 subsets, we (i) compute det(Ms) at ((IS)3) and (ii) scan and accumulate negative entries form the penalty at ((SS))) which asymptotically dominated by the determinant cost. Hence the overall complexity remains
O(C + Nd 2 2-
24 N²).
This is acceptable for small N, and when N is small relative model size,
OCC + Nd - 22) OCC).
6. VolDetPro Metric + Monte Carlo Sampling To scale to large-task regimes, we adopt Monte Carlo sampling with K < N subsets. After a single full training pass OCC), gradient normaliza- tion O ((N)), and Gram construction (N?dd) we evaluate K subsets. For each subset, deMs
20

---

Published as a conference paper at ICLR 2026
Table Comparison of Training Runs (TR), Determinant Calculations (DC), and Overall Compu- tational Complexity for Six Methods
Method
TR
DC
Overall Complexity
Am% (Exhaustive)
2
O 2 O(K C)
Am% + Monte Carlo
K
VolDet (Exhaustive)
2 O(N3)
O(C + Nd N No)
VolDet + Monte Carlo
K ((N3)
O(C Nd + N + { ((C)
VolDetPro (Exhaustive)
2 O(N3)
OCC + Nd 2 N3) O + Nd N KNO) O(C)
VolDetPro Monte Carlo
K O(N3)
costs O( and the penalty aggregation costs ([[S]3) (negligible in order). Therefore, the overall complexity is
O(C Nd + N + KN & O(C),
since in practice O(N3) < O(C) for deep MTL models, and K modest.
Algorithm 2 Monte Carlo Estimation of Shapley Values
1: Input: number of players N, utility function v(-), sample size K
2: Output: estimated Shapley values {
3: Initialize: < <- 0 for all i = I,...,N N
4: for 1 to do
5:
RandomPermutationn (1, N))
6:
S <-d 0
7:
for
to N do i < mli
8:
9:
A vSS U (i)) uS)
10:
o[ + 611 +
St SU {i}
12: end for
end for
14: for i = 1 to N do
15: Φ11 - olil/K K
16: end for
17: Return [
B.5 COMPARISON OF COMMON MTL CONFLICT METRICS
We will present the formal definitions of four widely used conflict metrics in MTL, along with comparative analysis of their advantages and limitations.
1: Cosine Similarity
g3 Definition: cos(gi.g;> |[ 1lg
Advantages:
Depends solely on direction; insensitive to gradient magnitude.
Computationally efficient: Od).
Limitations:
Measures only pairwise angular relations: unable capture higher-order task interactions.
Requires (NN2) comparisons as the number of tasks increases.
Cannot directly reflect the collective conflict within task subset.
21

---

Published as a conference paper at ICLR 2026
2: Inner Product
Definition: 9, 95-
Advantages:
Directly quantifies gradient projection.
More efficient than cosine similarity due to no normalization overhead.
Limitations:
Also limited to pairwise relations.
Sensitive to gradient norm differences; requires normalization or rescaling.
Lacks correspondence to global task conflict strength.
3: Performance Drop (Am)
Definition: Am% (-160
Mm. Mm.--Ma. Ma.k
100
(i.e.Eq. 4
Mo,
Advantages:
Reflects the actual performance impact task subset.
Semantically interpretable; aligns well with business objectives.
Limitations:
Sensitive to evaluation metrics, task scale, and other non-structural factors.
Requires training separate model for each subset; complexity O 22)).
Non- Hin-differemmiibbl; unsuitable for end-to-end learning integration.
4: VolDet
Definition: Let S (1....,N) .N be task subset. Define the normalized gradient matrix Gs and the corresponding Gram submatrix Ms (i.e. Definition
Ms = [g, òjkeees:
where v(S) = det (Ms).
Advantages:
Captures global geometric conflict among multiple tasks, beyond pairwise relations.
Requires only single round of gradient sampling and matrix construction; all subset utili- ties be computed via index-based operations.
Continuous, differentiable, and ocale-invariant solely depends gradient directions.
Limitations:
Exact evaluation for all subsets still requires O(2N DD); must be approximated via sampling (See the Appendix B.4).
5: VolDetPro
Definition: Let S C (1....,N) be task subset. Define the index set of antagonistic pairs on the strict upper triangle I(S) = [(i,j) | 1 < < (S), (Ms) < 0} (i.e. Definition
Upro (S) = det(Ms) / [[SS]++S
Σ (Ms)
|S|
((J)EI(()
Advantages:
22

---

Published as a conference paper at ICLR 2026
Sign-aware: distinguishes acute vs. obtuse configurations that share the same volume (re- solves VolDet's sign insensitivity).
Preserves VolDet's practicality: single-pass gradient sampling and Gram construction; subset utilities via index-based operations; Shapley-ready.
Scale-invariant (uses normalized gradients) and almost everywhere differentiable; mono- tone w.r.t. antagonism (and strength of negative pairs).
Lightweight overhead: per-subset penalty aggregation costs O[[S))), dominated by the determinant (IS))s: same big-O as VolDet.
Limitations:
The absolute-valus value term is non-smooth at 0 (subgradients apply), though this rarely affects optimization in practice.
6: Summary:
Local vs Global: determinant captures full-set geometry.
Cosine and inner product are limited pairwise relations, while the Gram
Efficiency & Single-pass: Replacing outcome-dependent Am with Gram-based volumes (VolDet/VolDDetPro) enables computing eS) for all subsets from single round of gradient sampling and Gram construction via indexing operations, greatly reducing the practical overhead of Shapley-value computation.
Robustesss & Differentiability: VolDet is norm-invariant and (almost everywhere) differ- entiable, making it suitable for end-to-end integration; it is also stable to gradient rescaling and rotations.
Sign-awareness (VolDetPro): VolDetPro resolves the sign insensitivity of pure volumes, distinguishing acute vs. obtuse configurations that share the same volume; it equals VolDet when I(S) = and increases with both the number and magnitude of negative similarities, while keeping the same big-O complexity as VolDet.
23

---

Published as a conference paper at ICLR 2026
C EXAMPLE: SHAPLEY VALUE COMPUTATION
We provide an example of Shapley value computation based on the VolDet metric, where the task number N is 4. Let gradient matrix G is:
0
0
0
G
911- 9N
0
0
0
1
0
1 1
We define the utility function of Shapely value is v(S) Vol det(Mss. (i..Eq. 5). where 9i is the gradient of task i, and M is the Gram matrix the normalized gradient matrix. Note. The Shapley-Vall Valuee computation is identical for VolDetPro; one only needs to replace Definition 2 with Definition 3 Since the set-function form and indexing operations remain unchanged and only the utility values differ by sign-aware additive term, present the worked example using VolDet for brevity.
C.I NORMALIZE GRADIENTS
The normalized gradient matrix T is:
1.0000
0.0000
0.0000
0.0000
G = [0..... 1..
0.7071
0.7071
0.0000
0.0000
0.7071
0.0000
0.7071
0.0000
0.5000
0.5000
0.5000
0.5000
where 01 = 1///l--
C.2
GRAM MATRIX M
The Gram matrix M is:
1.0000
0.7071
0.7071
0.5000
== -
0.7071
1.0000
0.5000
0.7071
0.7071
0.5000
1.0000
0.7071
0.5000
0.7071
0.7071
1.0000/
C.3 SHAPLEY VALUE FORMULA
For each task i, the Shapley value o; computed (i.e.
2
S - SS| 1)!
Φ:
Σ
v(S)).
(9)
N!
Since N 4, the weight function is:
w(k) k!(3 k)!
(0) 0.25, w(1) = 0.08333, w(2) = 0.08333, w(3) = 0.25.
4!
We compute the determinant v(S) of some subsets as examples:
1. Subset (1)
u((1)) det (M()).)) det ([Mii)) Mn 1.
2. Subset (2,33
[22
M23l
1
0.7071
o((2,3)) = det
det.
= 1 - (0.7071) 0.50.
M32
M33
0.7071
24

---

Published as a conference paper at ICLR 2026
Table 8: Shapley Value Contributions for |S|
S
((S)
v U (i))
A uSSu (0)) v(S)
Contribution=w(() A
(1)
0.5000
-0.5000
-0.04167
2
1
0.5000
- 0.5000
-0.04167
3)
0.7500
-0.2500
-0.02083
Table 9: Shapley Value Contributions for |S|
2
S
(S))
v(S U {i))
A
Contribution=w(() A
2]
0.7500
0.2500
-0.5000
-0.04167
1,3
0.5000
0.2500
-0.2500
-0.02083
2]
0.5000
0.2500
-0.2500
-0.02083
3. Subset (1,2.3
Mii
M12
M13
0.5000
0.7071
e((1,2,3)) = det
M21
M22
M23 = det
0.5000
1
0.7071
~ 0.25.
Msi
M32
M33.
(0.7071
0.7071
C.4 CONTRIBUTIONS FOR do
Let the "contribution" be denoted as w([S)) * [u(SU (()) - o(S),, and A = v(SU(()) v(S).
1. |S| 0
Weight: w00 = 0.25.
Difference: A v((0)) = -1 1.
Contribution: w0)) x = 0.25 0.25.
2. |S| = 1
Weight: w(1) = 12 = 12 0.08333. 4!
Difference: See the Table g
Contribution: Σ (ww(() (1) -.04 0.04167 0.02083 00.1047.
3. | = 2
Weight: w=2)
1 0.08333.
12
Difference: See the Table
Contribution: Σ w/ (2) 0.04167 0.02083 0.02083 0.08333.
4. | 3
Weight: w33) = 0.25.
Difference: ((,0,,,2,3) v((1,2,3) = 0.1250 0.2500
00.1250
Contribution: w(3)A = 0.25 x (-0.1250) = -0.03125.
C.5
SUMMARY OF do
Φ = 0.25 + (-0.10417) + (-0.08333) + 0.001225 - 0.03125.
Other d1. d2, and Ø3 follow the same procedure, and can be computed as:
Final Shapley Value: [0.03125, 0.03125, 0.03125, 0.03125
25

---

Published as a conference paper at ICLR 2026
C.6 SOFTMAX NORMALIZATION
According to Eq. D
with emperature parameter = 1, we have:
c
0.031225
Wi
4 0.0313 0.25.
Σ j== E
Final wi Value: [0.25, 0.25, 0.25, 0.25]
26

---

By applying the Published as a conference paper at ICLR 2026
D PROOF OF SVFAIR OPTIMIZATION PROBLEM
OPTIMIZATION OBJECTIVE
We consider the following SVFair gradient aggregation problem (See Eq. 7 in Section 4.3 for more details):
wi s.t. g. d > 0, Vi,
min
9,
We constrain the surface of an lz-ball of radius e, ie. Karush-Kuhn- Kuhn-Tucker (KKT) conditions, one obtains:
||||
Step 1: The optimum lies on the boundary: |d"|
Step 2: All inner-product constraints are strictly satisfied: 9, d' > 0, vi.
Step Introducing Lagrange multiplier yields the stationarity condition Vf(d')
Step 4: Finally, the stationarity condition simplifies to Eq. 10
eNws Ne. dj2 =
(10)
Proof of Step 1
Proof by contradiction. Assume that ||d*|| e. Since of d' for any n and n||d"|| S e, define
d d= d=
Thus, is evident that:
||||| = n|ld"|| e,
-f== nlg d²)
indicating that d is still feasible.
By scaling, we obtain:
N (nd*)
N
(())
Wi
Wi 1 s(d") < f(d'),
1
g. d'
11
which contradicts the optimality of d'
Therefore, the assumption does not hold, we must have:
||d =
Proof of Step 2
Proof by contradiction. Assume that there exists some index such that:
9f d = 0.
Then, we have:
N
f(d') = N Σ
Wi
2 Wj
=+00,
JT d=
N
27

---

Published as a conference paper at ICLR 2026
which contradicts the finiteness of the objective value at the optimal solution. Therefore, the assumption does not hold, we must have
gf d' > 0, V..
Proof of Step 3
1. KKT Preliminaries
The objective function (dd is convex over the feasible region {d: ||d|| S c, gf d 0}.
There exists strictly feasible point (e.g., taking ||d]| and all of d > satisfying Slater's condition, implying that the KKT conditions are both necessary and sufficient for optimality.
2. Lagrangian Function
Define the inequality constraints as:
(d) ||d|| - c, he(d) 0 = (....,,))
The Lagrangian function is then given by:
N
() = [(d) Mho(d) Σ
psha(d);
1=1
where A, u4 0. 3. Stationarity Condition
N
Valid'A,, = Vf(d)) +/V/ + Σ p+(-g)) = O.
d) Note that V||d" d' /|ld"| and d) -i-
4. Complementary Slackness and Feasibility
Since ||d" || = (from Step 1), it follows that À 2 0 and X A(I|d'| -) =0 - is feasible.
Since g.f d > 0 (from Step 2), hx(d") < = = 0 Vi.
5. Simplifying Stationarity
Substituting |dd'| = e and μ4 = O. we obtain:
d' + - Vf(d") = -
6. >> 0
If X O, then (dd') 0, indicating that the unconstrained optimal d= is a stationary point. How- ever, since ||d* & is not enforced, we can arbitrarily increase ||d]], making each term w d) smaller and unbounded, contradicting the finite optimal objective value. Hence, 0.
Proof of Step 4
For the function h;(d) = We the gradient given by:
(h,(d)
Wi
(,7 dj
Thus, the gradient of (d) can be expressed as:
28

---

Published as a conference paper at ICLR 2026
vd(d) 1
Vh((d)
1
W4
9i.
(g) d)
From Step 3, we have:
ff((d) - d"
Substituting the gradient expression above, we obtain:
W A d=
Di
dr.
N
gff
Multiplying both sides by (ANN, we get:
N
N Σ
e
wi
d'
AN
d )2
0s9i
1=1
where
EWI XN
Since w; and d.'d' O, we have a; > 0. Thus, d' located within the convex hull formed by {g:}.
29

---

Published as a conference paper at ICLR 2026
E SVFAIR ANALYSIS
E. SVFAIR CONVERGENCE ANALYSIS
Theorem 1
Suppose Assumption is satisfied. There exists a subsequence {0,} of the output
sequence { that converges a Pareto stationary point 8*, with convergence rate of (1/=²).
Proof: Step 1 According to Assumptions !
suppose that is L-smooth. We have:
I. 2 110+1 - 0ell?.
[(0++)) s ((0)) V2((1)) (0t+1 - 01) +
Since O2+1 O2 nede, we have:
L
((+t))-((0)) (())) d. + 2 11d.11
To guarantee
((0+41) S C(O,),
(1I)
the l/ should be satisfied:
2V.(()) de L||de|12
0<N 5
(12)
Meanwhile, we compute:
1 N VC(0,) d. = N
N
N
Σ (0))
To.
de
Witt
||de
2|
Wi,
N
aiit
Thus, obtain the upper bound:
N
Wi.t/as,t
N
1/t
N
t=1
2EN Vur.l/as LN EN vaurass
That is, selecting any value within the interval (0,
ensures Eq. 11 holds. For
NΣN /wrs/ase halved, i.e.
example, taking f)- LN EN, Awi,sas,
ΣN /WWie//a VL(00) de 1.1ck|12
11 LN 1i 10i,
(13)
LN
Step 2: Convergence
According to Eq. I and Eq. 13
we have:
((0,1)) < C(0,)
{C(0.))e2o
is monotonically non-increasing.
Since each individual loss (0) is bounded below, so:
1 N Σ (0) L OC.
L(0)
N
We conclude:
lim C(0,) =: L (a finite value),
1-000
30

---

Published as a conference paper at ICLR 2026
With Eq. 13 the Eq.
becomes:
3(V2(0))
((0,) [(0(+1) 2
2T||dd/2
Suppose [[))) A Then there exists >> 0 and subsequence {49 such that [V for all j. By Assumption 1 (boundedness of []d,]]). assume:
v((())) de, 0, ||de, sc
Let A L((,) [(0++1). then have:
3m
A
6 0.
2L
Infinitely many positive lower bounds ó would imply that [(0,) continues decreasing without bound, contradicting convergence to L. Therefore,
lim 8000
[VL(0)) = 0.
Since VL(0,) = D
-i= (0,), if there exists any task j such that (0)) A 0, then the individ- ual loss (0.) would continue decreasing, contradicting the boundedness of [(,,). Hence,
lim 11g (0:) O, Vi.
Since (0.)) + 0, by continuity we conclude gi (00) 0. Hence the gradient matrix 0" (0" a Pareto stationary point):
G( = (8*), IN (0*) 0
is the zero matrix (See the Preliminaries Section B for the definition Pareto stationary point).
Step 3: Iteration Complexity for Gradient Norm Accuracy
We aim to find the minimum iteration index T such that the gradient norm satisfies target aceu- racy 0:
V S.
When [L())) > e, by the Cauchy-Schwarz Schwarz inequality, we have:
VC(0,) 4 - [[(())]]
and therefore,
2L0 3/
322
A
8
2L|
2L
2L
Summing this lower bound from 1. to T - 1, we get:
T-1
[(10) - c-= Σ A.2.
which implies:
((0)) L (T(((0o) L*) 0(1/E3).
T
8
322
Hence, to achieve gradient norm smaller than s, the number of iterations is at most (1(=²²), which matches the classical convergence rate of first-order methods under non-convex L-smooth settings.
In non-convex L-smooth settings, gradient descent with a fixed step size y) = 1o2 achieves a classical convergence rate (1/=²) for reaching gradient norm accuracy 0. Denote the full objective L(0) and the update rule as:
71
L
31

---

Published as a conference paper at ICLR 2026
With the update 0141 0s VV(()) the descent lemma implies:
((0++1) - ((0.) 1
[V + L.
L II
(14)
L
= =((0) 2L VC(00))
(15)
Let A, = L(0,) [(0++1), then:
A 2 2L
Summing from = to 7' - 1, and using monotonicity and boundedness, with L = info L(0), we obtain:
T
T-1
[ & 2L
Σ A $ 2L(((()) L*].
Let i be the index such that [VL(0:]] is minimized. Then:
T-1
T VV <
[L(())) > 21(((0)) - L.).
1=0
To ensure |VL(0:)] & s, it suffices that:
(L(((00) L
7 2
0(1/6))
8²
32

---

Published as a conference paper at ICLR 2026
E.2
VOLDET-BASED FAIRNESS OF SVFAIR
E.2.1 FEASIBLE SETS AND REGULARITY
Per-task improvement for direction d is (,(d) = 0: d, i = 1....., N.
Fix > 0 and assume:
Assumption (Interior feasible direction) There exists d E TRP such that
gd 0, == N. ||||| S c,
Pick any 8 E (0, min; 9. d).
Definition 7 (Feasible direction and improvement sets) Define
D { R" ||d|| S e, 9. d 8, i 1, N}.
(16)
21 {u R. PE D, u = 1,....,
(17)
Then:
Definition 8 (Geometry of D and U) Under Assumption
1. D is non-empty, compact, and convex.
2. Let G 19 10i....,9M 9Nv] RPXN Then U G.D, hence U is non-empty, compact, and convex.
3. For all u U, 2 2>00 0.
Proof (1) D is the intersection of the closed ball {d: ||d| S { and finitely many closed half-spaces (d: g. d), hence closed and bounded (compact) and convex. Assumption and the choice of 5 imply d D, D # 0.
(2) GT is linear; linear images of convex (resp. compact, non-empty) sets are convex (resp. compact, non-empty), hence U1 has the desired properties.
(3) From Definition
g 2 for all D.
E.2.2 VOI.DET FAIRNESS FUNCTIONAL.
Definition (VolDet fairness functional) For u E R.N
+
F(u)
U1
Definition 10 (VolDet-fair allocation) A vector x² E l is VolDet-fair if
F(u') min.F(u)
well
If the minimizer is unique, is the unique VolDet-fair allocation.
Definition (Strict convexity) On RN u u+ F(u) strictly convex.
Proof For each i, f(()) = =i uj > 0 satisfies
2wh 0. (u.)
U
Thus fa is strictly convex and so is the sum F(u) E, f((e))
33

---

Published as a conference paper at ICLR 2026
E.2.3 UNCONSTRAINED STRUCTURE OF E, wi/u;
Proposition 2 (Closed form under linear sum constraint) Let 0, wi > Consider
W
min KER
S.f.
04
U.
(Pv)
Then the unique solution is
U
c//c.
Σ 2f=1
and hence
u = e
ii.
V 4ii
Proof Lagrangian:
C(u, A)
Σ u
U).
Ui
KKT:
Wi u
wi
+ \
ui
Sum constraint gives
+ Ví=
1
Σ
Ws.
U
i=1
Strict convexity (Definition 11 implies uniqueness.
E.2.4 SVFAIR AS UNIQUE VOLDET-FAIR DIRECTION
SVFair direction at this iteration solves
Wi
min
(Psv)
dED
9
Note that 9, d maps equation Psv to
min F(u). uEH
Theorem (SVFair unique VolDet-fair allocation) Under Assumption 2 and Definition
problem equation
Psv admits at least one solution dSV and the induced vector
SV u(dVV
satisfies
F(usV) min F(u). weld
Moreover, SV is unique, hence the unique VolDet-fair allocation.
Proof By Definition 8 U is non-empty and compact. By Definition II convex IR-Y in particular on U. Thus F attains unique minimizer sSV over U. Existence of at least one @S with u(d=V)= SV follows from the definition of U.
F continuous and strictly
(If needed, an additional non-degeneracy assumption on the linear map + GT ddd can be imposed ensure uniqueness of SSV itself.)
34

---

Published as a conference paper at ICLR 2026
E.2.5 FAIRGRAD UNDER VOLDET FAIRNESS
FairGrad-type methods solve, for some 7 0,
9, d)²-7
U
max
max WEM
(PrG)
dED
1-9
Let PG be any optimizer of equation PrG in U.
Assumption (Non-uniform VolDet-Shapley weights) There exist i, such that ws + Wi.
Theorem (SVFair vs. FairGrad under VolDet fairness) Under Assumptions and let SV be as in Theorem 2 and let u. PG solve equation PEG
Then
F(u < F(u
unless FG SV (degenerate coincidence).
proof By Theorem u SV is the unique minimizer of F on U. Thus, for any u E (usv].
Flusv) < F(u).
Apply this to FG
35

---

Published as a conference paper at ICLR 2026
F EXPERIMENTS DETAILS
F.1 EXPERIMENTS DETAILS FOR TOy EXAMPLE (DETAILS FOR SECTION 5.1
Following the setup in prior works (Liu al., 2023 Ban & Ji
2024 Qin al. 2025b). we employ three distinct two-task toy examples as presented in CAGrad (Liu et al.) 202 a). lay These examples
encompass varying levels of gradient conflict and optimization difficulty. Each two-task scenario (0) and L. (0)) is defined on 0 = (0), B2) E R allowing for a comprehensive comparison of MTL approaches across different task pairs. We use sky blue to highlight the differences among the three two-task settings.
Following the setup in prior works (Liu al. 2023 Ban & Ji 2024 Qin et al. 2025b). we use five distinct starting points {{ {--8.5, 7.5), (0, 0), (9.0,9.0), 9.0), 7.5. -055), 1.0) The Adam optimizer is employed with 1 x 10-3 learning rate. The result is shown in the Figs 4 Further- more, we compare our proposed SVFair method with the following methods in experiments: UW (Kendall et al.] 2018). RLW (Lin et al.; (021). DWA (Liu et al. 2019). IMTL-G (Liu et al. 202 MGDA (Sener & Koltun 2018), PCGrad (Yu et al. 2020a). CAGrad (Liu al.) 2021a),
Nash-MTL (Navon et al. 2022). FAMO (Liu et al. 2023). and FairGrad (Ban & Ji 2024). ). All ex-
periments follow the settings of prior works (Liu al. 2021a Navon al.; [002]
al. 2023
Ban & Ji. 2024 Qin et al. (2255)) Our code is available in the supplementary material. Here, 7 and 7 are the hyperparameters of their respective algorithms.
In all three two-task toy examples, SV-Fair requires only a single hyperparameter x 0.1-50 to smoothly and trace entirely out the Pareto front between Task and Task C. Its convergence is stable, exhibiting neither extreme oscillations nor bias toward any single task. By contrast, TAMO/Fairrra are highly sensitive to their trade-off parameter 7 and demand extensive tuning. while conventional baselines often oscillate in conflict regions or cover only one end of the front. Consequently, SV-Fair outperforms these methods regarding solution diversity, stability, and ease of
use.
Two-Task A From FAMO (Liu et al. 2023
(0) 0.1 0. (1) (0) (2))),,
C2(0) = ((000=
where the functions are defined as follows:
fi max(tanh(0.56))).; 0),
[2) max(lanl( 0),
91 (0) = log
(0000-0000000000000) - 7) - tanh (02) 0.000005)) + 6,
(2) log (max((0.5(-0)) 3) -tanha-0) + 2), 0.000005) + 6,
h²) ((-01 + 7) + +0.1 -0: -
20,
10
h2(0) ((-0) - 7)2 0.1 02 - 8)2) 20.
10
Two-Task B From FairGrad (Ban & Ji 2024
0.1 -100))111000000000 (0)
C2 = ((000=
where the functions are defined as follows:
(1) max 0).
f2( max 0),
91 (0) = log (max((0.5( 01 - 7) (a + 6,
(0) = log (max( + 3) - -tanh( 21,,000005)) + 6,
hg) ((-81 + 7)2 + 0.1( 01 - -8)2)
20,
10
36

---

Published as a conference paper at ICLR 2026
sherage
MISO
MIGI
Fame bwEat)
Peputtgetnr
Ninga lyeid
tequltgetors
No DN DNT DHEM
te www.iaateetteeee
Figure 4: Comparison of approcccs on challenging synthetic Two-Task A benchmark (From
FAMO (Liu et al. 2023 ). We visualize optimization trajectories w.r.. objective values (C1, C2). Black dots indicate starting points, and the Pareto front (see Section R is represented by thick gray lines.
37

---

Published as a conference paper at ICLR 2026
MISO
Aomustpete
Avnstpete
Ranguad (
Aangaa (yed
SATA WWG.INEEEOe brooud
Fal Debbd mett
WNGG bedoat
Safer SnD
SERERT CIG
Figure 5: Comparison of MTL approaches on a challenging synthetic Two- Two-Task benchmark (From FairGrad (Ban & Ji 2024h). We visualize optimization trajectories w.r.t. objective values C2),
38

---

Published as a conference paper at ICLR 2026
h2( ((-0) -722 + 0.1( 01 -8)3) 20.
10
Two-Task C From PIVRG (Qin et al., 2025b)
L, (0) fi (0) + (2))),
C2(0) f1(0))2(0) + (2(0))2(0),
where the functions are defined as follows:
f#) = max(tanh(0.56)).0) 0).
f2(0) max(tanh((-0550) 0),
91 (0) log (max((0.5f 01 - 7) - (a)h 0.00000)),000000005)) + 6,
9e( log (max (max((055(-0,
-Lanh-002)) + 2|, 0.000005) + 6,
(0) ((-01 + 7) + 022 -
20,
10
h2( ((-01 - 7) + 0.1( 02 - 8)3) 20.
10
39

---

Published as a conference paper at ICLR 2026
prawegge
MISG
Avsstpete
Aanguad (oN
Aasuad (red
1000- ST.
SAT SAGEOEEEEO brooud
Fal Debed mede
Safer Snoet
Figure 6: Comparison of MTL approaches on a challenging synthetic Two-Task benchmark (From PIVRG QQn et al. 2025b)). We visualize optimization trajectories w.r.t. objective values (C1, C2).
40

---

Published as a conference paper at ICLR 2026
F.2 BENCHMARK DESCRIPTIONS AND TRAINING DETAILS
All our experiments were conducted on RTX 3090 and RTX 4090 GPUs. Our code is available in the supplementary material.
NYU-v2 (Silberman et al. 2012 contains 1,449 densely annotated images focused on indoor scene understanding, supports one pixel-wise classification task (semantic segmentation) and two pixel- wise regression tasks (depth estimation and surface normal prediction), with depth estimation cover- ing 13 semantic categories. Following prior work, adopt the MTAN backbone based on SegNet, integrating task-specific attention modules. Our method is trained on NYU-v2 for 200 epochs with batch size of 2; the learning rate set to le-4 for the first 100 epochs and then decayed by half for the remaining epochs.
Cityscapes (Cordts et al. 2016) contains 5,000 urban street view images, designed for two tasks: 7-class semantic segmentation (classification) and depth estimation (regression). We adopt the same training setup as NYU-v2, but the batch size
CelebA (Liu et al. 2015) is a large-scale facial attribute dataset containing over 200,000 images, each annotated with 40 attributes. It can therefore be formulated as a MTL problem with 40 image- level classification tasks. Following prior work, layers and attach a separate linear output layer for each task. The model is trained using the Adam optimizer with learning rate of 3c-4 for 15 epochs and batch size of 256.
adopt CNN backbone with 9 convolutional
Office-31 (Saenko et al. 2010) contains 4,110 images spanning three domains (Amazon, DSLR, and Webcam), designed for image classification. Following prior work, we adopt pre-trained ResNet- 18 as the shared feature extractor, with task-specific linear heads. Input images are resized 224x224, and the model is trained for 100 epochs using the Adam optimizer with learning rate of le-4 and weight decay of le-5.
41

---

Published as a conference paper at ICLR 2026
F.3 EXPERIMENTS RESULT DETAILS (DETAILS FOR SECTION 5.2
We present the detailed experimental results corresponding to Section 5 Table 10
and Table I1
Table 10: Results on NYU-v2 (3-task) dataset. Each experiment repeated 3 times with different random seeds and the average is reported.
Segmentation
Depth
Surface Normal
Method
Angle
Within 8
MR
Am?
mIoU 1
PAce
AEr
REir 4
Mean
Median
11.25
30
STL
38.30
63.76
0.6754
0,2780
25.01
19.21
30.14
57.20
69.15
37.17
0.5759
0.2410
24.18
22.26
47.05
60.62
DWA
39.11
65.31
0.5510
0.2285
27.61
23.18
24.17
50.18
62.39
10.89
3.57
UW
36.87
63.17
0.5446
0.2260
27.04
22.61
23.54
49.05
63.65
10.78
4.06
MGDA
30.47
59.90
0.6070
0.2555
24.88
19.45
29.18
56.88
69.30 36
8.78
1.38
PCGrad
38.06
0.5550
0.2325
22.80
23.86
63.14
11.22
3.97
GradDrop
39.39
65.12
0.5455
01,2279
27.48
22.96
23.38
49.44
62.87
10.56
3.58
CAGrad
39.79
65.49
0.5486
0.2250
26.31
21.58
25.61
52.36
65.58
8.22
0.19
IMTL-G
39.35
65.60
0.5426
0.2256
26.02
21.19
26.20
53.13
66.24
7.44
-0.60
Nash-MIIL
40.13
0.5261
0.2171
28.40
68.15
FAMO
38.88
64.90
0.5474
0,2194
25.06
19.57
29.21
56.61
68.98
6.67
-4.10
FairGrad
39.74
66.01
0,5377
0.2236
24.84
19.60
29.26
56.58
69.16
4.78
-4.66
PIVRG
39.90
65.74
0,5365
0.2243
24.30
18.80
30.95
58.26
70.38
3.56
-6.50
SVFair(VolDet)
39.89
66.14
0.5195
0.21188
23.93
17.92
32.02
59.50
71.36
1.44
-8.81
VolleroVo
40.23
65.56
0.5245
0.2111
24.16
18.09
31.68
59.13
70.99
2.22
-8.29
Table 11: Results on Cityscapes (2-task) dataset. Each experiment is repeated times with different random seeds and the average is reported.
Segmentation
Depth
Method
MR 4
Am% +
mloU
Pix Acc 1
Abs Err +
Rel Err
STL
74.01
93.16
0.0125
27.77
RLW
74.57
93.41
0.0158
47.79
11.00
24.37
DWA
75.24
93.52
0.0160
44.37
9.00
21.43
UW
72.02
92.85
0.0140
30.13
9.25
5.88
MGDA
68.84
91.54
0.0309
33.50
12.25
44.14
PCGrad
75.13
93.48
0.0154
42.07
9.12
18.21
GradDrop
75.27
93.53
0.0157
47.54
8.25
23.67
CAGrad
75.16
93.48
0.0141
37.60
7.88
11.58
IMTL-G
75.33
93.49
0.0135
38.41
6.50
11.04
Nash-MTL
75.41
93.66
0.0129
35.02
4.25
6.72
FAMO
74.54
93.29
0.0145
32.59
8.75
8.13
FairGrad
75.72
93.68
0.0134
32.25
3.25
5.12
PIVRG
75.82
93.65
0.0126
27.87
2.50
-0.45
SVFair(VolDet)
73.42
92.92
0.0123
25.61
6.88
-2.08
(eoDeroo
74.16
93.13
0.0123
25.60
6.12
-2.40
42

---

Published as a conference paper at ICLR 2026
F.4 EXPERIMENTS DETAILS IN PINN: BURGERS EQUATION
To further validate the convergence and applicability of SVFair in broader multi-task scenarios, we conduct an additional experiment on physics-informed neural networks (PINNs). Compared with the vision and tabular tasks in the main paper, PINN training is typically more non-convex and exhibits stronger gradient stiffness: the gradients and convergence rates of the PDE residual, initial condition, and boundary condition losses are highly imbalanced.
1: Experimental Sctup: Multi-task PINN on the Burgers Equation
We adopt the time-dependent Burgers equation as our test case. Let the scalar field w(x, t) satisfy
du Ou
Ou dr
+ dx
01
where the viscosity coefficient 0.01/T, and the space-time domain is æ E [-1, L [0. The initial and boundary conditions follow the classical setting:
u(x 0) = sin(
-(1,))) =.
Within the PINN framework, we fully connected neural network ug 1) to approximate the solution u(x, t). Following the configuration in the ConFIG (Liu et al. (024) work, we define three loss terms for the PDE residual, boundary condition, and initial condition, denoted by LN, LB, Lr, and obtain their corresponding gradients IN. 98.9. 91 via automatic differentiation. To align with the two-task formulation used by existing gradient-bas based methods, we merge the boundary and initial losses into single term
Las = LB + Lr,
which yields two-task loss pair (LN, LB)) which we apply different multi-task optimization methods. All methods share the same PINN architecture, sampling strategy, and training budget; only the gradient aggregation / weight assignment mechanism is changed, ensuring a fair compari- son.
In SVFair, we treat (IN, LBI) as two "tasks", define the utility e(5) of any task subset S via VolDet/VolDetrro, and compute task-level Shapley values {øi} based on this utility. We then obtain per-task weights through
(T( exp(dr/T),
and use them to construct the final update direction d = Σ. wigi- Other implementation details (net- work depth, activation functions, number of collocation points, learning rate schedule, etc.) follow the main paper and the ConFIG paper, and are omitted here for brevity.
2: Baselines and Evaluation Metric
We compare SVFair against the following representative methods: Adam (Kingma 2014), PCGrad Yu al. 2020a).
IMTL-G (Liu al. 2021b). MinMax (Liu & Wang 2021). ReLoBRaLo (Bischof)
& Kraus 2025). ConFIG (Liu et al.) 2024), M-ConFIG (Liu et al. 2024), ParetoMTL (Lin et al.)
2019) MGDA (Sener Koltun 2018).
All methods are trained with the same set of random seeds, and we report the mean + standard deviation of the test error over three runs. The evaluation metric consistent with ConFIG (Liu ct al. 2024) and measures the numerical eиror of the Burgers solution at test points (lower is better). The results are summarized in Table [2] where the last row corresponds to our newly added SVFair.
From Table 12 we observe that: Among all methods, SVFair achieves the smallest test error, con- sistently improving over Adam as well as generic MTL baselines. Notably, SVFair also outperforms ConFIG and M-ConFIG, which were specifically designed for PINNs and have been shown to hibit strong convergence and numerical performance on Burgers the original work. Under the same PINN architecture and training budget, using VolDet with Shapley- -based weights construct the SVFair update direction further alleviates the gradient conflict between LN and Lgz and leads better multi-objective trade-off.
Overall, this additional experiment shows that SVFair is not merely "tuned to work" on standard vision MTL benchmarks, but also demonstrates stable convergence behavior and performance gains on highly stiff, strongly conflicting PINN training problems.
43

---

Published as a conference paper at ICLR 2026
Table 12: Results on Burgers.
Method
Adam
1,484 + 0.061
PCGrad
1.344
IMTL-G
1.024
MinMax
0.149
ReLoBRaLo
1.059
ConFIG
1.30800000
M-ConFIG
1.035
ParetoMTL
1.603 0.030
MGDA
1.706
SVFair
1,271 0.055
G DETAILS ABOUT THE MC SHAPLEY ESTIMATION
We provide additional details of the computation of the Shapley value discussed in Section 5.4 The analysis details of computational complexity can be found in Appendix [B.4 Appendix B.4 states that, as the number of tasks N grows, although VolDet (VolDetPro) can greatly reduce training costs, the overhead of computing determinants still increases exponentially each determinant evaluation has time complexity of ((N*). Therefore, introduce Monte Carlo sampling to reduce the overall computational cost from exponential to an acceptable linear scale. The procedure of Monte Carlo sampling is summarized in Algorithm
Algorithm Monte Carlo Estimation of Shapley Values
1: Input: number of players N, utility function v(-), sample size
2: Output: estimated Shapley values {
3: Initialize: Φ[1] < 0 for all i I....,N
4: for t 1 to K do
5: π f- RandomPermutation/ ((1...,((11 NJ)
6: S <<d 0
7:
for j = 1 to N do
8:
i +++t) T|i]
9:
A + vSS (SU
v(S)
01 + ø1 + S SU {i}
11:
end for
13: end for
to do 15: Φii] obi]/K
for
16: end for
17: Return (or)
G.1 DETAILS ABOUT THE EFFECTIVENESS, CONVERGENCE, AND COST-PERFORMANCE TRADE-OFF OF MC SHAPLEY ESTIMATION
1. Analysis of the Convergence of Monte Carlo Estimation for Shapley Values For any task subset S. the VolDet utility satisfies 0 S v(S) det(Ms) < 1. Hence each marginal contribution in the permutation-Be based MC estimator, A.(S) vSS J {t}) v(S), is bounded in an interval of width B < 2. Together with uniform sampling over permutations, this yields an unbiased and bounded MC estimator of the Shapley values. In Appendix G.2 we derive the following Hoeffding-
44

---

Published as a conference paper at ICLR 2026
style concentration bound:
2K
Pr (r 01100 >E)
- 2N exp(
(18)
132
This incquality shows that the deviation 18
211 shrinks at rate ((1/v as the number of MC
samples K increases. We provide more detail in Appendix G.2
2. Empirical results validate the effectiveness and convergence of MC Shapley Value Ta-
ble [3 in Appendix G.T reports experiment empirically validate the effectiveness and vergence of MC Shapley. We collect gradient Gram matrices from Office-31 (N = 3) and from CelebA (N 40). For CelebA, we randomly sample task subsets of size Naub E {10, 15, 20) and extract the corresponding VolDet submatrices. For each matrix, we compute both exact Shap- ley values and their MC approximation using our permutation- K (100, 250, 500, 1000, 2000). We then measure the MSE between exact and MC Shapley val- ues and average over 10 matrices.
estimator, for MC samples
Table 13: Results Office-31 and CelebA. Mean squared error between exact and Monte Carlo Shapley values under the VolDet utility on Office-31 (N = 3) and CelebA subsets (N = 15, 20) for different numbers of MC samples K. Results are averaged over 10 gradient Gram matrices for each setting.
Sample Size Office-31(N = 3)
CelebA(N = 10)
CelebA(N = 15)
CelebA(N = 20)
K = 100
9.18 x 10-5
1.96 x 10-5
5.95 x 10
3.19 x 10-6
K = 250
1.05 x
2.13 x 10 5
3.07 x 10-6
1.48 x 10-6
K 500
5.84 10-5
1.38 x 10
2.49 x 10-6
1.77 x 10
K = 1000
0.671 x 10-5
1.28 x 10-5
2.26 x 10-6
1.33 x 10-6
K 2000
5.36 10-5
1.25 x 10-5
1.87 x 10-6
1.28 x 10-6
Office
CalehA
CalehA_15
0.00008
CalebA_20
MSE
0.00006
0,00000
0,00002
250
500
750
10000
1250
1500
17500
2000
Num sanpassssllessssse
Figure 7: Visualization of Table
13
make two observations from the Table 13
A: for all large-task settings with task number Nub > 10, the MSE rapidly decreases K grows and essentially plateaus once K 1000. This confirms that our choice of K 1000 the CelebA experiments is well justified: further increasing K to 2000 yields almost no reduction in error, while nearly doubling the Shapley-related computation.
B: for the small-task case (Office-31, N = 3), computing exact Shapley values is already cheap and MC is not needed in practice. The slight non-monee non-monotonic "rebound" at K = 2000 is a typical Monte Carlo fluctuation at the 10 level rather than systematic issue, and it has no impact because we never rely on MC in such small-N settings. Overall, these results directly support the effectiveness and convergence of the MC Shapley estimator.
method
45

---

Published as a conference paper at ICLR 2026
3. Empirical results validate the MC samples K can improve the final performance Table 14 in Appendix [.1 reports an experiment to empirically validate that increasing the MC samples K improve the final performance. We run full training experiments on Office-31 and CclebA with different MC samples K. The results are summarized the following table:
Table 14:
Results on Office-31 (3-task) and CelebA (40-task) datasets. End-to-end results of SVFair with different Monte Carlo sample sizes K on Office-31 and CelebA benchmarks.
Office-31
CelebA
Sample Size
Amazon t
DSLR f
Webcam 1
Am% +
Am% +
SVFair (K = 100)
85.30
99.18
98.87
-1.43
7.12
SVFair (K 250)
84.84
99.45
99.07
-1.42
3.83
SVFair (K = =500)
85.58
99,45
98.52
-1.51
2.12
SVFair (K =1000)
85.41
99.73
98.70
-1.60
0.11
SVFair (K = 2000)
85.64
99.73
98.33
-1.56
0.06
SVFair (Precise)
86.04
99.45
98.33
-1.62
We make observation from the Table 14
CelebA Increasing K from 100 1000 dramatically improves fairness, with Am% dropping from 7.12 to 0.11. Further increasing K to 2000 yields only a very small improvement (0.11 0.06), while almost doubling the Shapley-related computation. This shows that K 1000 already lies on a cost-effective performance plateau, consistent with the MSE results in point 2 (i.e. 2.Empirical results validate the effectiveness and convergence of MC Shapley value). The Office-31 result also exhibits a similar experimental phenomenon. Overall, these results directly support that MC samples K can improve the final performance.
In summary, our analysis and experiments jointly demonstrate that the permutation- MC Shap- ley estimator is theoretically well-founded, empirically accurate, and practically efficient.
G.2 MONTE CARLO ESTIMATION OF SHAPLEY VALUES FOR VOLDET
In this section show that the Monte Carlo estimator (Rozemberezki et al. 2022 Mitchell al. 2022) used in SVFair provides an unbiased estimate of the task-wise Shapley Values and enjoys standard Hoeffding- type concentration bound when the utility is defined by VolDet. The procedure of Monte Carlo sampling is summarized Algorithm 2
Setup: Let T {l,..., .) denote the task set and v(S) be the utility of any subset S S T. For
VolDet, v(S) defined as the determinant of the Gram matrix constructed from the fe-normalieed gradients of tasks in S, vSS) det(Mss. By construction Ms is positive semidefinite and has unit diagonal. The Shapley value of task i
Φi Σ SCN(()
[S - |S| 1)!
(c(SU (i)) v(S)
(19)
N!
For convenience in writing, we rewrite Eq 19 as the following Eq 20
Φ N! Σ TES
(
(20)
where SN the set of all permutations of and (T( denotes the set of tasks that appear before under the permutation æ.
Definition 12 (Boundedness of VolDet utility) For any subset S T, , the VolDer utility satisfies:
v(S) det(Ms) 1.
(21)
Proof: Since My is Gram matrix of (z-normalized vectors, is positive semidefinite with diago- nal entries (Ms) 1. By Hadamard's 's inequality, the determinant of positive semidefinite matrix upper bounded by the product of its diagonal entries, hence det(Ms) x [Ilss(Ms)s Pos- itive semidefiniteness also implies (sM 2 0.
46

---

Published as a conference paper at ICLR 2026
As direct corollary, for any task i and subset S S T {i), have:
r(S U (i)), (SS) E [0, 1] A, = = (i)) - E E|-1,1]
(22)
by averaging marginal contributions over K i.idd random permutations. Specifically, let T(1), 3.... T (K) be i.id. samples from the uni- form distribution over Sv, and define
Permutation-base MC estimator. We approximate Eq 20
( = -
t= I= K,
(23)
and the MC estimator
a. 1 x A
(24)
K
Proposition 3 (Unbiasedness) For each task the MC estimator Ti is an unbiased estimator of the Shapley value 6z i..,
E
di-
(25)
Proof: The permutations ( are i.i.d. and uniformly distributed over Sx. and ( is a measurable function of x(1) Therefore,
B
Σ 1=
K E
EA!
Uni(((((ss))
(0) -
(26)
K
By the permutation definition of the Shapley value in Eq 20 the right-hand side equals &i, which proves the claim.
we know that v(S) E [0, 1], hence each marginal contribution satisfies A ; E 1- in Eq 22 Let B denote the width of this bounded interval,
Concentration bound.
From Definition
(4)
B = max (sup A - inf 1 2
(27)
8,
Theorem (Hoeffding-style concentration for VolDet) Let T = (a.,... 2x) be the MC esti- mator and Φ = (#1...., .NN be the true Shapley values. Assume the marginal contributions are bounded with width B as above. Then for any E 0,
2K
Pr (1Ø-
8) s 2N exp
(28)
B2
In particular, when v(.) is VolDet, we can take B < 2.
A Proof: Fix a task i E T. The random variables A(1) (.,....A) (K) interval width at most B. By Hoeffding's ' inequality,
are independent and bounded in an
2K
Pr (10, Φ5l > = s 2exp
(29)
B
Applying the union bound over all i = 1,..., ......NN N yields
Pr (1ल 21 > 2))
Pr (1or - 0,1 > 1 2N exp
2K
(30)
==
B
which is exactly Eq 28
The bound in Theorem shows that the deviation | - Φ11 shrinks at rate O1/v/)) as the number of MC samples K increases, and provides a conservative lower bound on the number of samples required to achieve desired accuracy level.
47

---

Published as a conference paper at ICLR 2026
H DETAILS FOR CHOICE OF T. (DETAILS FOR SECTION 5.4)
H.1 THEORETICAL PERSPECTIVE TO GUIDE THE CHOICE OF T.
In SVFair, Φ; denote the Shapley value. The task weights are computed
exp(oi/T) (:( Expx((0/T))
(31)
It can be shown (a formal derivation will be given in the Appendix exactly the solution of the following antropy-regulartn
H.2 that this softmax mapping optimization problem:
w(T) arg max (we b) + + H+)) WEA
(32)
where A is the probability simplex and entropy (w) - E; wi logw,. plays the role of regularization coefficient: the linear term (w, Φ) encourages the weights to follow the Shapley values (i.e., the "ideal faimess compensation"), while the term r rH(w) encourages high-entropy weight distribution, preventing over-concentration on a few tasks and improving training stability.
From this formulation, we obtain two key monotonicity properties:
1: If < 72, then
(w(())) < (T)()))
max w(() 2 (72);
(33)
i.e. smaller 7 yields sharper, more concentrated weights, whereas larger makes the weights closer to uniform.
2: For any pair of tasks (i,j),
wi(T) Φi و5 (T)
log W/
(34)
8o, for a given gap Φ; Φj. t precisely controls how strongly this gap is amplified in the weight space.
Based on this, we propose theoretically motivated guideline for choosing т. short warm-up phase, we estimate a representative set of Shapley values {ø:} and define k(T) = T max; w;(r) as a measure the amplification factor the largest task weight. practice, we can pre-specify an acceptable upper bound concentration, an entropy lower bound Hmin- We then choose the smallest T that satisfies x(T) S Emax or (w)()) 2 Hmins so that we maximize the Shapley- driven fairness adjustment while maintaining sufficient stability.
Intuitively, when Shapley values exhibit large gaps and gradient conflicts are sharp, a too small leads to very large k(T) and the training is dominated by few tasks; in this case, relatively larger r is preferable to keep entropy at a reasonable level. When Shapley values are closer to each other and tasks are more balanced, one can safely use smaller magnify mild unfairness and improve the performance of disadvantaged tasks.
H.2 SOFTMAX AS THE OPTIMAL SOLUTION OF ENTROPY-REGULARIZZD LINEAR UTILITY.
Proposition 4 (Softmax as the optimal solution of reguoay-re linear utility) Let Φ (øI...., ΦT) E R be a given vector of Shapley values, and let
T
A =
w
E
R W
2
O.
Σ
wi =
be the probability simplex. For any temperature parameter O, consider the optimization problem
max
F(w)
where F(w) = (w, Φ) + (H),,
WEA
with Shannon entropy
T H(w)= = Σ
wi log
1=1
48

---

Published as a conference paper at ICLR 2026
Then this problem admits unique maximizer. and the optimal solution is exactly the softmax
(T( exp(0i/T)
1=
proof:
Concavity and uniqueness. Note that
T
H(w) = we logw;
strictly concave on the relative interior of A, and the linear term (w:, Φ) is concave as well. Hence
F(w) (w, o) (w)
strictly concave A. Since A is compact and convex, strictly concave function A has most one global maximizer, and the maximizer exists and is unique. We now derive its closed-form expression.
2. Lagrangian and KKT conditions. Consider the constrained optimization problem
max WERT
Σ
wi Tw; log w/
s.t.
1, Wa 2 0.
We form the Lagrangian (let À be the multiplier for the equality constraint; the non-negativity straints will be automatically satisfied at the interior optimum):
T
((w, A) = Σ
(irdr Tw, logws) 1(1
1=1
1=1
Taking the partial derivative with respect to each wi and setting to zero yields the first-order optimality condition:
OC
Φ; 7(1 + (ogw;; - d = i= 1....., ..
dw;
Equivalently,
Φi 7 logwis
so
Øi -
01
A X+T
logw; w;
exp
exp
Define
C exp
T
then we can write
wi = C exp
I...,,T.
Normalization and softmax form. Using the normalization constraint Σ wi 1, we obtain
T
1
Σ
exp
Φi (
i=1
1=1
which implies
C
Σ exp
49

---

Published as a conference paper at ICLR 2026
Substituting this back into the expression for wi, get exp(d1/T) w exp((s/7))
i= 1=
4. Non-negativin and feasibility. Since the exponential function is strictly positive, for any >> and any Ø, E we have wi (T) and E; (T) Hence (=( A. and by Step must be the unique global maximizer.
Therefore, the unique optimal solution of the eetropy-regularized problem coincides with the soft- max weights.
H.3 EXPERIMENTS ABOUT THE CHOICE OF
The Eq 7 in SVFair relies on temperature parameter T:
expl0./T)
Wi Σ5= exp(()/T)
We provide additional results of different discussed in S.. Table To evaluate the effect of 7 the mean rank and overall performance drop (Am%), we conduct small-scale hyperpa- rameter sweep with (0.5, 1.0, 2.0, 5.0} on the Cityscapes, Office-31, and CelebA benchmarks. The detailed results are Table 13 and Table 16 The result shows that different values of 7 have vary- ing effects on different benchmarks. can be observed that as т changes, the algorithm prioritizes certain tasks, leading better average rankings and performance improvements.
Table 15: Results Cityscapes (2-task) dataset. Each experiment is repeated times with different random seeds and the average is reported.
Segmentation
Depth
Method
MR 4
Am%
mloU
Pix Acc f
Abs Err 4
Rel Err 4
(VFair(VolDe)) (r =0.5)
73.75
93.07
0.0126
26.01
7.88
-1.27
SVFair(VolDet) (1=1.0)
74.50
92.90
0.0127
27.54
8.75
0.10
SVFair(VolDet) (2.2.0)
74.76
93.36
0,0125
27.24
6.00
-0.78
SVFair(VolDet) (r =5.0)
74.27
93.19
0.0121
26.37
7.00
-2.16
SVFair( (VolDetPro) (r (0:5)
74.22
93.11
0,0126
26.93
11.12
-0.61
SVFair(VolDettPigg (r =1.0)
73.90
93.11
0.0127
25.52
11.00
-1.57
SVFair VollDetPo) (r =2.0)
74.04
93.05
0.0125
26.32
11.12
-1.29
#VFair(VolDettrro (r =5.0)
73.95
92.99
0.0124
26.26
11.00
-1.49
Table Results on Office-31 (3-task) and CelebA (40-task) datasets. Each experiment repeated 3 times with different random seeds and the average is reported.
Office-31
CelebA
Method
Amazon
DSLR
Webcam
MR 4.
Am%
MR
Am% 4
VVlair(VoDDet)rr (r =0.5))
85.13
99.19
98.70
4.83
-1.30
7.50
1.20
SVFair(VolDet) (7=1.0)
85.70
98.91
98.89
3.67
-1.49
7.68
0.22
SVFair(VolDet) (7=2.0)
85.99
99.18
98.15
4.00
-1.44
7.00
-0.10
SVFair(VolDet) (r=5.0)
84.50
98.91
98.71
7.00
-0.97
7.82
0.23
SVFair(Volee1tro)) (r (0.5)
85.64
98.91
98.70
5.17
-1.41
12.62
6.34
SVFair(VolDe1PPg) (r =1.0)
84.96
99.18
98.89
5.00
-1.30
10.60
3.31
SVFair =2.0)
85.53
98.09
99.26
4.33
-1.27
8.52
1.10
SVFair(VoDeetProo (7 =5.0)
85.35
99.18
98.89
4.00
-1.45
8.85
0.27
50

---

Published as a conference paper at ICLR 2026
I EXPERIMENTS DETAILS FOR VOLDET VS VOLDETPRO (DETAILS FOR SECTION 5.3
Experiments setting. We SVFair with the different temperature values shown in the legend (T 5.0 for Office-31 and Cityscapes, 7 = 2.0 for NYU-V2, r = 1.0 for CelebA). At eight evenly spaced checkpoints from 33.3% 100% training progress, we compute Proportion of Negative Cosine Pairs (PNCS) from the current task gradients by normalizing per-task gradients {ō: J and normalized gradient matrix G forming all pairwise cosines, and reporting the fraction with cosine Curves show the mean PNCS over runs; the dashed line marks the 0.25 threshold used in analysis. Office-31 stays mostly above 0.25, whereas NYU-V2, CelebA and Cityscapes remain below.
0.32
NYU-V2(T-2.0)
CitSSapes
Proportion Nfgenvvve Cosine
Office3 (T-5.0)
0.28
CelchA (T-1.0)
0.24
0.20
0.16
33.3%
50.0%
56.7%
83.3%
100.0%
Training Progress (%)
Figure Proportion of Negative Cosine Pairs (PNCS) during training.
Metric mechanism. VolDetPro extends VolDet by adding penalty on the number and magnitude of negative- cosine pairs. The penalty activates only when obtuse relations appear, making the metric more responsive directional opposition and allowing SVFair to expose conflicts earlier and adjust Shapley weights.
When VolDetPro is better. When tasks exhibit pronounced negative correlations, the volume alone cannot distinguish configurations with the same volume but more obtuse angles. The penalty VolDetPro increases monotonically with both the count and the absolute sum of negative cosines, therehy increasing sensitivity to true conflicts. During weight allocation, SVFair moves weight away from opposing tasks, reduces uugof-waat and improves the weakest task. Hence VolDetPro is more likely to achieve lower MR and better Am% in such regimes.
When VolDet is better. When overall correlations are positive or conflicts are weak, sign-sensitive penalty may induce slight over-correction and prematurely suppress useful direc- tional diversity. In this case the volume-only VolDet is more neutral and tends to yield steadier aggregate gains.
Experiments Diagnostic. We compute at multiple training stages the PNCS statistic, see Fig. A1. Office-31, PNCS stays above 0.25 most of the later stages, indicating that oppositional rela- tions dominate. Using VolDetPro in this regime amplifies true conflict signals and yields better MR and Am%. On NYU-V2, CelebA and Cityscapes, PNCS remains below 0.25 throughout training, which suggests mainly positive weakly conflicting relations; the volume term already captures the global geometry well, and VolDet is more stable and often better. This observation is fully consistent with the tables in the main text and provides practical diagnostic for selecting the metric.
51

---

Published as a conference paper at ICLR 2026
J THE LINK BETWEEN SHAPLEY VALUE WEIGHTING AND MTL OPTIMIZATION
SVFair builds the same fairness-aware aggregation principle introduced in Eq. (o-fair utility maximization) by FairGrad (Ban & Jij 2024) and PIVRG (Qin et al.] 2025b), which generalizes network a-fairness gradient combination. a "utility," and the algorithm seeks the descent direction that maximizes global fairness subject to remaining a valid descent for all tasks.
that framework, each task's gradient is treated as
In SVFair, we replace the original heuristic weights in Eq. with Shapley values o; computed from our VolDet (i.e. Definition [ conflict metric, the determinant of the gradient Gram matrix. VolDet measures the multi-dimensionas volume spanned by normalized gradients, flagging both overt and subtle conflicts that pairwise cosines can miss. task whose gradient contributes most to inflating this volume (i.c., injects new conflicting directions) receives higher o;, whereas gradients that align well receive lower weight.
Beyond VolDet, we introduce sign-sensitive extension, VolDetPro (i.e. Definition 6 which pre- serves the volume term while explicitly accounting for negative cosine relations. When (S) = Ø, vSS) reduces to VolDet; when obtuse alignments exist, the second term grows with both the num- ber and the magnitude of negative cosines, correcting the sign-agnosticity of VolDet and yielding fairer attributions under-represented opposed tasks. We compute Shapley values Φ; v(.), so that tasks that newly introduce strong disagreement receive larger @; while already represented directions receive smaller ones.
Solving Eq. 7 with x exp(ó:/T) yields a Shapley-balanced update: under-represented or heavily opposed tasks pull toward themselves, while naturally aligned tasks have less influence. Under Assumption 1 (Appendix E.II, iterating this update converges to Pareto-stationary point that is feasible for all tasks and realizes more balanced compromise on the Pareto front, rather than collapsing to an extreme corner.
From a cooperative-gane gariiieeegaeeeeeeee perspective, Shapley allocations for convex games lie in the core, meaning no subset of players (tasks) can improve their outcome by deviating. Analogously, our Shapley- weighted gradient direction is stable: no group of tasks can find better joint descent direction that helps themselves without harming others. This direct parallel to Pareto efficiency and coalition stability, together with the sign-aware design of VolDetPro, provides a clear theoretical justification for why Ciapley-VVoDee/Voone
weighting leads more balanced and robust Pareto solutions, not merely as an empirical artifact but grounded in fair- division principles al. 2025b).
(Ban Ji. 2024 Qin
52

---

Published as a conference paper at ICLR 2026
K POSITIONING OF SVFAIR IN THE LITERATURE FROM GEOMETRIC AND GAME- TAME-THEORETIC PERSPECTIVES
This appendix further clarifies where SVFair lies in the landscape of existing MTL optimization methods. We focus on two complementary viewpoints the gradient geometry perspective and the cooperative game-theoretic perspective - and highlight the conceptual advances of our work beyond prior approaches.
K.1 GEOMETRIC PERSPECTIVE
From geometric viewpoint, mainstream MTL optimization methods typically designed around the relative configuration of task gradients in vector space. Early approaches such as the MGDA family search, at each step, for direction in the convex hull of task gradients that minimizes the norm or satisfies certain constraints, thereby obtaining an update direction lying on (or approximat- ing) the Pareto front (Sener & Koltun 2018 Zhang et al., 2025). PCGrad (Yu et al. 2020a) instead relies on pairwise cosine similarities between task gradients, projecting out components that are mu- tually conflicting, so as to mitigate local conflicts. Subsequent methods such as FAMO (Liu et al.] 2023) similarly build on pairwise angles, gradient-norm ratios, or simple geometric regularizers to measure and alleviate task interference.
A common feature of this line of work is that the geometric quantities are mostly restricted to pairwise relationships between tasks (e.g., cosine similarity or angle deviations). Algorithmically, the focus is locally modifying gradients the current iteration, without explicitly constructing set function v(S) defined over all task subsets S. As a consequence, these geometric signals are not elevated to global object that can be directly coupled with principled notion of "fair allocation".
On top of this, SVFair introduces VolDet and VolDetPro as coalition-level geometric conffict mea- sures. For any task subset S, VolDet interprets the determinant of the Gram matrix of task gradients in S as the squared oriented volume of the corresponding gradient parallelotope, yielding a utility function v(S) defined on all coalitions. This quantity jointly encodes the conflict structure among all tasks in S, rather than only pairwise information, and is invariant to task ordering and to orthogonal reparameterizations.
Furthermore, VolDetPro recognizes that the standard volumetric measure is sign-agnostic with re- spect the gradient angle: acute and obtuse angles with the same absolute cosine yield the same volume. To remedy this, VolDetPro augments VolDet with an additional penalty term that is only activated when negative cosine similarities are present, assigning larger conflict scores to coalitions whose gradients form strongly antagonistic (obtuse) angles, while exactly reducing to VolDet when all pairwise cosines are non-negative. In this way, VolDet/VolDetPro are not merely a collection of heuristic pairwise metrics, but a mathematically well-defined, coalition-level volumetric function that enjoys both geometric invariance and sign sensitivity, and that serves as a unified foundation for the subsequent gheeethe modeling.
K.2 COOPERATIVE GAME-THEORETE PERSPECTIVE
From the perspective of cooperative games and fairness-aware optimization, existing work has pri- marily focused on designing reasonable trade-offs in loss space. A canonical example is Nash-MTL (Navon et al. 2022). which formulates MTL Nash bargaining problem and derives task weights by solving for a Nash solution that balances task losses. Rafii et al. (Rafi et al., 2024) provide comprehensive survey of constas--onstrainee optimization in federated and multi-tas settings. In these approaches, the notion of fairness mainly applies to task losses or high-level utilities, and the weighting rules are determined by the specific bargaining model or constraints, with relatively indirect connections gradient geometry.
On the other hand, Shapley-related work typically treats data points or features as players and studies their contribution to single-task performance, as in Data Shapley (Ghorbani & Zou 2019) and subsequent gradient-basee based approximations. In such methods, the utility function (SS) is usually defined as the performance obtainable when training only on the subset S, and Shapley values used for offline data/feature valuation or interpretability, rather than for real-time weighting of task gradients during training.
53

---

Published as a conference paper at ICLR 2026
SVFair's key innovation the cooperative game-theoretic side is to construct task-level cooper- ative game directly in gradient space. Here, the players are the tasks, and the utility function v(S) is the "gradient volume conflict" of the subset S as measured by VolDet/VolDetPro. Based this, we apply Shapley values to allocate this utility in a way that is uniquely determined by the standard fairness axioms (efficiency, symmetry, dummy, additivity), resulting in task contributions { that are then mapped via a softmax into aggregation weights wi (7) at each step.
Unlike Nash-MTL, which negotiates at the level of task losses, SVFair directly imposes fairness on the responsibility for gradient conflict: "how much task contributes to conflict" and "how large an update step should receive" are coupled within the same formal framework. At the same time, SVFair is clearly distinguished from data Shapley methods that treat samples or features as players and are mainly used for offline valuation.
In summary, SVFair is not simple combination of existing geometric or game-theoretis methods. Instead, it upgrades gradient geometry into coalition-level set function uSS) via VolDet/VolDetPrr, and then uses Shapley's ss fair cost sharing to map this back to task-level weights. This unifies geo- metric conflict quantification and game-theoretic fair allocation within a single theoretical object, and thereby constitutes conceptual advance over prior work along both perspectives.
54


Reviews:
[{"review_id": "JznIKJd95F", "reviewer": "Reviewer_563x", "summary": "This paper proposes SVFair, a Shapley value-based framework for fair multi-task learning (MTL) that systematically quantifies and mitigates gradient conflicts among tasks. The authors introduce two utility functions, named as VolDet and VolDetPro, to compute Shapley values in a single training pass. These values are then used to guide gradient aggregation, promoting balanced optimization across tasks. Extensive experiments on supervised and reinforcement learning benchmarks demonstrate that SVFair achieves state-of-the-art performance and improves existing MTL methods when integrated as a fairness module.", "questions": "Please see the weakness.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "++ The paper integrates Shapley values into MTL optimization for quantifying gradient deviation and conflict at a subset level. The proposed VolDet and VolDetPro metrics offer ageometric perspective on gradient interactions.\n\n++ The method is well-motivated and theoretically grounded, with convergence guarantees to Pareto stationary points under reasonable assumptions.\n\n++ SVFair demonstrates strong empirical performance across diverse benchmarks (e.g., NYU-v2, CelebA, MT10) and can be easily integrated into existing MTL methods, enhancing their fairness and performance. The framework is scalable and supports Monte Carlo sampling for large-task settings.", "weaknesses": "-- Although the complexity is dominated by the training cost, the exact Shapley value computation can be prohibitive for very large N (number of tasks). While Monte Carlo sampling is proposed, its effectiveness and convergence properties are not thoroughly analyzed or empirically validated across different task scales.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Zc5fszTqHj", "reviewer": "Reviewer_Mspw", "summary": "This paper proposes two novel geometric gradient conflict measures, VolDet and VolDetPro, which quantify the squared volume of a subset of normalized task gradients. It then uses Shapley values to attribute each task's contribution to gradient conflict, and this information is incorporated into an existing fairness-based MTL optimization framework. Experiments on a suite of MTL benchmarks demonstrate the effectiveness of the proposed method.", "questions": "See the discussion in weaknesses.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The introduced VolDet and VolDetPro metrics are novel, and they can capture the geometric misalignment across task gradients.\n2. The Shapley values further leverage the VolDet/VolDetPro and lead to the overall per-task weights quantifying contribution to the misalignment. This angular-driven measure is also novel.\n3. Theoretical analysis and extensive empirical studies are provided, which can demonstrate the effectiveness of the proposed SVFair framework.", "weaknesses": "1. Line 240-241 states that higher $\\phi_i$ values indicate task $i$ is more misaligned with others and should be assigned a lower influence. It may be a typo. It seems that such tasks should actually be assigned higher influence, as mentioned in Eq. (7) and Line 2093–2094.\n2. The volume metric is built on the normalized task gradients. It ignores the magnitude information. Although the term $g_i^\\top d$ in Eq. (7) considers magnitude information, it may lead to a suboptimal solution. For example, consider two approximately aligned task gradients, but with very different magnitudes. They have similar Shapley value weights, and Eq. (7) reduces to $\\arg\\min \\sum_i \\frac{1}{g_i^\\top d}$, which corresponds to the minimum potential delay (MPD) fairness in FairGrad [1].  This MPD fairness criterion may not be suitable for this case, thus leading to suboptimal solutions. \n3. The calculation of the Shapley value introduces additional cost at each training step. Though Table 5 shows that the cost can be mitigated by Monte Carlo subset sampling, an ablation study is still needed to demonstrate the necessity of using Shapley value weights. The VolDet and Shapley value are two independent and separable concepts. It would be informative to compare them with other simpler weighting strategies, such as $\\phi_i^\\prime=\\sum_{i, j\\neq i} M_{ij}$, where a larger value indicates better alignment. You do not necessarily need to use this example; what I want to express is that it would be better to provide ablation studies to show that the Shapley value provides sufficient benefit compared to its additional cost.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5vAzINZEa5", "reviewer": "Reviewer_gTqY", "summary": "This paper addresses the common issue of gradient conflicts in multi-task learning, which can lead to unfair optimization among tasks. The authors propose SVFair, a Shapley value–based framework designed to quantify and mitigate such unfairness. Specifically, they introduce two geometric conflict metrics: (1) VolDet, a Gram-determinant-based volume metric that measures the geometric diversity of task gradients, and (2) VolDetPro, a sign-aware extension that distinguishes antagonistic gradients. By integrating these metrics into Shapley value computation, SVFair evaluates each task’s deviation from the aggregated gradient and rebalances the updates to improve fairness. Extensive experiments across both supervised and reinforcement learning benchmarks demonstrate that SVFair achieves state-of-the-art performance and can serve as a fairness-enhancing plug-in module for existing multi-task optimization methods.", "questions": "1. The computational cost of Shapley value estimation grows exponentially with the number of tasks, as also acknowledged by the authors. To address this, the paper employs Monte Carlo subset sampling to approximate the Shapley values. However, increasing the number of sampled subsets should, in principle, improve the estimation accuracy. Would this lead to better downstream performance (e.g., in terms of fairness or overall task accuracy)? I encourage the authors to include an ablation study on the number of sampled subsets to clarify the trade-off between computational cost and performance.\n\n2. Referring to Weakness 2, while the authors conducted ablation experiments on the temperature coefficient $\\tau$, they did not provide further theoretical insights or practical guidance on how to choose this parameter. Could the authors elaborate on how $\\tau$ influences the optimization dynamics, and whether any general guidance or heuristic can be derived from the observed trends?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper proposes a novel approach that combines Shapley-value-based fairness reasoning with gradient conflict measurement in multi-task learning. \n\n2. The method is well-motivated and theoretically solid. The analysis is detailed and convincing, providing clear justification for the proposed metrics and their role in achieving fair optimization.\n\n3. The experimental evaluation is extensive, covering both supervised and reinforcement learning settings. Results demonstrate strong performance improvements and fairness benefits over several competitive baselines.\n\n4. The authors have released code, which greatly improves the reproducibility and credibility of the work.", "weaknesses": "1. The core contribution of the proposed SVFair framework lies in computing Shapley values based on the newly defined VolDet and VolDetPro conflict metrics. However, once these weights are obtained, the subsequent problem formulation and solution procedure (Sec. 4.3) largely follow prior works such as FairGrad and PIVRG, and the corresponding convergence analysis also remains very similar. This overlap somewhat weakens the theoretical novelty of the paper.\n\n2. Although the authors conducted an ablation study on the temperature coefficient $\\tau$ in Table 4, they did not provide further theoretical analysis or practical guidance for tuning this parameter. In particular, the optimal $\\tau$ varies across tasks and does not show a consistent trend (for example, in the Cityscapes experiment, performance first drops and then improves as $\\tau$ increases). I believe the authors should include a more thorough investigation and discussion to clarify the role of $\\tau$ and how it should be selected.\n\n3. The overall presentation could be improved. For instance, the abstract is overly concise and does not sufficiently describe the problem background or the core motivation behind introducing Shapley-value-based fairness reasoning. A slightly more detailed introduction would help readers better understand the context and significance of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uT4gZVHdJo", "reviewer": "Reviewer_j3fR", "summary": "This paper introduces SVFair, a gradient-based multi-task learning (MTL) method that aims to balance training across tasks in a fairness-aware manner. The authors define two gradient conflict metrics: VolDet, the volume of the parallelotope formed by task gradients (computed via the Gram determinant), and VolDetPro, a signed extension that penalizes negative cosine relations between gradients. These metrics quantify the degree of conflict or synergy among tasks. Using these as importance weights, the paper constructs a single-pass Shapley value approximation that determines each task’s contribution to the joint update. The resulting algorithm adaptively reweights task gradients to seek a Pareto-stationary point near the center of the trade-off front, improving fairness across tasks. Experiments on standard MTL benchmarks show small but consistent improvements in balance metrics compared to simple averaging and prior multi-gradient methods.", "questions": "Can you theoretically connect VolDetPro minimization to convergence to a Pareto stationary solution?\n\nHow does your Shapley approximation differ in complexity or quality from prior Shapley-MTL approaches (e.g., Yoon et al. 2022)?\n\nDid you test the sensitivity of the method to task scaling or normalization (since determinant magnitudes can vary drastically)?\n\nWould your framework work for a large number of tasks (the Gram determinant computation scales poorly)?\n\nCan you clarify whether “fairness” here has any formal definition (e.g., equal loss, equal gradient norm) or is purely heuristic?\n\nPlease provide statistical significance or confidence intervals for performance differences.\n\nWould you consider integrating ParetoMTL’s convex combination layer with VolDet weighting to strengthen the theoretical link?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Geometric intuition. The gradient volume concept (Gram determinant of task gradients) provides a tangible geometric measure of task conflict. It’s an appealing visualization tool and a natural extension of cosine-similarity-based conflict measures.\n\nUnified perspective on fairness in MTL. By linking gradient aggregation to task-level “fairness,” the paper situates multi-task balance as a fairness problem, offering conceptual coherence with broader fairness research.\n\nImplementation simplicity. SVFair requires only first-order gradient information and scales linearly in the number of tasks. It’s easy to implement atop existing MTL optimizers and could serve as a practical baseline.\n\nReadable and self-contained. The paper is decently written, with useful algorithm boxes and schematic figures clarifying how VolDet and VolDetPro differ. It’s easy for readers familiar with gradient manipulation to reproduce.", "weaknesses": "Lack of originality and differentiation. The key ingredients (gradient alignment metrics and Shapley weighting) are not new. Prior works such as PCGrad (Yu et al., NeurIPS 2020), IMTL-G (Liu et al., 2021), GradVac (Wang et al., 2022), and ParetoMTL (Navon et al., 2022) already addressed gradient conflict and Pareto fairness using similar geometric or cooperative-game ideas. VolDet and VolDetPro are essentially new metrics, not fundamentally new algorithms. Suggestion: The paper should clearly articulate what these new metrics provide beyond previous gradient-conflict measures and why the Shapley approximation adds value.\n\nWeak theoretical grounding. The claim that minimizing VolDetPro “drives the solution toward a front-center Pareto point” is heuristic, no theorem or guarantee is offered. Similarly, the Shapley-value approximation is described qualitatively but without convergence or unbiasedness analysis. Suggestion: Include theoretical propositions linking VolDet/VolDetPro to Pareto-stationarity or fairness guarantees.\n\nEmpirical evaluation is thin. Experiments are small-scale and mostly incremental. Improvements over baselines are minor (often <2%), and no statistical tests are reported. There are few ablations on the role of VolDet vs. VolDetPro or on Shapley-weight computation overhead. Suggestion: Expand experiments to more diverse MTL setups (e.g., computer vision or NLP) and provide detailed comparisons to ParetoMTL, PCGrad, MGDA, and IMTL-G.\n\n“Fairness” interpretation feels overstated. While the method balances gradients, calling it “fairness-aware” is somewhat misleading; the fairness concept here refers only to task balance, not fairness in the social or demographic sense. Suggestion: Use more neutral terminology like “balanced multi-task optimization” to avoid confusion.\n\nUnclear benefit of signed volume (VolDetPro). The proposed “signed” determinant variant is underexplained—why should penalizing negative cosine relations via determinant sign improve convergence? Empirical evidence is insufficient. Suggestion: Provide analytical insight or controlled experiments illustrating when VolDetPro outperforms VolDet.\n\nRelation to prior Shapley gradient methods. Shapley-based gradient allocation (e.g., Ghorbani & Zou, 2019; Yoon et al., 2022) has already been applied to task weighting and feature attribution. The “single-pass approximation” presented here is not clearly compared to these prior formulations. Suggestion: Include a subsection detailing differences in computational complexity and approximation bias.\n\nPositioning in literature. The related work section underplays existing geometric and cooperative-game perspectives on MTL optimization. Without a clearer comparison, readers may perceive SVFair as an incremental tweak rather than a conceptual advance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)