SlowGuess/NeurIPS_2023_OCR huggingface dataset:
It is straightforward to see that 2) can be satisfied as long as  $x \neq 0$ . Therefore, we will focus on the realization of 1). This further requires two conditions: i)  $I$  is a PO w.r.t.  $E_{\emptyset}$  and  $E_1$ , meaning that the normal vectors are non-negative; ii)  $I$  is a PO w.r.t.  $E_2$ , meaning that the non-negative orthant at  $I$  does not intersect with  $E_2$ .
Some simple calculations yield that i) is equivalent to
$$
q _ {2 2} \sqrt {y} \geq q _ {2 3} \sqrt {z} + q _ {1 2} \sqrt {x} \quad \text {a n d} \quad q _ {3 3} \sqrt {z} \geq q _ {2 3} \sqrt {y} + q _ {1 3} \sqrt {x}. \tag {49}
$$
Setting  $x = 0$ , we have
$$
q _ {2 2} \sqrt {y} > q _ {2 3} \sqrt {z} \quad \text {a n d} \quad q _ {3 3} \sqrt {z} > q _ {2 3} \sqrt {y} \tag {50}
$$
by Equation (47). Therefore, it suffices to show the non-negative orthant at  $(0,q_{13}^{2}/s,q_{12}^{2}/s)^{\top}$ $(s = q_{22}q_{13}^{2} + q_{33}q_{12}^{2} - 2q_{12}q_{23}q_{31})$  does not intersect with  $E_2$ , and by continuity we can find some  $x_0 > 0$  such that i) and ii) hold simultaneously.
To prove the above claim, suppose  $a \geq 0, b \geq q_{13}^2 / s, c \geq q_{12^2} / s$ . We will show  $(a, b, c)^\top$  must lie above the surface defined by  $E_2$ , i.e.,
$$
f (a, b, c) = q _ {1 1} a + q _ {2 2} b + q _ {3 3} c - 2 q _ {1 2} \sqrt {a b} - 2 q _ {1 3} \sqrt {a c} + 2 q _ {2 3} \sqrt {b c} > 1. \tag {51}
$$
Let  $a^* = \frac{q_{12}\sqrt{b} + q_{13}\sqrt{c}}{q_{11}}$ . We have
$$
\begin{array}{l} f (a, b, c) \geq f (a ^ {*}, b, c) \\ = \left(q _ {2 2} - \frac {q _ {1 2} ^ {2}}{q _ {1 1}}\right) b + \left(q _ {3 3} - \frac {q _ {1 3} ^ {2}}{q _ {1 1}}\right) c + 2 \left(\frac {q _ {1 1} q _ {2 3} - q _ {1 2} q _ {1 3}}{q _ {1 1}}\right) \sqrt {b c} \\ \geq \frac {q _ {1 1} \left(q _ {2 2} q _ {1 3} ^ {2} + q _ {3 3} q _ {1 2} ^ {2} + 2 q _ {1 2} q _ {2 3} q _ {3 1}\right) - 4 q _ {1 2} ^ {2} q _ {1 3} ^ {2}}{s q _ {1 1}} \\ > \frac {q _ {1 1} \left(q _ {2 2} q _ {1 3} ^ {2} + q _ {3 3} q _ {1 2} ^ {2} - 2 q _ {1 2} q _ {2 3} q _ {3 1}\right)}{s q _ {1 1}} \\ = 1. \\ \end{array}
$$
As a consequence, by choosing a small  $x_0$ , we can guarantee both 1) and 2).  $I$  is therefore a PO that cannot be achieved via scalarization.
C2 is necessary. Suppose C2 is not true. We can write
$$
Q = \left[ \begin{array}{c c c} q _ {1 1} & - q _ {1 2} & - q _ {1 3} \\ - q _ {2 1} & q _ {2 2} & - q _ {2 3} \\ - q _ {3 1} & - q _ {3 2} & q _ {3 3} \end{array} \right], \tag {52}
$$
where  $q_{ij} > 0$  for all pairs of  $(i,j)$ . The inner boundary of  $\mathcal{F}_1$  is formed by  $E_1, E_2, E_3$ , and the coordinates of their intersection  $I$  is given by  $v = (q_{23}^2 / r, q_{13^2} / r, q_{12}^2 / r)^\top$ , where  $r = q_{11}q_{23}^2 + q_{22}q_{13}^2 + q_{33}q_{12}^2 + 2q_{12}q_{23}q_{31}$ . Our goal is to show the corresponding  $I'$  on  $\mathcal{F}_2$  (i.e.,  $t - v$ ) is a PO which cannot be achieved by scalarization.
To see why this is true, first note that the gradients at  $I$  w.r.t. the three surfaces disagree, since  $q_{ij} > 0$  for all pairs of  $(i,j)$ . Now, to demonstrate  $I'$  is a PO of  $\mathcal{F}_2$ , it suffices to show the non-positive orthant of  $I$  intersects with  $\mathcal{F}_1$  only at  $I$ . By symmetry, it suffices to show that  $I$  does not dominate any point on  $E_1$ .
To prove the above claim, suppose  $a \leq q_{23}^2 / r, b \leq q_{13}^2 / r, c \leq q_{12}^2 / r$ . We will show,  $J = (a, b, c)^\top$  must lie below the surface defined by  $E_1$  unless  $J$  equals  $I$ . In fact, let
$$
g (a, b, c) = q _ {1 1} a + q _ {2 2} b + q _ {3 3} c + 2 q _ {1 2} \sqrt {a b} + 2 q _ {1 3} \sqrt {a c} - 2 q _ {2 3} \sqrt {b c}. \tag {53}
$$
$$
\begin{array}{l} g (a, b, c) \leq g \left(q _ {2 3} ^ {2} / r, b, c\right) \\ \leq g \left(q _ {2 3} ^ {2} / r, q _ {1 3} ^ {2} / r, c\right) \\ \leq g \left(q _ {2 3} ^ {2} / r, q _ {1 3} ^ {2} / r, q _ {1 2} ^ {2} / r\right) \\ = 1, \\ \end{array}
$$
where the equalities hold iff  $I = J$ . This finishes the proof as desired.
B.5 Proof of Theorem 3.10
We first present several useful lemmas regarding non-negative matrices.
Lemma B.7. Let  $A$  be a non-negative, irreducible and real symmetric matrix, then there only exists one eigenvector  $v$  of  $A$  (up to scaling), such that  $v > 0$ .
Proof. Since  $A$  is non-negative and irreducible, by Perron-Frobenius theorem (Shaked-Monderer and Berman, 2021), the eigenvector associated with the largest eigenvalue  $\rho(A)$  (a.k.a. Perron root) can be made positive, and is unique up to scaling. We denote it as  $v$ .
We will show the eigenvectors associated with other eigenvalues cannot be positive. In fact, assume  $u > 0$  is an eigenvector associated with  $\lambda < \rho(A)$ . Since  $A$  is symmetric, we have
$$
\rho (A) v ^ {\top} u = v ^ {\top} A ^ {\top} u = v ^ {\top} A u = \lambda v ^ {\top} u. \tag {54}
$$
This is a contradiction since  $v^\top u > 0$  and  $\rho(A) > \lambda$ .
Lemma B.8. Let  $A$  be a non-negative and real symmetric matrix. Suppose  $v$  is an eigenvector of  $A$  such that  $v > 0$ , then  $v$  corresponds to the largest eigenvalue of  $A$ .
Proof. Consider the normal form (Varga, 1999) of  $A$ :
$$
P A P ^ {\top} = \left[ \begin{array}{c c c c} B _ {1 1} & B _ {1 2} & \dots & B _ {1 h} \\ 0 & B _ {2 2} & \dots & B _ {2 h} \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \dots & B _ {h h} \end{array} \right] \triangleq B, \tag {55}
$$
where  $P$  is a permutation matrix and  $B_{ii}$  ( $i \in [h]$ ) are irreducible. It is straightforward to see that  $B$  is non-negative. Furthermore, since a permutation matrix is an orthogonal matrix,  $B$  is also symmetric (implying that  $B_{ij} = 0$  for  $i \neq j$ ) and has the same spectrum as  $A$ .
Now suppose  $Av = \lambda v$  for some  $v > 0$ . Since  $B = PAP^{\top}$ , we have  $B\psi = \lambda \psi$  with  $\psi = Pv > 0$ . Denote
$$
\psi = \left( \begin{array}{c} \psi_ {1} \\ \psi_ {2} \\ \vdots \\ \psi_ {n} \end{array} \right), \tag {56}
$$
where each  $\psi_{i}$  has the same dimension as  $B_{ii}$ . Now we have
$$
B _ {i i} \psi_ {i} = \lambda \psi_ {i}, \quad \forall i \in [ h ]. \tag {57}
$$
Since  $B_{ii}$  is non-negative, irreducible and real symmetric, by Lemma B.7,  $\psi_i$  corresponds to the largest eigenvalue of  $B_{ii}$ . Note  $B$  is a block diagonal matrix, so the spectrum of  $B$  is essentially the union of the spectrum of  $B_{ii}$ . As a consequence,
$$
\lambda_ {\max } (A) = \lambda_ {\max } (B) = \max  _ {i \in [ h ]} \lambda_ {\max } \left(B _ {i i}\right) = \lambda . \tag {58}
$$
Proof of Theorem 3.10. We prove the sufficiency of C1 and C2 separately.
C1 is sufficient. We have demonstrated in Lemma 3.4 that the Pareto front belongs to a single surface  $E$ , whose gram matrix is doubly non-negative. Let  $p = (a_{1},\dots ,a_{k})^{\top}$  be a PO of  $E$ , and denote  $t = \sqrt{p}$ . We will show that the tangent plane at this point lies above the feasible region, implying that it can be achieved by scalarization.
Concretely, the tangent plane is given by
$$
\sum_ {i \in [ k ]} \frac {\sum_ {j \in [ k ]} q _ {i j} \sqrt {a _ {j}}}{\sqrt {a _ {i}}} v _ {i} = 1. \tag {59}
$$
To show it lies above the feasible region, it suffices to prove that it lies above every surface in  $\mathcal{E}$ , i.e.,
$$
\sum_ {i \in [ k ]} \frac {\sum_ {j \in [ k ]} q _ {i j} \sqrt {a _ {j}}}{\sqrt {a _ {i}}} v _ {i} \leq \sum_ {i \in [ k ]} q _ {i i} ^ {i _ {1} \dots i _ {l}} v _ {i} + 2 \sum_ {1 \leq i <   j \leq k} q _ {i j} ^ {i _ {1} \dots i _ {l}} \sqrt {v _ {i} v _ {j}}, \tag {60}
$$
where  $q_{ij}^{i_1\dots i_l}$  represents the  $(i,j)$ -th entry of  $Q_{i_1\dots i_l}$ . Note Equation (60) can be treated as a quadratic form w.r.t.  $\sqrt{v}$ , so it suffices to prove the corresponding matrix is PSD. Writing compactly, it suffices to show
$$
Q _ {i _ {1} \dots i _ {l}} - \operatorname {d i a g} \{Q t \otimes t \} = D _ {i _ {1} \dots i _ {l}} (Q - \operatorname {d i a g} \{Q t \otimes t \}) D _ {i _ {1} \dots i _ {l}} \tag {61}
$$
is PSD. As a consequence, we only need to show the positive semi-definiteness of the following matrix
$$
T := Q - \operatorname {d i a g} \{Q t \oslash t \}. \tag {62}
$$
Since  $p$  is a PO of  $E$ , we have  $Qt \geq 0$ . Let  $w = Qt$ , we have
$$
T = G ^ {- 1} - \operatorname {d i a g} \{w \oslash G w \}. \tag {63}
$$
Assume w.l.o.g. that  $w > 0$  (otherwise we simply remove the corresponding rows and columns in  $G^{-1}$ ). Denote
$$
R = \sqrt {\operatorname {d i a g} \{w \odot G w \}} \quad \text {a n d} \quad \phi = \sqrt {w \odot G w} > 0. \tag {64}
$$
$$
\begin{array}{l} R G R \phi = R G \sqrt {\operatorname {d i a g} \{w \oslash G w \}} \sqrt {w \odot G w} \\ = R G w \\ = \sqrt {\operatorname {d i a g} \{w \oslash G w \}} G w \\ = \sqrt {w \odot G w} \\ = \phi . \\ \end{array}
$$
By Lemma B.8, we have  $\rho(RGR) = 1$ . Since  $RGR$  is positive definite, we have
$$
I \succeq R G R \succ 0. \tag {65}
$$
This further implies that  $R^{-2} \succeq G$ , and
$$
T = G ^ {- 1} - R ^ {2} \succeq 0. \tag {66}
$$
C2 is sufficient. We have demonstrated in Lemma 3.5 that the Pareto front belongs to a single surface  $I_{\emptyset}$ . Here we will further show  $I_{\emptyset}$  is concave, and apply the supporting hyperplane theorem (Boyd et al., 2004, Section 2.5.2) to conclude the proof. In fact,  $E_{\emptyset}$  is parametrized by
$$
\sum_ {i \in [ k ]} q _ {i i} v _ {i} + 2 \sum_ {1 \leq i <   j \leq k} q _ {i j} \sqrt {v _ {i} v _ {j}} = 1. \tag {67}
$$
Since the first term is a summation of linear functions,  $f(x,y) = \sqrt{xy}$  is a concave function and that  $q_{ij} \geq 0$ , we conclude that  $E_{\emptyset}$  is convex, and  $I_{\emptyset}$  is concave. This finishes the proof as desired.
C A polynomial algorithm for checking C1
We explain how Algorithm 1 works.
Essentially, the goal is to find a set of flipping signs  $\{s_i\}_{i\in [k]}\in \{+, - \}$  that ensure positive correlation between all pairs in  $\{s_i\hat{y}_i\}_{i\in [k]}$  (except for those pairs that are not correlated). We state an observation which is critical for our algorithm—given a pair  $\hat{y}_i$  and  $\hat{y}_j$ , whether their signs are the same or opposite can be determined by whether  $\hat{y}_i$  and  $\hat{y}_j$  is positively or negatively correlated. Thus, given a determined  $s_i$ , the signs of its subsequent predictors  $s_j$ ,  $j > i$  can be uniquely determined by evaluating  $\langle \hat{y}_i,\hat{y}_j\rangle$ . This allows us to determine the signs for the predictors  $s_i$  in ascending order, and check for potential conflicts in the mean time.
We state one loop invariant in our algorithm—when entering iteration  $i$  in the outer loop,  $s_i$  has been determined by its preceding predictors and no conflict has occurred, or it has remained undetermined (i.e., None) all the way because it is not correlated with any preceding predictors. At this point, what is left to be done is to examine the subsequent predictors. If  $s_i$  is undetermined (lines 2-7), we attempt to use its subsequent predictors to determine it (line 5), and break out of the loop once it gets determined. Now that we make sure that  $s_i$  is determined after line 9, we can proceed to check the remaining subsequent predictors, to either determine their signs (lines 12-13) or check for potential conflicts (lines 14-15).
We comment that caution needs to be taken for the pairs that are not correlated (i.e.,  $\langle \hat{y}_i, \hat{y}_j \rangle = 0$ ). In this case, one cannot determine the sign of  $s_j$  by the relationship between these two predictors; basically, the sign has no influence on their correlation. Thus, the predictor  $\hat{y}_j$  retains the flexibility to be determined by other predictors other than  $\hat{y}_i$ . Additionally, it is possible that at the end of line 9  $s_j$  is still undetermined; this only occurs when  $\hat{y}_i$  is not correlated with any other optimal predictor. In this case,  $t$  will take the value of  $k + 1$  and the following loop (lines 10-15) will not be executed.
D Additional experimental details and results
D.1 Additional experimental details
Dataset. The SARCOS dataset (Vijayakumar and Schaal, 2000) consists of the configurations (position, velocity, acceleration) of the robotic arms, and the problem is to predict the torque of the
Algorithm 1: An  $O(k^2)$  algorithm of checking C1
Input: Optimal predictors  $\hat{y}_1,\hat{y}_2,\dots ,\hat{y}_k$    
Output: True if C1 is met, and False otherwise   
Initialize:  $s_1 = 1,s_2 = \dots = s_k =$  None Define:  $\operatorname {sgn}(x) = \left\{ \begin{array}{ll}1, & \text{if} x > 0\\ -1, & \text{if} x <   0\\ \text{None,} & \text{if} x = 0 \end{array} \right.$    
1 for  $i = 1$  to  $k - 1$  do Check whether  $s_i$  has been determined by its preceding optimal predictors  $\{\hat{y}_1,\ldots ,\hat{y}_{i - 1}\}$  ; if not, determine it using the subsequent ones.   
2 if  $s_i =$  None then for  $j = i + 1$  to  $k$  do if  $s_j\neq$  None and  $\operatorname {sgn}(\langle \hat{y}_i,\hat{y}_j\rangle)\neq$  None then break   
6 t←j+1   
7 else t←i+1   
9 △ Now that  $s_i$  has been determined, we proceed to check the remaining optimal predictors to either determine the sign for them or check for conflicts. for  $j = t$  to k do if  $\operatorname {sgn}(\langle \hat{y}_i,\hat{y}_j\rangle)\neq$  None then if  $s_j =$  None then sj←si·sgn(⟨yi,hatj） else if  $s_i\cdot s_j\cdot \mathrm{sgn}(\langle \hat{y}_i,\hat{y}_j\rangle) = -1$  then return False   
16 return True
respective arm given its triplet configuration. Since our study concentrates on the training procedure and does not concern generalization, we conduct the experiments on the training set alone, where the size of the training set of the SARCOS dataset (Vijayakumar and Schaal, 2000) is 40,000. Regression tasks normally benefit from standardization as a pro-processing step (Hastie et al., 2009), so we standardize the dataset to zero mean and unit variance.
Tasks. Following our theoretical analysis in Section 3, we study the scenario where  $q = 1$ . In this case, it is straightforward to see that the minimal  $k$  that may lead to a violation of C1 is 3. To show the violation of C1 leading to the failure of scalarization exploring the Pareto Front, we constrain our study to  $k \geq 3$ ; furthermore, for ease of visualization, we focus on  $k = 3$ . In our experiments, we take the arms 3, 4, and 5 for which do not satisfy C1.
Network. We use a two-layer linear network for the reason explained in Section 2. We merge the bias term  $b$  into the weight matrix  $W$  by adding an additional all-one column to the input  $X$ . The input dimension of the network is 22 (seven (position, velocity, acceleration) triplets plus the dummy feature induced by  $b$ ), the hidden size is  $q = 1$ , and the output size is 1 (i.e., the predicted torque). The same network architecture is used for both experiments in scalarization and that in SMTOs.
Implementation of linear scalarization. For linear scalarization, we uniformly sample 100,000 sets of convex coefficients from a three-dimensional simplex (i.e., a tetrahedron). Concretely, we first sample  $m_{1}, m_{2} \stackrel{i.i.d.}{\sim} U(0,1)$ , and then craft  $\lambda = (\lambda_{1}, \lambda_{2}, \lambda_{3}) = (\min(m_{1}, m_{2}), \max(m_{1}, m_{2}) - \min(m_{1}, m_{2}), 1 - \max(m_{1}, m_{2}) + \min(m_{1}, m_{2}))$  as the weights. For a given set of convex coefficients, we calculate the optimal value directly based on the analysis in Section 2 instead of performing actual training. More specifically, the optimum of the scalarization objective is given by  $\hat{Y}_{q,\Lambda}\Lambda := \sum_{i=1}^{q} \sigma_{i}u_{i}v_{i}^{\top}$ , from which we can compute the corresponding MSE for each task.
SMTOs. We consider two state-of-the-art SMTOs, MGDA (Désidéri, 2012) and MGDA-UB (Sener and Koltun, 2018). MGDA is an optimization algorithm specifically devised for handling multiple objectives concurrently. It utilizes a composite gradient direction to facilitate simultaneous progress across all objectives. MGDA-UB is an efficient variant of MGDA that focuses on maximizing the minimum improvement among all objectives. It is a gradient-based multi-objective optimization algorithm aimed at achieving balanced optimization outcomes across all objectives.
Implementation of SMTOs. Our code is based on the released implementation<sup>5</sup> of MGDA-UB, which also includes the code for MGDA. We apply their code on the SARCOS dataset. For both methods, we use vanilla gradient descent with a learning rate of 0.5 for 100 epochs, following the default choice in the released implementation. We comment that early stopping can also be adopted, i.e., terminate once the minimum norm of the convex hull of gradients is smaller than a threshold, for which we set to be  $10^{-3}$ .
D.2 Additional experiments on random initialization
To eliminate the influence of random initialization, we perform 300 trials for each algorithm using different random seeds. We filter out solutions whose maximum MSE is larger than 1 for clearer presentation, which leads to 198 solutions for MGDA and 240 solutions for MGDA-UB. We plot all these solutions in Figure 4. We see that MGDA and MGDA-UB are consistently capable of finding balanced solutions regardless of the random seed.
(a) MGDA with multiple initializations
(b) MGDA-UB with multiple initializations
Figure 4: More solutions of MGDA and MGDA-UB obtained by varying the initializations. Both algorithms tend to find solutions located at the interior of the Pareto front.
D.3 Additional experiments on randomization
To verify the effectiveness of randomization as discussed in Section 4.2, we plot the region achievable by randomized linear scalarization. Specifically, according to Equation (13), the region can be expressed as the collection of the convex combination of two points from the feasible region. To this end, we randomly sample 100,000 weight pairs (the same number as in Figure 3). For each weight pair  $(w_{1},w_{2})$ , we uniformly draw  $t\sim U(0,1)$  and get two corresponding optimal networks  $f_{1}$  and  $f_{2}$  by SVD. For each sample, with probability  $t$ , the model uses  $f_{1}$ , otherwise  $f_{2}$ , to calculate the MSE. The final result is demonstrated in Figure 5. It is straightforward to see that randomization allows scalarization to trace out the PF since there is no hole within the blue region, thus validating our theoretical analysis. We additionally comment that randomization convexifies the feasible region, as such, the solutions found by MGDA and MGDA-UB are dominated.
<sup>5</sup>https://github.com/is1-org/MultiObjectiveOptimization
Figure 5: The region achievable by randomized linear scalarization