SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 2: Special case of Algorithm 1 for  $k = 2$
Input: Bi-weighted matroid  $(E,r,w_1,w_2)$   
Output:  $S, \Lambda$   
1  $S, \Lambda' \gets \emptyset$ ;  
2  $\Lambda \gets \{0,1\}$ ;  
3 while  $\Lambda \setminus \Lambda' \neq \emptyset$  do  
4 for  $\lambda \in \Lambda \setminus \Lambda'$  do  
5  $\begin{array}{c}a,b\leftarrow$  bases minimizing  $(1 - \lambda)w_{1} + \lambda w_{2}$  prioritizing  $w_{1}$  and  $w_{2}$ , respectively;  
6  $S \gets S \cup \{a,b\}$ ;  
7  $\Lambda' \gets \Lambda' \cup \Lambda$ ;  
8  $\Lambda \gets \emptyset$ ;  
9  $\pi \gets$  element indexes of  $S$  in increasing  $w_{1}(\cdot)$  order;  
10 for  $i \in \{1,\dots,|S| - 1\}$  do  
11  $\delta_1, \delta_2 \gets w_1S_{\pi(i+1)} - w_1S_{\pi(i)}, w_2S_{\pi(i)} - w_2S_{\pi(i+1)}$ ;  
12  $\Lambda \gets \Lambda \cup \{\delta_1 / (\delta_1 + \delta_2)\}$ ;
4 Exact Computation of Extreme Points
In this section, we describe a deterministic framework that finds a solution for each extreme point, as well as a complete trade-off set. This framework, modified from the algorithm proposed in [10] for bi-objective MST, is outlined in Algorithm 1. It calls another algorithm (e.g. Greedy) to find MWB to scalarized weights, and iteratively computes new extreme points based on information from previous ones. Intuitively, each subset of extreme points  $Z$  is such that its convex hull is "inside"  $\operatorname{Conv}(F)$  and contains, for each extreme point  $y \notin Z$ , a facet separating  $Z$  from  $y$ . This means  $y$  can be discovered by minimizing the weight scalarized along the normal direction of this facet, essentially expanding  $\operatorname{Conv}(Z)$  to "touch"  $y$ . This iterative process begins with an optimum in each objective, and ends when all new normal vectors are duplicates of ones found previously, indicating that the current convex hull cannot be expanded further and equals  $\operatorname{Conv}(F)$ . The special case of this algorithm for  $k = 2$  is given in Algorithm 2 which treats trade-offs as scalars.
Algorithm 1, naively implemented, requires  $O(\# (\mathrm{poly}(k) + k!m\log m))$  operations and  $O(k!\# m)$  calls to the matroid rank oracle where  $\#$  is the number of extreme points. Each iteration in the main loop adds at least one extreme point, and redundant points are excluded from future iterations via  $\Lambda'$ . Here, we assume updating trade-off for each new vertex takes poly(  $k$  ) operations.
We remark that exhaustive tie-breaking over all objectives is done at line 6 to ensure that the computed points are the vertices of  $\mathrm{Conv}(F)$  instead of interior points of its faces, and that all extreme points are accounted for when the termination criterion is met. Furthermore, if the trade-off assigns zero value to some objectives, this also guarantees that the resulted solutions are non-dominated. This subroutine can be improved by grouping together objectives whose sorting permutations agree (in relative orderings) among elements tied by  $w^{(\lambda)}$ .
Algorithm 3: MOEA/D for MOMWB
Input: A MOMWB instance, trade-off set  $\Lambda$ , neighborhood size  $N \geq 1$   
Output: S  
1  $\forall \lambda \in \Lambda, B_{\lambda} \gets N$  nearest neighbors of  $\lambda$  in  $\Lambda$  (Euclidean distance);  
2  $\forall \lambda \in \Lambda, P_{\lambda} \gets$  a random sample from  $\{0,1\}^m$ ;  
3  $S \gets \emptyset$ ;  
4 while stopping conditions not met do  
5 for  $\lambda \in \Lambda$  do  
6  $x \gets$  uniformly sampled from  $P_{\lambda}$ ;  
7  $y \gets$  independent bit flips on  $x$  with probability  $1/m$ ;  
8  $D \gets \{l \in B_{\lambda}: \forall z \in P_l, f_l(y) < f_l(z)\}$ ;  
9  $T \gets \{l \in B_{\lambda}: \forall z \in P_l, f_l(y) = f_l(z)\}$ ;  
10  $\forall l \in D, P_l \gets \{y\}$ ;  
11  $\forall l \in T, P_l \gets P_l \cup \{y\}$  removing solutions with duplicate images;  
12  $S \gets$  non-dominated individuals in  $S \cup \{y\}$  under relation  $\leq_g$ ;
5 MOEA/D With Weight Scalarization
Multi-Objective Evolutionary Algorithm based on Decomposition (MOEA/D), introduced in [33], is a co-evolutionary framework characterized by simultaneous optimization of single-objective subproblems in a multi-objective problem. While there are many approaches to decompose the multi-objective into single-objectives, we consider the classical approach that is weight scalarization [15], as hinted in preceding sections. This simple scheme is sufficient in approximating  $F$  and even enumerating  $\mathrm{Conv}(F)$ .
5.1 Description
MOEA/D uses two fitness functions, a scalar function formulated by the decomposition scheme and a vector function for dominance checking [33]. To account for the matroid base constraint, we use the penalty term formulated in [26], which was adapted from prior work on MST [19]. Letting  $w_{max} \coloneqq \max_{(i,e) \in \{1,\dots,k\} \times E} w_{i,e}$ , we have the fitness  $f_{\lambda}$  of  $x \in \{0,1\}^{m}$  at trade-off  $\lambda$ , and the fitness vector  $g$  for dominance checking where  $\mathbb{1}$  is the one vector.
$$
f _ {\lambda} (x) := m (n - r (x)) w _ {m a x} + w ^ {(\lambda)} x, \quad g (x) := m (n - r (x)) w _ {m a x} \mathbb {1} + w x \tag {1}
$$
The MOEA/D for the MOMWB problem is outlined in Algorithm 3. The fitness functions defined in Eq. (1) and the input trade-off set realize the decomposition, and the algorithm evolves a population for each scalarized subproblem with potential neighborhood-based collaboration. During the search, it maintains a non-dominated solution set  $S$ , which does not influence the search and is returned as output. An optimum to each scalarized subproblem is a supported solution. Note in this formulation, MOEA/D keeps ties in each subproblem, allowing all found optima to participate in mutation operations. This is to avoid having to flip more than two bits to jump from a supported solution to an uncollected point in Conv(F). We will see that while this may increase the population size, it does not affect the run-time to reach optimality in each subproblem.
5.2 Expected Time To Minimize Scalarized Weights
In the following, we do not assume a particular value of  $N$ , and the results hold for any  $N \geq 1$ . Furthermore, we exclude the trivial instances with  $n = m$  and  $n = 0$ , which admit exactly one base each.
Lemma 4. MOEA/D working on trade-off set  $\Lambda$  finds a base's superset for each  $\lambda \in \Lambda$  within  $O(|\Lambda|m\log n)$  expected search points.
Let  $OPT_{\lambda}$  be the optimal value to  $(E, r, w^{(\lambda)})$ , we have the following drift argument proven in [26] for the standard bit mutation in the MWB problem.
Lemma 5 ([26], Proposition 9). Given a trade-off  $\lambda \in [0,1]$  and  $x \in \{0,1\}^m$ , if  $x$  supersets a base, then there are  $n$  2-bit flips and  $m - n$  1-bit flips on  $x$  reducing  $f_{\lambda}(x)$  on average by  $(f_{\lambda}(x) - OPT_{\lambda}) / m$ .
We use the same ideas as the proof of Theorem 2 in [26], while sharpening an argument to derive a slightly tighter bound.
Theorem 3. MOEA/D working on trade-off set  $\Lambda$  finds MWBs to instances scalarized by trade-offs in  $\Lambda$  in  $O\left(|\Lambda |m\log n + \sum_{\lambda \in \Lambda}m^{2}(\log (m - n) + \log w_{max} + \log d_{\lambda})\right)$  expected search points where  $d_{\lambda}\coloneqq \min \{a > 0:a\lambda ,a(1 - \lambda)\in \mathbb{N}\}$ .
In order for MOEA/D to reach a  $k$ -approximation and not lose it afterwards, it suffices that  $\Lambda$  is sufficient and each scalarized subproblem admits optima with a unique image. As mentioned and from Lemma 1, this can be obtained by sampling from the interiors of convex polytopes in  $H_{A}$ . For  $k = 2$ , this can be done by taking a complete scalar trade-off set  $A$  (e.g. as returned by Algorithm 2) and include  $(a + b) / 2$  (which is an interior point) for each non-empty interval  $(a, b)$  bounded by consecutive elements in  $A \cup \{0, 1\}$ . Under the integer weights assumption, this method gives rational trade-offs whose integral denominators are  $O(w_{max}^{2})$ , so we have the following bound from Corollary 3.
Corollary 4. For a bi-objective instance, MOEA/D working on a minimal sufficient trade-off set finds a sufficient solution set within  $O(m^2 \sqrt{\min\{n, m - n\}} (m(\log(m - n) + 3 \log w_{max}) + \log n))$  expected number of search points.
This method can be generalized to higher dimensions. Instead of taking an average of two consecutive elements, we can take the average of the vectors normal to  $k$  facets of  $\operatorname{Conv}(F)$  that meet at an extreme point, as this ensures the resulted trade-off  $\lambda$  is not normal<sup>5</sup> to any  $h$ -faces of  $\operatorname{Conv}(F)$  for all  $h \in [1,k]$ , and that said extreme point minimizes  $w^{(\lambda)}$ . Since each facet is determinable by  $k$  points with integral coordinates, each such (1-norm unit) vector admits rational coordinates with denominator at most  $kw_{max}$ . Therefore, the trade-offs derived by this method admit rational representations whose denominators are  $O(k^{k+1}w_{max}^k)$ , giving the run-time upper bound from Corollary 1 under the assumption that  $k$  is sufficiently small.
Corollary 5. Given a  $k$ -objective instance where  $k \in o(m)$  and  $k \in o(w_{max})$ , MOEA/D working on a minimal sufficient trade-off set guarantees  $k$ -approximation within  $O(m^{2k-1}(m(\log(m-n) + (k+1)\log w_{max}) + \log n))$  expected number of search points.
As a side note, since MOEA/D uses standard bit mutation, we can replace  $w_{max}$  with  $m^m$  and remove  $\log d_{\lambda}$  from the bound in Theorem 3 to arrive at weight-free asymptotic bounds [27].
5.3 Expected Time To Enumerate Conv(F)
We see from Corollary 2 that MOEA/D with a complete trade-off set can collect all points in  $\mathrm{Conv}(F)$  with 2-bit flips starting from an optimum to each subproblem. As mentioned, this is afforded by allowing all found optima to undergo mutation.
Theorem 4. Assuming distinct supported solutions have distinct images under  $w$ , MOEA/D working on a minimal complete trade-off set  $\Lambda$ , and starting from an optimum for each  $\lambda \in \Lambda$  enumerates  $C := \operatorname{Conv}(F)$  in  $O(|\Lambda||C|^2m^2)$  expected number of search points.
With this, Theorem 3 and Corollary 3 and 1, we have the following expected run-time bounds under the distinct image assumption. Note this assumption can be removed by having MOEA/D keep duplicate images at line 11.
Corollary 6. For a bi-objective instance, MOEA/D working on a minimal complete trade-off set enumerates  $C \coloneqq \operatorname{Conv}(F)$  in expected time  $O(m^2\sqrt{\min\{n,m - n\}} (m(\log (m - n) + 3\log w_{max} + |C|^2) + \log n))$ .
Corollary 7. Given a  $k$ -objective instance where  $k \in o(m)$  and  $k \in o(w_{max})$ , MOEA/D working on a minimal complete trade-off set enumerates  $C \coloneqq \operatorname{Conv}(F)$  in expected time  $O(m^{2k-1}(m(\log(m - n) + (k+1)\log w_{max} + |C|^2) + \log n))$ .
If  $\lambda$  is normal to a face of  $\mathrm{Conv}(F)$ , then all points in this face minimize  $w^{(\lambda)}$ . A convex combination of vectors normal to some faces is normal to their (assumed non-empty) intersection, the intersection of  $h$  adjacent  $d$ -faces is a  $(d - h + 1)$ -face, and an extreme point or vertex is a 0-face.
6 Experimental Investigation
In this section, we perform computational runs of MOEA/D on various bi-objective minimum spanning tree instances. Spanning trees in a connected graph  $G = (V,E)$  are bases of the graphic matroid defined by  $G$  admitting the ground set  $E$ . The rank of such a matroid (i.e. the size of the spanning tree) equals  $|V| - 1$  and its rank function is defined with  $r(x) = |V| - cc(x)$  where  $cc(x)$  is the number of connected components in  $(V,x)$ . In notations, we have  $n = |V| - 1$  and  $m = |E|$ . We use simple undirected graphs in our experiments, and the edge-set representation of solutions in the implementations of the algorithms [25].
6.1 Setting and Performance Metrics
We uniformly sample graphs with  $|V| \in \{26, 51, 101\}$  and  $|E| \in \{150, 300\}$ . In this procedure, edges are added randomly into an empty graph up to the desired edge count, and this is repeated until a connected graph is obtained. Each edge weight is an integer sampled independently from  $\mathcal{U}(1, 100)$ . We generate two weighted graphs with each setting, making 12 instances in total.
With this experiment, we aim to measure the algorithms' performances in finding solutions mapped to all extreme points, we denote this set of target points with  $R$ . We compare MOEA/D against GSEMO, previously studied for its performance in bi-objective MST [18]. For GSEMO, we use the fitness function  $g$  defined in Eq. (1). Since GSEMO checks for dominance in each iteration across its entire population, we set  $N := |\Lambda|$  for MOEA/D to match. Here, the input  $\Lambda$  to MOEA/D is derived from a complete trade-off set output by Algorithm 2 in the manner described in Section 5.2. The points given by Algorithm 1 are the target points  $R$ , and each run is terminated if all target points are hit. Additionally, each run is terminated after at most  $\lceil 3|R|m^2\log(m-n)\rceil$  evaluations (each evaluation is a call to function  $g$ ). Each algorithm is run on each instance 10 times. Their performances are measured with the followings ( $S$  is the final population returned by the algorithm):
- Success rate: The number of runs where all target points are hit within the evaluation budget.
- Cover rate: The proportion of hit target points after termination,  $|R \cap \{wx : x \in S\}| / |R|$ . A run is successful if this reaches 100%.
- Modified inverted generational distance (IGD+): The distance between the output and the target points [16],  $\sum_{y\in R}\min_{x\in S}\sqrt{\sum_{i = 1}^{k}(\max\{(wx)_i - y_i,0\})^2} / |R|$ . A run is successful if this reaches 0.
- T: The number of evaluations until all target points are hit.
We remark that the fitnesses  $f_{\lambda}$  can be quickly computed alongside  $g$ , incurring minimal overheads. In fact, the run-time bottleneck is in checking the number of connected components.
The results are shown in Table 1, with IGD+ and cover rate from MOEA/D omitted due to them being 0 and  $100\%$  across all instances, respectively. These are contextualized by the listed instance-specific parameters. Of note is the number of target points  $|R|$  which is smaller than the upper bound in Theorem 2 in all instances.
Immediately, we see the GSEMO failed to hit all target points within the evaluation budget in most runs, while MOEA/D succeeded in every run. In most cases, GSEMO hit at most one target point. Inspecting the output points and IGD+ reveals that its population converged well to the non-dominated front, yet somehow misses most extreme points. In contrast, MOEA/D hit all target points within up to  $92\%$  of the evaluation budget, though there are significant relative variances in the run-times.
Inspecting the run-times of MOEA/D in relation to the evaluation budgets, we see that the means of ratios remain fairly stable across instances. This suggests the asymptotic bound in Theorem 3 is not overly pessimistic. Instances 5 and 6 are particularly interesting as they are ostensibly the easiest due to the small number of extreme points, yet MOEA/D seems to require the most fractions of the budget. Given that these instances exhibit the smallest  $m - n$ , this can be explained by the interference of lower-order terms in the asymptotic bound, which are not counted in the budgets.
Table 1: Means and standard deviations of performance statistics from GSEMO and MOEA/D on bi-objective MST instances. Means and standard deviations of  $T$  are computed over successful runs only. All differences are statistically significant.
Id	m	n	|R|	max eval.	GSEMO	MOEA/D
Suc. rate	Cover rate	IGD+	T/max eval.	Suc. rate	T/max eval.
1	150	25	39	12710536	2/10	2.5±0.06%	0.13±0.11	79±11%	10/10	50±15%
2	150	25	31	10103247	0/10	3.1±0.14%	0.25±0.32	N/A	10/10	43±11%
3	150	50	45	13988205	0/10	1.9±0.1%	1.7±0.7	N/A	10/10	52±11%
4	150	50	45	13988205	0/10	2±0.083%	0.92±0.23	N/A	10/10	47±12%
5	150	100	35	9242155	0/10	2.5±0.13%	2.6±1.2	N/A	10/10	63±15%
6	150	100	36	9506216	0/10	2.2±0.21%	3.7±2.3	N/A	10/10	58±11%
7	300	25	45	68243769	3/10	2.2±0.031%	0.06±0.072	86±13%	10/10	41±11%
8	300	25	49	74309882	1/10	2±0.031%	0.076±0.083	94±0%	10/10	40±8.1%
9	300	50	66	98392434	0/10	1.4±0.048%	0.49±0.13	N/A	10/10	50±11%
10	300	50	63	93920051	0/10	1.5±0.045%	0.8±0.24	N/A	10/10	60±15%
11	300	100	79	113013110	0/10	1.1±0.043%	2±0.53	N/A	10/10	60±8.3%
12	300	100	80	114443656	0/10	1.1±0.072%	3.1±1	N/A	10/10	58±11%
In this study, we contribute to the theoretical analyses of evolutionary multi-objective optimization in the context of non-trivial combinatorial problems. We give the first run-time analysis of the MOEA/D algorithm for a broad problem class that is multi-objective minimum weight base problem. In particular, we show a fixed-parameter polynomial expected run-time for approximating the nondominated front, simultaneously extending existing pseudo-polynomial bounds for GSEMO to arbitrary number of objectives and broader combinatorial structures. Our experiments in random bi-objective minimum spanning tree instances indicate that MOEA/D significantly outperforms GSEMO in the computing extreme points under an appropriate decomposition. Along the way, we prove properties that give further insight into the problem of interest.
This work was supported by Australian Research Council grants DP190103894 and FT200100536, and by National Science Foundation grant 2144080.
[1] C. Bazgan, S. Ruzika, C. Thielen, and D. Vanderpooten, “The power of the weighted sum scalarization for approximating multiobjective optimization problems,” Theory of Computing Systems, vol. 66, no. 1, pp. 395–415, Nov. 2021.
[2] C. Bian, C. Qian, and K. Tang, "A general approach to running time analysis of multi-objective evolutionary algorithms," in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization, Jul. 2018, pp. 1405-1411.
[3] R. A. Brualdi, “Comments on bases in dependence structures,” Bulletin of the Australian Mathematical Society, vol. 1, no. 2, pp. 161–167, Aug. 1969.
[4] S. Cerf, B. Doerr, B. Hebras, Y. Kahane, and S. Wietheger, “The first proven performance guarantees for the non-dominated sorting genetic algorithm II (NSGA-II) on a combinatorial optimization problem,” in Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization, Aug. 2023.
[5] A. V. Do and F. Neumann, “Maximizing submodular or monotone functions under partition matroid constraints by multi-objective evolutionary algorithms,” in Parallel Problem Solving from Nature – PPSN XVI. Springer International Publishing, 2020, pp. 588–603.
[25] G. R. Raidl, "An efficient evolutionary algorithm for the degree-constrained minimum spanning tree problem," in Proceedings of the 2000 Congress on Evolutionary Computation. CEC 2000, vol. 1. IEEE, 2000, pp. 104-111.
[26] J. Reichel and M. Skutella, "Evolutionary algorithms and matroid optimization problems," Algorithmica, vol. 57, no. 1, pp. 187-206, Dec. 2008.
[27] ——, “On the size of weights in randomized search heuristics,” in Foundations of Genetic Algorithms, 10th ACM SIGEVO International Workshop, FOGA 2009, Proceedings. ACM, Jan. 2009, pp. 21–28.
[28] V. Roostapour, J. Bossek, and F. Neumann, "Runtime analysis of evolutionary algorithms with biased mutation for the multi-objective minimum spanning tree problem," in Proceedings of the 2020 Genetic and Evolutionary Computation Conference. New York, NY, USA: ACM, Jun. 2020, p. 551-559.
[29] A. Trivedi, D. Srinivasan, K. Sanyal, and A. Ghosh, "A survey of multiobjective evolutionary algorithms based on decomposition," IEEE Transactions on Evolutionary Computation, vol. 21, no. 3, pp. 440-462, 2016.
[30] Z. Wang, Y. Pei, and J. Li, "A survey on search strategy of evolutionary multi-objective optimization algorithms," Applied Sciences, vol. 13, no. 7, p. 4643, Apr. 2023.
[31] Q. Xu, Z. Xu, and T. Ma, "A survey of multiobjective evolutionary algorithms based on decomposition: Variants, challenges and future directions," IEEE Access, vol. 8, pp. 41588-41614, 2020.
[32] T. Zaslavsky, “Facing up to arrangements: face-count formulas for partitions of space by hyperplanes,” Memoirs of the American Mathematical Society, vol. 1, no. 154, pp. 0–0, 1975.
[33] Q. Zhang and H. Li, "MOEA/D: A multiobjective evolutionary algorithm based on decomposition," IEEE Transactions on Evolutionary Computation, vol. 11, no. 6, pp. 712-731, Dec. 2007.
[34] W. Zheng, Y. Liu, and B. Doerr, “A first mathematical runtime analysis of the non-dominated sorting genetic algorithm II (NSGA-II),” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 9, pp. 10408-10416, Jun. 2022.
[35] A. Zhou, B.-Y. Qu, H. Li, S.-Z. Zhao, P. N. Suganthan, and Q. Zhang, "Multiobjective evolutionary algorithms: A survey of the state of the art," Swarm and Evolutionary Computation, vol. 1, no. 1, pp. 32-49, Mar. 2011.
Proof of Theorem 1. Let  $f \coloneqq (f_i)_{i=1}^k$  be the objective function vector,  $z$  be any solution,  $Z = \{i : f_i(z) = 0\}$ , if  $|Z| = k$  then  $z$  is an extreme solution, so  $P$  1-approximates  $z$ . Assume otherwise, we define  $\lambda \in (0,1]^k$  where  $\lambda_i \coloneqq \epsilon / \delta_i$  if  $i \in Z$  and  $\lambda_i \coloneqq \epsilon / [(k - |Z|) f_i(z)]$  otherwise for some sufficiently small  $\epsilon > 0$ . By definition of sufficient solution set and Observation 1, there is  $x \in P$  minimizing  $\lambda^\top f$ , i.e.,  $\lambda^\top f(x) \leq \lambda^\top f(z) = \epsilon$ . If  $f_i(x) > 0$  for some  $i \in Z$  or  $f_i(x) > (k - |Z|) f_i(z)$  for some  $i \notin Z$ , then since  $f(x) \in \mathbb{R}_{\geq 0}^k$ , we have  $\lambda^\top f(x) > \epsilon$ , a contradiction. Therefore,  $x$ , and by extension  $P$ ,  $(k - |Z|)$ -approximates  $z$ . Since  $z$  can assume positive values in all objectives<sup>6</sup>, this factor simplifies to  $k$ .
We show tightness by construction. Let  $\epsilon \in (0,k)$ ,  $m \coloneqq k^2$ ,  $\theta_i \coloneqq \sum_{j=0}^{k-1} e_{ik-j}$  for  $i = 1, \ldots, k$  where  $e_j$  is the  $j$ th unit vector in  $\mathbb{R}^m$ , we define a non-negative  $k$ -objective instance over  $\{0,1\}^m$ :  $\min_x \{f(x) \coloneqq (\theta_i^\top x - \epsilon \prod_{j=0}^{k-1} x_{ik-j})_i^k : |x| \geq k\}$ . We see that the set of all supported solutions is precisely  $S \coloneqq \{\theta_i\}_{i=1}^k$ . Let  $z \coloneqq \sum_{i=0}^{k-1} e_{ik+1}$  be a solution, for all  $i = 1, \ldots, k$ ,  $f_i(\theta_i) = k - \epsilon \geq (k - \epsilon)f_i(z)$  (equality holds if  $k > 1$ ). This means  $S$  fails to  $(k - \epsilon - \varepsilon)$ -approximate  $z$  for any  $\varepsilon > 0$  and  $\epsilon$  can be arbitrarily small. Since  $S$  is a complete solution set, the claim follows.
Proof of Lemma 1. Let  $c$  be any point in  $\operatorname{Int}(Q)$ , by definition of  $A$ ,  $w_i^{(c)} = w_j^{(c)}$  iff  $\delta_{i,j} = 0$ , and  $w^{(c)}$  admits multiple minima iff they contain different elements among those sharing weights in  $w^{(c)}$ , while sharing all other elements. Indeed, let  $x$  and  $y$  be a pair of minima violating this condition, they must contain different sets of weights so for all bijection  $\gamma$  between  $x \setminus y$  and  $y \setminus x$ , there is  $u \in x \setminus y$  where  $w_u^{(c)} \neq w_{\gamma(u)}^{(c)}$ ; this leads to a contradiction when combined with the base exchange property. This means these optima share image under  $w$ , and bases not having the same image do not minimize  $w^{(c)}$ .