SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Definition 2.1 (Stochastic Convex Optimization in  $\ell_{\infty}$ ). In the stochastic convex optimization in  $\ell_{\infty}$  problem we have a (stochastic) subgradient oracle  $g: \mathbb{R}^n \to \mathbb{R}^n$  such that  $\mathbb{E}g(x) \in \partial f(x)$  and  $\mathbb{E}\|g(x)\|_2^2 \leq \sigma^2$  for a convex function  $f: \mathbb{R}^n \to \mathbb{R}$  that is  $L$ -Lipschitz in  $\ell_{\infty}$ . Given the guarantee that  $f$  has a minimizer  $x_* \in \mathbb{R}^n$  with  $\|x_*\|_2 \leq R$  our goal is to compute an  $\varepsilon$ -approximate minimizer of  $f$ , i.e.,  $x \in \mathbb{R}^n$  with  $f(x) \leq f(x_*) + \varepsilon$ .
Note that in Definition 2.1,  $\ell_{\infty}$  appears only to determine the norm in which  $f$  is Lipschitz. However, the bound on  $x_{*}$  in  $\ell_{2}$  that can be easily converted to one in terms of  $\ell_{\infty}$  by using that  $\| x_{*}\|_{2} \leq \sqrt{n}\| x_{*}\|_{\infty}$ . Furthermore, a convex function  $f: \mathbb{R}^{n} \to \mathbb{R}$  is  $L$ -Lipschitz in  $\ell_{\infty}$  if and only if  $\| g\|_{1} \leq L$  for all  $g \in \partial (x)$  for  $x \in \mathbb{R}^{n}$ . Since  $\| g\|_{2} \leq \| g\|_{1}$  we see that this stochastic convex optimization problem subsumes the (non-stochastic) problem of computing an  $\varepsilon$ -approximate minimizer to a convex function that is  $L$ -Lipschitz in  $\ell_{\infty}$  given a (deterministic) subgradient oracle. We define this more general problem as, interestingly, our algorithm tolerates this weaker stochastic oracle without any loss (as we discussed).
Our main result regarding stochastic convex optimization in  $\ell_{\infty}$  is given in the following theorem.
Theorem 2.2 (Stochastic Convex Optimization in  $\ell_{\infty}$ ). There is an algorithm that solves the stochastic convex optimization problem (Definition 2.1) in  $\ell_{\infty}$  (Definition 2.1) in  $\widetilde{O}((LR / \varepsilon)^{2/3})$  rounds and  $\widetilde{O}((\sigma R / \varepsilon)^2)$  queries.
Due to the aforementioned connection between  $\ell_{\infty}$ -Lipschitz continuity and bounds on the subgradient, and the fact that  $\| x_{*}\|_{\infty} \leq 1$  implies  $\| x_{*}\|_{2} \leq \sqrt{n}$ , Theorem 2.2 immediately yields a  $\tilde{O}(n^{1/3}\varepsilon^{-2/3})$ -round,  $\tilde{O}(n\varepsilon^{-2})$ -query algorithm for the problem of minimizing a convex function that is 1-Lipschitz in  $\ell_{\infty}$  and minimized at a point with  $\ell_{\infty}$ -norm at most 1. As discussed in the introduction, the parallel complexity of this algorithm is near-optimal for constant  $\varepsilon$  [Nem94].
In Section 2.1, we show how to use Theorem 2.2 to obtain our results for SFM. We then present the ingredients in the proof of Theorem 2.2 (which we defer to Appendix A) in Section 2.2. As part of our reduction from SFM to Stochastic Convex Optimization in  $\ell_{\infty}$  in Theorem 2.2, we provide a general tool for reducing constrained to unconstrained minimization (Lemma 2.3); we use this lemma to facilitate our results in both sections.
2.1 From Unconstrained Convex Optimization in  $\ell_{\infty}$  to SFM
Here we show how to use Theorem 1.2 to prove the following theorem regarding SFM.
A key ingredient of our proof is the following general, simple technical tool which allows one to reduce constrained Lipschitz optimization over a ball in any norm to unconstrained minimization with only a very mild increase in parameters.
Lemma 2.3. Let  $f: \mathbb{R}^n \to \mathbb{R}$  be convex and  $L$ -Lipschitz with respect to norm  $\| \cdot \|: \mathbb{R}^n \to \mathbb{R}$ . For any  $c, x \in \mathbb{R}^n$  and  $r \in \mathbb{R}$  let
$$
f _ {\text {r e g}} ^ {c, r} (x) := f (x) + 2 L \cdot \max  \{0, \| x - c \| - r \}. \tag {1}
$$
Then  $f_{\mathrm{reg}}^{c,r}(x)$  is convex and 3L-Lipschitz with respect to  $\| \cdot \|$ . Additionally, for any  $y \in \mathbb{R}^n$  for which  $\| y - c \| \geq r$ , define  $y^{c,r} := c + \frac{r}{\|y - c\|} (y - c)$ . Then,
$$
\left\| y ^ {c, r} - c \right\| = r \text {a n d} f _ {\operatorname {r e g}} ^ {c, r} \left(y ^ {c, r}\right) = f \left(y ^ {c, r}\right) \leq f _ {\operatorname {r e g}} ^ {c, r} (y) - L \left(\left\| y - c \right\| - r\right). \tag {2}
$$
Consequently,  $f_{\mathrm{reg}}^{c,r}(x)$  has an unconstrained minimizer  $x_{*}^{c,r}$  and all such minimizers satisfy
$$
\left\| x _ {*} ^ {c, r} - c \right\| \leq r \text {a n d} f _ {\operatorname {r e g}} ^ {c, r} \left(x _ {*} ^ {c, r}\right) = f \left(x _ {*} ^ {c, r}\right) = \min  _ {x \in \mathbb {R} ^ {n} | \| x - c \| \leq r} f (x). \tag {3}
$$
Lemma 2.3 implies that minimizing  $f$  subject to a distance constraint  $\| x - c \| \leq r$  reduces to unconstrained minimization of  $f_{\mathrm{reg}}^{c,r}$ . More formally, to compute an  $\epsilon$ -optimal solution to the constrained minimization problem,  $\min_{x \in \mathbb{R}^n: \| x - c \| \leq r} f(x)$ , it suffices to instead compute an  $\varepsilon$ -optimal solution,  $x_{\varepsilon}$ , to the unconstrained minimization problem  $\min_x f_{\mathrm{reg}}^{c,r}(x)$ , and then output that point  $x_{\varepsilon}$  if  $\| x_{\varepsilon} - c \| \leq r$  and output  $x_{\varepsilon}^{r,c}$ -otherwise. From (2), we get  $f(x_{\varepsilon}^{r,c}) \leq f_{\mathrm{reg}}^{c,r}(x_{\varepsilon}) \leq \mathrm{opt} + \varepsilon$ , where  $\mathrm{opt} := \min_x f_{\mathrm{reg}}^{c,r}(x) = \min_{x \in \mathbb{R}^n | \| x - c \| \leq r} f(x)$  and the last equality follows from (3).
We remark that the  $2L$  in the definition of  $f_{\mathrm{reg}}^{c,r}$  can be changed to  $L + \delta$  for any  $\delta \geq 0$  with the only effect of turning the  $L$  in (2) to  $\delta$  and causing (3) to only hold for some minimizer (rather than all) if  $\delta = 0$ . The proof of Lemma 2.3 is deferred to Appendix A.
Next, we obtain Theorem 1.3 by applying Theorem 2.2 to the Lovász extension of the submodular function  $f$  extended to an unconstrained minimization problem by Lemma 2.3.
Given a submodular function  $f$  defined over subsets of an  $n$  element universe, the Lovász extension  $f_{\mathrm{Lov}}: \mathbb{R}^n \to \mathbb{R}$  is defined as follows:  $f_{\mathrm{Lov}}(x) := \sum_{i \in [n]} x_{\pi_x(i)} (f(S_{\pi_x,i}) - f(S_{\pi_x,i-1}))$ , where  $\pi_x : [n] \to [n]$  is the permutation such that  $x_{\pi_x(1)} \geq x_{\pi_x(2)} \geq \dots \geq x_{\pi_x(n)}$  (ties broken in an arbitrary but consistent manner), and  $S_{\pi_x,j}$  is the subset  $\{\pi_x(1), \ldots, \pi_x(j)\}$ .
Next we give standard properties of the Lovász extension and use them to prove Theorem 1.3.
Fact 2.4 (e.g., [GLS88, JB11]). The following are true about the Lovász extension  $f_{\mathrm{Lov}}$ :
1.  $f_{\mathsf{Lov}}$  is convex with  $\min_{x \in [0,1]^n} f_{\mathsf{Lov}}(x) = \min_{S \subseteq V} f(S)$ . Indeed, given any  $x \in [0,1]^n$ , in  $n$  queries one can find a subset  $S$  with  $f(S) \leq f_{\mathsf{Lov}}(x)$ .
2. Given any  $x \in \mathbb{R}^n$  and corresponding permutation  $\pi_x$ , the vector  $g \in \mathbb{R}^n$  where  $g(x)_{(\pi(i))} := f(S_{\pi_x, i}) - f(S_{\pi_x, i-1})$  is a subgradient of  $f_{\mathrm{Lov}}$  at  $x$ . Furthermore,  $g(x)$  can be computed in 1 round of  $n$  queries to an evaluation oracle for  $f$ .
3. If  $f$  has range in  $[-M, + M]$ , then the  $\ell_1$ -norm of the subgradient is bounded, in particular,  $\| g(x)\| _1\leq 3M$ . Equivalently,  $f_{\mathrm{Lov}}$  is  $3M$ -Lipschitz with respect to the  $\ell_{\infty}$ -norm.
As discussed in Section 1.1, to prove Theorem 1.3, it suffices to prove the following more general result regarding approximately minimizing a submodular function.
Theorem 2.5 ( $\epsilon$ -approximate minimizer for SFM). There is an algorithm that, when given an evaluation oracle for submodular  $f: 2^{[n]} \to \mathbb{R}$  with minimizer  $x_*$ ,  $f(\emptyset) = 0$  and  $|f(S)| \leq M, \forall S \subseteq [n]$ , finds a set  $S$  with  $f(S) \leq f(x_*) + \epsilon M$  in  $\widetilde{O}(n^{1/3} / \epsilon^{2/3})$  rounds and a total of  $\widetilde{O}(n^2 / \epsilon^2)$  queries.
Proof of Theorem 2.5. By Fact 2.4, SFM reduces to minimizing  $f_{\mathrm{Lov}}$  over  $x \in [0,1]^n$  which is the same set  $\{x \in \mathbb{R}^n : \| x - c \|_\infty \leq 0.5\}$  where  $c$  is the  $n$ -dimensional vector with all entries 0.5. By Lemma 2.3, we can do so by applying Theorem 2.2 to the regularized version  $f_{\mathrm{reg}}^{c,0.5}$  of  $f_{\mathrm{Lov}}$  with respect to the  $\ell_\infty$ -norm. This regularized function is guaranteed to have a minimizer  $x_*$  with  $\| x_* \|_2 \leq \sqrt{n}$  which also minimizes  $f_{\mathrm{Lov}}$  in  $[0,1]^n$ . The subgradient of this regularized function at any point can be computed from the subgradient of  $f_{\mathrm{Lov}}$  at the same point which takes  $n$  evaluation oracle queries to the submodular function. Hence, we obtain an  $\varepsilon M$ -approximate minimizer in  $\widetilde{O}(n^{1/3} / \varepsilon^{2/3})$  rounds and with a total of  $\widetilde{O}(n^2 / \varepsilon^2)$  queries to the evaluation oracle of  $f$ .
Now, we are ready to prove Theorem 1.3.
Proof of Theorem 1.3. Set  $\epsilon = \frac{1}{2M}$  and apply Theorem 2.5 to obtain, in  $\widetilde{O}(n^{1/3}M^{2/3})$  rounds and with a total of  $\widetilde{O}(n^2M^2)$  queries to the evaluation oracle of  $f$ , a  $x \in [0,1]^n$  with  $f_{\mathrm{Lov}}(x) \leq \min_{z \in [0,1]^n} f_{\mathrm{Lov}}(z) + \frac{1}{2}$ . Then by property 1 in Fact 2.4, one can get a subset  $A \subseteq V$  with  $f(A) \leq \min_{S \subseteq V} f(S) + \frac{1}{2}$ . As  $f$  is assumed to be integer valued,  $A$  must be the minimizer of the submodular function.
2.2 Parallel Stochastic Convex Optimization in  $\ell_{\infty}$
Here we present the key steps in proving Theorem 2.2 regarding our new parallel results for the stochastic convex optimization problem in  $\ell_{\infty}$  (Definition 2.1). Throughout this subsection, in our exposition, lemma statements, and proofs we assume that we are in the setting of Definition 2.1.
To prove Theorem 2.2 we apply the approach of  $\left[\mathrm{CJJ}^{+}23\right]$  with two modifications. First, we consider the convolution of  $f$  with a centered Gaussian density function with covariance  $\rho^2\mathbf{I}_n$ . However, we show that in our setting, it is possible to use a larger value of  $\rho$  without perturbing the function value too much, due to the  $\ell_{\infty}$  geometry. Unfortunately, the minimizer of the convolved function may move outside the box of radius  $R$ . Thus, the second modification we make is working with a regularized function,  $f_{\mathrm{reg}}$ , which is pointwise close to  $f$ , still  $L$ -Lipschitz in the  $\ell_{\infty}$  norm, and keeps the minimizer in the ball or radius  $R$  even after applying the convolution with a Gaussian.
In the remainder of this subsection we first present the ingredients going into the proof of Theorem 2.2, and then give a brief explanation for how they fit into the proof of Theorem 2.2, deferring the complete proof to the appendix. We start with our bound on function perturbation after adding a Gaussian with covariance  $\rho^2\mathbf{I}_n$  to an  $\ell_{\infty}$  Lipschitz function (Lemma 2.6). We then introduce the concept of a ball optimization oracle, along with a result on how to implement it in low depth for the special case of a function that is the result of Gaussian convolution (Proposition 2.8). Lastly, we present the result which allows us to use a ball optimization oracle black-box to obtain the desired depth (Proposition 2.9).
Now, we are ready to present the lemma which allows us to obtain a better dependence of depth on the dimension  $n$ , compared to the naive  $n^{2/3}$  obtained by directly applying the  $\ell_2$ -Lipschitz optimization result. We start with the definition of Gaussian convolution.
Gaussian Convolution. Let  $\gamma_{\rho} \coloneqq (2\pi \rho)^{-n/2} \exp \left(-\frac{1}{2\rho^2}\|x\|_2^2\right)$  be the probability density function of  $\mathcal{N}(0,\rho^2\mathbf{I}_n)$ . Given a function  $f: \mathbb{R}^n \to \mathbb{R}$ , we define its convolution with a Gaussian of covariance  $\rho^2\mathbf{I}_n$  by  $\widehat{f}_{\rho} \coloneqq f*\gamma_{\rho}$ , i.e.
$$
\widehat {f} _ {\rho} (x) := \mathbb {E} _ {y \sim \mathcal {N} \left(0, \rho^ {2} \mathbf {I} _ {n}\right)} [ f (x + y) ] = \int_ {\mathbb {R} ^ {n}} f (x - y) \gamma_ {\rho} (y) d y. \tag {4}
$$
Next, we present a lemma which allows us to obtain a better dependence on the dimension  $n$  in depth, compared to the naive  $n^{2/3}$  obtained by directly applying the  $\ell_2$ -Lipschitz optimization result. This lemma shows that we can perform more Gaussian smoothing (as compared to the  $\ell_2$ -setting) without perturbing the function too much (as mentioned in Section 1.1).
Lemma 2.6 (Gaussian Convolution Distortion Bound for  $\ell_{\infty}$ ). Let  $f:\mathbb{R}^n\to \mathbb{R}$  be L-Lipschitz with respect to the  $\ell_{\infty}$ -norm. Then for any point  $x\in \mathbb{R}^n$ , we have  $|\widehat{f}_{\rho}(x) - f(x)|\leq L\rho \cdot \sqrt{2\log n}$ .
Proof. Note that  $|\widehat{f}_{\rho}(x) - f(x)| \leq \int_{\mathbb{R}^n} |f(z) - f(x)| \gamma_{\rho}(x - z) \, \mathrm{d}z \leq L \int_{\mathbb{R}^n} \| x - z \|_{\infty} \gamma_{\rho}(x - z) \, \mathrm{d}z$  where the first inequality follows from the definition of  $\widehat{f}_{\rho}$  and the second follows as  $f$  is  $L$ -Lipschitz in  $\ell_{\infty}$ -norm. The RHS is simply the expected  $\ell_{\infty}$ -norm of a zero-mean random Gaussian vector with covariance  $\rho^2 \mathbf{I}_n$ , and this is  $\Theta(\rho \sqrt{\log n})$  (e.g., [Ver18]). More precisely, we get
$$
| \widehat {f} _ {\rho} (x) - f (x) | \leq L \cdot \mathbb {E} _ {y \sim \mathcal {N} (0, \rho^ {2} \mathbf {I} _ {n})} \| y \| _ {\infty} \leq L \rho \cdot \sqrt {2 \log n}.
$$
As mentioned in Section 1.1, by contrast, convolving a function  $f$  that is  $L$ -Lipschitz in  $\ell_2$  with a Gaussian of covariance  $\rho^2\mathbf{I}_n$  could change the function value by  $O(\rho \sqrt{n})$ . Hence, the  $\ell_{\infty}$  geometry allows us to add a larger amount of Gaussian smoothing without changing the function value by more than  $\varepsilon$ , which in turn allows for better rates.
Ball Optimization. A subroutine that we use for minimizing  $\widehat{f}_{\rho}$  is called a ball optimization oracle. As suggested by [CJJ+20], the concept of ball optimization oracle is related to the notion of trust regions, explored in several papers, such as [CGT00]. The particular ball optimization procedure we employ takes a function  $F: \mathbb{R}^n \to \mathbb{R}$  and a point  $\bar{x} \in \mathbb{R}^n$ , which is an approximate solution to  $F$  in a small ball of  $\bar{x}$ . More formally, we work with the following definition:
Definition 2.7 (Ball Optimization Oracle [CJJ+23]). Let  $F: \mathbb{R}^n \to \mathbb{R}$  be a convex function.  $\mathcal{O}_{\mathrm{bo}}$  is an  $(\phi, \lambda, r)$ -ball optimization oracle for  $F$  if given any  $\bar{x} \in \mathbb{R}^n$ , it returns an  $x \in \mathbb{R}^n$  with the property
$$
\mathbb {E} \left[ F (x) + \frac {\lambda}{2} \cdot \| x - \bar {x} \| _ {2} ^ {2} \right] \leq F (x _ {\mathrm {l o c}} ^ {\star}) + \frac {\lambda}{2} \| x _ {\mathrm {l o c}} ^ {\star} - \bar {x} \| _ {2} ^ {2} + \phi ,
$$
where  $x_{\mathrm{loc}}^{\star} = \arg \min_{x\in B_{\bar{x}}(r)}(F(x) + \frac{\lambda}{2}\| x - \bar{x}\| _2^2)$ .
From  $\left[\mathrm{CJJ}^{+}20\right]$  it is known is that for any Lipschitz convex function  $f$ , any stochastic subgradient oracle  $g$  as above, and any  $\rho$ , if we set  $r = \rho$ , then efficient ball-optimization oracles exist. More formally, we use the following proposition from  $\left[\mathrm{CJJ}^{+}23\right]$  which is in turn inspired from  $\left[\mathrm{ACJ}^{+}21\right]$ .
Proposition 2.8 (Proposition 3, [CJJ+23]). Let  $f: \mathbb{R}^n \to \mathbb{R}$  be convex and  $g: \mathbb{R}^n \to \mathbb{R}^n$  be a stochastic subgradient oracle satisfying  $\mathbb{E}[g(x)] \in \partial f(x)$  and  $\mathbb{E}\|g(x)\|_2^2 \leq \sigma^2$  for all  $x \in \mathbb{R}^n$ . Let  $\widehat{f}_{\rho} := f * \gamma_{\rho}$ , i.e.,
$$
\widehat {f} _ {\rho} (x) := \mathbb {E} _ {y \sim \mathcal {N} \left(0, \rho^ {2} \mathbf {I} _ {n}\right)} [ f (x - y) ] = \int_ {\mathbb {R} ^ {n}} f (x - y) \gamma_ {\rho} (y) \mathrm {d} y. \tag {5}
$$
Then there is a  $(\phi, \lambda, \rho)$ -ball optimization oracle for  $\widehat{f}_{\rho}$  which makes  $O\left(\frac{\sigma^2}{\phi\lambda}\right)$  total queries to  $g$  in a constant number of rounds.
Highly Parallel Optimization. As shown in  $\left[\mathrm{CJJ}^{+}23\right]$ , the ball optimization oracle above can be used for highly-parallel optimization as follows.
Proposition 2.9 (Proposition 2 in [CJJ+23]). Fix a function  $F: \mathbb{R}^n \to \mathbb{R}$  which is  $L$ -Lipschitz with respect to the  $\ell_2$ -norm and convex. Suppose  $R \geq 0$  is a parameter such that  $x^{\star} \in \arg \min_{x} F(x)$  satisfies  $\| x^{\star}\|_{2} \leq R$ . Let  $r \in (0, R]$  and  $\epsilon_{\mathrm{opt}} \in (0, LR]$  be two parameters. Define the following quantities
$$
\kappa := \frac {L R}{\epsilon_ {\mathrm {o p t}}}, \quad K := \left(\frac {R}{r}\right) ^ {\frac {2}{3}}, \quad a n d \quad \lambda_ {*} := \frac {\epsilon_ {\mathrm {o p t}} K ^ {2}}{R ^ {2}} \log^ {2} \kappa . \tag {6}
$$
Then, there exists a universal constant  $C > 0$  and an algorithm BallAccel which runs in  $CK\log \kappa$  iterations and produces a point  $x$  such that  $\mathbb{E}F(x)\leq F(x^{\star}) + \epsilon_{\mathrm{opt}}$ . Moreover,
1. Each iteration makes at most  $C \log^2\left(\frac{R\kappa}{r}\right)$  calls to  $\left(\frac{\lambda r^2}{C}, \lambda, r\right)$ -ball optimization oracle with values of  $\lambda \in [\frac{\lambda_*}{C}, \frac{CL}{\epsilon_{\mathrm{opt}}}]$ .
2. For each  $j \in [\lceil \log_2 K + C \rceil]$ , at most  $C^2 \cdot 2^{-j} K \log \left(\frac{R\kappa}{r}\right)$  iterations query a  $\left(\frac{\lambda r^2}{C^{2j}} \cdot \log^{-2}\left(\frac{R\kappa}{r}\right), \lambda, r\right)$ -ball optimization oracle for some  $\lambda \in \left[\frac{\lambda_*}{C}, \frac{CL}{\epsilon_{\mathrm{opt}}}\right]$ .
While the proof of Theorem 2.2 is deferred to Appendix A, we provide some intuition for how to use the stated components to obtain the result. A natural approach would be to apply Proposition 2.9 to  $F \coloneqq \widehat{f}_{\rho}$ . However, to ensure that the minimizer of  $F$  has  $\ell_2$ -norm at most  $R$ , we will instead work with  $F \coloneqq \widehat{f}_{\mathrm{reg}_{\rho}}^{c,R}$  as defined in Lemma 2.3. The idea is to invoke Lemma 2.3 and apply BallAccel on the function  $F \coloneqq \widehat{f}_{\mathrm{reg}_{\rho}}^{c,R}$  with respect to  $\| \cdot \|_2$ ,  $c$  being the origin, and  $\rho \coloneqq \frac{\epsilon_{\mathrm{opt}}}{L\sqrt{2}\log n}$ . With this choice of  $\rho$ , by Lemma 2.6, we know  $|F(x) - f_{\mathrm{reg}}^{c,R}(x)| \leq \epsilon_{\mathrm{opt}}$  everywhere. Noting that  $F$  has a minimizer  $x_*$  with  $\| x_* \|_2 \leq 3R$ , this enables us to apply Proposition 2.9 to  $F$ , obtaining the stated bounds on the number of rounds and query complexity.
3 2-Round  $O(n^{M + 1})$ -Query Algorithm for SFM
Here we present our 2-round,  $O(n^{M + 1})$ -query algorithm for SFM. The algorithm AugmentingSets, given in Algorithm 1, iterates over every  $M$ -sparse  $S \subseteq [n]$  (i.e.  $|S| \leq M$ ) (denoted  $\mathcal{F}$ ). For every such  $S$  the algorithm then builds the augmented set  $A(S)$ , consisted of the union of  $S$  and all elements  $i$  that have non-positive marginal with respect to  $S$ , i.e.,  $f(S \cup \{i\}) \leq f(S)$ . The algorithm then outputs the set  $A(S)$  that has the smallest value.
As we show below, computing all the  $A(S)$  for  $M$ -sparse sets  $S$  can be done in 1 round and  $O(n^{M + 1})$ -queries, and then computing an element of  $A(S)$  with the smallest value can be done in another round and  $O(n^{M + 1})$ -queries. The correctness of the algorithm is guaranteed by the fact that  $A(S)$ , for some  $|S| \leq M$ , is the maximal minimizer of  $f$ , and therefore the algorithm outputs a set with the optimum value<sup>6</sup>.
Our main result of this section is the following theorem.
Note, however, that the algorithm doesn't necessarily output the maximal minimizer itself.
Algorithm 1: Augmenting Sets Algorithm
Input: Submodular function  $f:2^{[n]}\to \mathbb{Z}$  and  $M\in \mathbb{Z}_{>0}$  such that  $|f(S)|\leq M$  for all  $S\subseteq [n]$  and  $f(\emptyset) = 0$    
Output:  $S_{\mathrm{out}}$  , a minimizer of function  $f$    
1 Function AugmentingSets  $(f,M)$  ..   
2  $\mathcal{F}\gets \{S\subset [n]\mid |S|\leq M\}$    
3 for  $S\in \mathcal{F}$  do   
4  $\begin{array}{rlr}{|A(S)}{\leftarrow}S\cup \{i\notin S\mid f(S\cup \{i\})\leq f(S)\} & \text{/} & \text{Compute the augmentation of } S \end{array}$    
5 end   
6 return  $S_{\mathrm{out}}\in \operatorname *{argmin}_{S\in \mathcal{F}}f(A(S))$
Theorem 1.4 (Two-round Parallel SFM). There is an algorithm (Algorithm 1) that when given an evaluation oracle for submodular  $f: 2^{[n]} \to \mathbb{Z}$  with  $f(\emptyset) = 0$  and  $|f(S)| \leq M$  for all  $S \subseteq [n]$  finds a minimizer of  $f$  in 2 rounds and  $O(n^{M + 1})$  queries.
Proof. First, we bound the parallel and query complexity of the algorithm. Line 2 to Line 5 can be implemented in 1 round as they simply evaluate  $f$  on all subsets of  $[n]$  of size  $\leq M + 1$ . Line 6 can be implemented in 1 round by evaluating  $f(A(S))$  for each  $S \in \mathcal{F}$  in parallel. Consequently, the algorithm is implementable in 2 rounds. To bound the query complexity, note that  $|\mathcal{F}| \leq \sum_{k=0}^{M} \binom{n}{k} = O(n^M)$  and the algorithm only makes  $O(n)$  queries for each  $S \in \mathcal{F}$  ( $O(n)$  in Line 4 and 1 in Line 6). Consequently, the algorithm makes  $O(n^{M+1})$  total queries.
It only remains to show that the algorithm outputs a minimizer of  $f$ . We prove this by showing that for some  $S \in \mathcal{F}$ , its augmented set  $A(S)$  is the maximal minimizer of  $f$ , i.e., the union of all minimizers, which is a minimizer itself by submodularity. This suffices as the algorithm outputs the  $A(S)$  for  $S \in \mathcal{F}$  of smallest value. Let  $S_*$  be the maximal minimizer of  $f$ . We build a subset  $T \subseteq S_*$  of size  $|T| \leq M$ , which we call anchor, as follows. Start with  $T = \emptyset$  and an arbitrary ordering of elements of  $S_*$ . For each element  $i \in S_*$  in this order, we add it to the current  $T$  if  $f(T \cup \{i\}) > f(T)$ . Since  $f$  only takes integer values, this means that whenever we add an element  $i$  to  $T$ , the value of  $f(T)$  goes up by at least 1. At the end of the process we have  $f(T) \geq |T|$ . Since  $f(T) \leq M$ , it follows that  $|T| \leq M$  and therefore  $T \in \mathcal{F}$ .
We now claim that  $A(T) = S_{*}$ . For any element  $i \in S_{*} \setminus T$ , we didn't add  $i$  to  $T$  because  $f(T_i \cup \{i\}) - f(T_i) \leq 0$ , where  $T_i \subseteq T$  is the value of  $T$  when element  $i$  is visited in the procedure above. By submodularity,  $f(T \cup \{i\}) - f(T) \leq f(T_i \cup \{i\}) - f(T_i) \leq 0$ . This implies that  $S_{*} \subseteq A(T)$ . Also note that for any  $j \notin S_{*}$ , we have  $f(S_{*} \cup \{j\}) > f(S_{*})$  by the maximality of  $S_{*}$ . It again follows from submodularity and  $T \subseteq S_{*}$  that  $f(T \cup \{j\}) > f(T)$ , which implies  $j \notin A(T)$ . This proves  $A(T) = S^{*}$  and completes the proof of the theorem.