Title: Discretely beyond $1/e$: Guided Combinatorial Algortihms for Submodular Maximization
Venue: NeurIPS 2024
Accepted: yes
Keywords: combinatorial algorithms

deterministic algorithms

submodular optimization
Abstract: For constrained, not necessarily monotone submodular maximization, all known approximation algorithms with ratio greater than $1/e$ require continuous ideas, such as queries to the multilinear extension of a submodular function and its gradient, which are typically expensive to simulate with the original set function. For combinatorial algorithms, the best known approximation ratios for both size and matroid constraint are obtained by a simple randomized greedy algorithm of Buchbinder et al. [9]: $1/e \approx 0.367$ for size constraint and $0.281$ for the matroid constraint in $\mathcal O (kn)$ queries, where $k$ is the rank of the matroid. In this work, we develop the first combinatorial algorithms to break the $1/e$ barrier: we obtain approximation ratio of $0.385$ in $\mathcal O (kn)$ queries to the submodular set function for size constraint, and $0.305$ for a general matroid constraint. These are achieved by guiding the randomized greedy algorithm with a fast local search algorithm. Further, we develop deterministic versions of these algorithms, maintaining the same ratio and asymptotic time complexity. Finally, we develop a deterministic, nearly linear time algorithm with ratio $0.377$.

Discretely Beyond 1 /e : Guided Combinatorial Algorithms for Submodular Maximization

Abstract

For constrained, not necessarily monotone submodular maximization, all known approximation algorithms with ratio greater than 1 /e require continuous ideas, such as queries to the multilinear extension of a submodular function and its gradient, which are typically expensive to simulate with the original set function. For combinatorial algorithms, the best known approximation ratios for both size and matroid constraint are obtained by a simple randomized greedy algorithm of Buchbinder et al. [9]: 1 /e ≈ 0 . 367 for size constraint and 0 . 281 for the matroid constraint in O ( kn ) queries, where k is the rank of the matroid. In this work, we develop the first combinatorial algorithms to break the 1 /e barrier: we obtain approximation ratio of 0 . 385 in O ( kn ) queries to the submodular set function for size constraint, and 0 . 305 for a general matroid constraint. These are achieved by guiding the randomized greedy algorithm with a fast local search algorithm. Further, we develop deterministic versions of these algorithms, maintaining the same ratio and asymptotic time complexity. Finally, we develop a deterministic, nearly linear time algorithm with ratio 0 . 377 .

1 Introduction

A nonnegative set function f : 2 U → R + is submodular iff for all S ⊆ T ⊆ U , x ∈ U \ T , f ( S ∪ { x } ) -f ( S ) ≥ f ( T ∪ { x } ) -f ( T ) ; and f is monotone iff f ( S ) ≤ f ( T ) for all S ⊆ T ⊆ U . Submodular optimization plays an important role in data science and machine learning [3], particularly in tasks that involve selecting a representative subset of data or features. Its diminishing returns property makes it ideal for scenarios where the incremental benefit of adding an element to a set decreases as the set grows. Applications include sensor placement for environmental monitoring [19, 27], where the goal is to maximize coverage with limited sensors, feature selection [22, 18, 2] in machine learning to improve model performance and reduce overfitting, and data summarization [23, 31] for creating concise and informative summaries of large datasets. Further, many of these applications employ submodular objective functions that are non-monotone, e.g. Mirzasoleiman et al. [23], Tschiatschek et al. [31]. Formally, we study the optimization problem (SM): max f ( S ) , s.t. S ∈ I , where f is nonnegative, submodular and not necessarily monotone; and I ⊆ 2 U is a family of feasible subsets. Specifically, we consider two cases: when I is a size constraint (all sets of size at most k ); and more generally, when I is an arbitrary matroid of rank k .

In this field, algorithms typically assume access to a value oracle for the submodular function f , and the efficiency of an algorithm is measured by the number of queries to the oracle, because evaluation of the submodular function is typically expensive and dominates other parts of the computation. In the general, not necessarily monotone case, the approximability of constrained submodular optimization in the value oracle model is not well understood. For several years, 1 /e ≈ 0 . 367 was conjectured

to be the best ratio, as this ratio is obtained by the measured continuous greedy [15] algorithm that also gets the 1 -1 /e ratio in the monotone setting, which is known to be optimal [24]. However, in several landmark works, the 1 /e barrier was broken: first to 0 . 371 by Buchbinder et al. [9] (for size constraint only) and subsequently to 0 . 372 by Ene and Nguyen [13], then 0 . 385 by Buchbinder and Feldman [6]. Very recently, the best known approximation factor has been improved to 0.401 [7]. On the other hand, the best hardness result is 0 . 478 [16, 28].

All of the algorithms improving on the 1 /e ratio use oracle queries to the multilinear extension of a submodular function and its gradient. The multilinear extension relaxes the submodular set function to allow choosing an element with probability in [0 , 1] . Although this is a powerful technique, the multilinear extension must be approximated by polynomially many random samples of the original set function oracle. Unfortunately, this leads to a high query complexity for these algorithms, which we term continuous algorithms ; typically, the query complexity to the original submodular function is left uncomputed. As an illustration, we compute in Appendix B that the continuous algorithm of Buchbinder and Feldman [6] achieves ratio of 0 . 385 with query complexity of O ( n 11 log( n ) ) to the set function oracle. Consequently, these algorithms are of mostly theoretical interest - the time cost of running on tiny instances (say, n < 100 ) is already prohibitive, as demonstrated by Chen and Kuhnle [12] where a continuous algorithm required more than 10 9 queries to the set function on an instance with n = 87 , k = 10 .

For size and matroid constraints, the current state-of-the-art approximation ratio for a combinatorial algorithm ( i.e. not continuous) is obtained by the RANDOMGREEDY algorithm (Algorithm 1) of Buchbinder et al. [9]. RANDOMGREEDY achieves ratio 1 /e ≈ 0 . 367 for size constraint,

and 0 . 283 -ε ratio for matroid constraint; its query complexity is O ( kn ) . Thus, there is no known combinatorial algorithm that breaks the 1 /e barrier; and therefore, no such algorithm is available to be used in practice on any of the applications of SM described above.

Moreover, closing the gap between ratios achieved by deterministic and randomized algorithms for SM has been the focus of a number of recent works [5, 17, 11, 8]. In addition to theoretical interest, deterministic algorithms are desirable in practice, as a ratio that holds in expectation may fail on any given run with constant probability. Buchbinder and Feldman [5] introduced a linear programming method to derandomize the RANDOMGREEDY algorithm (at the expense of additional time complexity), meaning that the best known ratios for deterministic algorithms are again given by RANDOMGREEDY. There is no known method to derandomize continuous

algorithms, as the only known way to approximate the multilinear extension of a general submodular set function relies on random sampling methods. Özcan et al. [26], however, introduced a deterministic estimation via Taylor series approximation, but this approach is limited to a specific class of submodular functions that can be expressed as weighted compositions of analytic and multilinear functions. Therefore, there is no known deterministic algorithm that breaks the 1 /e barrier. The best known ratio in each category of continuous, combinatorial, and deterministic algorithms is summarized in Table 1. In this work, we consider the following questions:

Can combinatorial algorithms, and separately, deterministic algorithms, obtain approximation ratios for SM beyond 1 /e ? If so, are the resulting algorithms practical and do they yield empirical improvements in objective value over existing algorithms?

1.1 Contributions

In this work, we improve the best known ratio for a combinatorial algorithm for size-constrained SM to 0 . 385 -ε ≈ 1 /e +0 . 018 . This is achieved by using the result of a novel local search algorithm to guide the RANDOMGREEDY algorithm. Overall, we obtain query complexity of O ( kn/ε ) , which is at worst quadratic in the size of the ground set, since k ≤ n . Thus, this algorithm is practical and can run on moderate instance sizes; the first algorithm with ratio beyond 1 /e for which this is possible. Further, we extend this algorithm to the matroid constraint, where it improves the best known ratio of a combinatorial algorithm for a general matroid constraint from 0 . 283 of RANDOMGREEDY to 0 . 305 -ε .

Secondly, we obtain these same approximation ratios with deterministic algorithms. The ideas are similar to the randomized case, except we leverage a recently formulated algorithm INTERPOLATEDGREEDY [11] as a replacement for guided RANDOMGREEDY. The analysis of INTERPOLATEDGREEDY has similar recurrences (up to low order terms) and the algorithm can be guided in a similar fashion to RANDOMGREEDY, but is amenable to derandomization. The derandomization only adds a constant factor, albeit one that is exponential in (1 /ε ) .

Next, we seek to lower the query complexity further, while still improving the 1 /e ratio. As INTERPOLATEDGREEDY can be sped up to O ε ( n log k ) , the bottleneck becomes the local search procedure. Thus, we develop a faster way to produce the guiding set Z by exploiting a run of (unguided) INTERPOLATEDGREEDY and demonstrating that a decent guiding set is produced if the algorithm exhibits nearly worst-case behavior. With this method, we achieve a deterministic algorithm with ratio 0 . 377 ≈ 1 /e +0 . 01 in O ε ( n log k ) query complexity, which is nearly linear in the size of the ground set (since k = O ( n ) ).

Finally, we demonstrate the practical utility of our combinatorial 0 . 385 -approximation algorithm by implementing it and evaluating in the context of two applications of size-constrained SM on moderate instance sizes (up to n = 10 4 ). We evaluate it with parameters set to enforce a ratio > 1 /e . It outperforms both the standard greedy algorithm and RANDOMGREEDY by a significant margin in terms of objective value; moreover, it uses about twice the queries of RANDOMGREEDY and is orders of magnitude faster than existing local search algorithms.

1.2 Additional Related Work

Derandomization. Buchbinder and Feldman [5] introduced a linear programming (LP) method to derandomize the RANDOMGREEDY algorithm, thereby obtaining ratio 1 /e with a deterministic algorithm. Further, Sun et al. [30] were able to apply this technique to RANDOMGREEDY for matroids. A disadvantage of this approach is an increase in the query complexity over the original randomized algorithm. Moreover, we attempted to use this method to derandomize our guided RANDOMGREEDY algorithm, but were unsuccessful. Instead, we obtained our deterministic algorithms by guiding the INTERPOLATEDGREEDY algorithm instead of RANDOMGREEDY; this algorithm is easier to derandomize, notably without increasing the asymptotic query complexity.

Relationship to Buchbinder and Feldman [6]. The continuous, 0 . 385 -approximation algorithm of Buchbinder and Feldman [6] guides the measured continuous greedy algorithm using the output of a continuous local search algorithm, in analogous fashion to how we guide RANDOMGREEDY with the output of a combinatorial local search. However, the analysis of RANDOMGREEDY is much different from that of measured continuous greedy, although the resulting approximation factor is the same.

Specifically, Buchbinder and Feldman [6] obtain their ratio by optimizing a linear program mixing the continous local search and guided measured continous greedy; in contrast, we use submodularity and the output of our fast local search to formulate new recurrences for guided RANDOMGREEDY, which we then solve.

Local search algorithms. Local search is a technique widely used in combinatorial optimization. Nemhauser et al. [25] introduced a local search algorithm for monotone functions under size constraint; they showed a ratio of 1 / 2 , but noted that their algorithm may run in exponential time. Subsequently, local search has been found to be useful, especially for non-monotone functions. Feige et al. [14] proposed a 1 / 3 approximation algorithm with O ( n 4 /ε ) queries for the unconstrained submodular maximization problem utilizing local search. Meanwhile, Lee et al. [21] proposed a local search algorithm for general SM with matroid constraint, attaining 1 / 4 -ε approximation ratio with a query complexity of O ( k 5 log( k ) n/ε ) . We propose our own FASTLS in Section 2.1, yielding a ratio of 1 / 2 for monotone cases and 1 / 4 for non-monotone cases through repeated applications of FASTLS, while running in O ( kn/ε ) queries.

Fast approximation algorithms. Buchbinder et al. [10] developed a faster version of RANDOMGREEDY for size constraint that reduces the query complexity to O ε ( n ) with ratio of 1 /e -ε . Chen and Kuhnle [11] proposed LINEARCARD, the first deterministic, linear-time algorithm with an 1 / 11 . 657 -approximation ratio for size constraints. Also, Han et al. [17] introduced TWINGREEDY, a 0 . 25 -approximation algorithm with a query complexity of O ( kn ) for matroid constraints. These algorithms are fast enough to be used as building blocks for our FASTLS, which requires as an input a constant-factor approximation in O ( kn ) queries.

Relationship to Tukan et al. [32]. During the submission of this paper, we noticed an independent and parallel work by Tukan et al. [32], which proposed a different 0 . 385 -approximation algorithm. Both papers start from a similar idea-guiding the random greedy algorithm with a fast algorithm to find a local optimum. However, Tukan et al. [32] only considered size constraint and focused on algorithm speedup. They introduced a randomized local search algorithm and used its output to guide the stochastic greedy of Buchbinder et al. [9], achieving a query complexity of O ε ( n + k 2 ) . On the other hand, we 1) address a more general constraint-matroid constraint; 2) for size constraint, present an asymptotically faster algorithm that uses a novel way of guiding with partial solutions from random greedy itself, which are not local optima, thereby achieving ratio 0 . 377 -ε with O ε ( n log( k )) queries; and 3) derandomize these algorithms.

1.3 Preliminaries

Notation. In this section, we establish the notations employed throughout the paper. We denote the marginal gain of adding A to B by ∆( A | B ) = f ( A ∪ B ) -f ( B ) . For every set S ⊆ U and an element x ∈ U , we denote S ∪ { x } by S + x , and S \{ x } by S -x . Given a constraint and its related feasible sets I , let O ∈ arg max S ∈I f ( S ) ; that is, O is an optimal solution. To simplify the pseudocode and the analysis, we add k dummy elements into the ground set, where the dummy element serves as a null element with zero marginal gain when added to any set. The symbol e 0 is utilized to represent a dummy element.

Submodularity. A set function f : 2 U → R + is submodular, if ∆( x | S ) ≥ ∆( x | T ) for all S ⊆ T ⊆ U and x ∈ U \ T , or equivalently, for all A,B ⊆ U , it holds that f ( A ) + f ( B ) ≥ f ( A ∪ B ) + f ( A ∩ B ) .

Constraints. In this paper, our focus lies on two constraints: size constraint and matroid constraint. For size constraint, we define the feasible subsets as I ( k ) = { S ⊆ U : | S | ≤ k } , where k is an input parameter. The matroid constraint is defined in Appendix A.

Organization. Our randomized algorithms are described in Section 2, with two subroutines, FASTLS and GUIDEDRG, in Section 2.1 and 2.2, respectively. Due to space constraints, we provide only a sketch of the analysis for size constraint in the main text. The full pseudocodes and formal proofs for both size and matroid constraint are provided in Appendix C. Then, we briefly sketch the deterministic approximation algorithms in Section 2.3, with full details provided in Appendix D. Next, we introduce the nearly linear-time deterministic algorithm in Section 3, with omitted analysis provided in Appendix E. Our empirical evaluation is summarized in Section 4. In Section 5, we discuss limitations and future directions.

Algorithm 2: Randomized combinatorial approximation algorithm.

2 A Randomized (0 . 385 -ε ) -approximation in O ( kn/ε ) Queries

In this section, we present our randomized approximation algorithm (Alg. 2) for both size and matroid constraints. This algorithm improves the state-of-the-art, combinatorial approximation ratio to 0 . 385 -ε ≈ 1 /e + 0 . 018 for size constraint, and to 0 . 305 -ε ≈ 0 . 283 + 0 . 022 for matroid constraint.

Algorithm Overview. In overview, Alg. 2 consists of two components, which are detailed below. The first component is a local search algorithm, FASTLS (Alg. 4 in Appendix C.1), described in detail in Section 2.1. In brief, the local search algorithm takes an accuracy parameter ε > 0 and a constant-factor, approximate solution Z 0 as input, which may be produced by any approximation algorithm with better than O ( kn ) query complexity. The second component is a random greedy algorithm, GUIDEDRG (Alg. 6 in Appendix C.2), that is guided by the output Z of the local search, described in detail in Section 2.2. Also, GUIDEDRG takes a parameter t ∈ [0 , 1] , which is the switching time (as fraction of the budget or rank k ) from guided to unguided behavior. The candidate with best f value from the two subroutines is returned.

If f ( Z ) < α OPT (otherwise, there is nothing to show), then the local search set satisfies our definition of ((1 + ε ) α, α ) -guidance set (Def. 2.1 below). Under this guidance, we show that GUIDEDRG produces a superior solution compared to its unguided counterpart. The two components, FASTLS and GUIDEDRG are described in Sections 2.1 and 2.2, respectively. The following theorem is proven in Section 2.2 (size constraint) and Appendix C.2.2 (matroid constraint).

Theorem 2.1. Let ( f, I ) be an instance of SM. Let ε > 0 , and k ≥ 1 /ε . Algorithm 2 achieves an expected (0 . 385 -ε ) -approximation ratio for size constraint with t = 0 . 372 , and an expected (0 . 305 -ε ) -approximation ratio for matroid constraint with t = 0 . 559 . The query complexity of the algorithm is O ( kn/ε ) .

2.1 The Fast Local Search Algorithm

In this section, we introduce FASTLS (Alg. 4), which is the same for size or matroid constraints. There are several innovations in FASTLS that result in O ( kn/ε ) time complexity, where k is the maximum size of a feasible set, and ε > 0 is an input accuracy parameter.

In overview, the algorithm maintains a feasible set Z ; initially, Z = Z 0 , where Z 0 is an input set which is a constant approximation to OPT. The value of Z is iteratively improved via swapping, which is done in the following way. For each element a ∈ U , we compute ∆( a | Z \ a ) ; if a ̸∈ Z , this is just the gain of a to Z ; this requires O ( n ) queries. Then, if a ∈ Z and e ̸∈ Z such that Z \ a + e is feasible, and ∆( e | Z ) -∆( a | Z \ a ) ≥ ε k f ( Z ) , then a is swapped in favor of e . If no such swap exists, the algorithm terminates.

One can show that, for each swap, the value of Z increases by at least a multiplicative (1 + ε/k ) factor. Since f ( Z ) is initialized to a constant fraction of OPT, it follows that we make at most O ( k/ε ) swaps. Since each swap requires O ( n ) queries, this yields the query complexity of the algorithm: O ( kn/ε ) . In addition, if f is monotone, FASTLS gets ratio of nearly 1 / 2 for FASTLS. A second repetition of FASTLS yields a ratio of 1 / 4 in the case of general (non-monotone) f , as shown in Appendix C.1.2. Thus, FASTLS may be of independent interest, as local search obtains good objective values empirically and is commonly used in applications.

For our purposes, we want to use the output of FASTLS to guide RANDOMGREEDY. Since we will also use another algorithm for a similar purpose in Section 3, we abstract the properties needed for such a guidance set. Intuitively, a set Z is a good guidance set if it has a low f -value and also ensures that the value of its intersection and union with an optimal solution are poor.

0

k

(a)

1

e

f

(

O

)

f

(

O

)

[

f

(

O

Ai

)]

[

f

(

Ai

)]

0

k

(b)

1

e

f

(

O

)

f

(

O

\

Z

)

f

(

O

)

[

f

(

O

Ai

)]

[

f

((

O

\

Z

)

Ai

)]

0

tk

k

(c)

1

e

f

(

O

)

f

(

O

\

Z

)

f

(

O

)

[

f

(

O

Ai

)]

[

f

((

O

\

Z

)

Ai

)]

[

f

(

Ai

)]

Definition 2.1. A set Z is a ( α, β )-guidance set, if given constants α, β ∈ (0 , 0 . 5) and optimum solution O , it holds that: 1) f ( Z ) < αf ( O ) ; 2) f ( O ∩ Z ) ≤ αf ( O ) ; 3) f ( O ∪ Z ) ≤ βf ( O ) , or alternatively, 3 ′ ) f ( O ∩ Z ) + f ( O ∪ Z ) ≤ ( α + β ) f ( O ) .

Lemma 2.2 (proved in Appendix C.1.1) implies that for the FASTLS output Z , if f ( Z ) < α OPT, then Z is ((1 + ε ) α, α ) -guidance set.

Lemma 2.2. Let ε > 0 , and let ( f, I ( M )) be an instance of SM. The input set Z 0 is an α 0 -approximate solution to ( f, I ( M )) . FASTLS (Alg. 4) returns a solution Z with O ( kn log(1 /α 0 ) /ε ) queries such that f ( S ∪ Z ) + f ( S ∩ Z ) < (2 + ε ) f ( Z ) , where S ∈ I ( M ) .

2.2 Guiding the RANDOMGREEDY Algorithm

In this section, we discuss the guided RANDOMGREEDY algorithm (Alg. 6) using an ((1 + ε ) α, α ) -guidance set Z returned by FASTLS. Due to space constraints, we only consider the size constraint in the main text. The ideas for the matroid constraint are similar, although the final recurrences obtained differ. The version for matroid constraints is given in Appendix C.2.2.

The algorithm GUIDEDRG is simple to describe: it maintains a partial solution A , initially empty. It takes as parameters the switching time t and guidance set Z . While the partial solution satisfies | A | < tk , the algorithm operates as RANDOMGREEDY with ground set U \ Z ; after | A | ≥ tk , it operates as RANDOMGREEDY with ground set U . Pseudocode is provided in Appendix C.2.

Overview of analysis. For clarity, we first describe the (unguided) RANDOMGREEDY analysis from Buchbinder et al. [9]. There are two recurrences: the first is the greedy gain:

Intuitively, the gain at iteration i is at least a 1 /k fraction of the difference between f ( O ∪ A ) and A , in expectation, where A is the partial solution. If f were monotone, the right hand side would be at least ( OPT -f ( A )) /k . However, in the case that f is not monotone, the set O ∪ A may have value smaller than OPT.

To handle this case, it can be shown that the expected value of f ( O ∪ A ) satisfies a second recurrence:

where M i is the set of elements with the top k marginal gains at iteration i , (a) is from submodularity, and (b) is from nonnegativity. Thus, this expected value, while initially OPT (since A 0 = ∅ ), may degrade but is bounded.

Both of these recurrences are solved together to prove the expected ratio of 1 /e for RANDOMGREEDY: the worst-case evolution of the expected values of f ( A i ) , f ( O ∪ A i ) , according to this analysis,

is illustrated in Fig. 1(a). Observe that f ( A i ) converges to OPT /e (as required for the ratio), and observe that f ( O ∪ A i ) also converges to OPT /e . Thus, very little gain is obtained in the later stages of the algorithm, as illustrated in the plot. The overarching idea of the guided version of the algorithm is to obtain a better degradation of E [ f ( O ∪ A i )] , leading to better gains later in the algorithm that improve the worst-case ratio. In the following, we elaborate on this goal, the achievement of which is illustrated in Fig. 1(c).

Stage 1: Recurrences when avoiding Z . Suppose Z is an ( α, β ) -guidance set, and that RANDOMGREEDY selects elements as before, but excluding Z from the ground set. Then, the recurrences change as follows. The recurrence for the gain becomes:

where O \ Z replaces O since we select elements outside of the set Z . For the second recurrence, we can lower bound the term E [ f ( O ∪ A i -1 ∪ M i )] using submodularity and the fact that Z ∩ A i -1 = ∅ :

Finally, a similar recurrence to (2) also holds for f (( O \ Z ) ∪ A i ) ; both are needed for the analysis. Since Z is a guidance set, by submodularity, f ( O \ Z ) ≥ f ( O ) -f ( O ∩ Z ) ≥ (1 -α ) OPT , which ensures that some gain is available by selection outside of Z . And f ( O ) -f ( O ∪ Z ) ≥ (1 -β ) OPT, which means that the degradation recurrences are improved.

The blue line in Figure 1(b) depicts this improved degradation with the size of the partial solution. However, this improvement comes at a cost: a smaller increase in E [ f ( A i )] is obtained over the unguided version. Therefore, to obtain an improved ratio we switch back to the regular behavior of RANDOMGREEDY - intuitively, this shifts the relatively good, earlier behavior of RANDOMGREEDY to later in the algorithm.

Stage 2: Switching back to selection from whole ground set. After the switch, the recurrences revert back to the original ones, but with different starting values. Since E [ f ( O ∪ A i )] was significantly enhanced in the first stage, in the final analysis we get an overall improvement over the unguided version. The blue line in Figure 1(c) demonstrates the degradation of E [ f ( O ∪ A i )] over two stages, while the orange line depicts how the approximation ratio converges to a value 0 . 385 > 1 /e .

The above analysis sketch can be formalized and the resulting recurrences solved: the results are stated in the following lemma, which is formally proven in Appendix C.2.1.

From Lemma 2.3, we can directly prove the main result for size constraint.

Proof of Theorem 2.1 under size constraint. Let ( f, I ) be an instance of SM, with optimal solution set O . If f ( Z ) ≥ (0 . 385 -ε ) f ( O ) under size constraint, the approximation ratio holds immediately. Otherwise, by Lemma 2.2, FASTLS returns a set Z which is an ((1 + ε ) α, α ) -guidance set, where α = 0 . 385 -ε . By Lemma 2.3,

2.3 Deterministic approximation algorithms

In this section, we outline the deterministic algorithms, for size and matroid constraints. The main idea is similar, but we replace GUIDEDRG with a deterministic subroutine. For simplicity, we

present a randomized version in Appendix D.2 as Alg. 10, which we then derandomize (Alg. 11 in Appendix D.3). Further discussion is provided in Appendix D.

Algorithm overview. Chen and Kuhnle [11] proposed a randomized algorithm, INTERPOLATEDGREEDY, which may be thought of as an interpolation between standard greedy [25] and RANDOMGREEDY [9]. Instead of picking k elements, each randomly chosen from the top k marginal gains, it picks ℓ = O (1 /ε ) sets randomly from O ( ℓ ) candidates. Although it uses only a constant number of rounds, the recurrences for INTERPOLATEDGREEDY are similar to the RANDOMGREEDY ones discussed above, so we can guide it similarly.

To select the candidate sets in each iteration, we replace INTERLACEGREEDY (the subroutine of INTERPOLATEDGREEDY proposed in Chen and Kuhnle [11]) with a guided version: GUIDEDIG-S (Alg. 9 in Appendix D.1.1) for size constraint, and GUIDEDIG-M (Alg. 8 in Appendix D.1) for matroid constraint. Since only ℓ random choices are made, each from O ( ℓ ) sets, there are at most O ( ℓ O ( ℓ ) ) possible solutions, where ℓ is a constant number depending on ε . Notably, we are still able to obtain the same approximation factors as in Section 2. The proof of Theorem 2.4 is provided in Appendices D.2 and D.3.

Theorem 2.4. Let ( f, k ) be an instance of SM, with the optimal solution set O . Alg. 11 achieves a deterministic (0 . 385 -ε ) approximation ratio with t = 0 . 372 , and a deterministic (0 . 305 -ε ) approximation ratio with t = 0 . 559 . The query complexity of the algorithm is O ( knℓ 2 ℓ -1 ) where ℓ = 10 9 ε .

3 Deterministic Algorithm with Nearly Linear Query Complexity

In this section, we sketch a deterministic algorithm with (0 . 377 -ε ) approximation ratio and O ε ( n log( k )) query complexity for the size constraint. A full pseudocode (Alg. 14) and analysis is provided in Appendix E.

Description of algorithm. Our goal is to improve the asymptotic O ε ( kn ) query complexity. Recall that in Section 2, we described a deterministic algorithm that employed the output of local search to guide the INTERPOLATEDGREEDY algorithm, which obeys similar recurrences to RANDOMGREEDY. To produce the ℓ candidate sets for each iteration of INTERPOLATEDGREEDY, a greedy algorithm (guided INTERLACEGREEDY) is used. These algorithms can be sped up using a descending thresholds technique. This results in THRESHGUIDEDIG (Alg. 12 in Appendix E.1), which achieves O ε ( n log k ) query complexity for the guided part of our algorithm.

However, the local search FASTLS still requires O ( kn/ε ) queries, so we seek to find a guidance set in a faster way. Recall that, in the definition of guidance set Z , the value f ( Z ) needs to dominate both f ( O ∩ Z ) and f ( O ∪ Z ) . To achieve this with faster query complexity, we employ a run of unguided INTERPOLATEDGREEDY. Consider the recurrences plotted in Fig. 1(a) - if the worst-case degradation occurs, then at some point the value of f ( A i ) becomes close to f ( O ∪ A i ) . On the other hand, if the worst-case degradation does not occur, then the approximation factor of INTERPOLATEDGREEDY is improved (see Fig. 2). Moreover, if we ensure that at every stage, A i contains no elements that contribute a negative gain, then we will also have f ( A i ) ≥ f ( O ∩ A i ) .

To execute this idea, we run (derandomized, unguided) INTERPOLATEDGREEDY, and consider all O ( ℓ ℓ ) intermediate solutions. Each one of these is pruned (by which we mean, any element with negative contribution is dis-

E[f(O uA;)]

βf(O)

E[f(A;)]

0

carded until none such remain). Then, the guided part of our algorithm executes with every possible candidate intermediate solution as the guiding set; finally, the feasible set encountered with maximum f value is returned.

8

10

12

14

16

Number of frames selected

0.97

0.98

0.99

1.00

1.01

1.02

Objective / Greedy

(a) Video Summarization,

solution value

8

10

12

14

16

Number of frames selected

10 0

10 1

Queries / Greedy

(b) Video Summarization,

queries

0.0

0.2

0.4

k/n

0.85

0.90

0.95

1.00

Objective / Greedy

(c) Maximum Cut (ER),

solution value

0.0

0.2

0.4

k/n

10 0

10 1

10 2

Queries / Greedy

FastLS+GuidedRG

Random

(d) Maximum Cut (ER),

queries

The tradeoff between the first and second parts of the algorithm is optimized with α = 0 . 377 and β = 0 . 46 . That is, if INTERPOLATEDGREEDY produces an ( α, β ) -guidance set, the guided part of our algorithm achieves ratio 0 . 377 ; otherwise, INTERPOLATEDGREEDY has ratio at least 0 . 377 . We have the following theorem. The algorithms and analysis sketched here are formalized in Appendix E.

Theorem 3.1. Let ( f, k ) be an instance of SM, with the optimal solution set O . Algorithm 14 achieves a deterministic (0 . 377 -ε ) approximation ratio with O ( n log( k ) ℓ 1 2 ℓ 1 ℓ 2 2 ℓ 2 -1 ) queries, where ℓ 1 = 10 3 ε and ℓ 2 = 5 ε .

4 Empirical Evaluation

In this section, along with Appendix F, we implement and empirically evaluate our randomized (0 . 385 -ε ) -approximation algorithm (Alg. 2, FASTLS+GUIDEDRG) on two applications of sizeconstrained SM, and compare to several baselines in terms of objective value of solution and number of queries to f . In summary, our algorithm uses roughly twice the queries as the standard greedy algorithm, but obtains competitive objective values with an expensive local search that uses one to two orders of magnitude more queries. 1

Baselines. 1) STANDARDGREEDY: the classical greedy algorithm [25], which often performs well empirically on non-monotone objectives despite having no theoretical guarantee. 2) RANDOMGREEDY, the current state-of-the-art combinatorial algorithm as discussed extensively above. 3) The local search algorithm of Lee et al. [21], which is the only prior polynomial-time local search algorithm with a theoretical guarantee: ratio 1 / 4 -ε in O ( k 5 log( k ) n/ε ) queries. As our emphasis is on theoretical guarantees above 1 /e , we set ε = 0 . 01 for our algorithm, which yields ratio at least 0 . 375 in this evaluation. For Lee et al. [21], we set ε = 0 . 1 , which is the standard value of the accuracy parameter in the literature - running their algorithm with ε = 0 . 01 produced identical results.

Applications and datasets. For instances of SM upon which to evaluate, we chose video summarization and maximum cut (MC). For video summarization, our objective is to select a subset of frames from a video to create a summary. As in Banihashem et al. [1] , we use a Determinantal Point Process objective function to select a diverse set of elements [20]. Maximum cut is a classical example of a non-monotone, submodular objective function. We run experiments on unweighted Erd˝ os-Rényi (ER), Barabási-Albert (BA) and Watts-Strogatz (WS) graphs which have been used to model many real-world networks. The formal definition of problems, details of datasets, and hyperparameters of graph generation can be found in the Appendix F. In video summarization, there are n = 100 frames. On all the instances of maximum cut, the number of vertices n = 10000 . The mean of 20 independent runs is plotted, and the shaded region represents one standard deviation about the mean.

Results. As shown in Figure 3 in this section, and Figure 5 and 6 in Appendix F, on both applications, FASTLS +GUIDEDRG produces solutions of higher objective value than STANDARDGREEDY, and also higher than RANDOMGREEDY. The objective values of FASTLS +GUIDEDRG often matches with Lee et al. [21] which performs the best; this agrees with the intuition that, empirically, local search is nearly optimal. In terms of queries, our algorithm uses roughly twice the number of queries

as STANDARDGREEDY, but we improve on Lee et al. [21] typically by at least a factor of 10 and often by more than a factor of 100 .

5 Discussion and Limitations

Prior to this work, the state-of-the-art combinatorial ratios were 1 /e ≈ 0 . 367 and 0 . 283 for size constrained and matroid constrained SM, respectively, both achieved by the RANDOMGREEDY algorithm. In this work, we show how to guide RANDOMGREEDY with a fast local search algorithm to achieve ratios 0 . 385 and 0 . 305 , respectively, in O ( kn/ε ) queries. The resulting algorithm is practical and empirically outperforms both RANDOMGREEDY and standard greedy in objective value on several applications of SM. However, if k is on the order of n , the query complexity is quadratic in n , which is too slow for modern data sizes. Therefore, an interesting question for future work is whether further improvements in the query complexity to achieve these ratios (or better ones) could be made.

In addition, we achieve the same approximation ratios and asymptotic query complexity with deterministic algorithms, achieved by guiding a different algorithm; moreover, we speed up the deterministic algorithm to O ε ( n log k ) by obtaining the guidance set in another way. This result is a partial answer to the limitation in the previous paragraph, as we achieve a ratio beyond 1 /e in nearly linear query complexity. However, for all of our deterministic algorithms, there is an exponential dependence on 1 /ε , which makes these algorithms impractical and mostly of theoretical interest.

References

Banihashem, K., Biabani, L., Goudarzi, S., Hajiaghayi, M., Jabbarzade, P., and Monemizadeh, M. (2023). Dynamic non-monotone submodular maximization. Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023 .

Bao, W., Hang, J., and Zhang, M. (2022). Submodular feature selection for partial label learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2022 .

Bilmes, J. A. (2022). Submodularity in machine learning and artificial intelligence. arXiv preprint arXiv:2202.00132 .

Brualdi, R. A. (1969). Comments on bases in dependence structures. Bulletin of the Australian Mathematical Society , 1(2):161-167.

Buchbinder, N. and Feldman, M. (2018). Deterministic algorithms for submodular maximization problems. ACM Trans. Algorithms , 14(3):32:1-32:20.

Buchbinder, N. and Feldman, M. (2019). Constrained submodular maximization via a nonsymmetric technique. Math. Oper. Res. , 44(3):988-1005.

Buchbinder, N. and Feldman, M. (2023). Constrained submodular maximization via new bounds for dr-submodular functions. arXiv preprint arXiv:2311.01129 .

Buchbinder, N., Feldman, M., and Garg, M. (2023). Deterministic (1/2 + ϵ )-approximation for submodular maximization over a matroid. SIAM J. Comput. , 52(4):945-967.

Buchbinder, N., Feldman, M., Naor, J., and Schwartz, R. (2014). Submodular maximization with cardinality constraints. In Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2014 .

Buchbinder, N., Feldman, M., and Schwartz, R. (2017). Comparing apples and oranges: Query trade-off in submodular maximization. Math. Oper. Res. , 42(2):308-329.

Chen, Y. and Kuhnle, A. (2023). Approximation algorithms for size-constrained non-monotone submodular maximization in deterministic linear time. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2023 .

Chen, Y. and Kuhnle, A. (2024). Practical and parallelizable algorithms for non-monotone submodular maximization with size constraint. J. Artif. Intell. Res. , 79:599-637.

Ene, A. and Nguyen, H. L. (2016). Constrained submodular maximization: Beyond 1/e. In IEEE 57th Annual Symposium on Foundations of Computer Science, FOCS 2016 .

Feige, U., Mirrokni, V. S., and Vondrák, J. (2011). Maximizing non-monotone submodular functions. SIAM J. Comput. , 40(4):1133-1153.

Feldman, M., Naor, J., and Schwartz, R. (2011). A unified continuous greedy algorithm for submodular maximization. In IEEE 52nd Annual Symposium on Foundations of Computer Science, FOCS 2011 .

Gharan, S. O. and Vondrák, J. (2011). Submodular maximization by simulated annealing. In Proceedings of the Twenty-Second Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2011 .

Han, K., Cao, Z., Cui, S., and Wu, B. (2020). Deterministic approximation for submodular maximization over a matroid in nearly linear time. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020 .

Khanna, R., Elenberg, E. R., Dimakis, A. G., Negahban, S. N., and Ghosh, J. (2017). Scalable greedy feature selection via weak submodularity. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017 .

Krause, A., Singh, A. P., and Guestrin, C. (2008). Near-optimal sensor placements in gaussian processes: Theory, efficient algorithms and empirical studies. J. Mach. Learn. Res. , 9:235-284.

Kulesza, A., Taskar, B., et al. (2012). Determinantal point processes for machine learning. Foundations and Trends® in Machine Learning , 5(2-3):123-286.

Lee, J., Mirrokni, V. S., Nagarajan, V., and Sviridenko, M. (2009). Non-monotone submodular maximization under matroid and knapsack constraints. In Proceedings of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009 .

Liu, Y., Wei, K., Kirchhoff, K., Song, Y., and Bilmes, J. A. (2013). Submodular feature selection for high-dimensional acoustic score spaces. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2013 .

Mirzasoleiman, B., Jegelka, S., and Krause, A. (2018). Streaming non-monotone submodular maximization: Personalized video summarization on the fly. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, AAAI 2018 .

Nemhauser, G. L. and Wolsey, L. A. (1978). Best algorithms for approximating the maximum of a submodular set function. Math. Oper. Res. , 3(3):177-188.

Nemhauser, G. L., Wolsey, L. A., and Fisher, M. L. (1978). An analysis of approximations for maximizing submodular set functions - I. Math. Program. , 14(1):265-294.

Özcan, G., Moharrer, A., and Ioannidis, S. (2021). Submodular maximization via taylor series approximation. In Demeniconi, C. and Davidson, I., editors, Proceedings of the 2021 SIAM International Conference on Data Mining, SDM 2021, Virtual Event, April 29 - May 1, 2021 , pages 423-431. SIAM.

Powers, T., Bilmes, J. A., Krout, D. W., and Atlas, L. E. (2016). Constrained robust submodular sensor selection with applications to multistatic sonar arrays. In 19th International Conference on Information Fusion, FUSION 2016 .

Qi, B. (2022). On maximizing sums of non-monotone submodular and linear functions. In 33rd International Symposium on Algorithms and Computation, ISAAC 2022 .

Roy (2022). Cooking video. https://youtu.be/voUDP4rUKvQ?si=ZUezR4jMVzOz5Kcw.

Sun, X., Zhang, J., Zhang, S., and Zhang, Z. (2022). Improved deterministic algorithms for non-monotone submodular maximization. In Computing and Combinatorics - 28th International Conference, COCOON 2022 .

Tschiatschek, S., Iyer, R. K., Wei, H., and Bilmes, J. A. (2014). Learning mixtures of submodular functions for image collection summarization. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, NeurIPS 2014 .

Tukan, M., Mualem, L., and Feldman, M. (2024). Practical 0 . 385 -approximation for submodular maximization subject to a cardinality constraint. arXiv preprint arXiv:2405.13994 .

A Additional Preliminaries

A.1 Constraints

In this paper, our focus lies on two constraints: size constraint and matroid constraint. For size constraint, we define the feasible subsets as I ( k ) = { S ⊆ U : | S | ≤ k } . For matroid constraint, the definition is as follows:

Definition A.1. A matroid M is a pair ( U , I ) , where U is the ground set and I is the independent sets with the following properties: (1) ∅ ∈ I ; (2) hereditary property: A ∈ I ⇒ B ∈ I , ∀ B ⊆ A ; (3) exchange property: A,B ∈ I , | A | > | B |⇒∃ x ∈ A \ B, s.t. B + x ∈ I .

Specifically, we use I ( M ) to represent the independent sets of matroid M . A maximal independent set in I ( M ) is called a basis. Let k be the size of the maximal independent set.

In the following, we consider an extended matroid with k dummy elements added to the ground set M ′ = ( U ′ , I ′ ) . We show that SM on M ′ return the same solution as on M .

Lemma A.1. Let M = ( U , I ) be a matroid, and U ′ = U ∪ E , where E = { e 1 0 , . . . , e k 0 } and e i 0 is a dummy element for each i ∈ [ k ] . Let I ′ = ⋃ S ∈I { S, S ∪ { e 1 0 } , . . . , S ∪ { e 1 0 , . . . , e k -| S | 0 }} . Then, M ′ = ( U ′ , I ′ ) is also a matroid and max S ∈I f ( S ) = max S ∈I ′ f ( S ) .

Proof. Firstly, we prove that M ′ = ( U ′ , I ′ ) is also a matroid by Definiton A.1.

Since ∅ ∈ I and I ⊆ I ′ , it holds that ∅ ∈ I ′ .

Let A ′ ∈ I ′ , and A = A ′ \ E . Then, A ∈ I . For every B ′ ⊆ A ′ , since ( U , I ) is a matroid, B = B ′ \ E ⊆ A ∈ I . Since | B ′ ∩ E | ≤ | A ′ ∩ E | ≤ k -| A | ≤ k -| B | , it holds that B ∈ I ′ by the construction of I ′ .

Let A ′ , B ′ ∈ I ′ and | A ′ | > | B ′ | . Let A = A ′ \ E and B = B ′ \ E . Then, A,B ∈ I . If | A | > | B | , by Def. A.1, there exists x ∈ A \ B s.t. B + x ∈ I . Since ( B ′ + x ) \ ( B + x ) ⊆ E and | B ′ | < | A ′ | ≤ k , B ′ + x ∈ I ′ . Otherwise, | A | < | B | , which indicates that | A ′ ∩ E | > | B ′ ∩ E | . Since | B ′ | < | A ′ | ≤ k , by adding a dummy element e 0 ∈ A ′ ∩ E \ B ′ to B ′ , it holds that B ′ + e 0 ∈ I ′ .

Thus, by Def. A.1, M ′ = ( U ′ , I ′ ) is also a matroid.

As for SM on I ′ , since dummy element does not contribute to the objective value, it holds that, for every S ∈ I , f ( S ) = f ( S ∪ E ′ ) , where E ′ ⊆ E . Then, { f ( S ) : S ∈ I} = { f ( S ) : S ∈ I ′ } . Further, max S ∈I f ( S ) = max S ∈I ′ f ( S ) .

A.2 Technical Lemma

Lemma A.2. (Brualdi [4]) If B 1 and B 2 are finite bases, then there exists a bijection σ : B 1 \ B 2 → B 2 \ B 1 such that B 2 + e -σ ( e ) is a basis for all e ∈ B 1 \ B 2

Lemma A.3. Let a ∈ R + , b, X 0 ∈ R . If X i ≥ aX i -1 + b for every i ∈ [ k ] , then

Proof. By repeatedly implementing that X i ≥ aX i -1 + b , we can bound X k as follows,

Lemma A.4.

B Query Complexity Analysis of the Continuous Algorithm in Buchbinder and Feldman [6]

Algorithm 3: Aided Measured Continuous Greedy ( f, P, Z, t s ) [6]

In this section, we analyze the query complexity of Aided Measured Continuous Greedy (Alg. 3) proposed by Buchbinder and Feldman [6], which is O ( n 11 log( n ) ) .

In Alg. 3, queries to the oracle f only occur on Line 5. For each element u in the ground set U , r = ⌈ 48 n 6 log(2 n ) ⌉ queries are made. These queries correspond to r independent samples of R ( y ( t )) to estimate E [∆( u | R ( y ( t )))] . Therefore, there are nr = O ( n 7 log( n ) ) queries for each iteration of the while loop (Line 3-11).

Time variable t is increased by ¯ δ 1 = t s · n -4 , when t < t s , and is increased by ¯ δ 2 = (1 -t s ) · n -4 , when t ≥ t s . Thus, there are a total of 2 n 4 iterations within the while loop. In conclusion, the total number of queries made by the algorithm is O ( n 11 log( n ) ) .

C Analysis of Randomized Approximation Algorithm, Alg. 2

In this section, we provide a detailed analysis of our randomized approximation algorithm and its two components, FASTLS and GUIDEDRG. This section is organized as follows: Appendix C.1 analyzes the theoretical guarantee of single run of FASTLS (Appendix C.1.1), which is needed to show that it finds a good guidance set. Then, although it is not needed for our results, we show the FASTLS independently achieves an approximation ratio achieved for SMCC under monotone and non-monotone objectives (Appendix C.1.2).

In Appendix C.2, we provide pseudocode and formally prove the results for GUIDEDRG. Specifically, we solve the recurrences for both size and matroid constraint.

Algorithm 4: A fast local search algorithm with query complexity O ( kn/ε ) .

C.1 Analysis of FASTLS (Alg. 4)

Pseudocode for FASTLS is provided in Alg. 4.

C.1.1 Finding A Good Guidance Set - Proofs for Lemma 2.2 of Alg. 4 in Section 2.1

Recall that FASTLS (Alg. 4) takes a matroid constraint I ( M ) and an approximation result Z 0 as inputs, and outputs a local optimum Z . Here, we restate the theoretical guarantees of FASTLS (Lemma 2.2). Using the conclusions drawn from Lemma 2.2, we demonstrate in Corollary C.1 that Z is a ((1 + ε ) α, α ) -guidance set. At the end of this section, we provide the proof for Lemma 2.2.

Lemma 2.2. Let ε > 0 , and let ( f, I ( M )) be an instance of SM. The input set Z 0 is an α 0 -approximate solution to ( f, I ( M )) . FASTLS (Alg. 4) returns a solution Z with O ( kn log(1 /α 0 ) /ε ) queries such that f ( S ∪ Z ) + f ( S ∩ Z ) < (2 + ε ) f ( Z ) , where S ∈ I ( M ) .

Corollary C.1. Let Z be the solution of FASTLS ( f, I ( M ) , Z 0 , ε ) . If f ( Z ) < α OPT, Z is a ((1 + ε ) α, α ) -guidance set.

Proof of Corollary C.1. By Lemma 2.2, let S = O and S = O ∩ Z respectively, it holds that

By Definition 2.1, Z is a ((1 + ε ) α, α )

In the following, we prove Lemma 2.2 for Alg. 4.

Proof of Lemma 2.2. Query Complexity. For each successful replacement of elements on Line 6, it holds that ∆( e | Z ) -∆( a | Z -a ) ≥ ε k f ( Z ) . By submodularity,

Hence, the oracle value f ( Z ) is increased by a factor of at least (1 + ε/k ) after the swap. Therefore, there are at most ⌈ log 1+ ε/k ( f ( O ) f ( Z 0 ) )⌉ iterations, since otherwise, it would entail f ( Z ) > f ( O ) , which contradicts the fact that O is the optimal solution. Then, since the algorithm makes at most O ( n ) queries at each iteration, the query complexity can be bounded as follows,

Objective Value. For any S ∈ I ( M ) , we consider adding dummy elements into S until | S | = k . By Lemma A.1, we consider Z and S are bases of matroid M with or without dummy elements.

If f ( Z ) < α OPT, then

Then, by Lemma A.2, there exits a bijection σ : S \ Z → Z \ S such that Z + e -σ ( e ) is a basis for all e ∈ S \ Z . After the algorithm terminates, for every e ∈ S \ Z , it holds that, ∆( e | Z ) -∆( σ ( e ) | Z -σ ( e )) < ε k f ( Z ) . Then,

C.1.2 Approximation Ratio achieved by FASTLS

In this section, we show that FASTLS can be employed independently to achieves approximation ratios of nearly 1 / 2 and 1 / 4 for both the monotone and non-monotone versions of the problem, respectively.

Monotone submodular functions. By employing FASTLS once, it returns an 1 2+ ε -approximation result in monotone cases.

Theorem C.1. Let ε > 0 , and let ( f, I ( M )) be an instance of SM, where f is monotone. The input set Z 0 is an α 0 -approximate solution to ( f, I ( M )) . FASTLS (Alg. 4) returns a solution Z such that f ( Z ) ≥ f ( O ) / (2 + ε ) with O ( kn log(1 /α 0 ) /ε ) queries.

Proof. By Lemma 2.2, set S = O , it holds that

where the last inequality follows by monotonicity and non-negativity.

Non-monotone submodular functions. For the non-monotone problem, 2 repetitions of FASTLS (Alg. 5) yields a ratio of 1 4+2 ε . The theoretical guarantees and the corresponding analysis are provided as follows. We remark that this is a primitive implementation of the guiding idea: the second run of FASTLS avoids the output of the first one.

Algorithm 5: An 1 / (4 + 2 ε ) -approximation algorithm with O ( kn/ε )

1 Input: oracle f , constraint I , an approximation result Z , switch point t , error rate ε

Theorem C.2. Let ε > 0 , and let ( f, I ( M )) be an instance of SM, where f is not necessarily monotone. The input set Z 0 is an α 0 -approximate solution to ( f, I ( M )) . Alg. 5 returns a solution Z such that f ( Z ) ≥ f ( O ) / (4 + 2 ε ) with O ( kn log(1 /α 0 ) /ε ) queries.

Proof. By repeated application of Lemma 2.2 for the two calls of FASTLS in Alg. 5, it holds that

By summing up the above two inequalities, it holds that

C.2 Pseudocode of GUIDEDRG (Alg. 6) and its Analysis

In this section, we present the pseudocode for GUIDEDRG as Alg. 6. Then, we provide the detailed proof of Lemma 2.3 in Appendix C.2.1, which addresses size constraints. Finally, we analyze the algorithm under matroid constraints and provide the guarantees and its analysis in Appendix C.2.2.

C.2.1 GUIDEDRG under Size Constraints

In Sec. 2.2, we introduce the intuition behind GUIDEDRG under size constraint. Below, we reiterate theoretical guarantees achieved by GUIDEDRG under size constraints and provide the detailed analysis.

We provide the recurrence of f (( O \ Z ) ∪ A i ) , f ( O ∪ A i ) and f ( A i ) in Lemmata C.3 and C.4 and their analysis below to help prove Lemma 2.3 under size constraint.

Lemma C.3. When 0 < i ≤ t · k , it holds that

When t · k < i ≤ k , it holds that

Proof. At iteration i , condition on a given A i -1 . When i ≤ tk , A i -1 ∩ Z = ∅ and M i is selected out of A i -1 ∪ Z , so

Then,

When i > tk , it holds that

By unconditioning A

i -1 , the lemma is proved.

Lemma C.4. When 0 < i ≤ t · k , it holds that

When t · k < i ≤ k , it holds that

Proof. Given A i -1 at iteration i . When i ≤ t · k , it holds that

When i > t · k , it holds that

By unconditioning A i -1 , the lemma is proved.

Proof of Lemma 2.3. It follows from Lemma C.3 and the closed form for a recurrence provided in Lemma A.3 that

With the above inequalities, we can solve the recursion in Lemma C.4 as follows,

C.2.2 GUIDEDRG under Matroid Constraints

Discussion about Intuition behind GUIDEDRG under Matroid Constraints. The pseudocode for RANDOMGREEDY under matroid constraints is provided in Alg. 7. To deal with the feasibility for matroid constraints, Alg. 7 starts with an arbitrary basis and builds the solution by randomly swapping the elements in ground set with a candidate basis. The analysis of it proceeds according to two main recurrences.

Fig. 4(a) depicts the worse-case behavior of E [ f ( A i )] and E [ f ( O ∪ A i )] . As discussed in Section 2.2, we consider improving the degradation of E [ f ( O ∪ A i )] by selecting elements from outside of an ( α + ε, α ) -guidance set Z to enhance the lower bound of E [ f ( O ∪ A i -1 ∪ M i )] . The blue line in Fig. 4(b) illustrated the improvement of E [ f ( O ∪ A i )] with an ( α + ε, α ) -guidance set. However, restricting the selection only to elements outside Z restricts the increase in E [ f ( A i )] to the difference between E [ f (( O \ Z ) ∪ A i -1 )] and E [ f ( A i -1 )] . This restriction is illustrated by the red line in Figure 4(b), indicating degradation in E [ f (( O \ Z ) ∪ A i )] .

To benefit from the improved degradation of E [ f ( O ∪ A i )] , we consider transitioning to selecting elements from the whole ground set at a suitable point. The blue line in Fig. 4(b) illustrates how E [ f ( O ∪ A i )] degrades before and after we switch, and the orange line illustrates the evolution of E [ f ( A i )] . Even starting with an inferior selection at the first stage, we still get an overall improvement on the objective value.

We provide the updated recursions for f (( O \ Z ) ∪ A i ) , f ( O ∪ A i ) and f ( A i ) in Lemma C.5 and C.6 below. Then, the closed form of the solution value, derived from these lemmata, is presented in Lemma C.7. After that, we prove the approximation ratio of the randomized algorithm under matroid constraint.

Lemma C.5. When 0 < i ≤ t · k , it holds that

When t · k < i ≤ k , it holds that

0

k

(a)

1+

e

2

4

f

(

O

)

f

(

O

)

[

f

(

O

Ai

)]

[

f

(

Ai

)]

0

k

(b)

1+

e

2

4

f

(

O

)

f

(

O

\

Z

)

f

(

O

)

[

f

(

O

Ai

)]

[

f

((

O

\

Z

)

Ai

)]

0

tk

k

(c)

1+

e

2

4

f

(

O

)

f

(

O

\

Z

)

f

(

O

)

[

f

(

O

Ai

)]

[

f

((

O

\

Z

)

Ai

)]

[

f

(

Ai

)]

Proof. When i ≤ tk , it holds that

When i > tk , it holds that

By unconditioning A i -1 , the lemma is proved.

Lemma C.6. When 0 < i ≤ t · k , it holds that

When t · k < i ≤ k , it holds that

Proof. Given A i -1 at iteration i . When i ≤ t · k , since O is a base, O \ ( A i -1 ∪ Z ) with dummy elements is also a base. It holds that

When i > t · k , it holds that

Lemma C.7. With an input matroid constraint I and a ((1 + ε ) α, α ) -guidance set Z , GUIDEDRG returns set A k with O ( kn ) queries, s.t. E [ f ( A k )] ≥ 1 2 ( 1 2 + ( 3 2 -t -1 k ) ( 1 -2 k ) e 2( t -1) -e -2 -(1 + ε ) α ( ( 1 -2 k ) 2 e 2( t -1) -e -2 ) -α (( 1 2 + 1 -t 1 -2 k ) e 2( t -1) -( 3 2 -1 k ) e -2 )) f ( O ) .

Proof. It follows from Lemma C.5 and the closed form for a recurrence provided in Lemma A.3 that

Then, by sovling the recursion in Lemma C.6 with the above inequalities, it holds that

where Inequality ( a ) and ( b ) follow from Inequality 6, Lemma C.6 and A.3.

Proof of Theorem 2.1 under matroid constraint. Let ( f, I ) be an instance of SM, with optimal solution set O . If f ( Z ) ≥ (0 . 305 -ε ) f ( O ) under matroid constraint, the approximation ratio holds immediately. Otherwise, by Corollary C.1, FASTLS returns a set Z which is an ((1+ ε ) α, α ) -guidance set, where α = 0 . 305 -ε .

By Lemma C.7 and Z is an ((1 + ε ) α, α ) -guidance set with α = 0 . 305 -ε ,

D Analysis of Deterministic Approximation Algorithm

In this section, we present the pseudocode of deterministic algorithm and its analysis. The organization of this section is as follows: firstly, in Appendix D.1, we introduce the deterministic subroutine, GUIDEDIG-S and GUIDEDIG-M, along with their analysis; then, in Appendix D.2, we provide a randomized version of the deterministic algorithm for analytical purposes; finally, in Appendix D.3, we provide the deterministic algorithm and its theoretical guarantee.

D.1 Deterministic Subroutines - GUIDEDIG-S and GUIDEDIG-M

Inspired by INTERLACEGREEDY algorithm, a subroutine of INTERPOLATEDGREEDY, proposed by Chen and Kuhnle [11], we introduce guided versions of it for both size and matroid constraints.

The algorithm for the size constraint closely resembles INTERLACEGREEDY in Chen and Kuhnle [11]. Hence, we provide the pseudocode (Alg. 9), guarantees, and analysis in Appendix D.1.1. In this section, we focus on presenting GUIDEDIG-M for matroid constraints as Alg. 8. This algorithm addresses the feasibility issue by incorporating INTERLACEGREEDY into matroid constraints. Moreover, while it compromises the approximation ratio over size constraint to some extent, it no longer has the drawback of low success probability, which the size-constrained version has.

Algorithm overview. Under size constraints, INTERPOLATEDGREEDY [11] constructs the solution with ℓ iterations, where each iteration involves calling the INTERLACEGREEDY subroutine and adding k/ℓ elements into the solution. However, this approach is not applicable to matroid constaint due to the feasibility problem. Consequently, we propose GUIDEDIG-M for matroid constraints designed as follows: 1) consider adding a basis ( k elements) A to ℓ solution sets, where each addition dominates the gain of a distinct element in O ; 2) by exchange property, establish a bijection between the basis A and the starting set G ; 3) delete elements in each solution set that are mapped to by the basis A . This procedure avoids the extensive guessing of GUIDEDIG-S for size constraints and reduces the number of potential solutions from ℓ ( ℓ + 1) to ℓ . We provide the theoretical guarantees and the detailed analysis below.

Lemma D.1. Let O ∈ I ( M ) , and suppose GUIDEDIG-M(Alg. 8) is called with ( f, M , Z, G, ℓ ) , where Z ∩ G = ∅ . Then GUIDEDIG-M outputs ℓ candidate sets with O ( ℓkn ) queries. Moreover, a randomly selected set G ′ from the output satisfies that:

Proof. A = { a ∗ 1 , a ∗ 2 , . . . , a ∗ k } be the sequence with the order of elements being added. Since A and O \ Z are basis of M (by adding dummy elements into O \ Z ), we can order O \ Z = { o 1 , o 2 , . . . , o k } s.t. for any 1 ≤ i ≤ k , { a ∗ 1 , . . . , a ∗ i -1 , o i } is an independent set. Thus, o i ∈ X i . Let A ( i ) j be A j after i -th iteration. Therefore, for any 1 ≤ j ≤ ℓ , by submodularity,

By summing up the above inequality with 1 ≤ j ≤ ℓ ,

Then, we can prove the first inequality as follows,

By submodularity, nonnegativity, and Z ∩ A = ∅ , the last two inequalities can be proved as follows,

D.1.1 GUIDEDIG-S and its Analysis

In this section, we provide the pseudocode, guarantees and analysis of GUIDEDIG-S, which highly resembles INTERLACEGREEDY in Chen and Kuhnle [11].

Lemma D.2. Let O ⊆ U be any set of size at most k , and suppose GUIDEDIG-S(Alg. 9) is called with ( f, k, Z, G, ℓ ) . Then GUIDEDIG-S outputs ℓ ( ℓ +1) candidate sets with O ( ℓkn ) queries. Moreover, with a probability of ( ℓ +1) -1 , a randomly selected set A from the output satisfies that:

Algorithm 9: A ( ≈ ℓ ) -approximation that interlaces ℓ greedy procedures together and uses only 1 /ℓ fraction of the budget.

Proof. Let o max = arg max o ∈ O \ ( G ∪ Z ) ∆( o | G ) , and let { a 1 , . . . , a ℓ } be the largest ℓ elements of { ∆( x | G ) : x ∈ U \ ( G ∪ Z ) } , as chosen on Line 3. We consider the following two cases.

Case ( O \ ( G ∪ Z )) ∩{ a 1 , . . . , a ℓ } = ∅ . Then, o max ̸∈ { a 1 , . . . , a ℓ } which implies that ∆( a u | G ) ≥ ∆( o | G ) , for every 1 ≤ u ≤ ℓ and o ∈ O \ ( G ∪ Z ) ; and, after the first iteration of the for loop on Line 10 of Alg. 9, none of the elements in O \ ( G ∪ Z ) is added into any of { A 0 ,i } ℓ i =1 . We will analyze the iteration of the for loop on Line 4 with u = 0 .

Since none of the elements in O \ ( G ∪ Z ) is added into the collection when j = 0 , we can order O \ ( G ∪ Z ) = { o 1 , o 2 , . . . } such that the first ℓ elements are not selected in any set before we get to j = 1 , the next ℓ elements are not selected in any set before we get to j = 2 , and so on. Let i ∈ { 1 , . . . , ℓ } . Let A j 0 ,i be the value of A 0 ,i after j elements are added into it, and define A 0 ,i = A k/ℓ 0 ,i , the final value. Finally, denote by δ j the value ∆ ( x j,i | A j 0 ,i ) . Then,

where the last inequality follows from the ordering of O and the selection of elements into the sets. By summing up the above inequality with all 1 ≤ i ≤ ℓ , it holds that,

̸

Since A 0 ,i 1 ∩ A 0 ,i 2 = G for any 1 ≤ i 1 = i 2 ≤ ℓ , and each x j,i is selected outside of Z , by repeated application of submodularity, it can be shown that

Therefore, if we select a random set from { A 0 ,i : 1 ≤ i ≤ ℓ } , the three inequalities in the Lemma hold and we have probability 1 / ( ℓ +1) of this happening.

Case ( O \ ( G ∪ Z )) ∩ { a 1 , . . . , a ℓ } ̸ = ∅ . Then o max ∈ { a 1 , . . . , a ℓ } , so a u = o max, for some u ∈ 1 , . . . , ℓ . We analyze the iteration u of the for loop on Line 4. Similar to the previous case, let i ∈ { 1 , . . . , ℓ } , define A j u,i be the value of A u,i after we add j elements into it, and we will use A u,i for A k/ℓ u,i , Also, let δ j = ∆ ( x j,i | A j -1 u,i ) . Finally, let x 1 ,i = a u and observe A 1 u,i = G ∪ { a u } , for any i ∈ { 1 , . . . , ℓ } .

Then, we can order O \ G = { o 1 , o 2 , . . . } such that: 1) for the first ℓ elements { o l } ℓ l =1 , ∆( o l | G ) ≤ ∆( o max | G ) = δ 1 ; 2) the next ℓ elements { o l } 2 ℓ l = ℓ +1 are not selected by any set before we get to j = 2 , which implies that ∆ ( o l | A 1 u,i ) ≤ δ 2 , and so on. Therefore, analagous to the the previous case, we have that

̸

Since, A u,i 1 ∩ A u,i 2 = G ∪ { a u } for any 1 ≤ i 1 = i 2 ≤ ℓ , a u ∈ O \ Z , and each x j,i is selected outside of Z , by submodularity and nonnegativity of f , it holds that

Therefore, if we select a random set from { A u,i : 1 ≤ i ≤ ℓ } , the three inequalities in the lemma holds, and this happens with probability ( ℓ +1) -1 .

D.2 Randomized Version of our Deterministic Algorithm

In this section, we provide the randomized version (Alg 10) of our deterministic algorithm (Alg. 11, provided in Appendix D). The deterministic version simply evaluates all possible paths and returns the best solution. In the following, we provide the theoretical guarantee and its analysis under different constraints.

Theorem D.3. Let ( f, I ) be an instance of SM, with the optimal solution set O . Algorithm 10 achieves an expected (0 . 385 -ε ) approximation ratio with ( ℓ +1) -ℓ success probability and input t = 0 . 372 under size constraint, where ℓ = 10 9 ε . Moreover, it achieves an expected (0 . 305 -ε ) approximation ratio with t = 0 . 559 under matroid constraint. The query complexity of the algorithm is O ( kn/ε ) .

D.2.1 Size constraints

By Lemma D.2 in Appendix D.1.1 and the closed form for a recurrence provided in Lemma A.3, the following corollary holds,

Corollary D.1. After iteration i of the for loop in Alg. 10, the following inequalities hold with a probability of ( ℓ +1) -i

Proof of approximation ratio. If f ( Z ) ≥ (0 . 385 -ε ) f ( O ) , the approximation ratio holds immediately. So, we analyze the case f ( Z ) < (0 . 385 -ε ) f ( O ) in the following.

Recall in Corollary C.1 that Z is a (1 + ε ) α, α ) -guidance set, it holds that f ( O ∪ Z ) + f ( O ∩ Z ) < (0 . 77 -1 . 615 ε ) f ( O ) and f ( O ∩ Z ) < (0 . 385 -0 . 615 ε ) f ( O ) .

By repeatedly implementing Lemma A.3 with the recursion in Corollary D.1, it holds that

D.2.2 Matroid Constraints

By Lemma D.1 in Appendix D.1 and the closed form for a recurrence provided in Lemma A.3, the following corollary holds,

Corollary D.2. After iteration i of the for loop in Alg. 10, the following inequalities hold,

Proof of approximation ratio. If f ( Z ) ≥ (0 . 305 -ε ) f ( O ) , the approximation ratio holds immediately. So, we analyze the case f ( Z ) < (0 . 305 -ε ) f ( O ) in the following.

Recall that Z is a (1 + ε ) α, α ) -guidance set in Corollary C.1, it holds that f ( O ∪ Z ) + f ( O ∩ Z ) < (0 . 61 -1 . 695 ε ) f ( O ) and f ( O ∩ Z ) < (0 . 305 -0 . 695 ε ) f ( O ) .

By repeatedly implementing Lemma A.3 with the recursion in Corollary D.2, it holds that

D.3 Derandomize Alg. 10 in Section 2.3

In this section, we present the deranmized version of Alg. 10, which simply evaluates all possible paths and returns the best solution. We reiterate the guarantee as follows.

Theorem 2.4. Let ( f, k ) be an instance of SM, with the optimal solution set O . Alg. 11 achieves a deterministic (0 . 385 -ε ) approximation ratio with t = 0 . 372 , and a deterministic (0 . 305 -ε ) approximation ratio with t = 0 . 559 . The query complexity of the algorithm is O ( knℓ 2 ℓ -1 ) where ℓ = 10 9 ε .

E Proofs for Nearly Linear Time Deterministic Algorithm in Section 3

In this section, we provide the pseudocode and additional analysis of our nearly linear time deterministic algorithm introduced in Section 3. We organize this section as follows: in Appendix E.1, we

provide a speedup version of GUIDEDIG-S which queries O ( n log( k )) times; in Appendix E.2, we analyze the subroutine, PRUNE; at last, in Appendix E.3, we provide the pseudocode of nearly linear time deterministic algorithm and its analysis.

E.1 GUIDEDIG-S Speedup

In this section, we provide the algorithm THRESHGUIDEDIG, which combines the guiding and descending threshold greedy techniques with INTERLACEGREEDY [11].

Lemma E.1. Let O ⊆ U be any set of size at most k , and suppose THRESHGUIDEDIG(Alg. 12) is called with ( f, k, Z, G, ℓ ) . Then THRESHGUIDEDIG outputs ℓ ( ℓ + 1) candidate sets with O ( ℓ 2 n log( k ) /ε ) queries. Moreover, with a probability of ( ℓ + 1) -1 , a randomly selected set A from the output satisfies that:

Proof. Let o max = arg max o ∈ O \ ( G ∪ Z ) ∆( o | G ) , and let { a 1 , . . . , a ℓ } be the largest ℓ elements of { ∆( x | G ) : x ∈ U \ ( G ∪ Z ) } , as chosen on Line 13. We consider the following two cases.

Case ( O \ ( G ∪ Z )) ∩{ a 1 , . . . , a ℓ } = ∅ . Then, o max ̸∈ { a 1 , . . . , a ℓ } which implies that ∆( a u | G ) ≥ ∆( o | G ) , for every 1 ≤ u ≤ ℓ and o ∈ O \ ( G ∪ Z ) ; and, after the first iteration of the while loop on

Line 13, none of the elements in O \ ( G ∪ Z ) is added into any of { A 0 ,i } ℓ i =1 . We will analyze the iteration of the for loop on Line 4 with u = 0 .

For any l ∈ [ ℓ ] , let A ( j ) 0 ,l be A 0 ,l after we add j elements into it, τ ( j ) l be τ l when we adopt j -th elements into A 0 ,l , and τ (1) l = M . By Line 7, it holds that A (1) 0 ,l = G ∪{ a l } . Since ( O \ ( G ∪ Z )) ∪ { a 1 , . . . , a ℓ } = ∅ , and we add elements to each set in turn, we can order O \ ( G ∪ Z ) = { o 1 , o 2 , . . . } such that the first ℓ elements are not selected by any set before we get A (1) 0 ,l , the next ℓ elements are not selected in any set before we get A (2) 0 ,l , and so on. Therefore, for any j ≤ | A 0 ,l \ G | and ℓ ( j -1) + 1 ≤ i ≤ ℓj , o i are filtered out by A 0 ,l with threshold τ ( j ) l / (1 -ε ) , which follows that ∆ ( o i | A ( j ) 0 ,l ) < τ ( j ) l / (1 -ε ) ≤ ( f ( A ( j ) 0 ,l ) -f ( A ( j -1) 0 ,l )) / (1 -ε ) ; for any ℓ | A 0 ,l \ G | < i ≤ | O \ ( G ∪ Z ) | , o i are filtered out by A 0 ,l with threshold εM k , which follows that ∆( o i | A 0 ,l ) < εM/k . Thus,

By summing up the above inequality with all 1 ≤ l ≤ ℓ , it holds that

̸

Since A u,l 1 ∩ A u,l 2 = G for any 1 ≤ l 1 = l 2 ≤ ℓ it holds that (( O \ Z ) ∪ A u,l 1 ) ∩ (( O \ Z ) ∪ A u,l 2 ) = ( O \ Z ) ∪ G . By repeated application of submodularity, it holds that

Therefore, the last two inequalities in the theorem hold.

Case ( O \ ( G ∪ Z )) ∩ { a 1 , . . . , a ℓ } ̸ = ∅ . Then o max ∈ { a 1 , . . . , a ℓ } . Suppose that a u = o max. We analyze that lemma holds with sets { A u,l } ℓ l =1 .

Similar to the analysis of the previous case, let A ( j ) 0 ,l be A 0 ,l after we add j elements into it, τ ( j ) l be τ l when we adopt j -th elements into A 0 ,l , and τ (1) l = M . By Line 10, it holds that A (1) u,l = G ∪ { a u } . Then, we can order O \ ( G ∪ Z ∪ { a u } ) = { o 1 , o 2 , . . . } such that the first ℓ elements are not selected by any set before we get A (1) u,l , the next ℓ elements are not selected in any set before we get A (2) u,l , and so on. Therefore, Inequality 8 also holds in this case, where A is a random set from { A u,l } ℓ l =1 .

̸

Since, a u ∈ O , and A u,l 1 ∩ A u,l 2 = G ∪ { a u } for any 1 ≤ l 1 = l 2 ≤ ℓ , it holds that (( O \ Z ) ∪ A u,i 1 ) ∩ (( O \ Z ) ∪ A u,i 2 ) = O ∪ G . Following the proof of case ( O \ ( G ∪ Z )) ∩ { a 1 , . . . , a ℓ } = ∅ , the last two inequalities still hold in this case.

Overall, since either one of the above cases happens, the theorem holds.

E.2 Pruning Subroutine

PRUNE is used in Alg. 14 to help construct the ( α, β ) -guidance set, where α = 0 . 377 and β = 0 . 46 . In this section, we provide the pseudocode, guarantee and its analysis.

Lemma E.2. Suppose PRUNE(Alg. 13) is called with ( f, A ) and returns the set A ′ . For every A ∈ A and its related output set A ′ ∈ A ′ , it holds that,

Algorithm 13: A pruning algorithm which deletes element with negative marginal gain

Proof. Let A i be A after we delete i -th element x i , A 0 be the input set A , A m be the output set A ′ . Since any element x i being deleted follows that ∆( x i | A i ) < 0 , it holds that f ( A i ) > f ( A i + x i ) = f ( A i -1 ) . Therefore, f ( A ′ ) = f ( A m ) > . . . > f ( A 0 ) = f ( A ) . The first inequality holds.

For any x i ∈ A \ A ′ , it holds that ∆( x i | A i ) < 0 . By submodularity,

The second inequality holds.

For any y ∈ A ′ \ T , since it is not deleted, there exists 0 ≤ i y ≤ m such that ∆ ( y | A i y ) ≥ 0 . By submodularity,

The third inequality holds.

E.3 Proofs for Theorem 3.1 of Alg. 14

Prior to delving into the proof of Theorem 3.1, we provide the following corollary first. It demonstrates the progression of the intermediate solution of THRESHGUIDEDIG, after the pruning process, relying on Lemma E.2 and E.1.

Corollary E.1. Let O ⊆ U be any set of size at most k . Then PRUNE ( THRESHGUIDEDIG ( f, k, ∅ , G, ℓ )) outputs ℓ ( ℓ +1) candidate sets with O ( ℓ 2 n log( k ) /ε ) queries. Moreover, with a probability of ( ℓ + 1) -1 , a randomly selected set A from the output satisfies that:

Theorem 3.1. Let ( f, k ) be an instance of SM, with the optimal solution set O . Algorithm 14 achieves a deterministic (0 . 377 -ε ) approximation ratio with O ( n log( k ) ℓ 1 2 ℓ 1 ℓ 2 2 ℓ 2 -1 ) queries, where ℓ 1 = 10 3 ε and ℓ 2 = 5 ε .

Proof. Following the proof of Theorem 3.1, we consider two cases of the algorithm.

Case 1. For every A ∈ Z , it holds that f ( O ∪ A ) ≥ 0 . 46 f ( O ) . Then, we prove that max C ∈ Z f ( C ) ≥ (0 . 377 -ε ) f ( O ) .

In the following, we prove the theorem by analyzing the random case of the algorithm, where we randomly select a set from the output of PRUNE (THRESHGUIDEDIG). Suppose that we successfullly select a set where the inequalities in Corollary E.1 hold. Let A i and A i -1 be random sets in Z i

and Z i -1 , respectively. Let ( 1 -1 ℓ 1 ) i ∗ -1 > 0 . 46 ≥ ( 1 -1 ℓ 1 ) i ∗ . Then, by Inequality (2) in Corollary E.1, when i < i ∗ , E [ f ( O ∪ A i )] ≥ ( 1 -1 ℓ 1 ) i f ( O ) ; when i ≥ i ∗ , f ( O ∪ A i ) ≥ 0 . 46 f ( O ) by assumption. By applying Inequality (1) in Corollary E.1,

Since we return the best solution in Z and G , it holds that f ( C ∗ ) ≥ E [ f ( A ℓ 1 )] ≥ (0 . 377 -ε ) f ( O ) . Case 2. There exists A ∈ Z , such that f ( O ∪ A ) < 0 . 46 f ( O ) . Then, we prove that max C ∈ G f ( C ) ≥ (0 . 377 -ε ) f ( O ) .

Suppose that f ( A ) < 0 . 377 f ( O ) . Otherwise, f ( C ∗ ) ≥ 0 . 377 f ( O ) immediately. By Inequality (3) in Corollary E.2, it holds that f ( O ∩ A ) ≤ f ( A ) < 0 . 377 f ( O ) . Let B ℓ 2 be a randomly selected set in G ℓ 2 , where we calculate G ℓ 2 with the guidance set A and inequalities in Theorem D.2 hold successfully. In the following, we also consider that randomized version of the algorithm, where we randomly select a set B i from all the solution set returned by THRESHGUIDEDIG. Suppose that we successfully select a set where the inequalities in Lemma E.1 hold. Then, the recursion of E [ f ( B i )] can be calculated as follows,

Then, by solving the above recursion, it holds that

F Experiments

Experimental setup We run all experiments on an Intel Xeon(R) W5-2445 CPU at 3.10 GHz with 20 cores, 64 GB of memory, and one NVIDIA RTX A4000 with 16 GB of memory. For Maximum Cut experiments, we use the standard multiprocessing provided in Python, which takes about 20 minutes to complete, while the video summarization finishes in under a minute.

F.1 Additional tables and plots

In this section, you can find the tables and plots omitted in the main paper due to space constraints. In Figure 5, we compare the frames selected by FASTLS +RANDOMGREEDY and STANDARDGREEDY, and in Figure 6, we report the results for Barab'asi-Albert and Watts-Strogatz models for Maximum Cut.

F.2 Problem Formulation

In this section, we formally introduce video summarization and Maximum Cut.

F.2.1 Video summarization

Formally, given n frames from a video, we present each frame by a p -dimensional vector. Let X ∈ R n × n be the Gramian matrix of the n resulting vectors so X ij quantifies the similarity between two vectors through their inner product. The Determinantal Point Process (DPP) objective function is defined by the determinant function f : 2 n →R : f ( S ) = log ( det ( X S ) + 1) , where X S is the principal submatrix of X indexed by S following Banihashem et al. [1] to make the objective function f a non-monotone non-negative submodular function.

Eg4pieces

Like Shaee sesubscribe

Like Share & Subscribe

(a) FASTLS +GUIDEDRG

(b) STANDARDGREEDY

F.2.2 Maximum cut

Given an undirected graph G ( V, E ) , where V represents the set of vertices, E denotes the set of edges and weights w ( u, v ) on the edges ( u, v ) ∈ E , the goal of the Maximum Cut problem is to find a subset of nodes S ⊆ V that maximizes the objective function, f ( S ) = ∑ u ∈ S,v ∈ V \ S w ( u, v ) .

F.3 Hyperparameters

For all experiments, we set the error rate, ϵ , to 0 . 01 for FASTLS +GUIDEDRG and to 0 . 1 for Lee et al. [21]. Additionally, for video summarization, we run RANDOMGREEDY 20 times and report the standard deviation of these runs. For all other experiments, we run the algorithms once per instance and report the standard deviation over instances.

0.0

0.2

0.4

k/n

0.8

0.9

1.0

Objective / Greedy

FastLS+GuidedRG

(a) BA, solution value

0.0

0.2

0.4

k/n

10 0

10 1

10 2

Queries / Greedy

(b) BA, queries

0.0

0.2

0.4

k/n

0.950

0.975

1.000

1.025

Objective / Greedy

0.0

0.2

0.4

k/n

10 0

10 1

10 2

10 3

Queries / Greedy

F.4 Datasets

The video we select [29] (available under CC BY license) for video summarization lasts for roughly 4 minutes, and we uniformly sample 100 frames from the video to form the ground set. For maximum cut, we run experiments on synthetic random graphs, each distribution consisting of 20 graphs of size 10 , 000 vertices generated using the Erd˝ os-Renyi (ER), Barabasi-Albert (BA), and Watts-Strogatz (WS) models. The ER graphs are generated with p = 0 . 001 , while the WS graphs are created with p = 0 . 001 and 10 edges per node. For the BA model, graphs are generated by adding m = 2 edges in each iteration. Data and code are provided in the supplementary material to regenerate the empirical results provided in the paper.

F.5 Implementation of FASTLS+GUIDEDRG

For our implementation of FASTLS, we take the solution of STANDARDGREEDY as our initial solution Z 0 ; the theoretical guarantee is thus f ( Z 0 ) > OPT /k , since Z 0 has higher f -value than the maximum singleton. This increases the theoretical query complexity of our algorithm as implemented to O ( kn ε log( k ε ) ) . Then, for each swap, we find the best candidates to remove from and add to the current solution set (including the dummy element), rather than any pair that satifies the criterion. For guided GUIDEDRG, we implement exactly as in the pseudocode (Alg. 6) - we remark that we could instead use (a guided version of) the linear-time variant of RANDOMGREEDY [10] to reduce the empirical number of queries further, but for simplicity we did not do this in our evaluation.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: All claims made in the abstract and introduction are matched by theorem statements with detailed proofs in the main text and appendices or by empirical evaluation.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: All formal theorem statements include the assumed hypotheses, and theoretical and empirical limitations are discussed explicitly in Section 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All assumptions are clearly specified. In the main text, we discuss the intuition behind the analysis and sketch the main arguments, while complete detailed proofs are provided in the appendices.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: All parameter settings and implementation choices for the evaluated algorithms are reported. Further, the source code and documentation is provided, with instructions for how to reproduce the results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The complete data, source code, and instructions to reproduce the experimental results are provided in the supplementary material. Once this manuscript is published, we plan to make this available on an open-source repository.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: No training is involved in the evaluation. All hyperparameters for random graph generation and for the algorithms are provided.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: In out plots, we plot the mean over 20 independent repetitions. The shaded regions of the plots indicate one standard deviation about the mean.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Our experiments are not large-scale, and modern desktop hardware should be sufficient to reproduce all results in a few hours.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conforms in every respect to the code of ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [No]

Justification: Ours is foundational research, not tied to any particular application or deployment. The applications upon which we implemented and empirically evaluated our algorithms are proof-of-concept and thus our provided implementations are unlikely to lead directly to negative societal impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We do not release any data or models that have a high risk for misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The only pre-existing asset used is the video in the summarization experiment. We explicitly mention the license in Appendix F and properly respect the terms.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The released source code is well documented.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the fundamental problem of improving the approximation factor for non-monotone submodular maximization subject to cardinality or matroid constraints. In particular, the focus of the work is on designing combinatorial algorithms as opposed to continuous methods based on the multilinear extension. \n\nThe paper has the following contributions:\nfor cardinality constraints:\n- a combinatorial $0.385 - \\varepsilon$ approximation is proposed, which runs in $O(nk/\\varepsilon)$. The algorithm is randomized but can be derandomized with a multiplicative deterioration of the query complexity which is exponential in $1/\\varepsilon$\n- a deterministic $0.377 - \\varepsilon$ approximation is obtained, with running time $O(n \\log k\\cdot C_{\\varepsilon})$, where $C_{\\varepsilon}$ is exponential in $1/\\varepsilon$. \n- for context, the previous best combinatorial algorithm provided a $0.367$ approximation, while the overall state of the art is a continuous $0.41$-approximation. \nfor matroid constraints: \n- a combinatorial $0.305 - \\varepsilon$ approximation is proposed, which runs in $O(nk/\\varepsilon)$. The algorithm is randomized but can be derandomized with a multiplicative deterioration of the query complexity, which is exponential in $1/\\varepsilon$\n- the previous best combinatorial algorithm provided a $0.283$ approximation, while the overall state of the art is a continuous $0.41$-approximation.\n\nSTRENGTHS:\n- Submodular maximization is an important problem to the ICML and NeurIPS  community, with many practical applications and a rich literature in these conferences\n- Closing the approximation gap for non-monotone submodular functions is an exciting research question. To get some context, the current state of the art, i.e., the 0.401 approximation by Buchbinder and Feldman, has recently been presented at STOC, one of the top conferences in theoretical computer science\n- finding combinatorial algorithms is important as continuous algorithms are impractical to implement and extremely expensive in terms of queries to the submodular function\n\nWEAKNESSES:\nthe deterministic algorithms are impractical, as they feature an exponential dependence on the approximation parameter $1/\\varepsilon$. Their relevance is mainly theoretical.\n\nQUESTIONS:\nNone\n\nLIMITATIONS:\nThe paper has no potential negative societal impact", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors develop a combinatorial algorithm for non-monotone submodular maximization under both size and matroid constraints. Their algorithm uses local search to guide RANDOM GREEDY and INTERPOLATEDGREEDY, providing both a randomized algorithm and a deterministic algorithm. For size constraints, the algorithm achieves an approximation ratio of 0.385, and for matroid constraints, it achieves an approximation ratio of 0.305. It is the first combinatorial algorithm with an approximation ratio greater than 1/e for size constraints. The technique in this paper is similar to that in [6]. However, while the algorithm in [6] is a continuous algorithm requiring a query complexity of $\\Omega(n^{10}) $, the algorithm presented in this paper only requires $O(kn)$ queries of a value oracle. They also propose another algorithm based on this approach, which replaces the greedy part in INTERPOLATEDGREEDY with the descending thresholds technique. This algorithm achieves an approximation ratio of 0.377 for size constraints in nearly linear query time $O(k \\log n)$, which is slightly greater than 1/e.\n\nSTRENGTHS:\n1. The paper provides two new deterministic algorithms with approximation ratio better than 1/e.\n2. The paper provides practical randomized algorithm with both theoretical analysis and experimental evaluation.\n\nWEAKNESSES:\n1.\tThe results in the paper are not strong enough. For the randomized algorithm, the paper an algorithm with query complexity $O(kn/\\epsilon)$ and approximation ratio 0.385. It is worse than the most recent paper (https://arxiv.org/abs/2405.13994). For two deterministic algorithms, they are impractical since the dependence of $\\epsilon$ is too large.\n\nQUESTIONS:\nBoth the results and techniques in this paper are somewhat similar to those in the recent paper (https://arxiv.org/abs/2405.13994). It might be beneficial to explain the relationship between your paper and theirs, especially the difference in the techinique ideas.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the classical problem of maximizing a non-monotone submodular function subject to a cardinality and a matroid constraint.\nA long line of work has developed approximation algorithms for these problems which achieve a 0.401approximation; however, every algorithm which achieves better than $1/e = 0.368$ relies on the multilinear extension and is primarily of theoretical interest, as the run times are very large (but polynomial).\nThe more practical combinatorial algorithms have not been able to break the $1/e$ barrier.\n\nThe main contribution is a suite of algorithms which work via an \"guided\" local search procedure.\nThe main algorithm works as follows: a local search algorithm obtains a first solution and then an \"guided\" local search uses this first solution to obtain a second one. \nIf the parameters of the algorithm is appropriately set, at least one of these two solutions is of high (expected) quality.\nThe main result is that this (randomized) algorithm achieves a $0.385-\\epsilon$-approximation for cardinality constraint and $0.305-\\epsilon$ approximation for matroid constraint and runs in $\\mathcal{O}(n k / \\epsilon)$ time.\n\nThere are a variety of other results, including several de-randomizations of this algorithm, though those are mostly of theoretical interest because their run time is exponential in $\\epsilon$.\nSimulations are run comparing the algorithms to two reasonable benchmarks.\n\nSTRENGTHS:\nThe main strength of the work is in the development of simple algorithms with improved approximation ratios for submodular maximization.\nIn particular, the randomized algorithm achieving $0.385-\\epsilon$-approximation for cardinality constraint and $0.305-\\epsilon$ approximation for matroid constraint is the main contribution of the paper.\nOf particular note is that this algorithm is analyzed for both cardinality and matroid constraints, though different switching times $t$ should be used.\n\nI really enjoyed the clarity of the writing in Section 2.2 which provided intuition for the theoretical results.\n\nWEAKNESSES:\n**Paper Clarity**: For the most part, the paper is not well organized and can be challenging to read. With the exception of Section 2.2, the writing is a bit rushed and lacking focus. Many times I had to go back and forth in the paper just to read it. Perhaps the reason for this is that several algorithms are being presented with various guarantees and settings. While the results are great, the paper requires some serious efforts in improving the readability.\n\n**Simulations**: While I appreciate the authors' work in constructing simulations, they are admittedly quite weak or at the very least poorly explained. This may be due in large part to space constraints. I summarize the issues here:\n\n- The objective functions are not presented and the movie summarization dataset is not discussed.\n- It is claimed that max-cut is run on 3 different random graph models -- but then how does only one line appear in the Figure 3c?\n- If FastLS + GuidedRG is random, then why does it not also have some uncertainty quantification (i.e. standard error bars) on its objective value and run time?\n- The simulations are weak because in all of these instances, the more sophisticated algorithms provide very little improvement over the greedy algorithm. I'm sympathetic to this phenomenon, I know it can happen. But then I think that the experiments should focus on practical instances where the sophisticated algorithms do actually improve over the greedy algorithm.\n\nQUESTIONS:\nI thank the authors for their manuscript and invite them to respond to the points raised in my review. \nHowever, I do not feel that I have any questions to raise at this time.\n\nLIMITATIONS:\nAll limitations sufficiently addressed.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates combinatorial approximation algorithms for constrained submodular maximization problems. Observing the algorithms that pass the $1/e$ threshold are generally carries the problem to the continuous domain, they investigate the answer to the following question: \"Is it possible to obtain approximation ratios that are better than $1/e$ for constrained submodular maximization problems with combinatorial algorithms?\" They answer this question affirmatively by combining Buchinder et al.'s RandomGreedy algorithm with a novel fast local search algorithm. Additionally, they propose derandomized versions of these algorithms by altering the InterpolatedGreedy algorithm by Chen and Kuhnle.\n\nSTRENGTHS:\nThe paper improves the known existing approximation ratios without resorting to continuous algorithms while using the same number of queries. More specifically, they improve the combinatorial approximation ratios of Buchbinder et al. from $1/e \\approx 0.367$ to $0.385 - \\varepsilon \\approx 1/e + 0.018$ for the cardinality constraint and from $0.283$ to $0.305 - \\varepsilon$ for matroid constraints while still requiring $\\mathcal{O}(kn)$ queries. The idea of guiding the RandomgGreedy algorithm with the Fast Local Search (FastLS) is original. The ideas are conveyed clearly and the improvement over the state of the art for combinatorial algorithms is quantifiable.\n\nWEAKNESSES:\n- On lines 73-75, you mention \"Unfortunately, there is no known method to derandomize continuous algorithms, as the only known way to approximate the multilinear extension relies on random sampling methods.\" You may want to revise this sentence because as far as I know, there is a work that proposes estimating the multilinear relaxation of coverage-like functions with Taylor series expansions instead of sampling. Citation: Özcan, Gözde, Armin Moharrer, and Stratis Ioannidis. \"Submodular maximization via Taylor series approximation.\" Proceedings of the 2021 SIAM International Conference on Data Mining (SDM). Society for Industrial and Applied Mathematics, 2021.\n\n- On line 283, you mention an InterlaceGreedy algorithm for the first time as far as I can tell. Is this a typo or a different algorithm? If it is a different algorithm, where it is described?\n\nQUESTIONS:\n- On lines 114-115, you mention using an LP method for derandomizing the RandomGreedy algorithm and how this method did not work in this case. Could you please elaborate more on why this was the case? Was the problem asymptotic query complexity or something else?\n\n- Why the dummy elements are needed?\n\n-\n\nLIMITATIONS:\nBoth in Sections 1.1 and 5, the authors discuss the limitations of their work fairly.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-17"}